Creating a Data Driven Test Suite in Katalon Studio
In order to extend the reach of my regression tests, I wanted to run a Test Suite multiple times for different users, one after the other. My goal is to log in as User A, run the regression suite, log in as User B, run the regression script again.
For example, I want to log in as John, check multiple pages worth of sales figures to perform calculations and comparisons. I then want to log in as Jane and do the same thing.
I currently have a Login script that I edit between each run with the name of the user I want to be. It works, but I want it to run for a list of users rather than me having to make an edit between each run.
Katalon supports Data Driven tests, where you can supply information to be read from a file. However, that applies to a single Test Case, not a Test Suite. It means the Test Case will iterate through all the data in the file before moving on to the next Test Case in the list.
Based on the scenario above, it would log in for User A, then immediately log in for User B, then User C, and continue do this for each name in the file. At the end of the list, the next test in the suite would be executed. It would read the sales data for the last person in the list.
What I need is for the data to be applied at the Suite level, which isn’t currently supported. However, there is a way to make this work that only takes a couple of steps. In essence, the Login script call numerous Test Cases after reading in the name of the user. It’s not quite as drag and drop as building a Test Suite, but it works.
To convert my Test Suite to work in my Test Case I took the following steps. Since the Test Suite is an XML file it can be easily read. Using a Terminal session and grep, I grabbed all the <testCaseId> lines and wrote them to a file.
grep "Test Cases" QA\ Regression\ Test.ts > testcases.txt
I now have a list of all my tests in the form:
<testCaseId>Test Cases/Sales Dashboard Tests/Main Dashboard Tests/Check Import Date</testCaseId>
The next step is open the testcases.txt file into an editor or spreadsheet such as LibreOffice. Using the Search and Replace function, strip out the <testCaseId> and </testCaseId> tags. This is now the list of Test Cases we want to run and will be added to the Login script.
From here, it’s easy to combine the callTestCase function and append the name of all the test cases to it.
In a spreadsheet, paste all the Test Cases into Column A.
In Column B enter:
In Column C, enter:
'), [:], FailureHandling.CONTINUE_ON_FAILURE)
In Column D, enter
This takes the callTestCase command, appends the name of the actual test case, then appends the closing syntax. This generates:
WebUI.callTestCase(findTestCase('Test Cases/Sales Dashboard Tests/Main Dashboard Tests/Check Import Date'), [:], FailureHandling.CONTINUE_ON_FAILURE)
Fill this formula down for each Test Case.
With all the commands complete, copy the entire list into the Test Case you want to iterate through, in my case, Login.
The final test case will look similar to:
WebUI.navigateToUrl(GlobalVariable.baseurl + '/login') WebUI.setText(findTestObject('Page_/input_login_Field'), impersonateUser) WebUI.delay(1) WebUI.click(findTestObject('Page_/btn_Submit-Login-Credentials')) WebUI.delay(1) WebUI.callTestCase(findTestCase('Test Cases/Sales Dashboard Tests/Main Dashboard Tests/Check Import Date'), [:], FailureHandling.CONTINUE_ON_FAILURE) WebUI.callTestCase(findTestCase('Test Cases/Sales Dashboard Tests/Main Dashboard Tests/Confirm Dashboard Link'), [:], FailureHandling.CONTINUE_ON_FAILURE)
I have 50 callTestCase commands, the full list of my Test Suite, in the Login Test Case. This means my new Test Suite only has 2 Test Cases listed. The first is to load the web page details. The second is the Login test. Login now replaces the previous Test Suite.
When I run this new test, it gets to the Login script, loads a user, then executes 50 regressions tests. When that cycle is complete, it loads the next user and repeats the process.
Also note the FailureHandling.CONTINUE_ON_FAILURE that has been added to each Test Case. This means the regression tests will keep running if one of them fails. Without this, as soon as an error condition is reached, all the tests will be halted. This includes an element missing, the markFailed command and quite a few others.
While not quite as robust as a regular Test Suite, this method works and will allow a series of tests to be run multiple times. It took less than 10 minutes to convert my Test Suite to this method and get it working. For now, it’s a pretty reasonable workaround.