Adding Timing Flags to Test Cases within Katalon

I've made reference to this before, but due to the success I've had using it, I wanted to bring it up again.

Within my tests cases I have been adding markers to time how long it takes to complete an action. This is usually connected with the Save action or something similar. For example, how long does it takes to save the form data? Or how long does it take to move from one page to another.

Using the "feature flag" idea, I place the start of the code right before there is an action to click the button. After the click action, there is a waitForElementVisible action, which waits for an email address input field to become visible. Once we can take action on it, the Save action is complete and we can record the time between those two events.

if (timedTest==true){
    timeStart = new Date()
}
WebUI.click(findTestObject('btn-Save Cart))
WebUI.waitForElementVisible(findTestObject(input-Email Address'), 90)
if (timedTest==true){
    timeStop = new Date()
    TimeDuration duration = TimeCategory.minus(timeStop, timeStart)
    log.logWarning("Execution Time: " + duration)
}

The same process works when moving through pages. Click the link or button that moves to the next page and wait for an item to appear that proves the page has been loaded. Take the time between those two events and output it.

Additionally, I have been outputting these times to a text file so I can gather some benchmark data. As an example, the following is listed within the timedTest flag.

    File file1 = new File('/FileLocation/savetimes.txt')
    file1 << "Execution Time: " + timeStop +"," + duration"

This has been a huge benefit for our performance testing. I can run the same test dozens of times of and record how long it takes. I don't have to guess or write it all down manually. It all becomes part of the test and gets tracked over time.

After we make a change, I run the same tests again and record the new results. This easily creates the Before and After benchmarks.

You can even add a small bit of code to give you a visual cue things are going south.

    String timeCounter=duration
    timeCounter=timeCounter.replaceAll(' seconds','')
    float timeValue=Float.valueOf(timeCounter)
    if (timeValue>15){
        log.logError('ERROR: The save time for the item is higher than expected')
        KeywordUtil.markError('ERROR: The save time for the item is higher than expected')
    }

Katalon Studio may not be a performance or load balancing tool, but you can easily track metrics and show processes that are taking longer than normal. This probably isn't part of a regression test, but by combining the feature flags with the TimeDuration library you can record all sorts of metrics about your test which can be incredibly value.