quinta-feira, 12 de abril de 2012

JMeter - Using BSF Assertion to fail samples based on JSON response

I've tested some web applications whose unexpected errors or exceptions were handled by the application code and encoded in the response body as JSON.

A hypothetical response is shown below:

    data: null,
    message: "TimeoutException: Transaction rolled back after 3000 seconds.",
    success: false

The problem with this approach is that most of the times the HTTP response code is 200 (OK) which  jmeter interprets as a successful sample.
I'm not a big fan of this solution I'd rather have it returning 4XX or 5XX response codes, in which case jmeter would fail the sample.

In order to workaround this, I normally use a BSF Assertion containing javascript code similar to:

try {
   eval('var response = ' + prev.getResponseDataAsString());
   if (!response.success) {
} catch(e) {
      prev.setResponseMessage("Invalid response. Expected a valid JSON.");

First I try to evaluate the response string into a valid javascript object. If that succeeds I check the success flag and handle it properly, otherwise, in the catch block, I force the sample to fail due to a bad response format.
Note that in both failure scenarios I invoke the setSuccessful and the setResponseMessage methods to signal a sample failure and have better detailed error messages respectively.

And that's it!
Happy load testing!