How to Calculate Percentiles Response Time From JMeter Aggregate Reports

In one of my project we used JMeter to test the performance of our web application.

As we know JMeter provide the percentile response time [default 90%] in the form of aggregate report, but our requirement was to generate the report having request and its given percentile response time [default 90%] in millisecond for current build and compare it with the benchmark report [STD response time set] and log the bug if percentile response time for particular request is greater than the expected.

So in this scenario we had needed to find out the way how JMeter calculate the response time for particular request.

Below are the steps and code I have written to achieve this.

·        When JMeter run the performance test it creates the resultAggregateReport_TestName.xml files in testResult folder

·        Below is the sample xml.

<?xml version="1.0" encoding="UTF-8"?> <testResults version="1.2"> <httpSample t="2443" lt="2091" ts="1361266219161" s="false" lb="login initial" rc="200" rm="OK" tn="Login Logout Many Thread Group 1-1" dt="text" by="44691" ng="1" na="6">   <assertionResult>     <name>Response Assertion</name>     <failure>false</failure>     <error>false</error>   </assertionResult>   <assertionResult>     <name>Duration Assertion</name>     <failure>true</failure>     <error>false</error>     <failureMessage>The operation lasted too long: It took 2,443 milliseconds, but should not have lasted longer than 1,500 milliseconds.</failureMessage>   </assertionResult> </httpSample> </testResults>

·        So for particular request JMeter capture the response time for lb from <httpSample> tags t attributes (where lb is label and t is Response/ Elapsed time (milliseconds)) and calculate the percentile response time which is by default 90 %

·        Below is the sample aggregate report showing % response time for particular lable/request

So we have the test result data available in the form of XML to calculate the percentile response time.

Below is the code to parse the aggregate report xml.

/**          * This method will parse the aggregate test result XML and return the          * HASHMAP having request id/label as key and "," separated response time as          * value          *          * @param path          * @return          */         public HashMap<String, String> parseAggregateXML(File file,                        HashMap<String, String> reportMap) {                try {                        tempReportMap = reportMap;                        SAXParserFactory factory = SAXParserFactory.newInstance();                        SAXParser saxParser = factory.newSAXParser();                        DefaultHandler handler = new DefaultHandler() {                               public void startElement(String uri, String localName,                                              String qName, Attributes attributes)                                              throws SAXException {                                       if (qName.equals("httpSample")) {                                              int length = attributes.getLength();                                              // Each attribute                                              for (int i = 0; i < length; i++) {                                                      if (attributes.getQName(i).equals("lb")) {                                                             key = attributes.getValue(i);                                                      }                                                      if (attributes.getQName(i).equals("t")) {                                                             responseTime = attributes.getValue(i);                                                      }                                              }                                              if (key != null && tempReportMap.containsKey(key)) {                                                      String responseValues = tempReportMap.get(key);                                                      responseValues = responseValues + ","                                                                     + responseTime;                                                      tempReportMap.put(key, responseValues);                                                                                                   } else if (key != null && responseTime != null) {                                                      tempReportMap.put(key, responseTime);                                                                                                   }                                       }                               }                               public void endElement(String uri, String localName,                                              String qName) throws SAXException {                                       if (qName.equals("httpSample")) {                                                             key = null;                                       }                               }                               public void characters(char ch[], int start, int length)                                              throws SAXException {                               }                        };                        InputStream in3 = new FileInputStream(file);                        saxParser.parse(in3, handler);                } catch (Exception e) {                }                return tempReportMap;         }

As we have many xml as a part of aggregate test result we pass those xml to above parser which return the hash map as result having label/request url as key and “,” separated response time captured from all Xml for particular request.

Now we are ready with the data, next we will see how to calculate the percentile response time from this available data.

In below method we will pass the HashMap created in above method. Which convert the “,” separated value in double array and pass It to Percentile.evaluate(double [] array)  . The resultant double array map will hold the percentile response time for particular request.

I used org.apache.commons.math.stat.descriptive.rank.Percentile to calculate the percentile value.

With this custom method we can calculate up to 99.99 percentile response time for any request

public HashMap<String, String> generatePercentileReportMap(HashMap<String, String> reportMap,                        double percentage) {                Iterator iter = reportMap.entrySet().iterator();                Percentile responsePercentile = new Percentile(percentage);                while (iter.hasNext()) {                        Map.Entry pairs = (Map.Entry) iter.next();                        String key = (String) pairs.getKey();                        String values = (String) pairs.getValue();                        double[] responseArray = ConversionUtil                                       .convertStringArrayToDouble(values); //find it below                        reportMap.put(key, Double.toString(ConversionUtil                                       .roundTwoDecimals(responsePercentile                                                      .evaluate(responseArray))));                }                               return reportMap;                        }

/**         * To convert string array to double array         * @param array         * @return         */        public static double[] convertStringArrayToDouble(String array){                           String[] responseTimeArray = array.split(",");              int length = responseTimeArray.length;            final double[] responseArray = new double[length];            for (int i=0; i < length; i++) {               responseArray[i] = Double.parseDouble(responseTimeArray[i]);                         }            return responseArray;               }