Archive | LoadRunner RSS feed for this section

Compression,Decompression,Mobile Performance and LoadRunner

4 Feb

Recently I inherited some of the  LR scripts from one of my colleagues,it was all about building the json calls for stressing the backend spring security framework which was first layer of entry into the mobile infrastructure.Those scripts were simple scripts  built using the custom request with json string as a body part.One of the things that really surprised me as part of this effort was that web custom request in itself was taking close to 100ms to 300ms to do decompression of the server response during load testing.

Okay first let me give you some background,servers were configured to send the response compressed in gzip format with content encoding header as gzip.The functionality under scope had SLA of 1 sec max and quite a few functionality in scope also had SLA that was less than 500ms.Quite a challenging SLA’s I would say.But again these functionality were supposed to be accessed over the mobile device,so probably less the response time better it is for users.

Most of the response coming from the server for all functionality was served as chunked bytes,so what it means is that server sends initially some bytes as response in compressed gzip format,LR decompresses  those bytes in 5 to 10ms and then again server sends next range of bytes as chunked gzip response and then again LR will spend close to 5 to 10ms to decompress those bytes and like wise the process continues till we have final set of bytes.All these process happens in the single connection and connection never closes with the server.In case if you do have some server response validation in place, then expect that it will add another 10ms to do that validation.

Now I have measured all these times in the single iteration of vugen,these times increase exponentially when we are running the Load Test in controller or PC and this overhead of decoding the gzip content becomes a quite an issue when response time SLA are in ms.

Here is how it looks when you see the behavior in LR Vugen with decompression on in the script.You can see that it takes 5ms to decode the 154 bytes of response.Now imagine the normal webpage will have size of 2mb of data gzipped,so you can see the impact of this decoding  when size of page increase specially when response is coming as chunked bytes with no fixed content length from the server.

pic1

 

I think HP LR team might also be aware of this behavior and probably that the reason as why they might have come up with function to disable this.Use Web set option with decode content flag turned off if you are running the scripts which do not require validation and has response time SLA’s in ms.The drawback of disabling this feature is that all your correlation and other checks for server response will fail since server response will show up as binary content like below.

pic3

 

I would suggest you to disable this feature if you can and do the response validation by using the other techniques like verifying server logs etc.By disabling this you will gain close to 15 to 20% reduction in response time reported by LR.

Is this expected behavior of LoadRunner ?, I think they have to do this,unless they decode the response, none of the other function like web reg save param or web reg find will work and these functions are core functions of LoadRunner.Probably the right way is that LR should not add these decompression timing in their transaction markers.These timing really pollute the results specially for web applications or probably they can increase the speed of this decompression library what they are using in LoadRunner.

Advertisements

Performance Testing Web based Ajax Applications–Then Read this

23 Jun

From past couple of days,lot of people have asked me about Ajax,Ajax protocol and how to Load Test Ajaxified Applications. So I though let me write a short code and try explain my thoughts about this. Ajax is often used in the web site to do quick and short interaction with servers for implementing some of the if then condition to implement some requirement.Ajax helps a lot in bringing some interactivity or I can say it creates VOW experience.Of course I am not the hard core front end engineer ,but yes having written couple  of lines of CSS/HTML/Javascript, I can visualize and make a sense as how to write the front end code which brings some interactivity to the site.

For the sake of this post, I have used Jquery library which has rich methods for making Ajax calls.It also has excellent methods of interacting with CSS and HTML markup elements.With Jquery we can easily write the call back function and write the response data to the DOM on the fly based on some conditions.Few drawbacks Which I noticed using  Jquery library is that it makes you lazy (If you really want to become good front end engineer, then having good gasp of Raw javascript is must) and just for implementing one or 2 functionality , I have to import 10 k lines of code.But again with Jquery benefits outweighs the risks.

Implementation of Ajax with Jquery looks something like below

$(document).ready(function(){
    $(“form”).submit(function(event){  
         event.preventDefault();
        var aemail = $(“#idtxt1”).val();
        var apassword = $(“#idtxt2”).val();
           alert(aemail);
           alert(apassword);
        var request = $.ajax({  
            url: “TestAjax.do”,
            type: “POST”,
            data: {
               email:ae,password:ap
            },
            cache: false,
            ifModifiedBoolean:false,
            beforeSend:function(){
              $(“#idspan”).val(“”);  
            },
            success:function (data){
//                $(“#idspan”).append(data);
                if(data == “Success”){ //redirect…
                      window.location = “/mysite/mypage.php”;
                } else { //report failure…
                       $(“#idspan”).append(data);
                }

            },
            error: function(data) {
               $(“span”).append(“data”);
                 
            }
        })            
    });
});

 

The granular level of details of Jquery Ajax function can be found here. So you might be thinking as why I have written this piece of code when it’s already available. This post is not about Jquery or how to use Jquery, this post is about understanding Ajax and how they are implemented and based on that coming up with proper solution for load testing Ajax based web applications.

If you look at the above code closely,you can observe that it does post request on the form submit event.As soon as user clicks on the submit button it does the call to the backend program.This is how browsers operate.Browsers bring interactivity to the site using event driven methods.This understanding is the key to know as how Ajax calls are made, every Ajax call is associated with some user driven event on the HTML Element. Events could be on Click, on Submit, on Hover,onMousein,onMouseOut,Focus in,Focus out,onKeyPressup and onKeyPressdown.These are some of the events which are associated with the HTML Elements and Javascript is used bring some interactivity to the site on occurrence of the these events.So every Ajax call has some event associated with it.Please note that there are many many events associated with each HTML element and complete reference of those can be found ECMA Javascript Guide or Web Standards document.

So continuing further, the above code makes the Ajax post call to TestAjax. do function which resides on the backend server.In above Ajax function call I am collecting value of 2 text fields name #idtxt1 and #idtxt 2 and passing those values as post body request to my backend J2ee program.Most of the Ajax calls irrespective of the library used do these things in the similar way(almost 99.99% of time),they capture the user inputs with Javascript methods and then post the data to the backend program which resides on some application servers.The program which resides on the backend servers communicates to the Ajax call whether the request has passed or succeeded.If the request succeeds , for example in above code if my backend program sends me the data as “success”, I redirect the request to mypage.php and if requests fails, I write some error text to the HTML  page within Span Element.

The browsers developers tools also helps in understanding the way Ajax often works,

      image

If you capture the network trace with Browser’s developer’s tool, the trace would look something like above, in the trace you are clearly see that Initiator of the call was  XMLHttpRequest. This is nothing but the core method which implements Ajax calls in the browsers.Since the above request was of type GET, values were appended to the url of the request and send to the backend program.In addition to this these tools bar also give lot of other information like http status code, which event initiated the request etc.However I will not suggest you to measure the response time of Ajax calls with these tools as I believe they somewhat give incomplete picture of response time.

The response data received from the server can also be viewed via browser developer’s tool and in my case it was looking some thing like below,

image

This response data(“Username not available”) is later appended to the page elements which is viewed the end users.

So coming back to original purpose of this post, I keep hearing from various Performance Engineers that existing Web Http protocol is insufficient in testing Ajax based web applications, now having implemented real time Ajax with fat datasets, I believe there isn’t much challenge to load test Ajax based applications.We just need to keep some key things in mind while working with Ajax based applications,

  • Understand the functionality of your application from technical prospective.
  • Ask developers explicitly, which functionality in the application is using Ajax call, is the Ajax call synchronous or Asynchronous.With Synchronous Calls,one cannot proceed unless he receive the data from the server for his Ajax call and with Asynchronous calls, one can work on the other parts of the page(Technically with Async Calls, browser need not wait for server response to build DOM Tree and with Sync calls, it has to wait for server response).
  • If you believe some events for your applications are not getting recorded via regular HTTP protocol, then probably you are not triggering the Ajax call at all.Remember to trigger Ajax call, you need to Trigger an event on the html element and it could be that you need to tab out of the element or bring focus in that element etc etc.Ask your developer as how to trigger a Ajax call for the business process under scope.
  • If you believe that you are not able to record Ajax call, then probably your Ajax request is cached.Ajax calls are heavily cached by browsers since they contain all JS/CSS files in them.Clear browser cache/cookies etc etc and Try again.Use Developer tools bars to debug such cases,ensure that you get 200 status for all your resources.
  • Ajax calls irrespective of libraries or technology uses regular GET/POST types which is nothing but http calls.Http calls should and must be recorded if tool claims to support HTTP Protocol.
  • If you see some unique values getting generated at the client side and these values are not seen in the server response, don’t get scared or nervous, they might be Unix style timestamp or Microsoft tick timestamp(However If you get Tick, you have solid reason to worry and it could be your pure good luck if your application is not using complete power of Tick.If it heavily uses Tick, then probably you need to go temple and prey God.You will surely require his blessing.Most of the current set of tools don’t go beyond Unix style timestamps and Tick is much more powerful timestamp format than Unix style timestamp).These values are generated by JS library in order to force browser not to cache the key JS files.However lot depends on the headers as well.
  • Thick client web based applications often uses chained JS calls to build different parts of the page, All you need to do in these cases is to ensure that you follow the right steps, trigger the events which chains many events during recording and then do your regular steps.
  • Remember for Load Testing Ajax based application still the goal would be capture the network traffic which goes out of the application to the server and stress the server for those calls.If your ajax calls are slow, then it gives an impression that front end is taking more time,most often it is never true.Spinning wheel which keeps spinning for minutest or seconds indicates server bottleneck and not client side bottlenecks.
  • Remember client side performance measurement metrics/techniques are different than Server side performance measurement/techniques.They require different skillsets and tools.Just having lot of Ajax calls do not necessarily mean that you have client side performance issues.However it does mean that you do lot of DOM Work and always browser needs to keep working or guessing so as to where and how much some space for response data.So repainting and refills of DOM happens quite often.

So finally you must be wondering if Ajax can be done with regular http protocol, then why are companies like HP coming up with new Ajax based protocol etc for Load Testing.

Answer is simple, they want to save some time for scripting and time is money in corporate world.But again if you have money, it does not always mean that you will save time.

How much time it saves ?

Again I cannot say as I have never used these protocols myself.But I really have a doubt whether they can successfully emulate the calls for all events.There are lot many ways to trigger Ajax calls.If you use Ajax protocol without understanding the fundamentals of Ajax, then I would say probably you have miles to go before you become a  Performance Engineer.There exists a high risk that your text checks will always succeed not matter what as most tools do not interact with DOM , so they cannot not read or write to the DOM on the fly.With Ajax most of the time error validation is done without browser refresh.So you have extra cautious here.

If you still believe we cannot test Ajax based application with regular http protocol, then I would like to hear from you about such cases and would appreciate your feedback with some sample test use case.

Performance Testing Web based Ajax Applications–Then Read this

23 Jun

From past couple of days,lot of people have asked me about Ajax,Ajax protocol and how to Load Test Ajaxified Applications. So I though let me write a short code and try explain my thoughts about this. Ajax is often used in the web site to do quick and short interaction with servers for implementing some of the if then condition to implement some requirement.Ajax helps a lot in bringing some interactivity or I can say it creates VOW experience.Of course I am not the hard core front end engineer ,but yes having written couple  of lines of CSS/HTML/Javascript, I can visualize and make a sense as how to write the front end code which brings some interactivity to the site.

For the sake of this post, I have used Jquery library which has rich methods for making Ajax calls.It also has excellent methods of interacting with CSS and HTML markup elements.With Jquery we can easily write the call back function and write the response data to the DOM on the fly based on some conditions.Few drawbacks Which I noticed using  Jquery library is that it makes you lazy (If you really want to become good front end engineer, then having good gasp of Raw javascript is must) and just for implementing one or 2 functionality , I have to import 10 k lines of code.But again with Jquery benefits outweighs the risks.

Implementation of Ajax with Jquery looks something like below

$(document).ready(function(){
    $(“form”).submit(function(event){  
         event.preventDefault();
        var ae = $(“#idtxt1”).val();
        var ap= $(“#idtxt2”).val();
           alert(ae);
           alert(ap);
        var request = $.ajax({  
            url: “TestAjax.do”,
            type: “POST”,
            data: {
               email:ae,password:ap
            },
            cache: false,
            ifModifiedBoolean:false,
            beforeSend:function(){
              $(“#idspan”).val(“”);  
            },
            success:function (data){
//                $(“#idspan”).append(data);
                if(data == “Success”){ //redirect…
                      window.location = “/mysite/mypage.php”;
                } else { //report failure…
                       $(“#idspan”).append(data);
                }

            },
            error: function(data) {
               $(“span”).append(“data”);
                 
            }
        })            
    });
});

 

The granular level of details of Jquery Ajax function can be found here. So you might be thinking as why I have written this piece of code when it’s already available. This post is not about Jquery or how to use Jquery, this post is about understanding Ajax and how they are implemented and based on that coming up with proper solution for load testing Ajax based web applications.

If you look at the above code closely,you can observe that it does post request on the form submit event.As soon as user clicks on the submit button it does the call to the backend program.This is how browsers operate.Browsers bring interactivity to the site using event driven methods.This understanding is the key to know as how Ajax calls are made, every Ajax call is associated with some user driven event on the HTML Element. Events could be on Click, on Submit, on Hover,onMousein,onMouseOut,Focus in,Focus out,onKeyPressup and onKeyPressdown.These are some of the events which are associated with the HTML Elements and Javascript is used bring some interactivity to the site on occurrence of the these events.So every Ajax call has some event associated with it.Please note that there are many many events associated with each HTML element and complete reference of those can be found ECMA Javascript Guide or Web Standards document.

So continuing further, the above code makes the Ajax post call to TestAjax. do function which resides on the backend server.In above Ajax function call I am collecting value of 2 text fields name #idtxt1 and #idtxt 2 and passing those values as post body request to my backend J2ee program.Most of the Ajax calls irrespective of the library used do these things in the similar way(almost 99.99% of time),they capture the user inputs with Javascript methods and then post the data to the backend program which resides on some application servers.The program which resides on the backend servers communicates to the Ajax call whether the request has passed or succeeded.If the request succeeds , for example in above code if my backend program sends me the data as “success”, I redirect the request to mypage.php and if requests fails, I write some error text to the HTML  page within Span Element.

The browsers developers tools also helps in understanding the way Ajax often works,

      image

If you capture the network trace with Browser’s developer’s tool, the trace would look something like above, in the trace you are clearly see that Initiator of the call was  XMLHttpRequest. This is nothing but the core method which implements Ajax calls in the browsers.Since the above request was of type GET, values were appended to the url of the request and send to the backend program.In addition to this these tools bar also give lot of other information like http status code, which event initiated the request etc.However I will not suggest you to measure the response time of Ajax calls with these tools as I believe they somewhat give incomplete picture of response time.

The response data received from the server can also be viewed via browser developer’s tool and in my case it was looking some thing like below,

image

This response data(“Username not available”) is later appended to the page elements which is viewed the end users.

So coming back to original purpose of this post, I keep hearing from various Performance Engineers that existing Web Http protocol is insufficient in testing Ajax based web applications, now having implemented real time Ajax with fat datasets, I believe there isn’t much challenge to load test Ajax based applications.We just need to keep some key things in mind while working with Ajax based applications,

  • Understand the functionality of your application from technical prospective.
  • Ask developers explicitly, which functionality in the application is using Ajax call, is the Ajax call synchronous or Asynchronous.With Synchronous Calls,one cannot proceed unless he receive the data from the server for his Ajax call and with Asynchronous calls, one can work on the other parts of the page(Technically with Async Calls, browser need not wait for server response to build DOM Tree and with Sync calls, it has to wait for server response).
  • If you believe some events for your applications are not getting recorded via regular HTTP protocol, then probably you are not triggering the Ajax call at all.Remember to trigger Ajax call, you need to Trigger an event on the html element and it could be that you need to tab out of the element or bring focus in that element etc etc.Ask your developer as how to trigger a Ajax call for the business process under scope.
  • If you believe that you are not able to record Ajax call, then probably your Ajax request is cached.Ajax calls are heavily cached by browsers since they contain all JS/CSS files in them.Clear browser cache/cookies etc etc and Try again.Use Developer tools bars to debug such cases,ensure that you get 200 status for all your resources.
  • Ajax calls irrespective of libraries or technology uses regular GET/POST types which is nothing but http calls.Http calls should and must be recorded if tool claims to support HTTP Protocol.
  • If you see some unique values getting generated at the client side and these values are not seen in the server response, don’t get scared or nervous, they might be Unix style timestamp or Microsoft tick timestamp(However If you get Tick, you have solid reason to worry and it could be your pure good luck if your application is not using complete power of Tick.If it heavily uses Tick, then probably you need to go temple and prey God.You will surely require his blessing.Most of the current set of tools don’t go beyond Unix style timestamps and Tick is much more powerful timestamp format than Unix style timestamp).These values are generated by JS library in order to force browser not to cache the key JS files.However lot depends on the headers as well.
  • Thick client web based applications often uses chained JS calls to build different parts of the page, All you need to do in these cases is to ensure that you follow the right steps, trigger the events which chains many events during recording and then do your regular steps.
  • Remember for Load Testing Ajax based application still the goal would be capture the network traffic which goes out of the application to the server and stress the server for those calls.If your Ajax calls are slow, then it gives an impression that front end is taking more time,most often it is never true.Spinning wheel which keeps spinning for minute or seconds indicates server bottleneck and not client side bottlenecks.
  • Remember client side performance measurement metrics/techniques are different than Server side performance measurement/techniques.They require different skillsets and tools.Just having lot of Ajax calls do not necessarily mean that you have client side performance issues.However it does mean that you do lot of DOM Work and always browser needs to keep working or guessing so as to where and how much some space for response data.So repainting and refills of DOM happens quite often.

So finally you must be wondering if Ajax can be done with regular http protocol, then why are companies like HP coming up with new Ajax based protocol etc for Load Testing.

Answer is simple, they want to save some time for scripting and time is money in corporate world.

How much time it saves ?

Again I cannot say as I have never used these protocols myself.But I have some doubt whether they can successfully emulate the calls for all events.There are lot many ways to trigger Ajax calls.If you use Ajax protocol without understanding the fundamentals of Ajax, then I would say probably its incorrect way of doing the Job.There exists a high risk that your text checks will always succeed not matter what as most tools do not have JS Engine or HTML Parser in them, so they have limited ability to read or write to the HTML document.With Ajax most of the time error validation is done without browser refresh based on some conditions.So you have extra cautious here.

If you still believe we cannot test Ajax based application with regular http protocol, then I would like to hear from you about such cases and would appreciate your feedback with some sample test use case.

Technorati Tags: ,,

Sessions, Client Side Javascript and Correlation

8 Jun

Recently I one of my good friend asked me a question ” How do I correlate the session id information which is created via javascript with LoadRunner”

Well this was an interesting question and of course he was not the first one to ask about this, there are lot many people who had asked this question in various online forums like LinkedIn etc.So I thought let me write some of my thoughts on this,

Http protocol as known today technically is the stateless protocol.It means that it cannot distinguish between the first time visitor or a regular visitors.It just forgets the client as soon as request is serviced and this behavior of the HTTP creates the need for maintaining the session information.This is reason as why session generation and maintenance is handled by the programming languages.

Session management with http is normally done by using below techniques

URL Rewriting

With this method of session handling, session ids are often appended to the query string of the url and the query string looks something like below,

http://www.mydomain.com/mysite/someproduct.aspx%7someurl?name1=value1&name2=value2&name3=value3…etc

Depending on the way we design the application, session information can pass as query string parameters namely in key/value or name/value format and this applies even to session ids.Also at times this method is often used whenever client do not accept cookies for some reason.In fact some servers like web logic etc automatically switches the session handling to this method once they detect that client is not accepting cookies from the server.

URL Rewriting often  leads to ugly urls  so at times these url’s are hidden from the client and replaced by good and short looking url’s.This trick is done by server side modules which can rewrite the urls on the fly(Apache mod rewrite is one such example).The best way to identify if your application uses this method of session handling is to block all cookies via browser and test it.Also there exists some restriction on the length of the url with regard to some browsers like IE.IE cannot take length of the url beyond 2k bytes.

http://support.microsoft.com/kb/208427

Also using this method at times could lead to security hole as referrer tag are leaked if one navigates in the via same browser window.

Hidden fields

In this method of session handling, extra form fields are sent to the client and these extra form fields always hidden from the client.The hidden fields look something like,

<input type=’hidden’ name=’id’ value=’a’/>

Hidden fields are also used to maintain the page information across the pages and is one of the frequently used method  to maintain session information.The good thing of this method is that url’s remains short and large data can be passed over the server for processing.In addition to session information, hidden fields can also be used to persists the client entered data across the pages.

Cookies

I feel this is simplest way to maintain the session information and definitely better than url rewriting and hidden fields.Most application as soon as they receive first request, server drop the cookies on the client computer and later on for every subsequent request , they check for its existence in client requests with regard to  further communication.So if we as performance engineer comment out these cookies, then obviously we are asking application to do more work and is definitely a wrong load model as well.Cookies are never commented out in real world scenario and should not be commented out during load testing as well.

HttpSession Objects

These are objects which are created on the server side to store session information.These session objects are never passed on the client.However there might be cases where unique identifier namely jsessionid is appended to the url and send across to the client,in case client do not accept cookies.In this method session is stored in the memory and if the session object is too large, then it gets serialized to the file system or to some disk.Personally I have seen good performance with this method of session handling whenever the user base is small.How ever storing session in the httpsession objects could lead to heavy resources consumption given that sessions cannot moved outside the servers.In other words, application using this method of session handling cannot effectively use  server farms or web gardens as sessions are inproc. Also using this method leads to inefficient usage of load balancers.

Now having discussed some basics on sessions,so where does client side javascript fit in all these ?

Storing or accessing the session via javascript is all big security hole. I don’t recollect even a single instance where javascript was used to maintain sessions for any of my projects.So I doubt if any one uses Javascript to create session or store session information.

So with javascript can we create or access the sessions or session ids ?

I would say yes.Javascript provides rich set of methods to access url’s/cookies/hidden fields and with Ajax , we can definitely create and maintain the session information with javascript.However interesting point here is that what if client turns off javascript in their browsers ? 

Knowing  these risks, do you still believe that people are going to use Javascript to store session information ?

If yes, then probably I need to write another post on how to get around these type of cases given that no load generating tool existing in the market today can execute or capture javascript events correctly as done by the browsers in real sense.

reCaptcha’s,Simple Captcha,Javascript Captcha’s and Testing/Performance Engineering

10 May

I know for sure that most of us browsing the internet has at some point of time  come across captcha images and validation.Captcha Images validation techniques are  used with HTML forms in  web sites to prevent automated spams.I had recently implemented one functionality which was using Google reCaptcha technique to prevent automated spams.I had used the Google reCaptcha Java/Jsp plugin for my functionality.While implementing this reCaptcha functionality I came across interesting observations which I feel can help Performance Engineering community since I believe there exists some knowledge gaps in the way performance engineers understands captchas.The reason why I say this is that I have come across few folks misunderstanding and  asking me as how to capture or correlate captchas or at times I have seen people claiming to have defeated the captchas during their load testing assignments , of course without disabling the captchas validation.

Google recaptcha is more of  an web service call to the Google servers and uses some interesting techniques in validating the users input.So in Google reCaptcha’s there is actually 2 things we need to take care, first displaying the captcha in the front end and this can be done using the via below code,and second is verifying the inputs given by the user at the server side by passing required number of the parameters.

 

<script type="text/javascript"
     src=http://www.google.com/recaptcha/api/challenge?k=yourpublickey>
  </script>
  <noscript>
     http://span<br>
     <textarea name="recaptcha_challenge_field" rows="3" cols="40">
     </textarea>
     <input type="hidden" name="recaptcha_response_field"
         value="manual_challenge">
  </noscript>

The above code will display the captcha image in your web page and will built container which will have Captcha details in it along with text area field to write the image words.I suggest you use the above code at the head section of the HTML.The reCaptcha should look like below,

 

image

The above captcha consists of 2 words here one of the word is verification word (9th) and another is scanned text from some book which might belong to some ancient book or famous book or flop book or some super technical publication stuff which only Google folks read and this scanned text is then distorted to ensure that no OCR can read it correctly, and once it is confirmed that this text has failed OCR test, only then these types of images are fed in reCaptcha’s image database.

Now in order to fetch these captcha images in the front end, we see minimum 8 calls ,

image

 

One of the calls among these 8 calls has challenge field(recaptcha_challenge_field) which corresponds to the verification word as seen in the front end.Again we just know that its a long unique string generated at server side to uniquely identify the image and its answer.

So if you believe that correlating or capturing this long string means that you have solved captcha puzzle, then probably I would say, you are terribly wrong,read this post further to know why,

Now coming to the server side of validation,Lets use servlets to understand this process,

The first thing to do in the servlet program is to import the below packages in your validating servlet,

import net.tanesha.recaptcha.ReCaptchaImpl;
import net.tanesha.recaptcha.ReCaptchaResponse;

These 2 packages has method definitions to display error message encountered during validation and has method that can show us whether our captcha validation succeeded or failed.

String remoteAddr = request.getRemoteAddr();
ReCaptchaImpl reCaptcha = new ReCaptchaImpl();
        reCaptcha.setPrivateKey(“yourprivatekey”);
        String challenge = request.getParameter(“recaptcha_challenge_field”);
        String uresponse = request.getParameter(“recaptcha_response_field”);
        ReCaptchaResponse reCaptchaResponse = reCaptcha.checkAnswer(remoteAddr, challenge, uresponse);
         System.out.println(“challenge is ” + challenge + “‘”);
         System.out.println(” uresponse is ” + uresponse + “‘”);
         System.out.println(” remoteAddr is ” + remoteAddr + “‘”);

The above piece of code can written in the doPost method or doGet method of the servlet depending on the method type used in the main form.I have added to couple of print statements to ensure that we are indeed getting back the correct values from the front end.

If you look closely at the above code, we can see that there are at least 4 parameters which we need and among these 4 there are some parameters which we are collecting from the users namely

  • Remote IP Address of the client
  • Private Key of the site
  • User Response given by the user after seeing the captcha image at the front end
  • Challenge field.(This is same long string generated at the front end)

After passing  these 4 parameters correctly to the Google reCaptcha servers in the format as expected  by them, only then we say that the user is verifed as Human and not the automated bot.

Now what this all things means,

  • If we have some captchas which are third party, then probably we need unique IP address , in case we decide to test them.
  • There exists an option where we can test captcha as an isolated component given that most captcha solution are server side programs.
  • From the front end, its impossible to read the text embedded in the image since most of the matured captcha libraries present in the market host only images or use images which are distorted and cannot be read by OCR software,so testing captcha from the front end not a technically feasible approach.

So if someone tells you that they have tested the application which had captchas functionality in it, then probably you need to ask, how did they read the text embedded in the image ?

So moving on, is it the right thing to exclude captchas from the testing effort, I would say no, and I have certain reasons for this like

  • Captcha are blocking functionality features which according to me means that failed Captcha can block other working functionality.If captcha goes down for any reasons, user cannot do the submit or just cannot do any transaction on the page that benefits the site.So its loss to the site.
  • Captcha often has the blocking UI.I have seen this at least in my environment with recaptcha where till the captcha is loaded, certain portion of the page remains blank.
  • Since Captcha are server side programs, they suffer from same performance issues as seen in most of the server programs like server unavailable, client abort exception, 503 errors etc.
  • If we are using custom pure client side captcha solution like generating captcha using Javascript, then exists a risk that some day some smart user will just disable the javascript at his end and then do multiple submits to the site.Yes its possible to bypass client side captcha solutions that uses javascript solution with no server side verification .So if we are using this type of solution, then obviously testing is required end to end.

Captcha are interesting techniques and they do help to reduce spams, but they keeping them outside the scope of testing is not the right thing to do.

Technorati Tags: ,

Importance of IIS HTTP ERR logs in Performance Engineering

18 Feb

One among many skills required to be good Performance Engineer is to know, understand, interpret and analyze the logs of the backend servers and trace back those events to the load test executed during that known time period. Many times it so happens that in spite of having perfect script, perfect workload model, response times of the tests do not meet the service level agreements or they are just beyond the range as expected by the business owners or application developers. There could be many reasons for this, but to pin point the exact cause and come up with conclusive data, it becomes extremely necessary that we understand importance of logging and analyzing the server logs for the backend servers. So in this post I would be focusing my effort in explaining the importance of HTTP ERR logs which is the part of IIS logging.

Since IIS handles applications which are HTTP based, errors originating from the HTTP API of the IIS are logged in HTTP ERR logs. The most common types of errors logged here are network drop connections, connection dropped by the application pool, Service Unavailable errors, bad client request etc. Best part of this log is that it gives you the precise information as which URL or application pool is responsible for incorrect behavior, something which is very valuable in case if you have 10s or 100’s of sites sitting on the same IIS box or sharing the same application pool. All these information one can easily correlate with load test results to build a strong case that the application under test has some issues which needs some attention from the concerned team and may possibly have performance impact.

HttpErr logs comes with standard fields as shown below.It has all the information that requires for troubleshooting any fatal issue that occurs during performance testing. One thing I would like to mention here is that in case if you have IIS server sitting behind the  load balancers, then it might happen that the IP what you see in these logs might belong to the load balancer rather than end client users.Again this could be due to your infrastructure/environment design.

Field

Description

Date

The Date field follows the W3C format. This field is based on Coordinated Universal Time (UTC). The Date field is always ten characters in the form of YYYY-MM-DD. For example, May 1, 2003 is expressed as 2003-05-01.

Time

The Time field follows the W3C format. This field is based on UTC. The time field is always eight characters in the form of MM: HH: SS. For example, 5:30 PM (UTC) is expressed as 17:30:00.

Client IP Address

The IP address of the affected client. The value in this field can be either an IPv4 address or an IPv6 address. If the client IP address is an IPv6 address, the ScopeId field is also included in the address.

Client Port

The port number for the affected client.

Server IP Address

The IP address of the affected server. The value in this field can be either an IPv4 address or an IPv6 address. If the server IP address is an IPv6 address, the ScopeId field is also included in the address.

Server Port

The port number of the affected server.

Protocol Version

The version of the protocol that is being used.
If the connection has not been parsed sufficiently to determine the protocol version, a hyphen (0x002D) is used as a placeholder for the empty field.
If either the major version number or the minor version number that is parsed is greater than or equal to 10, the version is logged as HTTP/?

Verb

The verb state that the last request that is parsed passes. Unknown verbs are included, but any verb that is more than 255 bytes is truncated to this length. If a verb is not available, a hyphen (0x002D) is used as a placeholder for the empty field.

CookedURL + Query

The URL and any query that is associated with it are logged as one field that is separated by a question mark (0x3F). This field is truncated at its length limit of 4096 bytes.
If this URL has been parsed (“cooked”), it is logged with local code page conversion, and is treated as a Unicode field.
If this URL has not been parsed (“cooked”) at the time of logging, it is copied exactly, without any Unicode conversion.
If the HTTP API cannot parse this URL, a hyphen (0x002D) is used as a placeholder for the empty field.

Protocol Status

The protocol status cannot be greater than 999.
If the protocol status of the response to a request is available, it is logged in this field.
If the protocol status is not available, a hyphen (0x002D) is used as a placeholder for the empty field.

Site ID

Not used in this version of the HTTP API. A placeholder hyphen (0x002D) always appears in this field.

Reason Phrase

This field contains a string that identifies the type of error that is being logged. This field is never left empty.

Queue Name

This the request queue name.

Another good thing about HTTP ERR logs is that they can be queried exactly the way we query IIS Access logs by using log parser.

Below is one such example query, we can use to query HTTP ERR logs is as below,

LogParser‘SELECT TO_STRING(date,'YYYY-MM-DD'),TO_STRING
(time,'hh:mm:ss'),c-ip,c-port,s-ip,s-port,cs-version,cs-method,
cs-uri,sc-status,s-siteid,s-reason,s-queuename
 From \\YOUR PATH TO HTTP ERROR LOG’

Most often the HTTPERR logs are written in SYSTEMROOT\SYSTEM32\Log Files\HTTPERR Folder of the IIS box, however since this location is configurable  so at times we might need to confirm the exact location with Server administrators.Also in some rare cases , these logs might also have been disabled due to various reasons, so in those cases we might need to check this with Server administrators and get those enabled though I feel no one really disables it for any reason.

Once you query these logs with log parser you should be able to get the details as shown below.

Httperror1

As you can see the wealth of the information lies in the s-reason codes which helps us in identifying the issues with the application and aids a lot in troubleshooting some of the hardest performance bottlenecks.

Below is the Microsoft definition of the reason codes that comes as part of HTTP ERR logs,

AppOffline: A service unavailable error occurred (an HTTP error 503). The service is not available because application errors caused the application to be taken offline.

AppPoolTimer: A service unavailable error occurred (an HTTP error 503). The service is not available because the application pool process is too busy to handle the request.

AppShutdown : A service unavailable error occurred (an HTTP error 503). The service is not available because the application shut down automatically in response to administrator policy.

Bad Request : A parse error occurred while processing a request.

Client_Reset : The connection between the client and the server was closed before the request could be assigned to a worker process. The most common cause of this behavior is that the client prematurely closes its connection to the server.

Connection_Abandoned_By_AppPool : A worker process from the application pool has quit unexpectedly or orphaned a pending request by closing its handle.

Connection_Abandoned_By_ReqQueue : A worker process from the application pool has quit unexpectedly or orphaned a pending request by closing its handle. Specific to Windows Vista and Windows Server 2008.

Connection_Dropped : The connection between the client and the server was closed before the server could send its final response packet. The most common cause of this behavior is that the client prematurely closes its connection to the server.

ConnLimit : A service unavailable error occurred (an HTTP error 503). The service is not available because the site level connection limit has been reached or exceeded.

Connections_Refused : The kernel NonPagedPool memory has dropped below 20MB and http.sys has stopped receiving new connections.

Disabled : A service unavailable error occurred (an HTTP error 503). The service is not available because an administrator has taken the application offline.

EntityTooLarge : An entity exceeded the maximum size that is permitted.

FieldLength : A field length limit was exceeded.

Forbidden : A forbidden element or sequence was encountered while parsing.

Header : A parse error occurred in a header.

Hostname : A parse error occurred while processing a Hostname.

Internal : An internal server error occurred (an HTTP error 500).

Invalid_CR/LF : An illegal carriage return or line feed occurred.

N/A : A service unavailable error occurred (an HTTP error 503). The service is not available because an internal error (such as a memory allocation failure) occurred.

N/I : A not-implemented error occurred (an HTTP error 501), or a service unavailable error occurred (an HTTP error 503) because of an unknown transfer encoding.

Number : A parse error occurred while processing a number.

Precondition : A required precondition was missing.

QueueFull : A service unavailable error occurred (an HTTP error 503). The service is not available because the application request queue is full.

RequestLength : A request length limit was exceeded.

Timer_AppPool:The connection expired because a request waited too long in an application pool queue for a server application to de queue and process it. This timeout duration is ConnectionTimeout. By default, this value is set to two minutes.

Timer_Connection Idle :The connection expired and remains idle. The default ConnectionTimeout duration is two minutes.

Timer_EntityBody :The connection expired before the request entity body arrived. When it is clear that a request has an entity body, the HTTP API turns on the Timer_EntityBody timer. Initially, the limit of this timer is set to the ConnectionTimeout value (typically 2 minutes). Each time another data indication is received on this request, the HTTP API resets the timer to give the connection two more minutes (or whatever is specified in ConnectionTimeout).

Timer_HeaderWait :The connection expired because the header parsing for a request took more time than the default limit of two minutes.

Timer_MinBytesPerSecond :The connection expired because the client was not receiving a response at a reasonable speed. The response send rate was slower than the default of 240 bytes/sec. This can be controlled with the MinFileBytesPerSec metabase property.

Timer_ReqQueue : The connection expired because a request waited too long in an application pool queue for a server application to dequeue. This timeout duration is ConnectionTimeout. By default, this value is set to two minutes. Specific to Windows Vista and Windows Server 2008.

Timer_Response : Reserved. Not currently used.

URL : A parse error occurred while processing a URL.

URL_Length : A URL exceeded the maximum permitted size.

Verb : A parse error occurred while processing a verb.

Version_N/S : A version-not-supported error occurred (an HTTP error 505).

So next time if you are load testing any .Net Web Application installed on IIS Web Server, make sure that you ask for access to IIS HTTP ERR logs as well in addition to the IIS Access logs.It will help you identifying issues in the fraction of seconds compared to any other methods used quite often.

Siteminder and Some Performance Issues

11 Feb

Sometimes back I had written about some interesting issues encountered during the load testing application integrated with CA Siteminder. Load testing application integrated with Siteminder has its own challenges and limitations with regard to environment and scope of testing. Normally I have observed that most projects do not test Siteminder integration/approach/solutions as an independent component, but rather they club it with applications and prefer to test this with End to End approach as most people believe that it saves some time or they believe that it’s of little importance. I would say its incorrect approach or in cases a incomplete approach, if you have Siteminder integrated with your applications and you have 10’s of application with 1000’s of users, and then probably with performance testing Siteminder integration individually as a separate project might give you a lot of mileage in long run. Siteminder is a complex product and it does deserve performance testing individually given that it handles some of the critical functions of the business namely Access Management.

The purpose of my writing this post was to highlight one of the interesting bottlenecks we had encountered/investigated and fixed during Siteminder load testing for one of the projects. During load testing one of the applications we were constantly getting 401 errors and nature of the 401 errors were such that it was redirecting itself N number of times and in some cases going into infinite redirection loops. Reproducing this kind of error via browser is a harder job as browser often hides the redirection events and you just see progress activity bar in the browser status bar. With HTTP Debugging tool you can clearly see the redirection calls happening behind the scene, but again I believe browsers don’t really redirect exactly N number of times. So in browsers I have seen max redirection depth of 40 to 50 per tab, after that it use to stay still, may be browsers stops further redirections events. The redirection events looked like below in fiddler trace.

SM_Issue_001

In above image,we can clearly see that siteminder agent installed on the server is redirecting the request URL N number of times.Now during Load testing the issue was looking like below,

SM_Issue_002

We had number of calls with CA team and after we gave them this precise description of the  issue along with our vuser logs and fiddler traces we were seeing during our tests, they came back with below finding and fix,

One out-of four policy server had a bad value for the encryption key; this resulted in request sent to the bad Policy Serve to fail with “invalid key in use” – Looping observed in the fiddler traces for request failing to authorize.

Once we fixed in the encryption key in the sm.registry file all request processed as expected – no more looping (re-authentication process).

I am writing the exact text as given by the CA R &D team in case if any one faces this issue,they can try out this one as well.However please ensure that you see the same behavior as we have seen it and ensure that you do not have any issues with SM cookie collectors and they are doing their job fine.In addition to this make sure your users do not have access issues.

%d bloggers like this: