2015/04/03

Approaches to handling AJAX in Web Automation (Selenium)

Unfortunately pretty much places in web applications automation are based on "Magic". What do I mean by saying magic?

Example of Magic:
  1. Test is failing (or some operation is failing) and you don't know why
  2. You are trying something - and is works
  3. BUT you can't explain why the thing you done helps to fix the situation, and is it reliable or not

So, talking about JS/AJAX handling - it's mostly based on Magic for now. Nobody knows how to handle it properly, neither I do.

In this article I'll try to explain how to get rid of Magic in JS/AJAX handling as much as possible. Just theory, without code examples.



Why do we need to handle JS / AJAX in specific way?

Page loading can be divided into two parts: synchronous ans asynchronous.

Synchronous: 
  • Receiving most of the page content from server
  • All the elements are added"at once"
  • Basically it is what happens before "loading" indicator is stopped in browser (usually at the top of the browser UI)
  • How to handle:
    • In most cases we don't need to handle this specifically, because WebDriver waits for page to load before any action on elements. However some exceptions apply. I don't know whether the list of exceptions exist - so you probably will find it out the hard way.
    • Since all the elements are drawn (rendered) at once - to handle synchronous loading - you need to determine unique element on the page you want to work with. 
      • By waiting for this unique element - you will be sure that all synchronous elements are loaded as well. 
      • Element needs to be unique because otherwise your test may find this element on previous page and think that loading is already completed.

Asynchronous:
  • Any client-side processing - JS, AJAX
  • This may happen at any time, but usually is tied to some user interactions
  • Each element is added separately - after the processing is completed
  • How to handle:
    • Each case of asynchronous loading need to be handled separately.
    • More details on approaches below.

As you may see the main difference here is adding of elements (adding to DOM).

How JS is related here? - JS just is a way to add element on client-side (in browser).
How AJAX is related here? - AJAX gets some additional info from server, to be added on page via JS



How Asynchronous loading is handled?

I know following approach, which will be described in more details below:
  • Use Sleep to pause execution
  • Use ClickWithMouseOver or some other Magic method
  • Use JQuery or JS Event-based waiting
  • Use specific waiting for each case


Use Sleep to pause execution

There is some element which will be added now? Ok, I'll wait 5 sec for it, just like human do. Seems reasonable, but in fact it's not. 

This is the most wide-spread approach, and in fact the most straight-forward.

Human will stop waiting after he sees the element appeared. And if element is not appearing withing 5 seconds - he can wait more.

Sleep approach - instead determines some time to wait, This timeout time may be fixed (hopefully) or vary by each place. 

Bad:
  • It's slow. Every problem place will be just waiting.
  • It's not reliable. If test will still fail - the only thing you can do - increase Sleep time.
  • It's not effective. Your element may appear in 2 seconds, but you will still wait for full 5 seconds before continuing.
Good:
  • It's dead simple.
  • It's cheap to implement. (not counting test support time here)
Fortunately many people understand drawbacks of this approach, and try to eliminate it. But in some complex cases may fallback to this.


Use ClickWithMouseOver or some other Magic method

The second most used approach is pure Magic. 
The most simple example:
  1. You have a test case which was working, but now it fails on clicking a button.
  2. You run it locally - and it passes only sometimes (40% of executions).
  3. You have hard time thinking how may this happen.
  4. You try everything and occasionally try Clicking with Mouse over the button
  5. Hurray! it helps - test passes 100% of execution.
Do you do everything right here? Seems legit. But it may be not. 
In place of Clicking with Mouse over can be some other method which helps, like: 
  • Waiting for some element on page before Click
  • Checking of button attributes before Click
  • Even some kind of Waiting for AJAX (based on JQuery.active), more details in next section
  • Checking for some info in DB before click
What is common for all this functions? - They all are not doing anything helpful or needed for the test case.
How the hell then they helps up to pass the test? - They introduce some kind of delay. It may be even 0.5s to move the mouse pointer over the button, or to process some info from DB.

It appears that this approach is just a kind of unintentional Sleep calling. Instead of waiting for some specific time - you slow down your test just enough to make it passed.

Bad:
  • People usually don't understand when they do this
  • Extra unneeded actions performed
  • It's highly not stable, if test will fail later - you will just look for other Magic method which helps
  • You have confidence that you're not using sleeps, but in fact - you are
Good:
  • Tests are a bit faster then with using Sleeps
I won't recommend this approach in any cases. Even using Sleeps is better that this one - because you are using Sleep intentionally, without any Magic.


Use JQuery or JS Event-based waiting

Most of the sites use JQuery for different reasons, including performing AJAX calls. But this is not the panacea. Developers can implement AJAX without using JQuery.

If your site is using JQuery for AJAX - you can use JQuery to see whether AJAX call is ended or not. Seems ideal. But again it's not.

There are two approaches for handling JQuery AJAX calls ending, let's stop on them briefly:
  • .jQuery.active == 0
    • This counter shows how many requests are "in progress" now
    • This is not working, because there may be gaps between different requests.
  • ajaxStars and ajaxStop events
    • These events are triggered on Start and Stop of each request accordingly.
    • These events are commonly used to display "loading" icon
    • This is not working for the same reasons - there may be gaps between requests. Getting back to "loading" icon - you may see how it's flickering sometimes - this is a visualization of gaps between the requests.

Bad:
  • Tied to site implementation. May be not suitable for everyone
  • Gaps between requests can be handled correctly. Maybe only with some fallbacks to timeouts and sleeps. Like assuming that gap won't be longer than 3 seconds, and wait up to 3 seconds after active=0 before proceeding.
  • Involves low-level JS communication with site
  • A bit more complex to implement than previous approaches
Good:
  • It's slightly more reliable than previous approaches
  • It may work very well for sites with small amount of AJAX (when there is only 1 consequent request and may not be gaps)

Use specific waiting for each case

The most complex and the most suitable approach in my opinion.

I believe that you can't handle all the JS/AJAX cases in one way.

Instead of trying to achieve it - dig deeper in each specific AJAX call.

AJAX calls usually are tend to get some info from server, that means that as a result - something new will be displayed on page. You need to find out which new element will be displayed the last - and wait for this specific element. 

Bad:
  • Digging into details of AJAX calls may be complex
  • Code base is significantly bigger - since you write separate waiting for each place
  • Implementation may change, and you will need to support waiting
  • It touches pretty low-level interactions of elements on page
  • Implementations of this is pretty time consuming
Good:
  • You perfectly know how it works in every case. Finally no Magic here
  • No sleeps or timeouts involved, you wait for exact condition
  • It's stable to application slow downs (due to some performance issues or high-load)

In summary

I hope this article gives you more detailed view on AJAX in Web Applications and how to handle it in Automated testing.
Please post any other approaches in comments.