Free Automated Cross-Browser JavaScript Testing

JavaScript has become the lingua franca of web development. But like any language, there are different dialects depending on just where you’re speaking. One of the most challenging tasks when developing sophisticated JavaScript applications is making sure they work across users’ sundry browsers.

For the Parse JavaScript SDK, we’ve put together a framework that automates just this. For every release of the JavaScript SDK and backend, we run our entire test suite across five different browsers. Here’s how.

Basics

This guide assumes you are running your tests on Mac OS X. Much of what we describe will work on other systems, but Mac OS X is a requirement if you want to run tests on the iOS simulator. We also assume your tests are written in QUnit. Using a different test framework should be a straightforward.

Mozilla Firefox

Firefox is the simplest browser to start building your automated testing framework. We recommend using Selenium WebDriver to do the actual browser automation. WebDriver has lots of language bindings that you can use to write your tests. To automatically run tests using the ruby driver, do something like:

require 'selenium-webdriver'
driver = Selenium::WebDriver.for :firefox
driver.get url
wait = Selenium::WebDriver::Wait.new(:timeout => timeout)
begin
    wait.until { driver.title.downcase.start_with? "u2714" }
    puts "* Passed! :)"
rescue Selenium::WebDriver::Error::TimeOutError => e
    puts "* Failed! :("
end
driver.quit

The driver.title.downcase.start_with? "u2714" condition is specific to qunit. It waits for the checkmark to appear in the page title. Depending on your test framework, you may want to use some other condition. It’s always good to have a timeout when testing JavaScript, because many JavaScript errors will cause your tests to hang indefinitely.

Google Chrome

Automating Google Chrome is only slightly more difficult. You just need to install the WebDriver driver for Chrome, and put it in your PATH. Then follow the Firefox instructions, changing :firefox to :chrome.

iOS

To get the iOS browser working, you just need to start up the iOS Simulator and run the WebDriver app inside it. The most reliable way to do that is to checkout the driver project and run it from Xcode.

svn checkout http://selenium.googlecode.com/svn/trunk/ selenium-read-only

Open the iphone/iWebDriver.xcodeproj project and click the play button. Then follow the Firefox instructions, changing :firefox to :iphone.

Android

Running tests in Android is a similar process to iOS. Install the Android SDK, create an emulator instance, and run it. Download the Android WebDriver APK and install it and run it inside the emulator. If you run into problems connecting to the driver, try different versions of Android, as the driver is most stable in some than others. We’ve had the most luck with version 2.3, Gingerbread.

Then follow the Firefox instructions, changing :firefox to :android. One extra caveat with virtual machines such as this: if you want to connect to another server running on your test machine, you can’t connect to localhost. Instead, you have to connect to the default gateway, typically 10.0.2.2.

Internet Explorer

Okay, so Chrome, iOS, and Android are all Webkit-based. If your JavaScript works in one of them, it probably works in all of them. But Internet Explorer is the black sheep of the browser world. If your tests fail in just one browser, well, you get the picture. Accordingly, automating testing in Internet Explorer is much more involved than any other browser.

Start by downloading and installing VirtualBox, which will host the virtual machine where you can run Windows. Now, you can install your own copy of Windows in a new virtual machine, or you could use the free images Microsoft provides for Internet Explorer testing, using ievms. These Windows installs are already expired, so you can’t use them for very long at a time, but they work fine for automated JavaScript tests.

To set up the Windows environment, start up the virtual machine image. Install the VirtualBox Guest Additions from the menu. Download and install the Java runtime from Oracle. Download the Selenium Server Standalone Driver and verify that it works by running it manually from the Command Prompt. You can automate running the driver with VirtualBox using a command such as:

VBoxManage guestcontrol IE8 exec 
      --image C:/Windows/System32/java.exe 
      --username IEUser --password Password1 
      -- -jar C:/Users/IEUser/Desktop/selenium-server-standalone-2.20.0.jar

You also need to set up port forwarding so that your driver can connect to the server. This should be as simple as:

VBoxManage modifyvm IE8 --natpf1 WebDriver,tcp,,4444,,4444

Finally, follow the Firefox instructions, changing the driver initialization to:

driver = Selenium::WebDriver.for(:remote, :desired_capabilities => :firefox)

As with the Android virtual machine, if you want to connect to another server running on your test machine, you can’t connect to localhost. Instead, you have to connect to the default gateway, typically 10.0.2.2.

Conclusion

That’s all you have to do to run the test suite once. If you use GitHub, you can have your tests run after every commit with CruiseControl.

Getting automatic cross-browser JavaScript testing working is a complex affair, and bound to involve some amount of trial and error. We hope that this guide will help you get started, and save you some time by giving you a framework for success.

Bryan Klimt
June 28, 2012
blog comments powered by Disqus

Archives

Categories

RSS Feed Follow us Like us