Posted on

Painful Selenium Testing – Part 3 – Automated HTML Suite with Firefox

Facebooktwittergoogle_plusredditpinterestlinkedinmail

As mentioned in previous post, we now use the following archtiture to do the testing:

Selenium IDE + Selenium Server

The exact command we used for the execution of testing suite is:

java -jar c:/jenkins/selenium-server-standalone-2.33.0.jar -userExtensions C:\jenkins\bin\user-extensions.js -port 4520 -timeout 900 -firefoxProfileTemplate c:\FirefoxPortable201\Data\profile -htmlSuite "*firefox C:\FirefoxPortable201\App\Firefox\firefox.exe" http://example.com test_cases\suit_405_testing_example.html result_suit_405_testing_example.html -singlewindow

Some highlights here:

  1. The suit_405_testing_example was assumed to be exported from Selenium IDE, it is a set of HTML actions that are recorded on Selenium IDE.

  2. We set up a custom user-extensions.js for screen capture of error page in Firefox, it was not supported by default. Here are some details of the script.

  1. We use Firefox Portable instead of normal Firefox for two reasons:

    • We could get a more consistent version of Firefox for setting up suite (with Selenium IDE)
    • It's easier to deploy, just need to do copy and paste the folders. No need to worry about the configurations.

  2. The port is dynamic, because we need to run multiple testing instances at the same time.

  3. The result file name is also auto-generated via windows scripts.

  4. The timeout of 900 is fixed, Selenium server would kill the command after 15 minutes no matter what. So if you got a very long suite, you need to break it down into multiple smaller ones.

  5. I used a profile template folder in the command line, that makes sure every time the browser starts with a set of consistent default behaviors, e.g. auto save the csv files and pdf files.

    The profile easy to be created, I normally did some manual actions first (e.g. check auto-save the file next time), and then copy out the template folder inside portable Firefox for future use.

  6. THEORETICALLY, it would work for IE as well. However, IE execution turns out a nightmare. Eventually we just give up this one.

  7. Once the command works during manual testing, we could deploy it to Jenkins slave node using windows batch command.

  8. There is a plugin in Jenkins to parse and analyze the selenium result file, so it's quite convenient.

  9. We used a groovy script to send out SMS alerts, in case the suite is critical and it fails 5 times continuously.

  10. We set up testing instances on the cloud server. When we need to run more testing instances, we could simply increase the server power, or clone out more servers.