IE testing on Selenium IDE is a real nightmare.
In short, after struggled for a long time, I still failed to work it out.
Record the test suites using Selenium IDE of Firefox
Use Selenium Server to run the suites on IE
One problem with selenium (using either IDE, RC or Webdriver) is that you don't know what actually happened during the execution.
The selenium result would give some hints, e.g. if the verifyTextPresent failed then you know there is a missing text during execution. But you would like to know what actually appears on that screen? Did the page show up an error page? Or just the value is changed, or probably the server died?
The nature of automated testing is that you are unlikely to watch the executions all the time. Otherwise it's pointless to do the automation.
Read more Painful Selenium Testing – Part 4 – Firefox Screenshot
As mentioned in previous post, we now use the following archtiture to do the testing:
The exact command we used for the execution of testing suite is:
java -jar c:/jenkins/selenium-server-standalone-2.33.0.jar -userExtensions C:\jenkins\bin\user-extensions.js -port 4520 -timeout 900 -firefoxProfileTemplate c:\FirefoxPortable201\Data\profile -htmlSuite "*firefox C:\FirefoxPortable201\App\Firefox\firefox.exe" http://example.com test_cases\suit_405_testing_example.html result_suit_405_testing_example.html -singlewindow
Some highlights here:
The suit_405_testing_example was assumed to be exported from Selenium IDE, it is a set of HTML actions that are recorded on Selenium IDE.
We set up a custom user-extensions.js for screen capture of error page in Firefox, it was not supported by default. Here are some details of the script.
Read more Painful Selenium Testing – Part 3 – Automated HTML Suite with Firefox
Firstly, here is the structure we are using right now:
Selenium IDE + Selenium RC HTML + Jenkins + Selenium Reporter + Groovy Script + Twilio Alert
This is how it works now:
Most long-term projects have automated HTML UI tests (e.g. open URL, click, type in, submit, etc..) run every night
It will check sites, as well as server healthiness.
If it's success, a blue light will appear in Jenkins.
If it's failed, a red line will appear.
If it's a critical suite (e.g. site failed to be opened), and it continuously failed for 5 attempts, the system will send SMS to alert the developer.
I restarted this blog to mark down the painful days working with selenium.
In the world of agile, scrum and test driven development, everyone agrees that automated testing is important. In fact, it is one of the few required items for practices, they are even not optional.
However, when we step into this field a few years ago, we are surprised to find that there are so few companies actually did the genuine automated tests, for both web and mobile apps. Some companies did the unit tests, but very few companies worked out the legitimate functional tests (or UI tests) with automation.
We attempt to tackle the website automated testing with selenium, which is one of the few popular tools in this area. Yet it is still so painful.