Since I have migrated the extension of Simple Gmail Notes as an official side project of the new company , I would like to keep a more rigid development process of this.
Very soon, the extension would need to support 4 different environments, it would be run for:
Say, we have a perfect automation system,and we could basically control anything on the device, precisely. Would that imply we are going to have a perfect testing system right away?
Maybe not. Because data is also relevant.
For most automated testing systems, there is a common pattern:
- actions / inputs to the system
- assertions on the output
- further actions
- further assertions
- ...
That's true for testing system for either web based testing system (e.g. selenium), or mobile applications (e.g. iOS automated testing).
If the system can be fully controlled, such as a standalone invoice system, it would be easy. However, for the other system, especially the mobile apps with dynamic data, that would not be the case.
Read more Testing On Mobile Device – Part 2 – Network Data Mocking
We are a small sized IT company, with around 50 staff and 30 developers.
Like most other IT companies in the world, we don't have full-time engineers work as QA.
Most engineers got more one project at a time. (One for development and one for maintenance.)
Engineers don't have the time (or they just don't care) to write the unit tests and functional tests, even if the tools are provided by iOS and Android development frameworks.
Our QA did most of the testing manually, but it's a boring task. And they are not capable of writing the codes for the unit tests and functional tests.
The Apps for iOS simulator and device have significantly different architecture, but the apps for Android simulator and device are the same.
If the tasks depend on the work of engineers, it normally means the tasks need cooperation of engineer and QA, and the timing would be a big problem.**
Allow our QA to set up the automated tests for the mobile applications, with zero or minimum efforts of engineering.
IE testing on Selenium IDE is a real nightmare.
In short, after struggled for a long time, I still failed to work it out.
Record the test suites using Selenium IDE of Firefox
Use Selenium Server to run the suites on IE
One problem with selenium (using either IDE, RC or Webdriver) is that you don't know what actually happened during the execution.
The selenium result would give some hints, e.g. if the verifyTextPresent failed then you know there is a missing text during execution. But you would like to know what actually appears on that screen? Did the page show up an error page? Or just the value is changed, or probably the server died?
The nature of automated testing is that you are unlikely to watch the executions all the time. Otherwise it's pointless to do the automation.
Read more Painful Selenium Testing – Part 4 – Firefox Screenshot