Thursday, February 09, 2012

Selenium antipatterns or What is Selenium the worst for (or 'donts' for Selenium)

You probably know a lot of cases when selenium will help you a lot and will beat any competitor which means it is really the best solution on the market. However I'd like to share the set of the problems the selenium is better to put away from (selenium anti-patterns).

  • Prepare the base state. If you include the base state preparation procedures in your selenium-based framework, be ready you may get them failed which will cause the test fails as well. That will not mean your product functionality does not work in proper way. That will probably mean your base state generation procedure failed due to some instability which is certainly still the sort of seleniums feature combined with the specifics of http servers or network collisions :)
  • Run load testing. You definitely can simulate some load using the selenium just because you can simulate user's work flow with the help of it. Selenium drives the GUI browsers which consumes plenty of resources so even if you use HtmlUnit driver that won't help you much. More over the load generating framework supposes strong discipline of multi-threading beacuse if you have no such one, you may measure your own leaks aka the problems in your framework performance, not of your tested application's. Do not invent the wheel - use JMeter.
  • Accuracy testing. Once your web application delivers some analytics you probably will want to automatically check if the values in your tables are correct. That is bad practice for sure. Depending on your browser's locale the numbers may have different representation, so that is might cause the problems in string casting to number formats. The same is true for date and currency date types. Moreover you likely will have to maintain such the tests quite frequently due to HTML dynamic structure changes because in the most cases such the tables are drawn with the help of dedicated GUI framework which may vary the HTML representation slightly keeping GUI look unchanged for the user. And do not forget that being introducing accuracy assertions you will have to make sure your last assertion procedure matches the business logic of  the tested application
  • Third-party graphics testing like one provided by adobe flash, MS silverlight etc. Use dedicated products to test such the interactions
So those are the tings I'd not recommend to use selenium for. All other web testing stuff matches selenium capabilities pretty good.