Wednesday, February 13, 2013

Semi-automated regression testing

Automated regression testing is pretty much expensive stuff, but what to do if you have few of resources but your you're in strong need to have some fast way to assess if your new build got affected with regression? You're lucky enough in case your application produces some standardized output like table-data in csv file format. 

The approach may look like this:
  1. Define the reliable output of the application version you're considering the stable one
  2. Keep that data as golden-set
  3. Each time you've got the new dev-build scope your new piece of data to make it relevant to your golden set
  4. Use some comparison tool to detect if your new piece of data has the differences against your golden-set
  5. Analyze the root-cause of the differences in case they are
Pro:
  • You save the resources from wasting
  • You know about the new problem in case your application has produced improper data but you have proper one in your golden set
  • You know about the old problem in case your application has produced proper data but you have improper one in your golden set
Con:
  • You do not know about the problem in case you've got improper data in both the sets