Monday, February 21, 2011

Eclipse. Perfoce. Error in change specification. Can't include file(s) not already opened. Error in change specification. Can't include file(s) not already opened.

Recently I faced the problem using P4-Eclipse integration. The message said like the one specified in this post's subject. So, googling gave nothing but only the that lot of people face the same kind of problem but actually of different reasons. Now after the short investigation I can share my one.
It seems eclipse gets mad if the current client spec is set up with some client view different from looking straight into the storage root. You should either to remap client view or to use another client spec.

Tuesday, February 15, 2011

Selenium 2 + WebDriver. Test drive and first impression

Several months ago I started implementation of automation framework intended for certain social network automated testing. The first time I set the requirement so the framework should be run against Win OS to save the time on setting up the automation context. However I faced the problem with starting selenium server over min Win7 machine so I had to explore the new ways of "click-engine" application. That way became Selenium 2 powered by WebDriver.

The obvious advantages are:
  • Local usage, so less configuration effort required and risks are estimated (may also be considered as disadvantage)
  • Automatic recognition of element type
  • Backward compatibility with Selenium
Disadvantages:
  • Yet not perfect functionality. Even using selenium emulation.
  • Buggy integration with the latest Microsoft products
  • API differs from the classic selenium one so it is pretty much hard to port the Selenium 2 tests to Selenium
Yeah.. So actually I had to move the framework to Linux platform after all as WebDriver stuck on sending long strings to the textboxes. However Linux framework had no such issues at all and it still works fine there.

Thank you!

UPD here

Monday, February 14, 2011

Pattern to measure the time of execution

It often happens that you need the way to measure the time of certain step execution in your scenarios or just your own scripts. What is the convenient way of doing this? The practice I use is to wrap the steps with ad-hoc class that is declared abstract and having the following structure:


This way will allow you to manage the measurement aspects without the necessity of all the code refactoring. Here is the example of how to use this approach:
Here it is presented how to measure the certain step's execution time. We should instantiate that abstract class right within the code to be measured so it will be wrapped with step() method implementation for each required part of code.
The provided example shows how I measured some files.

Saturday, February 12, 2011

Curse on record/playback approach

If you test your homepage or the web site of your rock-band you may use record-playback and feel happiness. But once you're moving to something serious you feel only the need to get help, right? :) The obvious disadvantages of record-playback are:

  • The resulting script requires corrections anyway
  • You have to be sure that you have no dynamic locators in your html. Like dynamic IDs.
  • You are restricted in using flow control of your scenario
  • You are restricted in driving your test with the custom data
  • You are restricted in verifying the result of your scenarios
  • You are restricted in embedding your test suites into some more complicated solutions
So what to say. Just learn the programming languages and switch to testing tools' API. That is less comfortable but gives really wide capabilities and flexibility.

Friday, February 11, 2011

Should I automate?

Automation is pretty much mod, isn't it? But should I automate only because of that reason? Sure not. The main criterion that should affect your decision is so called ROI. That means "Return On Investments". Actually different people give different definition of what it does mean. Anyway that is either [gain from investments - cost of investments]/[cost of investments] or [Gain from investments]/[cost of investments]. Depending on what definition we accept we work out the formal condition to define whether the decision is well-grounded.
Let's accept the second definition, so it is pretty much obvious that the condition is ROI>1. But what we should consider to be the ROI's parts. What is the gain and what is the cost?
Gain: The main gain's component is the time saving on manual testing reduction. That's why we may talk about some ROI only within some time bounds. So let's assume that the manual execution of set of required tests consumes N hours which are priced as $n/hour. Let's also assume that to develop the tests we should invest M hours of developers time that is priced as $m/hours. So, starting from the month the test have been implemented we should maintain them until the validity won't expire which will cost us $k/month. Thus the expression will look like:
ROI = [n*N]/[m*M+k*[ValidityPeriodInYears]*12]
Of course the usual expression is much more complicated as the real development process has lots of constraints and dependencies and moreover when we calculate some financial long-term flows we should consider discounting the future prices to the present values.
However the guideline is given. So, take a dare!

Thursday, February 10, 2011

QA Automation Essentials

What should be set up on your workstation to successfully handle the automation tasks and to successfully ensure the quality of the objects you're testing. Here is the must-have list for starting the test automations


  1. Builders: Apache ANT. Actually you may automate almost all the manual activities on your workstation, not only build processes.
  2. Heavy-weight web browsers: Selenium.
  3. Light-weight web browsers: JMeter
  4. Virtual machine: VirtualBox
  5. Environment to start your framework: JRE 
  6. IDE to develop anything: Eclipse IDE
  7. Version control repository: SVN or GIT
  8. Statistics visualization tool: BIRT
  9. Office suite to keep the documentation: LibreOffice
These tools will help you to test fast, deliver the high quality and what is the most important you will work efficiently!