Wednesday, September 21, 2011

QA under scrum in long-term project

A lot of software development teams move to scrum not just because the reality requires so.. I mean not because they cannot work effectively under some other methodology. They move just because that's the world trend and because Google does so :)
I've been working under scrum for more than a year and have been watching the successes and the fails and also have been comparing the product quality progress between the classical waterfall we had before and the scrum we're working under now.
So.. the obvious thing is that scrum is pretty much suitable for short-term projects only. That means it is the best methodology if only you should not put a lot of efforts for addressing legacy problems like regression testing and so on. Otherwise you will have to fix bugs instead of delivering product value to the customer.
The second thing I'd like to highlight is the role of QA under scrum. Theoretically scrum team should have cross-functional nature and there should not be test engineers or development engineers but everyone knows that's pretty much hard to form such the team. Especially if you're putting your processes from waterfall to agile rails.
And I was thinking a lot.. About the best way to use QA specialists in scrum. About how to gain the most value from their skills and simultaneously not to move too far away from scrum? And what I what I've come to is here.
The best way to use the skills capabilities of QA guys in scrum is to create a kind of the layer between the iteration outcomes and the customer. Classical scrum supposes delivering the value to the customer at the end of each iteration. Then following the classical approach the team should use the customer feedback to adjust the product if the feedback takes place. I think I will be useful to introduce to types of iteration result demonstrations. First demo should be delivered to QA layer. Once it is done the QA department takes some time to certify the quality of the product. That looks like the classical testing however it may provide the feedback faster and more understandable for the team than the customer usually does.
After the team addresses the feedback of QA layer it may deliver the finalized results to the real customer or stakeholder. In such the way the delivered product will keep less bugs as it passes the QA processes and will not annoy the customer providing the results they do not expect to see. That will be supported by the classical relationships (which work under waterfall) between QA, customer and owner of the requirements.
This is what I'm thinking about. Will it be working or not..

Monday, September 19, 2011

JMeter server does not send back the sampler results on Ubuntu linux

Trying to simulate distributed JMeter configuration over the virtual hosts network (virtualbox) I facedd the set of problems caused by the fact that JMeter does not correctly determine the ip address of the hosts it'r running at. For example I could not even connect to jmeter-server because it considered the local address as the localhost (127.0.0.1). Short investigation showed that the solution is placed inside the jmeter-server file. It says:

# If the client fails with:
# ERROR - jmeter.engine.ClientJMeterEngine: java.rmi.ConnectException: Connection refused to host: 127.0.0.1
# then it may be due to the server host returning 127.0.0.1 as its address

# One way to fix this is to define RMI_HOST_DEF below
#RMI_HOST_DEF=-Djava.rmi.server.hostname=xxx.xxx.xxx.xxx

${DIRNAME}/jmeter ${RMI_HOST_DEF} -Dserver_port=${SERVER_PORT:-1099} -s -j jmeter-server.log "$@"

So uncommenting the RMI_HOST_DEF parameter and specifying the correct address solved the problem. However there were another one. When I ran the scenario I observed the errors in jmeter-server log file. It said the server couln't send back the sampler results as it tried to send it to 127.0.0.1. I couldn't find any solution in jmeter forums so I just applied the same one from jmeter-server to jmeter client.

So one should just edit the jmeter client execution linux bash script so that the last two lines will look like this:
RMI_HOST_DEF=-Djava.rmi.server.hostname=xxx.xxx.xxx.xxx
java $ARGS $JVM_ARGS $RMI_HOST_DEF -jar `dirname $0`/ApacheJMeter.jar "$@"

Where xxx.xxx.xxx.xxx should keep the ip address you're running your jmeter client at.