Zephyr Desktop test management system review

Introduction

This post originates the series of reviews of test-case management systems. All the tools are available for free (some with restricted functionality). The first one is Zephyr Desktop. This tool asserts supporting set of common TCMS use-cases such as manage requirements, test-cases and executions. It also features some stuff that allows to integrate your TCMS system with test-case automation solution.
See also SquashTM test case management system review.

Web-site

Nice web-site. It is easy to navigate and to find required distribution. However it does not differ much from the competitors. Free version is available after few of registration steps here.

Installation

Zephyr Desktop Community Edition is supplied with built-in MySQL server and Apache Tomcat 5.5 servlet container. The installation process is straightforward and should not make a problem for anyone. After you've installed the system you get two new services running: Zephyr server and MySQL server with operational database. Of course you have to ensure all the ports you're going to use are not already used by some other processes.

Features

The free version I've tried offers a set of common-qa-practice use cases. However it limits the number of available project by one. This is not so bad as you're free to work-around such the problem by considering the release term as product-release term (later I'm explaining what all this stuff means). So, the application allows to:

• Manage the resources (add, remove or edit users)
• Specify requirements, structure them and map to the test-cases
• Break-down test-cases by purpose (like regression, user acceptance, etc)
• Schedule TC execution
• Analyze project statistics
• Integrate with bug-tracking system (not touched)
• Execute external processes (not touched)
• Interact with external tools via API (not touched)

Usability

So, what's the user flow as it is seen for me: Some test manager logs in and configure the project. They scope available resources and releases and probably fills the project with release-specific requirements. The tester covers the requirements with test-cases. They break-down the test-cases by their purpose. The test manager schedules the test execution for particular test-set type (e.g. regression testing). Tester passes the test cases. They set the status for each one so that once the execution done we have the set distinguished by PASSED/FAILED so we have the statistics to be analyzed.

Look&Feel

Graphical framework

UI looks a bit heavy  and is built on Flex technology. It implements dialog model. It works fast but sometimes it looks overdriven so you can get lost in the numerous dialogs like this

That is a good solution for desktop applications as we have a sort of task-bar in windows or linux-GUI along with capability of switching between the windows using short-cuts (aka hot-keys). However the same approach in Web UI doesn't look so organic.
The frameworks allows drag'n'drop items at some places. It is generally used in tree structures to move items from one folder to another. However it does not allow to move folders to subfolders. However such the feature would look quite useful for me.

Releases

As I said before we're not able to manage several projects in free version of Zephyr Desktop. However the tool offers the release management feature which separate the workspace into several isolated areas. This looks like the picture below:

Once you select particular release tab you're able to manage requirements, test cases and other stuff in isolation from other releases. Thus, you can establish the convention saying that release is not actually just a release but PRODUCT-RELEASE. Hence you may manage several projects in one.

Requirements

The tool allows managing the requirements. This is great as (isn't is strange?) not all the TCMSs have requirements as the basic entity in the structure. However the this area has a bit of drawbacks as well. Say, why not to allow the user creating test-cases straight from the requirement? Does that look okay when you have hundreds of requirements and then should create thousands of test-cases and only after all finally bind them to each other? Anyway this way is how all happens in Zephyr.

Test-case management

All the test-cases may exist either being bound to certain requirement or separately. Application offers useful feature called "Repository". The repository allows breaking the test-cases down by the purpose you'd like them to target. You may want to include the test-case in, say, user acceptance set along with regression set. This is what repository will help you to achieve. However I couldn't manage to map the test-case to several repos at once, so I had to clone them to move separated items each to their dedicated repository folder.

Test cases consist of the steps. Each step can be described in comprehensive way including attaching documents as files or specifying URLs for those resources. You also can keep the release-wide documents uploaded so they are visible from everywhere.

Another feature that I like is test-case change-history so you can track what changes have been made to the test-case since its creation.

Test cases (as actually most of the entities in the system) are stored in a sort of folder structure so you're free to order your stuff in the way you like. However I would expect I see (or have the option to see) all the items  in selected folder and its sub-folders. Now if you want to observe all the tests you have to bypass all the tree-folders manually.

Scheduling

This function seems to be the only one I couldn't cavil at. It allows scheduling the execution of your tests breaking it down by phases and sub-phases with responsible resources and timing boundary. It should look like this:

Reporting

After you (or your guys) have executed all the tests you've been scheduled for, you may check the statistics of your project. In free version it looks quite poor. It delivers some simple stat of what part of your tests failed and what didn't, which part of requirements is covered with the test-cases, etc. It represents as the release-specific statistics as the project-wide one. All that looks like this:

You may also observe the main dashboard but this has not a lot of sense in free version where you're not able to manage several projects.

Not covered features

As I mentioned before I had no chance to play with a few of features in Zephyr. However when I tried to integrate the tool with JIRA it warned me of I will not be able to change my choice in the future and this decision should be final. Doesnt look pretty flexible for me :)

Conclusion

Anyway despite a set of drawbacks found in the tool I find this TCMs pretty fitting the needs of the ordinary QA team of 5-10 persons which manages not very large test sets (say 300-400 test-cases against 50-100 requirements) and which does not plan to change bug-tracking system :)

This tool will also be found comfortable for the teams which do not spend time for deployment and configuring huge and complex systems and want to start managing the tests as soon as possible.

I give this TCMSs 7.0 points of totally 10.0.