SquashTM (Squash TM) 1.5.0 test management system review

Introduction

Let's  continue exploring test case management systems. This time I chose the one called SquashTM (Squash Test Management or Squash TM) - open-source pretty looking and of broad functional scope tool from French team. Unlike some other products SquashTM offers full scope of functions on free-of-charge basis which definitely gives some extra points to the system.

You also might want to visit my previous review of Zephyr Desktop TCMS. In such the case please follow this link.

Web-site

One of the weakest components of the product is the web-site. The face of a product intended to raise the popularity has the big problem with localization. Shame on me, but I really don't speak French and I guess the most part of the world doesn't. Coming to the site of a product pretending to be the world-wide famous one in some observable future I expect to see convenient English version. I believe that will help to develop and to distribute the product much faster than now.

Well.. Anyway I tried to use google translate and that helped me to manage to download the binary distribution. By the moment the latest available (and stable) version is marked as 1.5.0. The supported operating systems are:

• Windows
• Mac OS
• Linux

The site has a Wiki part. The wiki made me completely confused. Fortunately it is written in English but the table of content has the topics ordered alphabetically. Thus the installation guide can be found somewhere in the middle of the topic list, however the most convenient place I guess is somewhere at the top.

Installation

SquashTM (or Squash TM) is distributed as jar installer (for Windows OS at least). Despite I chose English version the installation wizard still supplied the dialog controls with the captions like "Suivant" and "Quitter". Fortunately the control icons didn't give me the chance to go the wrong way. Just click the button with big green forward-arrow every time and you will succeed.

SquashTM (or Squash TM) fits people who would like to play with the system before take the positive or negative decision and at the same time wouldn't like to spend much time for setting application up as it comes with default H2 database. So you can use it right after the installer finish working. 

Looks like there is no way to migrate the data from the default database to your custom one (MysSQL is supported). There is direct warning in the Wiki at least saying literally the following

 Caution : by switching to a MySQL database, all data previously entered in the on-board database will be lost.

You can also install the system as Windows service so you don't have to care of starting the application each time your server gets rebooted.

SquashTM (or Squash TM) distribution binds to 8080 port by default so don't forget to change it once you've got a lot of web services running at your server. But please be aware of the official instructions which acn e found on this page are not correct. In my case I had to change environment variable HTTP_PORT as well to make my instance working on non-default one.

Features

SquashTM (or Squash TM) offers the standard set of features (see official terms description here)

• Integrating with SquashTA (the review is coming)
• Managing the isolated projects
• Managing Requirements (with customizable fields and versioning)
• Managing test-cases (with customizable fields)
• Managing campaigns (with customizable fields)
• Test steps support fields customization
• Iterations support field customization
• Integration with mantis bug-tracking system
• Managing the users and the teams. Each user should be assigned with the predefined permission set called "Profile". See the details here.
• Reporting

Usability

 General controls

In general the application UI looks quite attractively. The layout consists of three vertical column. The leftmost one and the most narrow at the same time holds workspaces controls. Using it you can switch between requirements, test-cases, campaign and report workspaces. Once you choose one you see two other columns. 

The middle column holds the tree of entities which are currently under management according to workspace selected. You may add, edit or remove entity, arrange them by the folders and perform searching there.

The leftmost and the biggest column contains the particular entity's details. Another thing I like in the application is that once you edit anything you're provided with rich editor and able to format the text with the broad scope of formatting controls

Requirements and Test-cases

The approach of binding requirements and test-cases that is used in SquashTM (or Squash TM) doesn't look very comfortable to me. It is supposed that you create set of the requirements and set of the test-cases which are not related to each other originally. Those sets might be of a huge number of items. Then you may do one of the following

• either go through each requirement and bind it (every one) to one ore more test-cases
• or go through each test-case and bind it to one ore more requirements

This seems not introducing big problems until you have big project with a lot of requirements and a lot of test cases and very complex requirement hierarchy.

The much more convenient way would be creating the test-cases straight from the requirement which is not possible in the current version. Thus having the requirement created (not necessarily by the current user - probably by another one) the dedicated tester could cover it with the set of appropriate test-cases and, say, mark it somehow as covered.

Composing test-cases

Unlike the requirements the test-cases are not just the free-text but some complex entity. Each test-case has some general information containing:

• Name
• Description
• Weight: Very High, High, Medium, Low
• Nature: Undefined, Functional, Business, User, Non Functional, Performance, Security, ATDD (No idea about what it could mean :) )
• Type: Undefined, Compliance, Correction, Evolution, Regression, End-to-End, Partner
• Status: Work in progress, Under review, Approved, Obsolete, To be updated (not sure if there is some BPM engine embedded to drive these states automatically)

This is how the general view of the test-case looks like:

All the properties above have nothing to do with the most valuable part of any test-cases - test steps. The steps should be populated at the dedicated "Script" tab. Each step consists of the list of the actions and the expected result. 

You also can assign the particular step to one of the test-case requirements so that it implies that step verifies one. However I couldn't manage to make it work. First strange thing is that when you're managing the requirement set verified by the particular step you're requested to remove unnecessary ones from the entire list of the test-case requirements. IMHO the more obvious way is to mark ones I'd like to include into the step coverage. Anyway.. the second strange thing is that the requirement gets selected if only to click the cell with requirement number (1,2,3...). I would expect the entry is selected wherever I click in the row... The last but the main problem is that despite I choose the requirement and click "Remove from step" button it still remains in the list and not removed.

Here is the step management view:

In addition you can attach files to the test-case or to the particular step of the script. Here I experienced the problem as well as the application rejected my attempt to attach png file to the test-case as the one of not supported formats.

 Campaigns

The test-execution mechanism is supported with campaign concept. The manager can create a campaign - the aggregation for the set of test-cases. Each test-case can be assigned to some user. The campaign should be broken down by the iteration - the particular test run. The iteration may contain the subset of the entire campaign test-case set. If the test-case appears in several iterations of a campaign and gets the certain status in one of the iterations then it has the same status in other ones. This should look like this:

Once the tester which is assigned with the tests sees the iteration ready in their campaign workspace they may start it like it is shown on the screen-shot:

Clicking "Begin" button takes the user to the steps of the test-case until the steps are over. Once they are the user moves to the next test-case of the iteration automatically until there are no more test-cases in the scope.

Reporting

Reporting capabilities of the application leaves ambiguous impression. Officially the tool has quite functional reporting. You may generate the report to analyze your requirements and test cases regardless whether you performed any test execution or not. Each report you may export to PDF format to present the results to your management for example.

Its also possible to generate the report for certain executions (aka campaigns). That's all great however I met the problems again. When I tried to filter the report data by my test project only, the filter couldn't ender the project set in the filter dialog so I had to be content with the all the existing projects in my report.

Here are several examples of the reports I managed to generate:

Conclusion

I'm confused and it is quite hard to give the resolution on the tool. From the one hand from functional and usability standpoints it is the best free test-case management systems I worked with. From another hand it is still quite raw and buggy. Here are some problems I found which I couldn't assign to the topics above and which are mostly related to the user management functionality:

• Whatever profile I assigned to a user I always saw Advanced Tester one when I came back to the corresponding dialog. However when I log in as that user I see the actually assigned profile for them
• When I compose the team I see the users I recently deleted and able to get to their details by drilling from the team configuration dialog
• Annoying problem is that the trees in the workspaces are not always updated in time when you add some sub-entities there

Well. I think despite there are a lot of quite annoying problems in the product they are all eliminated with the fact this tool is freeware and provides good functional scope and nice usability. Yet it fits small and middle-size development teams but the suject has all the chances to become domain standard in the observable future. So I give this TCMS 8.5 points of totally 10. I would give more, but those annoying bugs...