Manual regression test management:

Question

I'm curious on what testers think about using an official test management organizing system vs keeping everything in Excel/Word/folders?

Background:

We have a weekly deploy on Wednesdays so we basically perform manual regression tests on Tuesdays.  We have lots of automation but there are still places where we need manual testing. For the past couple of years we have used a system of MS Word for test plans/charters/exploratory testing.  We also use Excel to keep track of all the tests done on a weekly basis and all of this is kept in a folder with the Gen# for the name.  Then we just copy and paste all of that into a new folder for the next gen.  All of this has worked perfectly fine for us without being too much of a hassle.

We've looked at and demo'd various test management software (QASymphony, Smart Bear, etc) and while I can see the positives of having an organized/structured program with Jira integration they all seem unnecessarily complicated and may not offer us any more efficiency over our current process.  

Has anyone gone through this process and been happy they made the switch? Were there unexpected process benefits that bubbled up?

Answer 1

I am, as a tester, totally with tools like JIRA. Those tools makes life much easier to maintain test cases. I have worked on projects where in one project we used to manage our test cases on Excel and on another we used Version1. We could easily differentiate in terms of speed of work. We had close to 2000 test cases for each product where we had automated 1700 test cases. We had to test close to 300 test cases manually. Below is my experience.

Experience with Excel:

• Maintenance is very cumbersome. (If you have copied same test case to different test suite, you will have to search same test cases on different test suite file in case of modification.)
• Tagging is difficult.(You literally have to copy test case if the same test case is used in different test suite.)
• Maintaining change of history is difficult. (With excel, it was difficult to track change in history. Mainly when tester missed to update the change history sheet.)
• Assigning test case is little difficult. (We have to modify name in each tester column in case of the tester is changed.)
• Measuring the matrix is difficult. (We had to go through  each file to come up with numbers. Also, we had to have an intern to match numbers with master file.)
• Searching of test case was difficult.(If story changes and if test  case happen to be changed or if we happen to search test case, it was really difficult. We had to first go through master file with list of test cases and then work on test case file.)

In case of tools like version one, what ever difficulties we had with Excel could easily be resolved. Also, V1 was accessible via various devices. So, it allowed us to mobilize. I was on a vacation and we had a new tester. He called me up as he had some doubt with test case and the regression was going on. I just logged into V1 through my tablet and explained him test case on the go. It was that easy.

I understand that its little complex system to learn. But, once we adapt it, it was breeze.

Let me know your views.

Comments

Thanks for the reply.

Test case modification is a good point.  Our testers know always to take the most recently tested version to copy forward.  However, if a mistake is make it’s possible to lose updated tests easily. I could see this as being a nice process improvement.

I am unsure what you mean by tagging?  Does this have to do with Jira tags and them showing what areas of the software have been changed?

---

We can tag particular testcase by testsuite name, module name or theme.

Example, I have a testcase which should be a part of smoke suite, regression suite and XYZ module. So  whenever I want to do smoke testing, I just have to filter my test cases with that tag name and I can get list of all test cases.
Now assume that there was a change in XYZ module and my manager asked me to execute test cases related to XYZ module. I just have to filter test cases with that tagname and I can see all related test cases.