It is that time for the second survey on the State of Testing! If you did not participate in the first survey, you will definitely want to participate this year! The first survey had 600 participants and this year we are hoping for at least 1,000! The survey seeks to identify the existing characteristics, practices and challenges facing the testing community in hopes to shed light and provoke a fruitful discussion towards improvement. Click here to participate.
If you are not sure if you want to participate, please take a few minutes to review last year’s survey results. A lot of valuable information on how other Testers are spending their time, how they plan, methodologies used, challenges faced and much more! I hope you will take some time to complete this valuable survey as I believe you will find the information helpful!
For photo credit click here.
This article was first published in May 2011 in Testing Circus. There are a lot of great articles that you can read in this and other editions of Testing Circus. This was my first published testing article and I thank Ajoy for allowing me the opportunity to write for his fine magazine. I have made revisions to my original article, which is based upon the acronym “PLAN”.
Imagine you have been assigned the responsibility to oversee the testing of a new module being added to the company’s flag-ship application! What a wonderful opportunity to challenge your skills and to show management you can handle the responsibility. Where do you start with this assignment? Do you dive in and start testing? Or do you take some time to lay out a testing strategy?
The test plan is a tool to develop an overall testing strategy and provides guideposts to measure progress. The key to planning is to consider the project’s scope and risk level to determine how much time to devote to the planning process. Plus considering the audience and how they will use the documentation is important. Keep the test plan light-weight by documenting information that is helpful in managing the project. Remember the test plan is a living document. As you learn from the test results your approaches will adjust accordingly. Most importantly, enjoy your testing assignment by allowing the test plan to become one of the tools you use to successfully complete your mission.
To make the process easier, translate the word “PLAN” into four actionable steps: Prepare, Layout, Analyze, and Navigate. These steps can help you plan and manage your next testing assignment.
Prepare by gathering information to form an initial understanding of what you are testing.
Understand Expectations and Deliverables
In defining a testing approach, understand the purpose of the assignment, clients’ expectations, deliverables and associated deadlines. It is helpful to understand how the client will use the module, importance to their business, and what features are more important to them. This will enable you to write better test guidelines and allocate the testing time. Sometimes a module is being created from a prototype perspective and actual business cases may not be available. In those situations, understand the intended functionality to define test cases.
Review Documentation and Identify Contacts
An important aspect of the planning process is to review available documentation to identify initial questions and to determine if critical information is missing. There may not be much or any documentation available. Conversations are always important to gather information and to gain a better understanding of the testing problem. If there is not any documentation, start a list of general questions to gather this information. At this stage, consider the optimal skill-sets required in your testing team and identify subject-matter experts to contact with questions.
Layout the test plan in a concise manner selecting a medium that closely aligns with the project’s scope and risk level.
Create the Test Plan
As you gather the information, start to create the test plan using a medium such as: word document, spreadsheet, wiki page, or other collaborative tools. The specific tool selected can be a personal preference as each has its pros and cons. When assembling the document, consider using tables and bullet points to make it easier to read and locate information. I prefer a concise one- to two-page test plan. Remember you want people to read and refer to the test plan. Think about the last time someone gave you a lengthy test plan to read. Did you really want to read it? Was there “noise” that did not provide any value?
Another avenue is the use of low-tech options such as: a white board, an easel board, or a notebook allowing you to quickly make changes and track progress. Whatever option is selected, keep the test plan concise to ensure it is used as a reference tool throughout testing.
Analyze the information to identify risks, milestones, communication, and testing approaches.
Identify Risks and Training
Review the information gathered to identify initial risks with approaches to minimize those risks and identify any training that is necessary. For example, the testers may need to be trained on a system that feeds into the module. Common risks include not having enough testers and insufficient testing time. This is not the time for panic but for creative solutions! Tap into the tester’s strengths; identify how to integrate testing approaches; and keep the approach agile by reducing heavy documentation. If there is a strong exploratory tester, consider allocating time upfront to identify the module’s weaker areas to assist in prioritizing the testing. Plus get those surface bugs to development as soon as possible. They will often block your ability to find the deeper ones.
Identify Testing Approaches and Milestones
It is important to identify testing approaches that could include functional, regression, integration, and performance testing. Balance testing approaches with testing time and available testers to plan out rough time estimates considering any overlap across approaches. From this information, identify testing milestones to help measure the testing progress against the deliverable date. Your rough plan may change as you learn more through testing but consider it your roadmap to keep you on target.
Review the Test Plan with the Stakeholders
While developing the test plan, review the test approach with the appropriate stakeholders to determine if anything important is missing and to gain buy-in. It is better to discuss differences earlier in the project than to try and bridge miscommunication at the end. This does not have to be a formal review as sometimes a meeting discussing the approach is sufficient. The key is to have communication with your stakeholders. How often and when you have conversations with the stakeholders is dependent upon the scope and complexity of the project.
Navigate through testing by referencing the test plan and adjusting approaches based upon knowledge gained through testing.
Test Plans and Approaches Evolve
A test plan will provide strategic oversight to manage your testing assignment. Refer to the test plan periodically to ensure that the objectives and deliverables are being met; review identified risks to assess likelihood; and review milestones against the team’s progress for any corrective action. When reviewing the test plan, continue to identify new risks and determine if there is a need to change testing direction.
A test plan evolves throughout testing because test approaches are refined based upon additional knowledge gained through the test results. As testing progresses, risks may change, unexpected problems are encountered, and certain phases of testing may take longer than originally anticipated. The planning process is interwoven throughout test execution as it allows you to deal with the anticipated problems and the unknowns discovered during testing.