Tag Archives: test planning

My Rewarding Experience as a Reviewer and Contributor to “More Agile Testing: Learning Journeys for the Whole Team” by Janet Gregory and Lisa Crispin

More Agile Testing
Picture courtesy of Amazon and where you can order this excellent book!

For many years I have written articles for the Testing Community and shared them through Testing Circus and Tea-Time with Testers. One warm summer day in July 2013, I received a pleasant email from Lisa Crispin asking if I would like to be one of the reviewers and contributors to their upcoming book that would compliment their existing book “Agile Testing: A Practical Guide for Testers and Agile Teams”. It was an honor to be asked and of course my response confirmed my interest. An international review team was brought together comprising of people with different experiences and expertise in their fields.

This was a rewarding experience both professionally and personally as there were many learning opportunities reviewing the chapters and discussions with the team. Checking my email was fun since there was a potential for an email from Lisa and Janet announcing what was ready for review or from a fellow reviewer with a question to foster a lively discussion. New chapters were quickly printed for when there would be time to review, digest, and comment on them. Lisa and Janet worked with the reviewers to help us identify where we could contribute sidebars to their new book. It was fun reading what the other reviewers were writing based upon their experience. For me, this is what really makes this book come alive – Lisa and Janet are sharing a lot of valuable information that is supplemented by real-life experiences. As an example I enjoyed reading how other Testers are using Session-Based Testing and Thread-Based Testing to meet the challenges of testing in a fast-paced environment!

It was a great opportunity to witness how a book comes together, the chapters under review might not be named nor are they in the order of the final publication. As the book evolved, chapters may be combined or information moved to a different chapter. After reviewing each chapter it was interesting to read the book cover to cover. It reminds me of a movie production where scenes are filmed out of order and then brought together to tell the story.

Once the draft book was completed, there was a great sense of accomplishment and a feeling of disappointment. Being part of a community working towards a common goal is fulfilling and exciting. It reminds me of working with a great project team. When the project is done there is pride of the work completed but yet a sadness that the project is completed.

The information and experiences within this book series is part of the overall testing picture. As an artist blends together different colors and uses different tools to create a beautiful piece of artwork, we do the same in Software Testing. We blend together different approaches and techniques to design and execute testing strategies to tackle complex testing problems.

As with the first Agile book, there are great learning opportunities for you to determine how to apply the information to further your testing career and tackle testing problems. As an example I love the “Pillars of Testing” model by David Evans designed with different factors that integrate to improve our confidence in the information we provide about the Quality of the Product we test. This model helped me think through the different factors to understand our strengths and weaknesses in each area of this model. Adam Knight’s discussion on “T-Shaped Skills” inspired me to think differently about training by identifying the “deeper” skills required for different flavors of technical testing and Product testing. At this time, my background includes limited automated testing experience; however, this book provides a foundation for me to better understand both the benefits and challenges to have a reasonable conversation. The chapter on “Thinking Skills for Testing” is excellent and in particular I like how Sharon Robson discussed that you need to use different type of thinking based upon the problem such as when to use critical thinking, analytical thinking, and creative thinking. Plus sample tools are suggested that can be researched further.

Lastly and most importantly, I would like to thank Janet and Lisa for this wonderful opportunity and to their continual dedication to the Testing Community. My recommendation for my readers is that you will add this book to your testing toolbox and discover how it will influence your testing world!

Articles I Wrote that were Published through Tea-Time With Testers

books

Over the years, I had the pleasure of writing articles and several series for Tea-Time with Testers. My series included: How to Develop a Strategy for Cross-Browser Testing, Career Development and Learning Strategies for Testers and Introducing Session-Based Testing into your Testing department. Below are my articles in their original format. The attachments are in PDF format and you can read a lot of wonderful articles on Tea-Time with Testers.

2013 Articles

Introducing Session-Based Testing to Your Team Part 2 – December 2013 / January 2014
Click here to read
This article is focused on introducing session-based testing in your testing group after attending training.

Introducing Session-Based Testing to Your Team Part 1 – October 2013
Click here to read
This series will discuss how you can use session-based testing and tools like mind maps while providing suggestions on how to implement change within your testing team or department.

Prepare for Promotion Now! – Women in Testing Special September 2013
Click here to read
We often hear how the number of women moving up the corporate ladder becomes smaller at each rung. There are a lot of good conversations discussing the various reasons that factor into this problem. This article is based upon my experience on a few things women can do to let their voices be heard, have more influence, and prepare for senior level positions.

Using Mind Maps to Organize your Cross-Browser Testing Part 5 – February / March 2013
Click here to read
In this part of the series, I shared sample mind maps that I find useful in organizing my testing.

2012 Articles

Balancing Time with Cross-Browser Testing Part 4 – December 2012
Click here to read
In this part of the series, I shared sample mind maps that I find useful in organizing my testing.

Importance of Social Networking Part 3 – August 2012
Click here to read
In this article I shared information I gathered through the Testing Community on how to perform cross-browser testing.

Understanding Browser Compatibility Strategies Part 2 – July 2012
Click here to read
In this article I shared information I gathered through the Testing Community on how to perform cross-browser testing.

Understanding Browser Compatibility Strategies Part 1 – June 2012
Click here to read
In this article I shared information I gathered through the Testing Community on how to perform cross-browser testing.

Contributing to the Testing Community – May 2012
Click here to read
In this article I discussed the importance of building relationships within the Testing Community and contributing.

Training Someone New to Testing Part 2 – April 2012
Click here to read
This series focus on training someone who is new to the Testing field.

Training Someone New to Testing Part 1 – March 2012
Click here to read
This series focus on training someone who is new to the Testing field.

Facilitating a Journal Club – February 2012
Click here to read
In this article I provided suggestions on how to start to start a journal club to provide opportunities for Testers to discuss current trends.

Developing a Career Plan – January 2012
Click here to read
The focus of this article is developing a career plan with approaches that a manager can implement for his department and testers can adopt for their own professional development.

2011 Articles

Onboarding New Testers – November 2011
Click here to read
The focus of this article is onboarding newly hired testers into your department.

Articles I Wrote That Were Published in Testing Circus #Testing

publicationsOver the years, I had the pleasure of writing articles on testing, training, career development and other topics for Testing Circus. Below are my articles in their original format. Some of the articles I have revised and posted on The Testers Edge based upon what I have learned since that time or to take a different approach on the material. The attachments are PDF and you can find the magazines with a lot of great articles on Testing Circus.

2013 Articles

Are You Listening to Your Intuition? – 2013 September
Click here to read.
A lot has been written on how to progress your testing skills but an area often overlooked is listening to our intuition.

How Do I Manage My Time When There Is So Much To Do, 2nd part – 2013 April
Click here to read.
This article is the second part of “How Do I Manage My Time When there is so Much to Do?”

How Do I Manage My Time When There Is So Much To Do, 1st part – 2013 March
Click here to read.
With this article I addressed a few questions and challenges I face as a Software Testing Manager.

Who do I ask questions? – 2013 February
Click here to read.
In this article I discussed the importance of building relationships to help you identify the right person to ask questions and gather information.

How do I know when to stop testing – 2013 January
Click here to read.
In this article I addressed the often-asked question “how do I know when to stop testing?”

2012 Articles

How can I learn to Test – 2012 December
Click here to read.
In this article I addressed the question “how can I learn to test?”

Do you think I could be a tester, Part 2 – 2012 November
Click here to read.
In this article I addressed the question “do you think I could be a tester?”

Do you think I could be a tester, Part 1 – 2012 October
Click here to read.
In this article I addressed the question “do you think I could be a tester?”

Performing Risk Assessment Part 3 – 2012 September
Click here to read.
The last part of this series discusses a post-testing risk assessment or what is sometimes called lesson learned.

Performing Risk Assessment Part 2 – 2012 August
Click here to read.
The second part of this series is focused on monitoring initial risks and identifying new risks based upon what is learned during testing.

Performing Risk Assessment Part 1 – 2012 June
Click here to read.
The first part of this series is focused on identifying risks prior to testing.

Can We Approach Metrics Differently? Part 3 – 2012 May
Click here to read.
This part addressed defining a goal, metric setting, and guidelines for presenting information to senior management.

Can We Approach Metrics Differently? Part 2 – 2012 April
Click here to read.
This part addressed “The Five Whys” which is a fact-based approach to identifying the root cause of a problem.

Can We Approach Metrics Differently? Part 1 – 2012 March
Click here to read.
This part addressed providing a foundation in analyzing information to define your problems.

Four Questions to Consider When Writing a Test Report – 2012 February
Click here to read.
This article discussed guidelines for writing a test report.

What is a Tester’s Role in Quality Part 2 – 2012 January
Click here to read.
The second part of this article discussed how the processes and testing approaches adopted by a Testing Department and how they impact the product’s quality

2011 Articles

Guidelines on Writing a Meaningful Problem Report – 2011 November
Click here to read.
General guidelines that a tester can follow when researching and developing a problem report

You Want it When? Part 2 – 2011 October
Click here to read.
This part discussed approaches to prepare for testing when timelines have been reduced which may require the testing team to work extended hours

You Want it When? Part 1 – 2011 September
Click here to read.
This part discussed how to prepare for testing to hit the ground running.

Regression Risk-Based Testing – 2011 August
Click here to read.
This article discussed different approaches to assessing risk that can be used to develop both a modified- and core-regression strategy.

Breaking the Testing Mindsets – 2011 July
Click here to read.
This article discussed different ways to approach a testing problem.

A Test Matrix Approach for Organizing Testing – 2011 June
Click here to read.
This article discussed how to use a test matrix instead of test cases.

Planning Your Next Testing Assignments – 2011 May
Click here to read.
This article provided suggestions on planning a testing assignment. Translate the word “PLAN” into four actionable steps: Prepare, Layout, Analyze, and Navigate.

A Test Matrix Approach for Organizing Your Testing

TestMatrix
For photo credit click here. This article was first published in June 2011 in Testing Circus. I have revised my original article for this posting.

To prepare for testing, some testers will translate the functional requirements to formal test cases defining the input and expected output in order to cover different types of tests such as: positive, negative, and boundary. Test cases can be time consuming to write and maintain plus will not always provide the flexibility to introduce different testing scenarios. Over time the test cases can provide diminishing returns since we may be retesting the same keystrokes whereas exploring different testing paths provides an opportunity to find new bugs.

Testers can perform exploratory testing to explore areas of the application they feel may be at risk. At times a more structured approach is needed to help the tester organize her testing to ensure they adequately cover the features while providing the flexibility of exploratory testing.

The goal of the test matrix is to provide organization and not become a checklist that is blindly followed. The value of creating a test matrix must outweigh the time to create and maintain if it is to be reused. A test matrix can be beneficial for regression testing and when testing a feature across multiple browsers or testing areas. In the end, how a tester organizes her tests can be a personal preference or company-driven especially in regulated environments.

What is a Test Matrix?
A testing matrix can be created for existing features to facilitate regression testing or for new features that require multiple rounds of testing. The goal is to reduce the need to gather this information each time the feature is tested allowing the focus on generating test ideas and performing testing. This approach is also helpful for features that are tested across multiple browsers or modules within the application.

A testing matrix can be created using a spreadsheet or table format to take advantage of the row and column relationship providing a way to keep track of completed testing. The rows define the feature’s options and the columns represent the testing areas or browsers to test. This approach reduces the time to create and update the testing matrix through consolidating the feature’s options across testing areas. The tester can bundle multiple tests into one test scenario or she can test them individually based upon her test ideas, experience level in testing, and knowledge of the feature. This allows her to change her approach when retesting a feature and allows other testers to take different testing paths providing better overall testing coverage.

Creating a Test Matrix
If you are testing a web-based application and you are testing the ability to print a report using different browsers, your testing matrix may look as follows:

Chart 1 test matrix
The above matrix provides initial testing guidelines defining the browsers to test. The columns can be modified to add or change browsers. If multiple reports need to be tested, this matrix can be copied for each report or additional columns can be added. It really is up to you on the best way to organize the testing. With this information coupled with knowledge of the application and history of fixed bugs, testing ideas can be generated. This matrix can be customized to include columns to document the testing ideas and notes. A testing matrix should be created based upon the needs of the features being tested, the risk level, and how much documentation is helpful.

Variation of a Test Matrix
The above test matrix example defines specific feature options to be tested allowing the tester to generate their test ideas. Another variation would allow the tester to document their testing ideas. Your testing matrix may look as follows:

Chart 2 test matrix

Benefits of This Approach
This approach can be helpful when there are a define set of features that must be tested such as in regression testing or when the same set of options are tested across multiple browsers or testing areas. While the tester is executing her testing ideas, this approach can help her organize tests and document what has been tested so she can review progress. Because it is not always possible to perform all test ideas across all browsers or testing areas, the tester can assess how much testing to perform. This information can be reviewed with stakeholders or other testers to discuss test coverage.

Disadvantages of this Approach
There is a cost associated with creating the test matrix by ensuring the correct information is captured and maintaining it when the features change. There is the risk that a tester will test only within the constraints of this structure and could miss important tests. Training needs to occur to ensure testers use the testing matrix as a way to organize their testing and not constrain the testing.

Planning Your Next Testing Assignment

Planning
For photo credit click here.
This article was first published in May 2011 in Testing Circus. There are a lot of great articles that you can read in this and other editions of Testing Circus. This was my first published testing article and I thank Ajoy for allowing me the opportunity to write for his fine magazine. I have made revisions to my original article, which is based upon the acronym “PLAN”.

Imagine you have been assigned the responsibility to oversee the testing of a new module being added to the company’s flag-ship application! What a wonderful opportunity to challenge your skills and to show management you can handle the responsibility.  Where do you start with this assignment? Do you dive in and start testing? Or do you take some time to lay out a testing strategy?

The test plan is a tool to develop an overall testing strategy and provides guideposts to measure progress. The key to planning is to consider the project’s scope and risk level to determine how much time to devote to the planning process. Plus considering the audience and how they will use the documentation is important. Keep the test plan light-weight by documenting information that is helpful in managing the project. Remember the test plan is a living document. As you learn from the test results your approaches will adjust accordingly. Most importantly, enjoy your testing assignment by allowing the test plan to become one of the tools you use to successfully complete your mission.

To make the process easier, translate the word “PLAN” into four actionable steps: Prepare, Layout, Analyze, and Navigate. These steps can help you plan and manage your next testing assignment.

PREPARE
Prepare by gathering information to form an initial understanding of what you are testing.

Understand Expectations and Deliverables
In defining a testing approach, understand the purpose of the assignment, clients’ expectations, deliverables and associated deadlines. It is helpful to understand how the client will use the module, importance to their business, and what features are more important to them. This will enable you to write better test guidelines and allocate the testing time. Sometimes a module is being created from a prototype perspective and actual business cases may not be available. In those situations, understand the intended functionality to define test cases.

Review Documentation and Identify Contacts
An important aspect of the planning process is to review available documentation to identify initial questions and to determine if critical information is missing. There may not be much or any documentation available. Conversations are always important to gather information and to gain a better understanding of the testing problem. If there is not any documentation, start a list of general questions to gather this information. At this stage, consider the optimal skill-sets required in your testing team and identify subject-matter experts to contact with questions.

LAYOUT
Layout the test plan in a concise manner selecting a medium that closely aligns with the project’s scope and risk level.

Create the Test Plan
As you gather the information, start to create the test plan using a medium such as: word document, spreadsheet, wiki page, or other collaborative tools. The specific tool selected can be a personal preference as each has its pros and cons. When assembling the document, consider using tables and bullet points to make it easier to read and locate information. I prefer a concise one- to two-page test plan. Remember you want people to read and refer to the test plan. Think about the last time someone gave you a lengthy test plan to read. Did you really want to read it? Was there “noise” that did not provide any value?

Another avenue is the use of low-tech options such as: a white board, an easel board, or a notebook allowing you to quickly make changes and track progress. Whatever option is selected, keep the test plan concise to ensure it is used as a reference tool throughout testing.

ANALYZE
Analyze the information to identify risks, milestones, communication, and testing approaches.

Identify Risks and Training
Review the information gathered to identify initial risks with approaches to minimize those risks and identify any training that is necessary. For example, the testers may need to be trained on a system that feeds into the module. Common risks include not having enough testers and insufficient testing time. This is not the time for panic but for creative solutions! Tap into the tester’s strengths; identify how to integrate testing approaches; and keep the approach agile by reducing heavy documentation. If there is a strong exploratory tester, consider allocating time upfront to identify the module’s weaker areas to assist in prioritizing the testing. Plus get those surface bugs to development as soon as possible. They will often block your ability to find the deeper ones.

Identify Testing Approaches and Milestones
It is important to identify testing approaches that could include functional, regression, integration, and performance testing. Balance testing approaches with testing time and available testers to plan out rough time estimates considering any overlap across approaches. From this information, identify testing milestones to help measure the testing progress against the deliverable date. Your rough plan may change as you learn more through testing but consider it your roadmap to keep you on target.

Review the Test Plan with the Stakeholders
While developing the test plan, review the test approach with the appropriate stakeholders to determine if anything important is missing and to gain buy-in. It is better to discuss differences earlier in the project than to try and bridge miscommunication at the end. This does not have to be a formal review as sometimes a meeting discussing the approach is sufficient. The key is to have communication with your stakeholders. How often and when you have conversations with the stakeholders is dependent upon the scope and complexity of the project.

NAVIGATE
Navigate through testing by referencing the test plan and adjusting approaches based upon knowledge gained through testing.

Test Plans and Approaches Evolve
A test plan will provide strategic oversight to manage your testing assignment. Refer to the test plan periodically to ensure that the objectives and deliverables are being met; review identified risks to assess likelihood; and review milestones against the team’s progress for any corrective action. When reviewing the test plan, continue to identify new risks and determine if there is a need to change testing direction.

A test plan evolves throughout testing because test approaches are refined based upon additional knowledge gained through the test results. As testing progresses, risks may change, unexpected problems are encountered, and certain phases of testing may take longer than originally anticipated.  The planning process is interwoven throughout test execution as it allows you to deal with the anticipated problems and the unknowns discovered during testing.