Tag Archives: Testing Circus

My Rewarding Experience as a Reviewer and Contributor to “More Agile Testing: Learning Journeys for the Whole Team” by Janet Gregory and Lisa Crispin

More Agile Testing
Picture courtesy of Amazon and where you can order this excellent book!

For many years I have written articles for the Testing Community and shared them through Testing Circus and Tea-Time with Testers. One warm summer day in July 2013, I received a pleasant email from Lisa Crispin asking if I would like to be one of the reviewers and contributors to their upcoming book that would compliment their existing book “Agile Testing: A Practical Guide for Testers and Agile Teams”. It was an honor to be asked and of course my response confirmed my interest. An international review team was brought together comprising of people with different experiences and expertise in their fields.

This was a rewarding experience both professionally and personally as there were many learning opportunities reviewing the chapters and discussions with the team. Checking my email was fun since there was a potential for an email from Lisa and Janet announcing what was ready for review or from a fellow reviewer with a question to foster a lively discussion. New chapters were quickly printed for when there would be time to review, digest, and comment on them. Lisa and Janet worked with the reviewers to help us identify where we could contribute sidebars to their new book. It was fun reading what the other reviewers were writing based upon their experience. For me, this is what really makes this book come alive – Lisa and Janet are sharing a lot of valuable information that is supplemented by real-life experiences. As an example I enjoyed reading how other Testers are using Session-Based Testing and Thread-Based Testing to meet the challenges of testing in a fast-paced environment!

It was a great opportunity to witness how a book comes together, the chapters under review might not be named nor are they in the order of the final publication. As the book evolved, chapters may be combined or information moved to a different chapter. After reviewing each chapter it was interesting to read the book cover to cover. It reminds me of a movie production where scenes are filmed out of order and then brought together to tell the story.

Once the draft book was completed, there was a great sense of accomplishment and a feeling of disappointment. Being part of a community working towards a common goal is fulfilling and exciting. It reminds me of working with a great project team. When the project is done there is pride of the work completed but yet a sadness that the project is completed.

The information and experiences within this book series is part of the overall testing picture. As an artist blends together different colors and uses different tools to create a beautiful piece of artwork, we do the same in Software Testing. We blend together different approaches and techniques to design and execute testing strategies to tackle complex testing problems.

As with the first Agile book, there are great learning opportunities for you to determine how to apply the information to further your testing career and tackle testing problems. As an example I love the “Pillars of Testing” model by David Evans designed with different factors that integrate to improve our confidence in the information we provide about the Quality of the Product we test. This model helped me think through the different factors to understand our strengths and weaknesses in each area of this model. Adam Knight’s discussion on “T-Shaped Skills” inspired me to think differently about training by identifying the “deeper” skills required for different flavors of technical testing and Product testing. At this time, my background includes limited automated testing experience; however, this book provides a foundation for me to better understand both the benefits and challenges to have a reasonable conversation. The chapter on “Thinking Skills for Testing” is excellent and in particular I like how Sharon Robson discussed that you need to use different type of thinking based upon the problem such as when to use critical thinking, analytical thinking, and creative thinking. Plus sample tools are suggested that can be researched further.

Lastly and most importantly, I would like to thank Janet and Lisa for this wonderful opportunity and to their continual dedication to the Testing Community. My recommendation for my readers is that you will add this book to your testing toolbox and discover how it will influence your testing world!

Articles I Wrote That Were Published in Testing Circus #Testing

publicationsOver the years, I had the pleasure of writing articles on testing, training, career development and other topics for Testing Circus. Below are my articles in their original format. Some of the articles I have revised and posted on The Testers Edge based upon what I have learned since that time or to take a different approach on the material. The attachments are PDF and you can find the magazines with a lot of great articles on Testing Circus.

2013 Articles

Are You Listening to Your Intuition? – 2013 September
Click here to read.
A lot has been written on how to progress your testing skills but an area often overlooked is listening to our intuition.

How Do I Manage My Time When There Is So Much To Do, 2nd part – 2013 April
Click here to read.
This article is the second part of “How Do I Manage My Time When there is so Much to Do?”

How Do I Manage My Time When There Is So Much To Do, 1st part – 2013 March
Click here to read.
With this article I addressed a few questions and challenges I face as a Software Testing Manager.

Who do I ask questions? – 2013 February
Click here to read.
In this article I discussed the importance of building relationships to help you identify the right person to ask questions and gather information.

How do I know when to stop testing – 2013 January
Click here to read.
In this article I addressed the often-asked question “how do I know when to stop testing?”

2012 Articles

How can I learn to Test – 2012 December
Click here to read.
In this article I addressed the question “how can I learn to test?”

Do you think I could be a tester, Part 2 – 2012 November
Click here to read.
In this article I addressed the question “do you think I could be a tester?”

Do you think I could be a tester, Part 1 – 2012 October
Click here to read.
In this article I addressed the question “do you think I could be a tester?”

Performing Risk Assessment Part 3 – 2012 September
Click here to read.
The last part of this series discusses a post-testing risk assessment or what is sometimes called lesson learned.

Performing Risk Assessment Part 2 – 2012 August
Click here to read.
The second part of this series is focused on monitoring initial risks and identifying new risks based upon what is learned during testing.

Performing Risk Assessment Part 1 – 2012 June
Click here to read.
The first part of this series is focused on identifying risks prior to testing.

Can We Approach Metrics Differently? Part 3 – 2012 May
Click here to read.
This part addressed defining a goal, metric setting, and guidelines for presenting information to senior management.

Can We Approach Metrics Differently? Part 2 – 2012 April
Click here to read.
This part addressed “The Five Whys” which is a fact-based approach to identifying the root cause of a problem.

Can We Approach Metrics Differently? Part 1 – 2012 March
Click here to read.
This part addressed providing a foundation in analyzing information to define your problems.

Four Questions to Consider When Writing a Test Report – 2012 February
Click here to read.
This article discussed guidelines for writing a test report.

What is a Tester’s Role in Quality Part 2 – 2012 January
Click here to read.
The second part of this article discussed how the processes and testing approaches adopted by a Testing Department and how they impact the product’s quality

2011 Articles

Guidelines on Writing a Meaningful Problem Report – 2011 November
Click here to read.
General guidelines that a tester can follow when researching and developing a problem report

You Want it When? Part 2 – 2011 October
Click here to read.
This part discussed approaches to prepare for testing when timelines have been reduced which may require the testing team to work extended hours

You Want it When? Part 1 – 2011 September
Click here to read.
This part discussed how to prepare for testing to hit the ground running.

Regression Risk-Based Testing – 2011 August
Click here to read.
This article discussed different approaches to assessing risk that can be used to develop both a modified- and core-regression strategy.

Breaking the Testing Mindsets – 2011 July
Click here to read.
This article discussed different ways to approach a testing problem.

A Test Matrix Approach for Organizing Testing – 2011 June
Click here to read.
This article discussed how to use a test matrix instead of test cases.

Planning Your Next Testing Assignments – 2011 May
Click here to read.
This article provided suggestions on planning a testing assignment. Translate the word “PLAN” into four actionable steps: Prepare, Layout, Analyze, and Navigate.

I am interested in a Testing Career – can you help me learn more about the field?

career development

Software Testing is one of the least known professions in the tech field. Most likely you have come across people who have performed a “checking” exercise and they equate it with “testing”. That really does dilute our value and it is important to take those teaching moments to let them know they performed a “checking” exercise to make sure that basic functionality did not break; however testing is much more complicated. This might lead to the question of:

  • Do you think I can be a Tester?
  • What does a Tester do?
  • Do you have a job opening in your Testing department?

These are great opportunities to educate people regardless if they become a Tester. We all need to come up with our elevator pitch when asked about the testing field. An elevator pitch is basically a short summary that can be provided in about 30-seconds.

In October and November 2012, I wrote a series for Testing Circus called: Do You Think I Could Be a Tester? Click here for October and here for November.

Here are a few ideas to get started and the above articles provide more details.

  1. What is his experience with Testers and what is his observation on their role? This can be helpful in correcting any misunderstandings as many people think Testers are the “Quality Police”. Another situation is the type of testing he witnessed at another company – for example that company may use test cases whereas another company is using session-based testing. Best to clarify any differences upfront.
  2. What does he enjoy doing? What are his career aspirations? If his goal is to implement company-wide solutions then a testing career might not be for him. But be careful about discouraging someone because a testing career could help him for other opportunities or he might find he really loves the field. A Tester’s skills crosses so many areas such as problem solving, critical thinking, risk analysis and mitigation, report writing, troubleshooting among many others.
  3. Take a personal approach to the conversation. Spend some time discussing why you became a Tester; why you enjoy the field; what do you find challenging about testing; what advice you would give a new Tester. Set a bit of background about the field before discussing how to perform testing. Have a few of the Testers share the same information. Perhaps have a small gathering of Testers meet with him to have this conversation. Make it informal so it does not feel forced and be sure to give the person opportunities to ask questions.
  4. Is he an employee of your company? Perhaps he can spend some time in the Testing department. Just be careful because if he does not understand testing then he might be overwhelmed with both what is testing and the product under test. There is a risk that this could discourage him from a career he might enjoy since he will not be going through the typical training program.
  5. Identify a tool or application he uses to demonstrate how to test a piece of functionality. Such as FaceBook, Twitter or anything that he is familiar with so his focus is on learning the testing piece.  In the November article are a couple of mind maps on bridging the training.

None of these suggestions are fool-proofed and it is important to tailor the approach to the person. Hopefully this will provide a few ideas  to prepare for these conversations. Plus be sure to come up with a Testing elevator pitch to be ready at any time to discuss “what is Testing”.


What is your Testing Elevator Pitch?

How do you explain a Tester’s responsibilities to someone who has no idea?

Breaking Testing Mindsets

For photo credit click here. This article was first published in June 2011 in Testing Circus. I have revised my original article for this posting.

Most testers have experienced the problem of needing to break testing mindsets to develop new testing ideas. Mindsets are basically traps we fall into such as testing the same keystrokes and solving problems using the same strategy. Making bad assumptions is another trap that can stop the tester from asking more questions or approaching the problem differently. Such as assuming something is not important to the client because it is a small feature. Over time, mindsets can become roadblocks to testing as we find less bugs believing the product is becoming more stable when in reality there are still bugs but we are not finding them. It is important for all testers to challenge how they approach testing.

We should learn from every mistake or lost testing opportunity by reflecting upon what we could have done differently in order to identify personalized learning buckets. Use this information to find new ways of testing in order to break those testing mindsets. Develop relationships with stakeholders to dive more deeply into the testing assignment to ask more questions and identify different ways to test. Continue to think outside the box by identifying what constrains or limits testing. Most importantly, always keep learning; challenge your skills; and never become complacent as a tester.

Identify Learning Buckets
Learn from missed bugs or new features not tested correctly to develop lessons learned to understand how the feature could have been tested and how that information might have been identified during testing. It is helpful to segment this information into personalized learning buckets. For example: training on the company’s product; knowledge on how the customer uses the product; deeper relationships with stakeholders; and problem-solving skills. Also identify the type of assumptions being made during testing. Through these learning buckets, identify strategies to improve testing skills in order to start removing testing roadblocks.

Product Training
It may not be possible to learn everything about the product; therefore, identify the subject-matter experts that can review the testing strategy to determine if anything is missing. Prior to meeting with the subject-matter expert, gather and review any requirement documents, training material, or other documentation provided to outline testing ideas and questions. This can help bridge both product and customer knowledge which can be brought into testing.

Challenge Assumptions
Identify the assumptions being made and then challenge those assumptions to determine if the testing approach changes. Do the assumptions have a pattern? For example, assuming a feature is not important because no one indicated its importance. Assumptions can become dangerous traps and over time may be filed away and not addressed during testing. For example, not having the test data to perform specific tests may have been acceptable in the past but as the feature evolves that assumption may no longer be correct. Identify old assumptions that no longer apply because the product or customer needs have changed. Whenever making an assumption, stop to identify the facts to determine if additional information or communication is required. When in doubt have a conversation with someone (ie., subject-matter expert, a test lead, developer) to review concerns and questions.

Think Outside the Testing Box
Identify what prevents or limits testing ideas and strategies. For example, a requirement document can be helpful in understanding intended functionality; however, how does it limit testing? Is this document being used as a checklist to perform testing within the box or is it being used to foster conversations with subject-matter experts on intended functionality to generate testing ideas that go beyond the written documentation. When reviewing the learning buckets, is one of them: “I did not test that because it was not in the requirement document.” If the answer is yes, then the requirement document is becoming a roadblock in generating testing ideas.

Is the testing based upon test cases that are providing diminishing-returns? If test cases are required, how can they be changed to become more fluid to avoid retesting the same key strokes, parameters, or test data? Consider rewriting test cases into testing ideas providing guidelines but not specific steps to test a feature.

Problem Solving Skills
Play games that require identifying different approaches to solve a problem. If you can, play them with friends to share and compare strategies to identify other ways of thinking through a problem. Identify your typical thinking patterns and understand how others are addressing the same problem differently. What can be learned from these differences to start removing testing barriers?

A Test Matrix Approach for Organizing Your Testing

For photo credit click here. This article was first published in June 2011 in Testing Circus. I have revised my original article for this posting.

To prepare for testing, some testers will translate the functional requirements to formal test cases defining the input and expected output in order to cover different types of tests such as: positive, negative, and boundary. Test cases can be time consuming to write and maintain plus will not always provide the flexibility to introduce different testing scenarios. Over time the test cases can provide diminishing returns since we may be retesting the same keystrokes whereas exploring different testing paths provides an opportunity to find new bugs.

Testers can perform exploratory testing to explore areas of the application they feel may be at risk. At times a more structured approach is needed to help the tester organize her testing to ensure they adequately cover the features while providing the flexibility of exploratory testing.

The goal of the test matrix is to provide organization and not become a checklist that is blindly followed. The value of creating a test matrix must outweigh the time to create and maintain if it is to be reused. A test matrix can be beneficial for regression testing and when testing a feature across multiple browsers or testing areas. In the end, how a tester organizes her tests can be a personal preference or company-driven especially in regulated environments.

What is a Test Matrix?
A testing matrix can be created for existing features to facilitate regression testing or for new features that require multiple rounds of testing. The goal is to reduce the need to gather this information each time the feature is tested allowing the focus on generating test ideas and performing testing. This approach is also helpful for features that are tested across multiple browsers or modules within the application.

A testing matrix can be created using a spreadsheet or table format to take advantage of the row and column relationship providing a way to keep track of completed testing. The rows define the feature’s options and the columns represent the testing areas or browsers to test. This approach reduces the time to create and update the testing matrix through consolidating the feature’s options across testing areas. The tester can bundle multiple tests into one test scenario or she can test them individually based upon her test ideas, experience level in testing, and knowledge of the feature. This allows her to change her approach when retesting a feature and allows other testers to take different testing paths providing better overall testing coverage.

Creating a Test Matrix
If you are testing a web-based application and you are testing the ability to print a report using different browsers, your testing matrix may look as follows:

Chart 1 test matrix
The above matrix provides initial testing guidelines defining the browsers to test. The columns can be modified to add or change browsers. If multiple reports need to be tested, this matrix can be copied for each report or additional columns can be added. It really is up to you on the best way to organize the testing. With this information coupled with knowledge of the application and history of fixed bugs, testing ideas can be generated. This matrix can be customized to include columns to document the testing ideas and notes. A testing matrix should be created based upon the needs of the features being tested, the risk level, and how much documentation is helpful.

Variation of a Test Matrix
The above test matrix example defines specific feature options to be tested allowing the tester to generate their test ideas. Another variation would allow the tester to document their testing ideas. Your testing matrix may look as follows:

Chart 2 test matrix

Benefits of This Approach
This approach can be helpful when there are a define set of features that must be tested such as in regression testing or when the same set of options are tested across multiple browsers or testing areas. While the tester is executing her testing ideas, this approach can help her organize tests and document what has been tested so she can review progress. Because it is not always possible to perform all test ideas across all browsers or testing areas, the tester can assess how much testing to perform. This information can be reviewed with stakeholders or other testers to discuss test coverage.

Disadvantages of this Approach
There is a cost associated with creating the test matrix by ensuring the correct information is captured and maintaining it when the features change. There is the risk that a tester will test only within the constraints of this structure and could miss important tests. Training needs to occur to ensure testers use the testing matrix as a way to organize their testing and not constrain the testing.

Planning Your Next Testing Assignment

For photo credit click here.
This article was first published in May 2011 in Testing Circus. There are a lot of great articles that you can read in this and other editions of Testing Circus. This was my first published testing article and I thank Ajoy for allowing me the opportunity to write for his fine magazine. I have made revisions to my original article, which is based upon the acronym “PLAN”.

Imagine you have been assigned the responsibility to oversee the testing of a new module being added to the company’s flag-ship application! What a wonderful opportunity to challenge your skills and to show management you can handle the responsibility.  Where do you start with this assignment? Do you dive in and start testing? Or do you take some time to lay out a testing strategy?

The test plan is a tool to develop an overall testing strategy and provides guideposts to measure progress. The key to planning is to consider the project’s scope and risk level to determine how much time to devote to the planning process. Plus considering the audience and how they will use the documentation is important. Keep the test plan light-weight by documenting information that is helpful in managing the project. Remember the test plan is a living document. As you learn from the test results your approaches will adjust accordingly. Most importantly, enjoy your testing assignment by allowing the test plan to become one of the tools you use to successfully complete your mission.

To make the process easier, translate the word “PLAN” into four actionable steps: Prepare, Layout, Analyze, and Navigate. These steps can help you plan and manage your next testing assignment.

Prepare by gathering information to form an initial understanding of what you are testing.

Understand Expectations and Deliverables
In defining a testing approach, understand the purpose of the assignment, clients’ expectations, deliverables and associated deadlines. It is helpful to understand how the client will use the module, importance to their business, and what features are more important to them. This will enable you to write better test guidelines and allocate the testing time. Sometimes a module is being created from a prototype perspective and actual business cases may not be available. In those situations, understand the intended functionality to define test cases.

Review Documentation and Identify Contacts
An important aspect of the planning process is to review available documentation to identify initial questions and to determine if critical information is missing. There may not be much or any documentation available. Conversations are always important to gather information and to gain a better understanding of the testing problem. If there is not any documentation, start a list of general questions to gather this information. At this stage, consider the optimal skill-sets required in your testing team and identify subject-matter experts to contact with questions.

Layout the test plan in a concise manner selecting a medium that closely aligns with the project’s scope and risk level.

Create the Test Plan
As you gather the information, start to create the test plan using a medium such as: word document, spreadsheet, wiki page, or other collaborative tools. The specific tool selected can be a personal preference as each has its pros and cons. When assembling the document, consider using tables and bullet points to make it easier to read and locate information. I prefer a concise one- to two-page test plan. Remember you want people to read and refer to the test plan. Think about the last time someone gave you a lengthy test plan to read. Did you really want to read it? Was there “noise” that did not provide any value?

Another avenue is the use of low-tech options such as: a white board, an easel board, or a notebook allowing you to quickly make changes and track progress. Whatever option is selected, keep the test plan concise to ensure it is used as a reference tool throughout testing.

Analyze the information to identify risks, milestones, communication, and testing approaches.

Identify Risks and Training
Review the information gathered to identify initial risks with approaches to minimize those risks and identify any training that is necessary. For example, the testers may need to be trained on a system that feeds into the module. Common risks include not having enough testers and insufficient testing time. This is not the time for panic but for creative solutions! Tap into the tester’s strengths; identify how to integrate testing approaches; and keep the approach agile by reducing heavy documentation. If there is a strong exploratory tester, consider allocating time upfront to identify the module’s weaker areas to assist in prioritizing the testing. Plus get those surface bugs to development as soon as possible. They will often block your ability to find the deeper ones.

Identify Testing Approaches and Milestones
It is important to identify testing approaches that could include functional, regression, integration, and performance testing. Balance testing approaches with testing time and available testers to plan out rough time estimates considering any overlap across approaches. From this information, identify testing milestones to help measure the testing progress against the deliverable date. Your rough plan may change as you learn more through testing but consider it your roadmap to keep you on target.

Review the Test Plan with the Stakeholders
While developing the test plan, review the test approach with the appropriate stakeholders to determine if anything important is missing and to gain buy-in. It is better to discuss differences earlier in the project than to try and bridge miscommunication at the end. This does not have to be a formal review as sometimes a meeting discussing the approach is sufficient. The key is to have communication with your stakeholders. How often and when you have conversations with the stakeholders is dependent upon the scope and complexity of the project.

Navigate through testing by referencing the test plan and adjusting approaches based upon knowledge gained through testing.

Test Plans and Approaches Evolve
A test plan will provide strategic oversight to manage your testing assignment. Refer to the test plan periodically to ensure that the objectives and deliverables are being met; review identified risks to assess likelihood; and review milestones against the team’s progress for any corrective action. When reviewing the test plan, continue to identify new risks and determine if there is a need to change testing direction.

A test plan evolves throughout testing because test approaches are refined based upon additional knowledge gained through the test results. As testing progresses, risks may change, unexpected problems are encountered, and certain phases of testing may take longer than originally anticipated.  The planning process is interwoven throughout test execution as it allows you to deal with the anticipated problems and the unknowns discovered during testing.