Image courtesy of Amazon
I like to think about Test Automation as providing tools that can execute important checks that might be tedious, time consuming, or there might not be enough time to manually perform those tests. Instead of performing these repetitive, time-consuming tests, the Testers can spend that time exploring risk areas where using their intelligence to make decisions on tests to perform is a better use of their time. (And a lot more fun than performing repetitive tests!)
Test Automation is not a trivial undertaking – research should be conducted to develop a strategy, identify tools, and determine the best approach to automate tests that are being performed manually. Test automation is not a silver bullet and it takes time to develop your approach over time by what you learned.
Do you want to learn more about Test Automation and what has worked and not worked so great at other companies? An excellent book to read that I highly recommend: Experiences of Test Automation Case Studies of Software Test Automation by Dorothy Graham and Mark Fewster. This book contains 29 case studies written by those involved with a test automation project. Each case study begins with a background including if the project was successful and if test automation is still being used at that organization. Consider starting a book club where everyone reads a different chapter(s), shares what they learned, and develop a strategic plan for your company on how you might approach test automation or improve your existing approach. (And consider adding food as part of your book club, food always make any event more fun!)
Key Take Aways:
- Test automation is not necessarily performed by Testers. It is performed by the person best qualified to use the tool selected that can include the ability to write code!
- Test automation does NOT replace Testers but should supplement your testing strategy.
- Test automation is basically maintaining a code base to test your Product’s code base.
Below are learning points; however the above resources provide a wealth of information that is not summarized within this posting.
- Make it a real project with dedicated people, time, and funding. Start with a small pilot with key objectives and gain buy in for your pilot by key stakeholders.
- Start with simple UI tests or smoke tests as a great way to get some initial protection around the code.
- Consider test automation strategy by: technology facing tests (unit, components, integration) and business facing tests (acceptance, business, GUI).
- Concentrate on testing 1 system first as a model – the system producing the most business (start small, focus on a single but important part of the system).
- It is expensive to automate all types of tests including edge cases. Identify what tests need to be in the automation suite. Think about the value of the test and the question it answers or information provided. What is the ROI?
- Tool training or employees who have experience with the tool is important to reduce expensive mistakes.
- Can Testers write the GUI tests and the Developers write the unit tests? Who will write the business layer tests? That depends on the tool selected and skills of the team.
- The developer may write the tests with the Testers running them.
- Be sure to include business people when selecting the business layer tests.
Tools & Approach:
- New code is often easier to automate than legacy code.
- Different tools / programming languages can be used for the test automation pyramid: Unit, acceptance tests, business layer, GUI.
- Test automation is good for mature code that is not constantly changing.
- Lean regression suites help keep maintenance costs low while still providing value.
- Be careful how you write your tests – reduce failures that really are not a problem.
- All tests running green do not mean there aren’t any problems.
- Once tests are automated, train Developers, Testers to understand how to maintain them.
- Determine if the tool needs to clean up after the test was executed.
- Ability to run the code across platforms, browsers, releases.
Managing Test Suites:
- Different type of test suites: nightly test suite, weekly test suite, candidate test suite.
- Keep the tests small and maintain the flexibility to run tests in logical test suites to reduce the time it takes to execute them. For example:
- you may need to execute some key tests during the day when time is running low.
- you may execute core tests with every new build for critical functionality.
- Strong management support is necessary to understand the cost of the pilot program and long-term support that includes ongoing maintenance of modifying and adding automated tests.
- Strong collaboration with Development is important – code needs to be written in a manner that it can have automated tests written.
- Make the benefits of test automation visible to stakeholders for continued support.
- Ideally Developers / Testers are dedicated to test automation to ensure that new and changed code is being reviewed to determine tests to add or tests to change.
- Identify different champions: tool champion, change agent, stakeholders, etc.
- Count number of tests for each test tool – trend over time to see how the tests increase in number.
- Send business users a calendar of green vs red if any of the tests are red.
- Build time is a critical metric since it includes the time to run the automated tests.
- On what O/S were more bugs found and frequency?
- Detection rate of bugs for tests. What tests found the most bugs? No bugs? Is the percentage of green tests trending upwards over time?
I wanted to let everyone know about a new website called: Women in Testing started by Jay Philips CEO & President of Project Realms, Inc. & TeamQualityPro.
If you are a woman in testing or if you know of any women in testing please let them know about this new website. It contains bios of women in testing; information on different events such as conferences and webinars; and a newsletter. Plus there is a Facebook page and a Twitter account – plus more is coming!
If you would like to be listed in the directory click here. I hope to see you listed on Women in Testing!
My Rewarding Experience as a Reviewer and Contributor to “More Agile Testing: Learning Journeys for the Whole Team” by Janet Gregory and Lisa Crispin
Picture courtesy of Amazon and where you can order this excellent book!
For many years I have written articles for the Testing Community and shared them through Testing Circus and Tea-Time with Testers. One warm summer day in July 2013, I received a pleasant email from Lisa Crispin asking if I would like to be one of the reviewers and contributors to their upcoming book that would compliment their existing book “Agile Testing: A Practical Guide for Testers and Agile Teams”. It was an honor to be asked and of course my response confirmed my interest. An international review team was brought together comprising of people with different experiences and expertise in their fields.
This was a rewarding experience both professionally and personally as there were many learning opportunities reviewing the chapters and discussions with the team. Checking my email was fun since there was a potential for an email from Lisa and Janet announcing what was ready for review or from a fellow reviewer with a question to foster a lively discussion. New chapters were quickly printed for when there would be time to review, digest, and comment on them. Lisa and Janet worked with the reviewers to help us identify where we could contribute sidebars to their new book. It was fun reading what the other reviewers were writing based upon their experience. For me, this is what really makes this book come alive – Lisa and Janet are sharing a lot of valuable information that is supplemented by real-life experiences. As an example I enjoyed reading how other Testers are using Session-Based Testing and Thread-Based Testing to meet the challenges of testing in a fast-paced environment!
It was a great opportunity to witness how a book comes together, the chapters under review might not be named nor are they in the order of the final publication. As the book evolved, chapters may be combined or information moved to a different chapter. After reviewing each chapter it was interesting to read the book cover to cover. It reminds me of a movie production where scenes are filmed out of order and then brought together to tell the story.
Once the draft book was completed, there was a great sense of accomplishment and a feeling of disappointment. Being part of a community working towards a common goal is fulfilling and exciting. It reminds me of working with a great project team. When the project is done there is pride of the work completed but yet a sadness that the project is completed.
The information and experiences within this book series is part of the overall testing picture. As an artist blends together different colors and uses different tools to create a beautiful piece of artwork, we do the same in Software Testing. We blend together different approaches and techniques to design and execute testing strategies to tackle complex testing problems.
As with the first Agile book, there are great learning opportunities for you to determine how to apply the information to further your testing career and tackle testing problems. As an example I love the “Pillars of Testing” model by David Evans designed with different factors that integrate to improve our confidence in the information we provide about the Quality of the Product we test. This model helped me think through the different factors to understand our strengths and weaknesses in each area of this model. Adam Knight’s discussion on “T-Shaped Skills” inspired me to think differently about training by identifying the “deeper” skills required for different flavors of technical testing and Product testing. At this time, my background includes limited automated testing experience; however, this book provides a foundation for me to better understand both the benefits and challenges to have a reasonable conversation. The chapter on “Thinking Skills for Testing” is excellent and in particular I like how Sharon Robson discussed that you need to use different type of thinking based upon the problem such as when to use critical thinking, analytical thinking, and creative thinking. Plus sample tools are suggested that can be researched further.
Lastly and most importantly, I would like to thank Janet and Lisa for this wonderful opportunity and to their continual dedication to the Testing Community. My recommendation for my readers is that you will add this book to your testing toolbox and discover how it will influence your testing world!
It is that time for the second survey on the State of Testing! If you did not participate in the first survey, you will definitely want to participate this year! The first survey had 600 participants and this year we are hoping for at least 1,000! The survey seeks to identify the existing characteristics, practices and challenges facing the testing community in hopes to shed light and provoke a fruitful discussion towards improvement. Click here to participate.
If you are not sure if you want to participate, please take a few minutes to review last year’s survey results. A lot of valuable information on how other Testers are spending their time, how they plan, methodologies used, challenges faced and much more! I hope you will take some time to complete this valuable survey as I believe you will find the information helpful!
Women Testers – October 2014 Edition is Out! How to Keep Yourself Organized When There are Too Many Demands on Your Time!
I hope you take an opportunity to read the latest edition of Women Testers! For this edition I have an article called: How to Keep Yourself Organized When There are Too Many Demands on Your Time! I plan on writing a second part to this article and would appreciate any questions or feedback that could be addressed in the next edition of Women Testers.
If you would like to submit an article for a future edition or help out in any way, please contact Jyothi Rangaiah at email@example.com and you can find more information by clicking here. Enjoy the magazine and please let people know about it including women who might be interested in a tech career!
Recently, I had a high school student spend two-days in the Testing department to gain a basic understanding of what testing is all about. Most of us know the challenges of explaining a Testers’ responsibilities without making it look too easy or so difficult that you completely lose them. Some of these challenges are discussed in a prior posting that you can find here. If you are hosting a shadowing day or any similar opportunity, below are a few ideas to get you started.
I would like to thank the following people who contributed to providing suggestions and links to potential training material. Griffin Jones, Teri Charles, Jennifer Hannon, and Stephen Becht.
- Create an initial agenda and identify a few alternatives. During the shadowing program there might be a need to change direction based upon how the person is reacting to the information. Some material might progress more quickly than expected. Other material you might decide is not appropriate once you get to know him better. Plus remember to have fun. Do not make it so serious or intense that you lose him! And make it interactive to keep him engaged.
- Create a Testers’ Profile by asking the Software Testers to answer a few questions on how they selected testing as a career, what they enjoy about their job, and what advice they would give a new Tester. Compile the information in a newsletter or some other fun format to provide as a take away. Host a conversation where the Testers can discuss their careers so he has an opportunity to ask questions. Try to maintain a casual environment. Look for a location that has comfortable chairs like a coffee break room or a conference room that is not the board room.
- Guru 99 has free software testing videos that last from about 1-minute to 5-minutes. They are basically snippets of information with some of them having small exercises to perform. Once the video is completed, discuss any questions and relate the material to something that he is familiar such as Facebook, Twitter etc. Reinforcing the material is important.
- Whodunnit? This video is a lot of fun providing some great opportunities for a conversation afterwards. Before playing the video instruct him to write down his observations. Do not watch the end of the video until he writes down observations on what has changed during the murder investigation. Consider replaying the beginning part of the video a few times so he can have a chance to find what has changed. Most people are surprised when they find out how much has changed during that short video.
- Basketball Awareness Test: Moonwalking Bear. From an awareness perspective a Tester may notice something that does not make sense or is out of scope. At that point, the Tester has to make a decision. Go down the new path to explore or quickly note the observation returning back to the original test. An interesting conversation – if you do not notice the moonwalking bear what does that mean? Is it good or is it bad?
- Easy Button Exploratory Testing by James and Jonathan Bach. This is a great video since it uses a lot of the testing terms in a fun manner. A great option for after lunch when he might be feeling that carb sleepiness. If possible purchase an Easy Button to reproduce some of the tests and relate them to your Testing department.
- Cast 2011 Promo 2 Video. Another video by James and Jonathan Bach. The Hidden Puzzle is very interesting since each Tester took different test paths to identify the picture. If possible, recreate the Hidden Puzzle exercise with him and include a few Testers. How did each person approach the testing? Who found the hidden picture? How many tests did it take? This could be a fun exercise while providing some real insight on testing.
- Steve McQueen, Consulting Software Tester. A short scene from the Towering Inferno with subtitles relating to Testing. It is best to play the video and then play it again but pausing at different times for a discussion.
- TestBash 2.0 – A-Galumphing We Go James Bach. A longer video. After watching it determine how you will evaluate if he is ready for the material.
- Open Lecture by James Bach on Software Testing. This video is longer and could work well if you stop it at different points for a discussion.
- Brain Focus and other similar exercises are great since they can parallel testing in terms of understanding the requirements, working through different testing paths, and the potential danger of making assumptions. When appropriate create a decision table / tree or other diagram similar to how your team approaches a testing problem.
- Create an exercise form with discussion questions for the videos or other material to be reviewed. This helps foster a conversation between each exercise and provides a place for him to take notes.
And most importantly… do not forget the food! Cookies, donuts, candy, going out for lunch… food is always a nice touch!
Let me know if you have any other ideas on how we might sponsor a shadowing or internship for a student whether high school or college to introduce them to Software Testing!
Software Testing is one of the least known professions in the tech field. Most likely you have come across people who have performed a “checking” exercise and they equate it with “testing”. That really does dilute our value and it is important to take those teaching moments to let them know they performed a “checking” exercise to make sure that basic functionality did not break; however testing is much more complicated. This might lead to the question of:
- Do you think I can be a Tester?
- What does a Tester do?
- Do you have a job opening in your Testing department?
These are great opportunities to educate people regardless if they become a Tester. We all need to come up with our elevator pitch when asked about the testing field. An elevator pitch is basically a short summary that can be provided in about 30-seconds.
Here are a few ideas to get started and the above articles provide more details.
- What is his experience with Testers and what is his observation on their role? This can be helpful in correcting any misunderstandings as many people think Testers are the “Quality Police”. Another situation is the type of testing he witnessed at another company – for example that company may use test cases whereas another company is using session-based testing. Best to clarify any differences upfront.
- What does he enjoy doing? What are his career aspirations? If his goal is to implement company-wide solutions then a testing career might not be for him. But be careful about discouraging someone because a testing career could help him for other opportunities or he might find he really loves the field. A Tester’s skills crosses so many areas such as problem solving, critical thinking, risk analysis and mitigation, report writing, troubleshooting among many others.
- Take a personal approach to the conversation. Spend some time discussing why you became a Tester; why you enjoy the field; what do you find challenging about testing; what advice you would give a new Tester. Set a bit of background about the field before discussing how to perform testing. Have a few of the Testers share the same information. Perhaps have a small gathering of Testers meet with him to have this conversation. Make it informal so it does not feel forced and be sure to give the person opportunities to ask questions.
- Is he an employee of your company? Perhaps he can spend some time in the Testing department. Just be careful because if he does not understand testing then he might be overwhelmed with both what is testing and the product under test. There is a risk that this could discourage him from a career he might enjoy since he will not be going through the typical training program.
- Identify a tool or application he uses to demonstrate how to test a piece of functionality. Such as FaceBook, Twitter or anything that he is familiar with so his focus is on learning the testing piece. In the November article are a couple of mind maps on bridging the training.
None of these suggestions are fool-proofed and it is important to tailor the approach to the person. Hopefully this will provide a few ideas to prepare for these conversations. Plus be sure to come up with a Testing elevator pitch to be ready at any time to discuss “what is Testing”.
What is your Testing Elevator Pitch?
How do you explain a Tester’s responsibilities to someone who has no idea?
As I have mentioned in a previous posting, I have a lot of admiration for Lalitkumar Bhamare and I was honored when he asked me for an interview for his excellent e-magazine. His interview provides some insight into my professional and personal life. So grab a cup of tea and perhaps a hot scone to not only read my interview but to also enjoy the wonderful articles written by Testers from around the world! My preference is loose Earl Grey Supreme by Harney and Sons. Speaking of scones… here is a great recipe for scones!
Click here for my interview.
Recruiting and onboarding new testers are two of the more difficult tasks in a Testing department. From a recruitment standpoint you are looking for someone who will be a good fit within the team with the ability to learn how to test your product and any technical skills required. Once you made the hiring decision it is important to develop and evolve an onboarding program. I find that you never create an onboarding program and it is done. Instead it evolves over time and you might need to make minor changes based upon the person you hire and their experience. For example, for someone who has never tested, you will need to provide training on the testing techniques used in your department. If the new tester has experience then you want to ensure you demonstrate how you apply testing techniques since different companies can use terminology to mean something different.
In Tea-time with Testers, I wrote the article “Onboarding New Testers” that you can find in their November 2011 edition. This posting is going to discuss onboarding testers from a slightly different perspective.
When developing your onboarding program, think of it in terms of what do they need to know and when do they need to know it. This can help you define a program that is manageable plus provides some guidelines for expectations. Consider creating a training log with anticipated timelines to help the tester manage his/her time and add self-study tasks to ensure down time is used wisely. For example you might set initial goals for the first 90-days, 6-months, and at 1-year. I will discuss this in more detail in a later posting.
Prior to the new tester starting, be sure his/her workstation is set up, a computer is available, access to the correct systems is approved, and any other appropriate activities are completed. Be sure the new tester knows what time to show up on his/her first day and who to ask for when arriving.
The first day is often about the employee completing HR paperwork and reviewing and signing documents such as security policies. The tester should log into email, calendar system, and any other important systems that they need to use immediately. It is helpful to work out any problems the first day with logging into systems, ensuring they can print documents, and other similar tasks. Some companies have a formal HR onboarding program where the employee may or may not have time with his/her new department. I have worked for companies where the morning is dedicated to a company onboarding program and in the afternoon you join your new department.
Determine what they should be learning during their first week. For example, this is a good time to pair with another tester to learn the testing process, meet the developers, BAs and other people they will have interaction. Define your goal for the first week and develop a plan around that goal.
For each month, define a goal of what they should know by the end of that month and then create a plan around that goal. This goal might be different depending upon the person you hire. During the interview process you want to identify how much testing experience and technical experience the candidate has in order to develop a training plan before their first day. Keep in mind this plan can change based upon how quickly or slowly the new employee learns. You may have to include additional training material once you assess the employee’s learning progress. Or you might find it will not take as long as originally anticipated and you can accelerate the learning process. Remember the onboarding program is an initial plan and it should be a living document.
Of course you will need to follow any Human Resources onboarding program, but they may not have a program specifically to train software testers. This is where you can develop your own program to help integrate the new tester into your team and company.
In this series on career development I am providing different suggestions for how you might manage your own career progression. In the last posting I shared how you could use a mind map to layout your vision, goals, actions, and how you might be accountable for achieving those goals. With the mind map you can add notes to each node to keep track of your progress. However, some people may prefer a written plan providing more details. In this posting, I am sharing a sample career development plan that you could adapt to your own situation regardless if you have a manager/mentor working with you. Remember never let the lack of a manager or mentor involvement to hinder you from developing your skills and knowledge. And if you are looking for a mentor and one is not available at work or in your own social settings – tap into social media. If you are building relationships through social media I am sure there are testers who will be happy to help you.
And remember always be prepared for that next work assignment or even promotion! When those opportunities become available it is too late to prepare for them – you must be ready.
Career Development Plan
For [enter name]
I. Career Goals and Commitments
- What are the employee’s career goals?
- What are the employee’s strengths that can help with furthering the career goals?
- What are the employee’s weaknesses that can hinder progression towards the career goals?
- Provide a brief overview of learning and skills required for the employee’s present position.
- Provide a brief overview of learning and skills required for future opportunities (if applicable).
- What are the top two or three development areas to focus on over the next year?
- What is the employee’s commitment to manage his or her career opportunities?
- How will the employee’s manager/mentor support his or her career growth?
II. Development Plan
Based upon Section I identify the learning opportunities and career building opportunities that realistically can be completed over the next six-months to a year to make progression towards the employee’s goals.
Career Building Opportunities:
Learning Opportunities can be defined as: internal training, external training, blogs, and virtual conferences.
Career Building Opportunities can be defined as: opportunities to develop skills and knowledge through work assignments or other similar opportunities.
III. Review Plan
Define how often the manager/mentor and employee will meet to review progress and make any changes to the plan. This should be a living plan that can change as progress is made and new opportunities arise.