Managing the Test Execution Process

(CSTP #4) or (CTM Elective)

This tutorial covers area 4 of the Certified Software Test Professional requirements. This tutorial also covers the Elective area of the Test Management Body of Knowledge (TMBOK) required for the Certified Test Manager (CTM) certification.

Overview

This interactive workshop addresses management aspects of carrying out tests and reporting test results and test status. Since test execution typically occupies the greatest portion of test time, it is essential to leverage limited resources to maximize defect detection and ejection. That means isolating and reporting defects so they can be removed quickly. It also means analyzing testing results to enable focusing economically on areas with the biggest payback. Finally, it involves appropriate use of automated tools and other techniques to organize and handle often huge volumes of tests and test results. Exercises enhance learning by allowing participants to practice techniques with an actual case.

This course is intended to complement the one-day Managing Testing as a Process course

Learning Objectives

Manual and automated techniques to structure and manage large volumes of testware
Writing defect and status reports that help assure the important bugs are removed before delivery
Methods to reliably keep testing efforts on track and economical
Measures to monitor both testing of particular software and overall test process effectivenes

Course Outline

DEFINING TEST CASES
- What a test case is
- Test case specification
- Simple and complex tests
- Black box traceability coverage
- White box tests and degrees of coverage
- Other test case dimensions—load, duration
- Providing data values
- Test script format
- Capturing test data in a matrix
- Screen images and file layouts
- Use cases
- Automated capture/replay scripts
- Exploratory testing and error guessing
- Repeatability and reusability of test cases
TESTING INFRASTRUCTURE--TECHNICAL
- Establishing the technical test environment
- Creating and maintaining test beds
- Hardware/software capabilities and versions
- Configuration management, essential tool
- Status-based defect tracking, also essential
- Tools for designing/generating test cases
- Tools to assist and monitor testing
- Automated test execution tools
- Load testing tools
- Test management tools
- Issues governing automation of testing
ISOLATING AND REPORTING DEFECTS
- Need for reproducibility
- Bug isolation vs. debugging
- Writing an effective bug report
- Categorizing defects, identifying trends
- Dealing with duplicates
- Prioritizing defects
- Getting the important bugs fixed
- Other information to capture
- Defect reports for the project manager
- Bug assignment and status
- Projecting when it will be “good enough”
- Counting test cycles
- Defect density, mean times to fail and fix
- Defect statistics, degrees of precision
- Estimating remaining defects
- Seeding and pooling techniques
RELATING TESTING PROJECT & PROCESS
- Defect age
- Measuring test detection effectiveness
- Monitoring test progress and earned value
- Key to improving accuracy of estimates
- Reporting test project status
- Measuring process via projects
- Evaluating effectiveness of tools, techniques
- Economical advantages of static reviews
- Identifying improvement opportunities

Biography

Robin Goldsmith JD is internationally recognized as an authority on business engineering and software acquisition/development quality, testing, and productivity. He is a frequent speaker at leading conferences and formerly International Vice President of the Association for Systems Management. Robin is the author of the book:" Discovering REAL Business Requirements for Software Project Success".