Tutorials: Series M
Monday, April 2nd, 2001

Six concurrent tutorials taught by nationally recognized quality experts. Each tutorial is a one full day of an in-depth instruction in a specific software quality topic.

   Series T - Tuesday, April 3, 2001

   Series W - Wednesday, April 4, 2001

   Series M - Thursday, April 5, 2001

   Series F - Friday, April 6, 2001   

Tutorial #M1 (Monday 8:30 - 4:30 p.m.)

Brian Hambling

This tutorial counts as one day of training towards the Certified Software Test Professional requirements.

Description

The Principles of Software Testing tutorial is aimed at those new to the software testing discipline, especially those who want to work towards the Certified Software Testing Professional qualification.

The seminar addresses the fundamental ideas that underpin the software testing discipline and the role that testing plays within the overall discipline of software development. As well as looking at general principles, the tutorial covers aspects of testing appropriate to specific environment such as mainframe, client/server, real-time software development, and e-business applications.

Outline

The tutorial addresses the basic principles of software testing:

• What software testing is and is not
• Basic testing vocabulary
• The basic testing process
• Verification and validation
• Static and dynamic testing
• Reviews and inspections
• The testing life cycle
  
  • Test strategy
  • Test planning
  • Test design
  • Test execution
  • Incident management
  • Testing metrics
    
• Testing in the development life cycle
• Alternative approaches to development and test
  
  • Mainframe systems
  • Real time systems
  • Client/server systems
  • Web/Internet/e-business systems
    
• Test automation

Management of the testing process

Learning Objectives

After attending this seminar delegates will be able to:

• Understand basic testing vocabulary
• Describe the basic testing process and relate it to the development process
• Describe the role of software testing in a number of alternative development approaches
• Identify the role and purpose of software test automation
• Describe the role and purpose of test management

Brian Hambling is Operations Director of ImagoQA Limited, a UK-based software testing consultancy. Brian has over 25 years experience in the IT industry, during which he has managed software development projects, taught software engineering, audited software quality management systems and acted as a strategic consultant to many organizations, both large and small, from the IT sector and from other industries. Brian’s role at ImagoQA includes quality management, research and development, development and delivery of consultancy services, and career development for a staff of 130 consultan

Top of Page Tutorials: Series M

Tutorial #M2 (Monday 8:30 - 4:30 p.m.)

This tutorial counts as one day of training towards the Certified Software Test Professional requirements.

Welcome to the wonderful world of the Web.  Along with exciting opportunities, we have impatient bosses and users, incompatible browsers, new (and buggy) tools, rapidly changing technologies, sites that tend to crash if you blink twice, eager hackers and interminable waits.  This session covers the basics of making sure that our bosses and users are not unpleasantly surprised after we say: "Yup. It's ready to go.  I'd swear my paycheck on it"

 Tutorial Objectives

 Attendees will learn the following: 

• The Common Risks and Quality Issues of Web Sites
• How to develop a Test Plan for a Web Site
• How to Test Web Site Features, Content and Navigability
• How to Test in an Iterative, Rapid Application Development World
• How to Manage the Web Test Facilities
• How Automated Tools for Web-based Testing Work
• How to Monitor Service Levels in Live Operation
• How to Manage Deadline Pressures 

Tutorial Outline: 

Overview of Web-Based Application Testing

   A Framework for Testing Web-Based Applications

   Web-Based Testing and Quality Issues

   Web Site Testing Approach

   Web-based Application Testing Checklist

   Web Page Test Case Checklist 

Web Testing Procedures

   The Web Site Test Plan

   Test Case Design Methods

   Evaluating Test Results

   Problem Reporting, Follow-up and Re-Testing

   “Web Time”: Handling Deadline Pressures 

Web Site Feature Testing

   Testing Features and Content

   Testing Process Flows, Navigability and Links

   Positive/Negative and Boundary Value Testing

   Risk-Based Testing

   Exploratory Testing

   Common Sources of Errors 

Testing in an Iterative, Rapid Application Development World

   Stability and Change Management

   Component Re-Use and Integration Testing

   Coordinating Development and Testing

   Change and Regression Testing

    Testing Changes

    Re-Testing Unaffected Features after Changes 

Establishing and Managing the Web-based Test Lab

   Equipment and Facilities Needed

   Test Tools

   Test Case Libraries

   Test Processes and Support Tools 

Automated Tools for Web-based Testing

   Functions of Automated Test Tools

   Why Automate Testing?

   Assessing Readiness for Test Automation

   Automated Test Tool Disadvantages

   Resource Impact of Automation

   Automated Tool Selector

   Load or Stress Testing Tools

   Test Data Generation Tools

   Compatibility Checking Tools

   Network Monitoring Tools

Managing Deadline Pressures 

Ross Collard is a software testing consultant.  His clients have included: Amazon.com, American Express, Anheuser-Busch, Banamex, Bank of America, Baxter Healthcare, Bechtel, Blue Cross/Blue Shield, Boeing, British Airways, the CIA, Ciba Geigy, Cisco, Citibank, Dell, EDS, Exxon, General Electric,  Goldman Sachs, the Federal Reserve Bank, Ford, Fijitsu, Hewlett-Packard, Intel, Johnson & Johnson, JP Morgan, Lucent, McGraw Hill, Merck, Microsoft, Motorola, NASA, Nortel, Oracle, Procter & Gamble, Prudential, IBM, the U.S. Air Force, Worldcom and Yahoo.  Ross has taught software testing for Harvard and U.C. Berkeley.

Top of Page Tutorials: Series M

Tutorial #M3 (Monday 8:30 - 4:30 p.m.)

This tutorial counts as one day of training towards the Certified Software Test Professional requirements.

This tutorial outlines the Automated Test Life-cycle Methodology, a structured process for designing, developing, executing and managing testing that parallels the System Development Life-cycle. It is based on the book titled "Automated Software Testing" co-authored by the instructor and published by AWL, ISBN 0-201-43287-0.

Automated Testing Life-Cycle Methodology

How test teams introduce an automated software test tool on a new project is nearly as important as the selection of the most appropriate test tool for the project. A tool is only as good as the process being used to implement the tool.

Over the last several years test teams have largely implemented automated testing tools on projects, without having a process or strategy in place describing in detail the steps involved in using the test tool productively. This approach commonly results in the development of test scripts that are not reusable, meaning that the test script serves a single test string but cannot be applied to a subsequent release of the software application. In the case of incremental software builds and as a result of software changes, these test scripts need to be recreated repeatedly and must be adjusted multiple times to accommodate minor software changes. This approach increases the testing effort and brings subsequent schedule increases and cost overruns.

The fallout from a bad experience with a test tool on a project can have a ripple effect throughout an organization. The experience may tarnish the reputation of the test group. Confidence in the tool by product and project managers may have been shaken to the point where the test team may have difficulty obtaining approval for use of a test tool on future efforts. Likewise, when budget pressures materialize, planned expenditures for test tool licenses and related tool support may be scratched.

By developing and following a strategy for rolling out and implement an automated test tool as part of the Automated Testing Life-cycle methodology, the test team can avoid having to make major unplanned adjustments throughout the test process. The tutorial "Automated Software Testing" addresses these various issues and their solutions.

The ATLM describes how and where "Automated Software Testing" fits into the system development life-cycle.

This tutorial will cover the following topics:

1. Overcoming False Expectations for Automated Test. This part of the tutorial will address the false expectations that automated testing will result in :
• Automatic Test Plan Generation
• Test Tool Fits All
• Immediate Test Effort reduction
• Immediate Schedule Reduction
• 100% Test Coverage
2. Outline Benefits of Automated test. This part of the tutorial will address the actual benefits of automated testing:
   • Production of a reliable System
   • Improvement of the quality of effort
   • Reduction of Test Effort and Minimization of Schedule
   • Some tests cannot be executed unless they are automated
     
3. Acquiring Management Support.  This part of the tutorial will address how to best acquire management support (i.e writing a proposal, content of the proposal, how to convince management, etc.)
   
4. Acquiring Tools. This part of the tutorial will address how to acquire an automated tool.
   • Review System Engineering Environment
   • Review Tools Available on the market
   • Tool Research and Evaluation
   • Reviewing the Test Life-Cycle Tools
   • Tool Purchase
     
5. Automated Testing Introduction Process. This part of the tutorial will address how to best introduce automated testing to a new project team, including:
   • Test Process Analysis
   • Process Overview
   • Goals and Objectives of Testing
   • Test Strategies
   • Test Tool Consideration
   • Test tool compatibility check
   • Review of Training requirements
   • Test Team Recruiting and Management
     
6. Test Planning, Design and Development. This part of the tutorial will address how to best address test planning, design and development of tests in an automated testing environment, to include:
   • Test plan documentation
   • Test Program Scope
   • Test Requirement Management
   • Test Environment
   • Test requirements Analysis
   • Development-Level Test Analysis
   • System-Level Test Analysis
   • Test Program Design
     • Review Test program design modules
     • White-Box Techniques ( Development-LevelTests)
     • Black-Box Techniques ( System-LevelTests)
     • Test Design Documentation – examples of Test Design using Orthogonal Array Testing Techniques.
     • Test procedure definition
     • Automated vs Manual Test Analysis – what to automate and what not to
     • Automated Test design standards
   • Test Development
     • Set up Test Environment
     • Automation Framework Reuse Analysis
     • Test Procedure Development/Execution Schedule
     • Calibration of the test tool
     • Compatibility and work around solutions
     • Manual Execution of Test Procedures
     • Test Procedure inspections and Peer reviews
     • Test Procedure Configuration management
     • Reusable Test procedures
       
7. Automated Test Execution and Management of it. This part of the tutorial will address how to best address automated test execution and the management of the testing execution cycle, to include:
   • Unit Test Execution and Evaluation
   • Integration Test Execution and Evaluation
   • System Test Execution and Evaluation
   • Test Analysis of Regression Testing
   • User Acceptance Test Execution and Evaluation
   • Testbed baseline
   • Creating Baseline data
   • Tools to Generate Data
     • Defect Tracking
     • Identify the priority of a defect
     • Assign a unique identifier to each defect
     • Link each defect to test procedure/build
     • Log all the appropriate dates for tracking
     • Create metrics for Defect Management
     • Test progress Tracking
     • Test Metrics
       
8. Post release. Test Process Improvement – This part of the tutorial will address the post release and evaluation period and how to improve the test process, to include:
   • Documenting Lessons learned
   • What worked and what did not?
   • How would you do things differently?
   • Reviewing standards for future projects

Various companies throughout the world have adopted the ATLM and incorporated it into their project.

Learning Objective

• Best practices for test automation
• A Case Study will be presented that covers how the ATLM was implemented on one particular project. This case study will address each phases of the Automated Testing Life-cycle. Students can bring their own project specific problems, which can be addressed during the tutorial.
• Acquiring management support
• Test tool evaluation and selection
• The automated testing introduction process
• Various tools used during the various life-cycle phases (i.e. Rational Rose during Architecture phase, ReqPro or DOORS during Requirements Management, Robot or Winrunner during Test Execution Phase, Performance Studio or E-Suite for performance testing)
• Automated and manual test planning and preparation
• Test procedure development guidelines
• Automation reuse analysis and reuse library

Elfriede Dustin is a SQA Certified test engineer and has supported test efforts for a multitude of applications. She is frequently a speaker at various Quality Assurance and Software Test Conferences. Elfriede has a BS degree in Computer Science and has performed as a Computer Systems Analyst/Programmer developing software applications and utilities, process and data modeling using CASE tools, and system design simulation models. 

In support of software test efforts, Elfriede has been responsible for implementing automated test, or has performed as the lead consultant guiding the implementation of automated software test.  Elfriede has lead the successful rollout of automated testing tools at three companies, and has applied her rollout strategy on over nine different projects.

Elfriede Dustin is the lead author of the best-selling book "Automated Software Testing", published by Addison-Wesley Pub Co; ISBN: 0201432870, July 1999. The book has been getting excellent reviews throughout the testing community (see also www.amazon.com).

Top of Page Tutorials: Series M

Robin F. Goldsmith JD

What will you Learn?

• 21 ways to test that business/user requirements are accurate and complete. 

• Finding previously overlooked problems when they are easiest and least expensive to fix.

• Recognizing, communicating, and gaining commitment to the importance of adequate requirements.  

• Evaluating the levels of quality embodied within the requirements.

• Testing techniques that enhance customers' involvement and communication with management.

• Allocating testing resources economically.

Course Outline:

VALUE OF TESTING UP-FRONT

•         Overcoming obstacles to improvement

•         Role of requirements in system problems

•         Big economic payoff of better requirements

•         Proactive TestingÔ Life Cycle Model

•         Survey on improving requirements quality

•         Keys to effective testing

•         Why up-front testing usually is so weak

•         CAT-Scan ApproachÔ secret to quality

  TESTING REQUIREMENTS FORMATS

•         The “Regular Way” we review requirements

•         Hidden weaknesses of traditional methods

•         Adding strength to subjective evaluations

•         Formal technical review

•         Inspection topics and standards

•         Making sure they are requirements

•         Assessing reviewability

•         Determining deliverability

•         Demonstrating testability

•         Testing structural completeness and clarity

•         Format for requirements deliverables

FINDING OVERLOOKED REQUIREMENTS

•         What we mean by system quality

•         Identifying all the stakeholders

•         Detecting all three Quality Dimensions

•         Design, Performance, Conformance needed

•         Addressing relevant quality factors

•         Candidate quality factor requiremements

•         Commonly overlooked deliverables

 ASSURING ACCURACY/COMPLETENESS

•         Checking importance and criticality

•         Finding Engineered Deliverable QualityÔ

•         Guidelines and conventions vs. IT standards

•         Engineering standards to do a job well

•         Ascertaining trade-off balances

•         Simulation and prototyping

•         Walking through requirements

•         Joint Application Development (JAD)

•         Defining acceptance criteria

•         Matching to independent definitions

•         Independent/expert validation

•         Measuring the "proof of the pudding"

This course shows ways to test requirements after they have been collected.   It is not intended to teach how to collect requirements, although the testing methods suggest collection methods which would be helpful.  This interactive session introduces 21 methods with increasing power.   Following the CAT-Scan Approach, participants apply the techniques successively to a real case and discover how each different method reveals additional, otherwise-overlooked defects when they are easiest and least expensive to fix.   Participants learn ways to find previously overlooked requirements, increase meaningful customer/user involvement, enhance communications and understanding, and truly test the adequacy of requirements definitions.

Robin Goldsmith JD is internationally recognized as an authority on business engineering and software acquisition/development quality, testing, and productivity.  He is a frequent speaker at leading conferences and formerly International Vice President of the Association for Systems Management.

Top of Page Tutorials: Series M

Tutorial #M5 (Monday 8:30 - 4:30 p.m.)

Patrick Johnson

This course shows how use case modeling and requirements management techniques can be used to define and document the requirements of a product that meets stakeholder needs.  Participants will learn how to elicit and manage changing requirements; analyze development problems, define the product vision and product features, define software requirements and requirement attributes, and maintain traceability for use in scope management, change management, and impact analysis.  Students gain an in-depth understanding of use case techniques through hands-on experience with actual use cases.

Recommended For

Technical, testing professional, and documentation managers, project leads, executives, application experts, business analysts, salespersons, and analysts/designers who are involved in requirements capture, specification, use, or management.

Objectives

§          Apply requirements management skills to create product requirements

§          Capture and document requirements with use case modeling techniques

§          Apply proven techniques for reaching and maintaining agreement with the stakeholders

§          Create a documentation hierarchy and standards for requirements

§          Understand the architecture of a simple requirements management system and maintain requirement attributes and traceability

§          Use requirement attributes and traceability links to manage scope and change throughout the life cycle of the product

§          Understand how requirements drive ongoing design, test, and user-documentation activities

Topics Covered

§          The requirements management process

§          The Rational Unified Process w/ UML

§          Analyzing the problem

§          Understanding stakeholders needs

§          Defining the systems: the vision, product features, and use case model

§          Finding actors and use cases

§          Managing system scope

§          Refiining the system definition

§          Detailing each use case

§          Managing changing requirements

§          Structuring the use case model using use case relationships (include, extend, generalization)

§          Requirements across the product life cycle

Prerequisites

A basic knowledge of project management and the software development life cycle is recommended.

Patrick Johnson has provided training and consulting on Rational’s products for several years.   His experiences range from modeling and analysis, requirements gathering to software quality automation, which enables him to provide expertise in each phase of the development and testing lifecycles… a unique skill set not common to the industry.   Also, he has effectively implemented Rational’s software quality methodology and tools on previous projects.  Patrick was instrumental in customizing Rational's testing methodology to develop a standardized testing methodology for large insurance company.  Because of Patrick’s expertise in Quality Structured Management Techniques and Quality Assurance for client/server applications, he was awarded the Quality Award by Shell’s management group in recognition for his innovative concepts and procedures that directly affected QUALITY.

Top of Page Tutorials: Series M

Tutorial #M6 (Monday 8:30 - 4:30 p.m.)

Linda Westfall

This tutorial introduces the reader to a practical process for establishing and tailoring a software metrics program that focuses on their goals and information needs. The objective of this tutorial is to provide a practical, 12-step, start-to-finish process of "how to" select, design and implement a tailored software metrics program. It outlines a cookbook method that attendees can use to simplify their journey from software metric concept to delivered information.

There is a multitude of possible software metrics. The first four steps in the 12-step process will teach the attendee how to identify their metric customers and then utilize the goal/question/metric paradigm to select the software metrics that match the information needs of those customers. Steps 5-10 walk the attendee through the design and tailoring of their selected metrics including standardizing definitions, selecting models, defining counting criteria, establishing benchmarks and objectives, choosing reporting mechanisms and selecting additional qualifiers. The last two steps help the attendee solve implementation issues including collecting data and minimizing the impact of human factors on their metrics.

Format

Information and skills taught in this tutorial are presented in an interactive format. Real-world metric examples are utilized to make the information relevant to the attendee. Throughout this tutorial, learned skills are practiced using both interactive and team exercises based on metrics currently used in the software industry and on the current information needs of the attendees. Templates are utilized to facilitate these exercises. The emphasis is on techniques that allow attendees to transition the skills learned in this tutorial to their own work environments.

• A set of student notes including copies of all presentation slides with annotated text
• Example answers to all exercises used in the tutorial
• An annotated bibliography of reference materials
• A set of software metrics templates that the attendees can tailor for their own use

Target Audience

This tutorial is designed for:

• Metrics Analysts who are responsible for designing and implementing software metrics
• Software Development and Test Managers & Engineers who want better information upon which to base their software product, project and process decisions
• Software Quality and Process Managers & Specialists who want qualitative and quantitative information to support their audits and assessments and to help identify process improvement opportunities.

This tutorial assumes a basic understanding of software development and maintenance practices and of software project management techniques. It is presented at an intermediate level.

Tutorial Outline

• Project Planning for Metrics Implementation
• Learning Lessons – Pilot the Metrics
• Implementing the 12 Steps

Linda Westfall is the President of The Westfall Team which provides Software Metrics and Software Quality Engineering training and consulting services. Prior to starting her own business, Linda was the Senior Manager of the Quality Metrics and Analysis at DSC Communications where her team designed and implemented a corporate wide metric program. Linda has twenty years of experience in real time software engineering, quality and metrics. She has worked as a Software Engineer, Systems Analyst, Software Process Engineer and Manager of Production Software.

Very active professionally, Linda Westfall is Chair Elect of the American Society for Quality (ASQ) Software Division. She has also served as the Software Division’s Program Chair and Certification Chair and on the ASQ National Certification Board. Linda wrote the Software Metrics and Software Project Management sections of the ASQ Software Quality Engineering course and co-authored the ASQ Software Metrics course.

Top of Page Tutorials: Series M