Tuesday Conference Sessions (8:30 AM - 5:00 PM)

KEYNOTE SPEAKER (Tuesday 8:30 - 9:45AM)

Estimating...It's too Important to be left to Managers Tim Lister

Concepts:

In order for a software project to deliver complex products successfully, we must learn how to court inherent risk, rather than try to avoid it, or even worse, ignore it. Every great software project is full of risk; maybe that is why it is worth doing in the first place. This talk will explain the basics of software risk management. First, why risk management is such a good deal even though it will cost you money and time. Second, the
case that software folks should view their projects on the risk-value scale rather than on the productivity-quality scale. Third, the nuts and bolts of risk management, an eight step procedure you can use immediately. Last, risk management in the context of your corporate culture, and the extra long-term benefits you can receive from practicing risk management.

Biography:

Tim Lister is a principal of the Atlantic Systems Guild, Inc., based in the New York office. He divides his time between consulting, teaching, and writing. Tim was the feature interview for IEEE Software’s March/April 1997 issue, had the Point article on Risk Management in the May/June 1997 issue, and wrote the Managers’ Column for the May/June 1998 issue.

            He is working on tailoring software development processes using software risk management techniques. He gave a keynote address at the Software Management Conference in Anaheim in 2002, and will keynote at the International Conference on Software Quality this fall in Ottawa. He was a guest speaker at the SEI Risk Management Conference in April 1997, and was a guest lecturer on the topic at City University, London in March 1998.  He is a member of the Airlie Software Council, a group of industry consultants, advising the DoD on best practices for software development and acquisition.

            Tim is co-author with Tom DeMarco of the immensely popular course sequences, Software Risk Management, Achieving Best in Class: Leading, Managing and Coaching the Modern Software Organization, and Controlling Software Projects: Management, Measurement, and Estimation.

            Tim Lister and Tom DeMarco are also co-authors of Peopleware: Productive Projects and Teams, 2nd ed. (Dorset House, 1999). This book has been a book club selection of the Library of Computer and Information Sciences, and has been translated into seven languages. Tim Lister and Tom DeMarco are also co-editors of Software State-of -the-Art: Selected Papers, a collection of 31 of the best papers on software published in the 1980’s (Dorset House, 1990).  The two partners have also produced a video entitled Productive Teams, also available through Dorset House. They have a new book, Software Risk Management, due out fall of 2002.            

            Tim Lister has 30 years of professional software development experience. Before the formation of the Atlantic Systems Guild, he worked at Yourdon Inc. from 1975 to 1983. At Yourdon he was an Executive Vice President and Fellow, in charge of all instructor/consultants, the technical content of all courses, and the quality of all consultations.

            Tim Lister lives in Manhattan.  He holds an A.B. from Brown University, and is a member of the I.E.E.E. and the A.C.M.   He also serves as a panelist for the American Arbitration Association, arbitrating disputes involving software and software services, and has served as an expert witness in litigation proceedings involving software problems.

Back to Conference Schedule

PSQT Featured Presentation (Tuesday 10:00 - 11:00 a.m.)

Golden Rules for Testing
Andy Redwood

Concepts:

Biography: 

Back to Conference Schedule

PSTT Track Presentations (Tuesday 10:00 - 11:00 a.m.)

Developing an ROI for Testing Process Improvmenets
Paul Selway

Concepts: 

As a testing professional, you are often called on to suggest better ways to do the testing process. These investment needed to make these improvements are increasing required to be supported by Return On Investment (ROI) statement. This presentation uses a testing improvement case study to illustrate how an ROI was created. The author shares with the audience his experiences of producing a credible ROI. The Cost of Quality, testing as a business investment, process mapping, payback period and calculation of an ROI is covered by the presentation.

Objectives Include:

·        Producing a ROI and Payback model

·        Testing improvement as a business investment

·        Being ready for common ROI pitfalls

·        Using Cost of Quality when creating an ROI model

Biography: 

Paul Selway is Managing Consultant for Spherion Technology Inc, Software Quality Management, Minneapolis, Minnesota. He is a practicing consultant with over 23 years practical experience. Paul is frequent speaker on quality and software process improvement. Paul has an MBA, BSc Computer Science and is certified (C.Eng/CSTE/CQA). Paul has helped many fortune 500 clients see the value-added of their IT test organization.

Back to Conference Schedule

Software Testing & PR - What they didn't Teach you in School
Robert Galen

Concepts: 

Many QA, Process Improvement and Test engineers feel that the “Business” or “Management” doesn’t understand, effectively support or sufficiently value their contributions.  You know what – they’re right!  However, once we get through that hurdle, the first question becomes what is the root cause?  I believe it’s our inability and ineffectiveness at communicating and PR – in other words “selling” ourselves, our abilities and our value proposition.  We believe that our work should speak for itself. Or, we think folks understand our “value proposition” and that the metrics and data should speak for themselves. Hogwash! Just as in any good interview situation, it’s up to the interviewee to make their case.  Otherwise, they don’t get the job.  As a discipline, we need to improve our overall salesmanship when it comes to our passion and our profession. 

This talk focuses on improving Communications & PR skills across your QA, Process and Test teams so that your key partners better understand your role and its importance.

What Attendees will Learn: 

·         Why PR is a fundamental skill for QA, Process Improvement and Testing professionals

·         5 broad techniques to improve your QA, Process Improvement & Test PR

·         To leverage unexpected PR opportunities – defects, quality assessments and hallway encounters

·         Why organizational training is an important part of the PR exercise

·         That “attitude” makes a tremendous difference in perceptions and PR

Biography: 

Robert Galen is employed at EMC Corporation in Research Triangle Park, NC as a Sr. QA & Test Manager. He has also recently started a consulting firm, RGalen Consulting Group, L.L.C., where he is Principal Consultant. Bob has held director, manager and contributor level positions in both software development and quality assurance organizations. He has over 20 years of experience working in Computer systems, Financial trading systems, Mail processing equipment, Medical diagnostics systems and Telecommunications & network analysis equipment.

Bob is an active member of ACM, ASQ, IEEE/CS, PMI and active (Program & Publicity Chair) in the local RTP-SPIN group - www.rtpspin.org. He is passionate about and committed to the profession of software engineering and product development.

Back to Conference Schedule

Runtime Synchronization
Jamie Mitchell

Concepts: 

Test automation scripts can fail to run successfully for many different reasons.  One of the most common failure modes is the inability of the script to synchronize successfully with the application under test (AUT.)  This is especially problematic when trying to use recorded or semi-recorded scripts. At its simplest, automation scripts work by feeding instructions (Windows messages) to the AUT to simulate a real user sitting at the keyboard.  Synchronization (timing) problems occur when the automation tool run-time application tries to feed instructions to the AUT when it is not ready to process them.  Good synchronization will allow a test to run correctly despite variances in the runtime performance of the workstation, AUT, network, servers, and other tiers on which the AUT depends.  Attendees will be presented with several tested strategies to ensure successful synchronization of their automation.  Included will be ways of dealing with both client/server and web based applications. 

Presentation Outline: 

Common causes of synchronization problems will be investigated, using several popular tool sets as examples.  Different solutions will be discussed, with both positive and negative points illuminated.  A return on investment (ROI) view will be used to discuss the differences, keeping in mind that some elegant solutions may be prohibitively expensive for the average automation group to use.  Code snippets will be used to illustrate specific concepts where applicable.   

·         Understand the reason for synchronization failures

·         Explore various synchronization strategies

·         Compare several tools facilities for synchronization

·         Compare push (reactive) vs. pull (proactive) strategies

·         Look at wrappers/shadow functions

·         Web synchronization

Biography: 

Jamie L. Mitchell is a Principal QA Consultant at BenchmarkQA. in Minneapolis, MN.  He is a contributing editor and columnist for “The Journal of Software Test Professionals.”   He previously was a Senior Consultant at CornerStone Consulting, and the Lead Automation Engineer for Distributed Integration Testing / Global for American Express Technologies Organization.  He has long been involved in test automation as automator, designer, architect, and mentor.  He has worked in test automation since the first automation tools were released in Windows 3.0. He earned the Master of Computer Science degree in 1992 from Lehigh University and is a QAI Certified Software Test Engineer. 

 Back to Conference Schedule

Workflow Driven Tests
Tim Nelson

Concepts: 

Data driven tests allow input and verification data to be accessed from a source other than the code.  Data driven tests are normally more flexible because one test can handle multiple data scenarios.  In addition, workload for code maintenance is reduced because the maintenance is done on the values in the data tables.  Manipulating data tables is normally much simpler than modifying programming code. 

A key use of data driven tests is in building tests that verify workflow processes.  A workflow is a logical sequence of tasks that are performed to exercise a business or test process.  Workflow tests simplify code maintenance by modularizing navigation, entry actions, and test data usage.  This presentation describes how to implement workflow driven tests. 

Learning Objectives: 

• How data driven tests can decrease code maintenance
• Components of workflow driven tests
• How workflow driven tests are implemented & maintained

Biography: 

Tim Nelson is a Senior QA Test Automation Specialist.  He has been in the test and QA industry for 25 years.  He has a BS in EE from the University of Minnesota, with minors in both computer science and business.  His experience has extended across several different engineering and business development sectors.  He has engineered diagnostic and automated test systems for hardware gate & block model simulators and designing custom enterprise wide test automation frameworks for client/server and Web based technologies.  He is now a principle owner in a Software Test and QA consulting firm (STAMP Technologies, LLC) that is developing a new generation of highly qualified software test and automation specialists. 

Michael R. Waller is a Senior QA Consultant.   He has over 20 years of IT experience including software QA, test automation, software development, and operations.  His experience spans manufacturing, financial, and software development companies.  He has an MS in Management from Florida Institute of Technology and a BS in Math/Computer Science from Boise State University.

Back to Conference Schedule

PSQT Track Presentations (Tuesday 1:00 - 2:00 PM)

Getting the Most from Your Change Process
Renata Johnson & Robert Daudt

Concepts: 

This presentation/paper covers a practical approach that can be easily applied to a change process that will help a project manager make major decisions throughout the software development lifecycle.  Making three adjustments to your change process will add value and ensure the quality of the products you deliver.  By concentrating on the problem reporting of your change management system, you can implement four basic elements to make a difference.  These are

1. Begin tracking the module or component name that the problem report is being reported against. 
2. Set a classification system for your problem reports that directly addresses the customers’ concerns regarding quality. 
3. Create a quality metric that will measure the quality based on your classification system.
4. Set quality metric goals for the product release(s).

 Objectives Include: 

• Understand how quality objectives can be met through the change management process.
• Understand and use a metric collection technique that is practical and easy to apply and that reflects the quality objectives and goals for your organization
• Understand the data once it is collected
• Understand how to report the data once collected 

Biographies: 

Ranata Johnson has over 14 years of experience in software engineering and management practices, with emphasis on software quality engineering and project management.  Since joining Pacific Northwest National Laboratory in 1991, Ms. Johnson has been a member of the Information Sciences and Engineering (IS&E) group.   Her responsibilities include project management and planning for IS&E projects, implementing configuration management, testing, reviews and inspections practices, metrics, ensuring successful customer support and system installations, and understanding and implementing the Software Systems Engineering Process (SSEP).  She also manages and promotes the SSEP, participates in and facilitates process improvement teams, and provides training to line/project staff on the SSEP.  Ms. Johnson has been a guest lecturer for Washington State University on Software Engineering and has been a trainer for the Quality Training Resource Center (QTRC) in the area of software quality assurance.   Ms. Johnson received her bachelor's degree in Computer Science from Chico State University.   She is certified by West Coast University in the area of configuration management.  She has previously sat on the board for the American Society for Quality Control (ASQC) as Secretary and Education Chair.  Ms. Johnson is also a Certified Lead Auditor. 

Robert Daudt has over 10 years of experience in systems engineering, software development, and project management.  Prior to his current role as Senior Software Quality Engineer at Pacific Northwest National Laboratory (PNNL), he served as Vice President of Quality and Information Technology for the Alistar Manufacturing Group.  Before that, he was a Systems Engineer for Cessna Aircraft Company, where he specialized in system safety assessment, failure mode analysis, and risk identification/reduction. Mr. Daudt’s application domain experience extends from aircraft flight control systems to mission critical enterprise systems to simple desktop applications.  His contributing roles have ranged from Project Manager to Programmer.  His current responsibilities include consulting on software engineering/management best practices in the areas of risk management, configuration management, test planning and execution, requirements management, software lifecycle tailoring, and performance measurement.  He is a member of the Software Systems Engineering Process (SSEP) team at PNNL, where he contributes as both a content provider and a software developer.  Mr. Daudt received his bachelor's degree in Aerospace Engineering from Embry-Riddle University.  He is currently completing the final requirements for a Master of Science in Computer Systems.  He is also an active member of the American Society for Quality (ASQ), and the IEEE Computer Society.

Back to Conference Schedule

Some Particularly Useful Metrics to Aid in Assessing Testing Status and Product Quality
Robert Vanderwall

Concepts: 

This presentation will discuss four metrics found to be particularly useful for understating the status of the test process and the quality of the product being tested.  The metrics will be presented in general terms, then described in detail.  The technique that was found effective for collecting the metrics is provided as well as cautions and gotcha’s.  Finally, practical solutions for analyzing the data and presenting those results is described. 

Objectives Include: 

·         How to collect and utilize the four presented metrics

·         How to recognize misuse of metrics

·         How to avoid collecting useless numbers

·         Ways to present and ‘sell’ the information

·        Tips for evaluating metrics before collecting them

Biography: 

Robert Vanderwall has more than 15 years in software testing and quality assurance.  During that time, he has been involved in every aspect of product realization from conception through deployment and field support.  Vanderwall was the test team leader on several projects and during that time, took every opportunity to learn the trade. Additionally, he has been an active, hands-on tester for much of his career.   He is currently finishing up a dissertation on software testing techniques.   While doing testing and managing testing, Vanderwall examined, tried and rejected a cornucopia of metrics before settling on the handful that have proven useful.  Those will be presented and discussed.

Back to Conference Schedule

Managing Quality During the Endgame
Bruce Schoor

Concepts: 

This presentation covers all the phenomena that one has to deal with in the period between code complete and ship in a professional software development environment.  Through a mix of pragmatic principles and hands-on examples from real-world projects, it is demonstrated how to plan the endgame, how to target the established quality bar, how to measure progress, how to use steering forces to drive progress and correct deviations, and how to deal with team morale issues during long stabilization efforts. 

Learning Objectives: 

Test and development focus after code complete

Dealing with risks

Using metrics to control risk

Preparing release candidates

Go/nogo 

Biography: 

Bruce Schoor, the founder of Schoor Software Consulting, is an expert in the field of development processes, quality assurance and project management.  He has mastered the art of shipping world-class products as QA Manager at Microsoft Corp., and applied his skills furthermore in dynamic environments such as Softimage/Avid (leading provider of high-end 3D animation & video editing software), Zero-Knowledge Systems (pioneer in internet privacy) and numerous consulting contexts. 

During more than 15 years in industry, Bruce has created highly effective R&D and QA infrastructures, and built dynamic motivated teams from the ground up.  Carefully selecting, planning and implementing the right software best practices have helped each of these large R&D teams become efficient and predictable at delivering high-quality releases.  Capability at CMM Level 3 and 4 was achieved successfully during these efforts while dealing with both the technical and people aspects of fast growth. 

Bruce has specific and extensive experience with endgame strategies, i.e. the phase from code complete until ship.  Through sophisticated but pragmatic practices in planning, metrics and using the right controls, Bruce has managed to ship products with high predictability, specified quality, and on-time delivery.

Back to Conference Schedule

A Do-It-Yourself Practical Software Project Assessment Method
Stephen Kan & Diane Manlove

Concepts:

Imagine...that you are a member of the quality team in your organization and your executive has asked you to lead the assessment of a specific project...or you are a software development manager (or project manager) and you are asked to conduct a peer review of a project...or you are a quality manager and you are asked to conduct an assessment of a handful of projects to establish the best practices of the projects and you don’t have the funding to hire external consultants...or you are just interested in learning a practical approach to conducting software project assessments!  If any of these situations apply to you, then you will want to attend this presentation and learn how to conduct efficient, effective, and fruitful assessments of software projects. 

This presentation discusses a 7-step, do-it-yourself software project assessment method, which was developed  based on the practical experiences of numerous project assessments by the authors over the past two decades.   In this presentation, the differences between process maturity assessments and software project assessments will be described, the software  process (or project) assessment cycle will be discussed, and a step by step discussion of the 7-step project assessment method will be provided.  Along with the description of the assessment method, the following methods and techniques will also be covered:

§          Process based Vs project based assessments

§          Examples and methods to change questions on “what’s” to questions on “how’s” and “why’s”

§          The vignette approach to questionnaire construction

§          Gap analysis

§          Ways to identify improvement opportunities

§          Ways to derive meaningful and useful recommendations

§          Examples from real-life project assessments

Biography:

Dr. Stephen Kan is a Senior Technical Staff Member and a technical manager in programming at IBM in Rochester, Minnesota.  He is responsible for the Quality Management Process in software development for IBM’s eServer iSeries (formerly the AS/400 computer system).  His responsibility covers all aspects of quality ranging from quality goal setting, supplier quality requirements, quality plans, in-process metrics and quality assessments, to reliability projections, field quality tracking, and customer satisfaction.  Dr. Kan has been the software quality focal point for the software system of the AS/400 since its initial release in 1988.  He is the author of the book Metrics and Models in Software Quality Engineering, numerous technical reports, and articles and chapters in the IBM Systems Journal, Encyclopedia of Computer Science and Technology, Encyclopedia of Microcomputers, and other professional journals.  Dr. Kan is also a faculty member of the University of Minnesota Master of Science in Software Engineering (MSSE) program.  Dr. Kan has conducted numerous software process and project assessments over the last two decades.

Diane Manlove is a Software Quality Engineer for IBM in Rochester, MN.   Her responsibilities include the management of release quality during product development, system quality improvement, product quality trend analysis and projections, and software project quality assessments.   Diane is certified by the American Society for Quality as a Software Quality Engineer, a Quality Manager, and a Reliability Engineer and by the Project Management Institute as a Project Management Professional.  She holds a Master's degree in Reliability and an undergraduate degree in Engineering.  Ms. Manlove has been employed by IBM since 1984 and has additional  experience in hardware and software test and in manufacturing quality.

Back to Conference Schedule

PSTT Featured Presentation (Tuesday 1:00 - 2:00 p.m.)

Determining the ROI of Testing for your Organization
Ed Adams

Concepts:

Biography:

Back to Conference Schedule

PSQT Featured Presentation (Tuesday 2:15 - 3:15 PM)

Minimum Metrics for Meaningful Management
Robin Goldsmith JD

Concepts:

To know what you are doing, you must meaningfully measure the right things at the right times in the right ways.  Clearly, IT's frequent poor project results are due in large part to poor measurement.  Not only do we tend to miss important measures, but we also often overwhelm ourselves with too many measurements.  This interactive presentation describes a minimum set of metrics that IT needs to know and the context needed to make the measures meaningful.  Special measures of testing effectiveness also are shown.  Techniques also are suggested for overcoming resistance when getting started. 

What would you measure if it REALLY mattered
How IT tends to measure what it does
The often-missing but essential measurement point
Why inadequate measures produce poor projects
What meaningful measurement would show
Economics of errors, can your organization tell?
Core set of measures
Keep measurement concepts and techniques
Avoiding the Hawthorne Effect
Secrets for overcoming resistance to metrics start-up
Reward vs. punishment
Measuring effectiveness of testing

Biography:

Robin Goldsmith JD is internationally recognized as an authority on business engineering and software acquisition/development quality, testing, and productivity.  He is a frequent speaker at leading conferences and formerly International Vice President of the Association for Systems Management.

Back to Conference Schedule

PSTT Track Presentations (Tuesday 2:15 - 3:15 PM)

Effectively Integrating Automation into a Manual Test Environment
Dr. Russ Shermer

Concepts: 

Roughly half of the purchased test automation tools live on the shelf. The other half of these tools are utilized poorly and rarely provide a significant portion of the test coverage. How can these tools be leveraged to yield a suitable return on investment?  The typical scenario: QA Manager spends a large sum of money on the latest and greatest tool. Upper management has great expectations and unrealistic application deployment deadlines. The testing team has a look of severe bewilderment when handed the testing tool package.  As testers we bock at having development test code or the business analysts design the software yet we break our own rules when it comes to writing good automated tests. 

Learning Objectives Include: 

Why standard test automation projects fail

How to build an effective test automation team from existing resources

What processes are necessary to ensure the success of a test automation project

A simple methodology to ensure the success of the testing effort

A set of metrics to measure the return on investment of any test automation effort. 

Outline: 

Introduction

Case Study of Automation Project Failures

Common Automation Failure Modes

Solving the Resource Issue

Defining an Effective Process

Work Product Standards

Metrics

Case Study of Process-Driven Projects

Discussion & Closing

Biography: 

Dr. Russ Shermer has a PhD in computational physics and has been developing computer-based solutions for fifteen years. He wrote a cross-platform test automation tool and has worked with several of the major testing tools for over five years. While at Spherion, he lead the effort to define and deploy a test automation methodology at the national level.  In his career, Russ has provided performance and test automation services in over 15 major corporations.

Back to Conference Schedule

Can a Testing Maturity Model Help Improve My Testing Process?
Thomas C. Staab

Concepts: 

The Software Testing Maturity Model (SW-TMM) is an exciting new tool that can help organization's make significant improvements in the testing process.  It is a companion to the Software Capability Maturity Model (SW-CMM). In order for an organization to improve their software testing process they should know their current testing maturity and how to progress to the next level.  This presentation will focus the Software Testing Maturity Model (SW-TMM) developed by the Illinois Institute of Technology. It will present the theory behind the SW-TMM, discuss why a company should use it and lastly discuss how to effectively utilize the SW-TMM.  This presentation will provide valuable testing maturity and assessment information that can be brought back to your organization and utilize immediately. 

Learning Objectives Include: 

1.      Discover the different Testing Maturity Models available

2.      Gain knowledge about the Software Testing Maturity Model (SW-TMM) developed by the Illinois Institute of Technology

3.      Explore the benefits a company will receive from using the SW-TMM

4.      Learn a process to use to evaluate your organizations' testing maturity 

Biography:

Thomas C. Staab owns an independent consulting firm, Wind Ridge International.  He has over 35 years experience.  Prior to opening his own consulting firm he worked for over 25 years in the quality assurance profession. He holds a Master of Science degree in Quality Systems. His consulting work incorporates his extensive quality assurance and information technology experience into every project.  He has developed the test plan and coordinated the testing of numerous systems for clients.  His expertise is in bringing this practical experience into the classroom.  

Mr. Staab is listed in the International Who’s Who of Information Technology.   He has currently published over 25 articles (one of which earned him Author of the Year Award from the Comptroller magazine) and presented over 50 speeches at regional, national and world conferences. He has developed and taught numerous training courses during his career and has always received excellent evaluations for his training courses and speeches.