Wednesday Conference Sessions (8:00 AM - 6:00 PM)

Keynote Presentation (Wednesday 8:30 - 9:45 AM)

Improving the Test Process Martin Pol

Concepts:

Since time-to-market, globalization and the quality of services, software included, are serious conditions for many companies to win the competition, traditional and time consuming test processes are unacceptable. For example, the high change rate of e-business applications combined with the more challenging quality requirements on security, usability and performance require adequate and more mature test solutions. Based on his experiences Martin Pol will explain why many companies around the world are improving their testing processes. He'll present the major business trends and technical innovations in the industry that put testing under pressure.  

Since Software Process Improvement models such as CMM/SEI more or less ignore testing, testing has to take care of its own improvements. Martin will discuss how the testing world can adequately improve the test processes in order to cope with the consequences of the beforementioned trends and innovations. Dedicated, practice based test process improvement guidelines should provide the frame of reference for continuous improvement of the test processes. Structured, continuous test process improvement programs are required to establish a stable foundation for the challenges to come. 

The goal of improving the test process is optimising the required time, money and quality of testing. The definition of measurable goals enables to make adequate measurements (quantified observations) to establish the status of the goals during and after the improvement process. Before starting the improvement activities, metrics should be selected and/or created for this purpose. The main areas for which metrics can be selected and measurements can take place will be explained in more detail.

Key Points

1.         Why improvement of the test process is necessary?

2.         How to improve?

3.         Measuring the results

Biography:

Martin Pol, with more than 25 years of experiences in structured testing is the (co-)author of many publications and books on structured testing in different languages. He is the CEO and a senior consultant for Polteq International Testing Services B.V. He was involved in the development of the structured testing approach TMap and the creation of TPI. Also because he's working in the testing practice every day Martin is a highly regarded lecturer and speaker at conferences throughout the world. He twice chaired the EuroSTAR conference and recently he received the European Testing Excellence Award for his contribution to the field of testing.

TPI^® and TMap^® are registered trademarks of IQUIP Informatica B.V.

Back to Conference Schedule

PSQT Featured Presentation (Wednesday 10:00 - 11:00 AM)

Defining Scope that Doesn't Creep
Robin Goldsmith JD

Concepts:

Most software developers consider a high degree of scope creep to be normal and unavoidable.  "That's not what I expected" and "Users don't know what they want" indeed are repeatedly predictable outcomes of the inadequate way requirements are defined conventionally, but they don't have to be.  This interactive presentation shows the primary-but seldom recognized--REAL reasons developers continue to get requirements and scope wrong, which is what actually causes creep.  Powerful techniques are described for discovering the REAL requirements and documenting scope in ways that can dramatically reduce creep. 

Three Key Elements of REAL Requirements
IT's View of Business and System Requirements
Functional Specs and Use Cases
REAL Business/User Requirements
Why IT Keeps Missing the REAL Requirements
Illusion of Requirements Gathering
Detective Approach to Requirements Discovery
Problem Statement Technique
Getting the Problem Right
7 Guidelines for Documenting Requirements
Conventional Scope Statements-That Creep
Scope Statements that Don't Creep
How to Do It Quickly

Biography:

Robin Goldsmith JD is internationally recognized as an authority on business engineering and software acquisition/development quality, testing, and productivity.  He is a frequent speaker at leading conferences and formerly International Vice President of the Association for Systems Management.

Back to Conference Schedule

PSTT Track Presentations (Wednesday 10:00 - 11:00 AM)

Excel'erating Test Status Reporting
Earl Burba & Jim Hazen

Concepts: 

This presentation will show how to implement a test status reporting tool and strategy using Microsoft Excel.  It will discuss how to use Excel as a tool to develop, manage, and report on testing status for the testing effort during a project.  It will show which of the features, and how to use them, in Excel to develop the test status reporting tool.  It will show how to build worksheets for test cases, summary reports, and status reports.  The presentation will discuss the benefits of effective test status reporting.  Also, when the tool needs to be built and implemented will be discussed.  Attendees will come away with a working model for their use. 

As a tester you are often asked how far along is the testing effort and when will it be done.  This is one of the tougher questions to answer, and usually the most nerve racking.  Especially when the testing effort for a project is just starting up, or close to being finished.  A tool is needed to help gather information and effectively report on this item.  Problem is a lot of companies cannot afford a complex commercial tool due to financial reasons, or time constraints to implement the tool.  A solution is available using commercial spreadsheet products, specifically Microsoft Excel.  Using the logic and formula functions along with a combination of linked worksheets an easy to implement and use test status report tool can be built.  The architecture of the system (worksheets and formula’s) will be shown and discussed during the presentation. 

Biography: 

Earl Burba has over 20 years experience in software development and testing.  Earl is a Sr. Project Manager with SysTest Labs in Denver, Colorado.  Earl has worked in various positions within industry.  Earl’s experience includes flight systems, telephony, database, business and embedded operating systems.  Earl is an award winning author, and holds patents on software testing tools & methodologies. 

Jim Hazen has 15 years experience in software development and testing.  Jim is a Sr. Test Engineer with SysTest Labs in Denver, Colorado.  Jim is a Certified Software Test Engineer through the Quality Assurance Institute.  Jim has worked in various capacities as a test lead / project manager, automation developer, and department manager.  He has worked with a wide variety of applications on DOS, OS/2, Windows, and Web environments.

Back to Conference Schedule

Test Documentation 101 Using IEEE 829
Bernard Homes

Concepts: 

Test documentation comprises both the planning and reporting documentation.  It documents the steps that were taken to ensure quality in the software being tested.

The test plan is the base of all the future tests for an SW application.  It is a contract between the test team, the management and the development team.  It details the strategy that will be used for testing and will enable:

-          the management to understand the added values of the tests and the remaining risks, the costs and requirements as well as the time frame,

-          the development team to focus on the areas that are most critical or where a required level of quality has been set,

-          the test team to develop and execute the test campaigns in order to reach the goals on time and within budget,

-          the test project manager to have a reference document to keep the project on course and on budget. 

This presentation first describes the different types of documents used in the IEEE 829 standard and their relations to one another. Building on this information it details the different steps and aspects of the test plan as described in the standard and the relations between the different chapters, how they complement each other in order to provide a complete solution. From there it shows how to use the plan for different types of tests such as functional / regression testing, performance testing and configuration tests. After this the test strategy is furthermore refined thru the different documents described in the standard. The presentation ends with practical example of application of this standard to cover different aspects of complex application testing. This original presentation is based on real life experience and implementation of test strategy developed using this and different methods and their comparison. 

At the end of this presentation/tutorial the audience should be able to: 

• Understand the basics of the IEEE 829 standard for Test documentation,
• Build a basic but complete test documentation for their application based on the IEEE 829,
• Have pointers to enhance the coverage of their test plans and test documentation
• Point to the specific parts lacking in their documentation coverage
• Know how to adapt the test documentation to different types of tests such as performance tests or functional tests, whether in an internet or an industrial environment.

The method focuses on practical and pragmatic approaches to setting up the test documentation and is illustrated with examples.

Biography:

Bernard Homès is founder and principal consultant for TESSCO ltd (Tests Evaluation Service & Solution, Conseil & Organisation limited).  After 20+ years in software development (banking, ERP, ...) and testing with different consulting firms, he set up the TESSCO group in Europe (France and UK) and Canada.  After helping customers in setting up software test centres, providing trainings and consultancy services worldwide, Bernard currently manages the qualification processes for a major telecom company. A speaker at international conferences, Mr Homès is also Assistant Vice-President Membership & Awards of the French section of IEEE.

Back to Conference Schedule

Simplicity Through Complexity
Jamie Mitchell

Concepts: 

The history of the computer age is rife with examples of making tasks easier for users by simplifying the interface of the task through complex programming.  At one time, all Windows programs were written in the C programming language.  This was highly technical, limiting it to a handful of programmers.  Language designers at Microsoft understood that the scarcity of good C programmers was likely to limit the growth of the platform.  By clever and complex programming, the designers came up with a revolutionary new language called Visual Basic.  This easy-to-use development environment has, in many ways, ensured the explosive growth of Windows. Test automation needs to be simple so that testers, business analysts, and domain experts can create automated tests easily. By taking a lesson from the VB designers, this presentation is going to propose creating a new class of tools which work in conjunction with the R/P tools to simplify automation.  The tools required to simplify automation will be discussed. 

Presentation Outline: 

Creating a computer program to test a computer program is much like trying to hit a bullet with another bullet: the complexity can be daunting.  Ignoring the complexity ensures that the test case will fail often and cause many maintenance problems.  Most people handle the complexity at the scripter level, ensuring that those who create the test cases must know programming.  This, however, creates its own problems.  This presentation will discuss managing the complexity while presenting the test creator and maintainer with a simplified interface; perhaps as easy as record and playback. 

The presentation will discuss the issues which must be solved and the tools that will be required to solve them.  Each phase of the automation test cycle and the tools required to facilitate them will be discussed: from analysis of the application to be tested to execution and maintenance.In addition, parallels will be drawn to other current automation methodologies which attempt to solve the same problems, including the TestFrame™ and Table-Driven methods of automation.

Biography: 

Jamie L. Mitchell is a Principal QA Consultant at BenchmarkQA. in Minneapolis, MN.  He is a contributing editor and columnist for “The Journal of Software Test Professionals.” He has long been involved in test automation as automator, designer, architect, and mentor.   He has worked in test automation since the first automation tools were released in Windows 3.0. He earned the Master of Computer Science degree in 1992 from Lehigh University and is a QAI Certified Software Test Engineer. 

Back to Conference Schedule

Selecting Test Environments for Web Applications
Russell Jaslow & Tom Beha

Concepts: 

Testing a web application can be a daunting undertaking when you consider the seemingly infinite combinations of web browsers, operating systems, and hardware specifications.  Then, there are the plug-ins for flash, audio, video, and so on.  Nobody has the time or resources to test every possible situation even if you use automated testing.  So, what do you do?  You have to pick a subset for your testing, and this session will explain how to analyze your user base and understand the technology you are testing, in order to select the most optimal test environments for your web applications. 

Objectives: 

• An understanding of how to select test environments for web applications.
• Knowing how to go about making compromises of which test environments to test on and which not to test on.
• Learn where to collect data on your users.
• Understand all the issues with test environments for web applications.

Biography: 

Russell Jaslow is currently the Software Quality Assurance and Testing Manager for the kodak.com group at Eastman Kodak Company.  There, he has built a QA and Testing group from scratch that specifically deals with testing web applications of all sorts (content, B2C, B2B, imaging, etc.) using the latest technology in web development.  Russell has worked in the software industry for 20 years, 15 of those in the QA and testing field for a wide range of products including factory automation systems, data communications, IT applications, OCR, microfilmers, scanners, digital cinema products, and now web applications.  Russell has a B.A. in Computer Science from the State University of New York/College at Potsdam.  In recent years, Russell’s personal activities have also revolved around the web.   He runs two auto racing related web sites, DeepThrottle.com and AutoRacingHistory.com, and also is a regular correspondent for US College Hockey Online (USCHO.com). 

Tom Beha was one of the first members in the kodak.com Software Quality Assurance and Testing Group as a Test Lead.  Tom tested various web imaging applications that Kodak developed for in-house use as well as for other web sites.  Tom was heavily involved in the redesign and update of the kodak.com test lab.  Previously, Tom worked as a support technician, database administrator, project lead, and lead test engineer on document storage and retrieval systems, digital cinema products, and medical software applications for IEC Electronics, Metscan, and Real Time Enterprises.  Tom has a B.T. in Electrical Engineering Technology from the Rochester Institute of Technology.

Back to Conference Schedule

PSQT Track Presentations (Wednesday 1:00 - 2:00 PM)

Use of Software Requirements Traceability Matrix in Function Point Estimation - A Practical Approach
Krishnamoorthy Gurusamy

Concepts:  

• Problems surfaced when maintaining Software Requirements Traceability Matrix and Function Point estimation separately
• Having one common place for both Requirements Traceability Matrix and Function Point Estimation worksheet
• Different solutions for combining these two aspects of project management and selecting the best option depends on the size of the project
• Extending the use of this Spreadsheet for Change impact analysis and new work product estimation in the post production scenario
• Advantages of having one common source – Only one version of requirement is maintained for tracking and estimation in the project, Estimation errors were reduced and time required for estimation is minimized. 

Learning Objectives: 

·         Complications involved when maintaining two versions of the same information for different purposes

·         Simplified process with less chances of estimation errors

·         Time saved when requirements traceability process and Function Points estimation process are combined

·         Deliverables of this new process

·         Selecting the solution from available options depends on the size of the project

Biography: 

Krishnamoorthy Gurusamy's areas of experience: Involved in all phases of Software development life cycle in the roles of Quality Assurance Facilitator, Testing Coordinator, Project Coordinator, Software Metrics Analyst and Software Engineer. I was part of ISO and CMM implementation team.

Presently working as a Consultant in Software Testing Services area for a client. Responsibilities include Test Planning, Test Coordination and Test Reporting. The extensive experience in the areas of Coordination and Metrics analysis helped me in identifying and removing the problems faced during test reporting. I always feel that simplified procedures and simple tools save a lot of time than using advanced tools with extensive procedures. This strong conviction from my experience forms the basis for presenting this paper.

Back to Conference Schedule

Early Lifecycle Identification of Software Quality Risk Factors
David Herron

Concepts: 

The capability to deliver software is based upon a variety of risk factors that influence a development organization’s ability to deliver software in a timely and economical fashion.  Success depends upon the developer’s ability to identify those risk factors early in the life cycle.  A quality risk factor assessment is a technique that can be readily applied and can produce results that benefit both the short terms needs of the project manager and long term process improvement needs of the organization. 

This presentation teaches practical and usable software quality risk management methods and techniques that software managers can readily adapt to their particular project environments.  The techniques learned include: early and effective evaluation of software quality risks; quantitative impact analysis of specific risk factors on quality; and the ability to effect change and improve the management of project resources and deliverables.  This interactive session is not a theoretical presentation on software quality risk management; instead it deals directly with the hands-on how to of managing software quality risks. 

Learning Objectives: 

• How to quantify the impact of specific software quality risk factors on productivity , performance and quality levels
• What organizations are doing to measure and improve project deliverables
• How to identify and measure the impact of current barriers to successful software delivery
• How to apply statistically proven algorithms to reduce project costs and shorten time to market

Biography: 

David Herron has over 25 years of experience in software development.  During the last ten years he has served as a consultant to Fortune 500 companies in the areas of software metrics and software risk management.  David has addressed audiences throughout the U.S. and Europe on functional measures and software estimating.  He is a Certified Function Point Specialist and serves on the Industry Benchmark and Management Reporting Committees of the International Function Point Users Group.  David is a Principal, with David Garmus, in the David Consulting Group, Inc. Together, they authored Measuring the Software Process: A Practical Guide To Functional Measurement, Prentice Hall, 1996 and Function Point Analysis, Measurement Practices for Successful Software Projects, Addison-Wesley, 2000.

Back to Conference Schedule

Successfully Managing Quality Change
Thomas C. Staab

Concepts: 

Management has committed to improving the quality process in the organization.  Now the challenge becomes, how to effectively implement and manage the quality change.  The first step is for management to emphatically demonstrate their commitment to quality improvement.   Management must also decide how to: manage the process, set realistic quality standards, overcome resistance to change, secure company-wide buy-in, and sustain the momentum.  This informative presentation will explore all of these issues and more.  Individuals attending this session will become knowledgeable in techniques that can be implemented immediately upon return. 

Objectives Include: 

1.      Explore the options for quality improvement design and implementation.

2.      Expand your knowledge of how to establish realistic quality standards and expectations.

3.      Learn techniques for management to demonstrate commitment to quality change.

4.      Become knowledgeable in ways to manage the process.

5.      Gain knowledge in how to conquer resistance to change. 

Biography: 

Thomas C. Staab owns an independent consulting firm, Wind Ridge International.  He has over 35 years experience.  Prior to opening his own consulting firm he worked for over 25 years in the quality assurance profession. He holds a Master of Science degree in Quality Systems. His consulting work incorporates his extensive quality assurance and information technology experience into every project.  He has developed the test plan and coordinated the testing of numerous systems for clients.  His expertise is in bringing this practical experience into the classroom.  

Mr. Staab is listed in the International Who’s Who of Information Technology.   He has currently published over 25 articles (one of which earned him Author of the Year Award from the Comptroller magazine) and presented over 50 speeches at regional, national and world conferences. He has developed and taught numerous training courses during his career and has always received excellent evaluations for his training courses and speeches. 

Mr. Staab holds a Bachelor of Arts degree from North Texas State University and a Master of Science degree from the University of Dallas

Back to Conference Schedule

Establishing Effective Measurements in Support of the SEI CMM
Koni Thompson

Concepts: 

Establishing effective measurements in support of the SEI-CMM is often a challenge to organizations. While measurements are required for every Key Process Area, the individual implementation and interpretation is left up to the organization.  There are many choices that must be made, and those choices can lead  to successful or unsuccessful results.  This presentation will help position an organization to be successful in their measurement program.  Based on case-studies and real-life experiences, successful recommendations will be provided that can ensure the measurements are compliant with SEI Levels 2 and 3 requirements.  In addition, since many organizations are just learning about CMMI, the presentation will provide a brief comparison on the CMM and CMMI measurement requirements, so that organizations can begin successful transition planning to that quality model.      

Learning Objectives: 

·                     To present recommendations for establishing an effective measurement program             compliant with SEI CMM Level 3 Key Process Areas (KPAs)

            Topics include:

¬    Steps to developing an effective and practical measurement program

¬    An overview of the SEI CMM measurement requirements for the Level 2 and 3 KPAs

¬    A recommended set of metrics which comply with the SEI requirements and promote process improvement

·                     To compare CMM and CMMI measurement requirements 

Biography: 

Koni Thompson is an international quality consultant with over 20 years experience in software development. Ms. Thompson's focus includes software measurement based on function point analysis (specializing in new environments, such as real-time systems), process assessments and process improvements, project estimating, benchmarking and outsourcing-based metrics, and software quality initiatives. She has worked with a number of Fortune 500 firms in the United States, Canada, and Europe over the past seven years, advancing SEI-CMM initiatives, process improvements, software measurements, and outsourced-based metrics programs.

Back to Conference Schedule

PSTT Featured Presentation (Wednesday 1:00 - 2:00 PM)

Team up with the Project Manager to Improve Quality and Testing
Dr. Rebecca Staton-Reinstein

Concepts:

Biography:

Back to Conference Schedule

PSQT Featured Presentation (Wednesday 2:15 - 3:15 PM)

Model-Based Requirements
Dr. Magdy Hanna

Concepts:

Biography:

Back to Conference Schedule

PSTT Track Presentations (Wednesday 2:15 - 3:15 PM)

The IEEE 829 Standard as a Support to the Management of the Testing Process
Adalberto Nociato Crespo

Concepts:

We describe an experience where the IEEE 829 Standard for Software Test Documentation was used in the acceptance testing of a software system contracted by a Brazilian public company, whose business is to provide services of waste treatment and disposal and water supply and distribution.   A software house  was contracted to implement a new integrated system to replace many independent systems, aiming at improving the quality of the services provided.

The Research Center “Renato Archer” – CenPRA was hired to carry out technical auditing to evaluate the actual operational conditions of the new system.  Auditing was done through software tests focusing on the billing functions, the most critical area of the company.  

An overview of the Standard is presented.   The proposal on how to use the Standard, focusing on the relationship among the documents is discussed.  Examples of problems concerning the operation of the new system are presented.  Finally, examples and results of our application are presented and discussed.

Learning Objectives:

·           Real application of the IEEE 829 Standard

·           Information necessary for creation of the documents of the IEEE 829 Standard

·           The IEEE 829 Standard supporting the management of the test process

·           Failures classification according to their impact on the system

·           Analysis of system failures and misuse of the system

Biography:

Adalberto Nobiato Crespo has a Doctor degree in Electrical Engineering from UNICAMP (State University of Campinas - 1997) and a Master degree in Engineering from UFRJ (Rio de Janeiro Federal University - 1980).  Software reliability is the subject of his doctorate dissertation, in which he proposes new software reliability models based on code coverage. Since 1998 he has worked at CenPRA, in the Software Test Group, which aims the development of a methodology for deployment of software testing processes in software organizations. (The documents proposed in the IEEE 829 Standard have been adopted as a basis of the methodology).  Dr. Adalberto is a co-author of papers presented in conferences on software engineering and software reliability (SBES - Brazilian Symposium on Software Engineering, ISSAT (International Society of Science and Applied Technologies) International Conference, ISSRE - International Symposium on Software Reliability Engineering.

Back to Conference Schedule

Managing the Test Effort Using Requriements-Based Testing Metrics
Gary Mogyorodi

Concepts: 

In many organizations, there is difficulty quantifying the true state of the test effort.   Often testing is measured by getting as much done as possible by an arbitrary deadline.  The true level of quality is never really known, and software is released to the next phase of development with portions of the code going untested.  This dramatically increases the risk of software failure. Ironically, when testing is properly deployed, with heavy emphasis on Requirements-Based Testing (RBT), it can have a major impact on overall project productivity as well as product quality.  

Presentation Outline: 

The Requirements-Based Testing Methodology provides a set of metrics throughout the software development cycle.  These metrics clearly provide the true state of the test effort at any point in time.  This presentation describes the RBT process and details the derivation of each RBT metric and its impact on the software development process.  This presentation addresses how the RBT process reduces the risk of delivering untested code, and provides project management with quantitative data on the test effort throughout the software development lifecycle.  The combined results are fewer tests with greater functional coverage, shortened time to delivery, reduced costs of development, and significantly improved quality.  

Biography: 

Gary Mogyorodi has over 28 years of experience in the computing industry. Currently as a Senior Consultant with Bloodworth Integrated Technology (BIT), Inc., Mr. Mogyorodi consults, trains and mentors in software testing, specializing in test case design. Mr. Mogyorodi began working for Bender & Associates in 1998, which merged with TBI in 1999, and was then acquired by Starbase Corp. in 2001. Prior to his tenure with Bender & Associates, the majority of Mr. Mogyorodi’s career was with Dofasco Inc.  From 1992, Mr. Mogyorodi was a Quality Assurance and Software Testing specialist, managing testing efforts, developing testing methodologies, and creating standards and procedures for quality assurance and testing. Prior to that, he worked at Dofasco as a Programmer, Systems Analyst and Manager of Software Development.