Tuesday Conference Sessions (8:30 AM - 6:00 PM)

KEYNOTE SPEAKER (Tuesday 8:30 - 9:45AM)

Beyond the Death of Capture/Playback Edward Kit

Concepts:

It's no secret that using capture / playback as the sole automation technology leads to failure.  Why could the first two generations of automation tooling not stand the test of time and what should we look for in 3rd Generation Automated Test Design and Automation Systems. Since Capture / playback as we know it is dying, Ed Kit presents an Automated Software Quality industry update and a vision of how we can create truly maintainable, effective test automation systems with few technical resources.

Biography:

Edward Kit, founder and president of Software Development Technologies, is well known as a test expert, author, and keynote speaker at testing conferences. His best-selling book, Software Testing in the Real World: Improving the Process, has been adopted as a standard by companies around the world such as Sun Microsystems, Exxon, Chase Manhattan Bank and Cisco Systems.  His feature articles in Software Development Magazine have outlined new state-of-the-practice test automation models that are currently being adopted around the world.  Mr. Kit continues to advise clients on bringing practical and proven software quality practices to their development efforts.  Ed holds a BSEE and MSEE from Purdue University.

Back to Conference Schedule

PSQT Featured Presentation (Tuesday 10:00 - 11:00 a.m.)

Project Manager Credibility
Robin Goldsmith JD

Concepts:

With up to 84 percent of projects late or over budget, and those that do finish delivering only half of required functionality, people may not believe project managers.  In this interactive session, Robin Goldsmith explains how lack of credibility may be limiting our effectiveness and how lack of awareness limits it further.  Discover ways to avoid common credibility-damaging traps which conventional wisdom often leads us into with

the best of intentions.  Learn how projects succeed or fail in the first 15 minutes and the critical factors you need for the successes that earn credibility. 

    *   How lack of credibility can limit our effectiveness. 

    *   Self-defeating traps when we're unaware credibility is an issue. 

    *   Projects succeed or fail in the first 15 minutes; how to choose success.

Biography: 

Robin Goldsmith JD is internationally recognized as an authority on business engineering and software acquisition/development quality, testing, and productivity. He is a frequent speaker at leading conferences and formerly International Vice President of the Association for Systems Management.

Back to Conference Schedule

PSTT Track Presentations (Tuesday 10:00 - 11:00 a.m.)

A Strategy for Designing and Executing an Effective Regression Test Program
Martin Gilleran

Concepts: 

The concepts covered include:

·        Regression Test – what is it, a common definition of regression testing, typical problems with regression testing.

·        Escaped Defect Analysis (EDA) - analysis of defects that escape to our customers.  The analysis is performed by our test team and seeks to understand why the problem escaped and why we did not detect it in our test phase.  Recommendations are usually made to update our process/procedures or make modifications to existing test suites to add coverage.

·        Root Cause Analysis (RCA) - analysis of all defects escaping the development phase.  Performed by the development organization and seeks to understand why the problem was introduced in the first place and also why it was not discovered during the development phase. Recommendations are usually made to update our process/procedures.

·        Problem Arrival Rates - what a theoretical curve should look like, what our curves actually looked like and why.

·        Applying the idea of requirements-based testing to regression testing without actually having specific product requirements to use.

·        Building flexibility in the regression program allowing it to adapt to product changes. 

Presentation Outline: 

Section 1 - Introduction

Section 2 - Understanding Our Problem

Section 3 – A Time For Change

Section 4 - The Regression Strategy

Section 5 – The Benefits

Section 6 – Issues

Section 7 - Summary & Conclusion 

Biography: 

Martin Gilleran graduated with a B.Sc. in Electrical/Electronic Engineering from Trinity College, Dublin and has spent the last 13 years working in industry.   Martin has worked with Measurex Inc. as a software development engineer responsible for developing, testing and installing measurement and control systems for the paper industry.   He has also worked with Motorola in different test roles, beginning as a test engineer responsible for new feature and regression testing on software release for Motorola's range of Mobile Switching Centres with additional experience working as a test project manager responsible for planning & tracking of the entire test phase for Motorola's software releases, coordinating test activities and as a Test Manager responsible for testing a range of Motorola's Network Products including Mobile Switching Centre (MSC) and Home Location Register/Visitor Location Register.  Martin is currently working as a Software Development Manager with Motorola.

Back to Conference Schedule

A Practical Worst-Case Software Test Methodology
Daniela Bright

Learning Objectives: 

·         Despite ISO and SEI CMM there are still worst-case test conditions found

·         Successful testing can be performed even under worst conditions

·         Focus of testing is finding defects, not demonstrating success

·         Checklist is versatile and may be only test tool applied 

Presentation Outline: 

Introduction

            Background

            Worst-case conditions

            Worst-case methodology

The Free-form Checklist

            Checklist history

            Checklist use

            Checklist details

Validation

            Application of worst-case methodology with checklist

Conclusion

            Valuable Tool

            Scaleable

            Scope

            Integration environments 

Biography: 

Daniela Bright was born and raised in Germany and immigrated to the United States.  She is the technical lead for the Integrated Space Command and Control Contract Enterprise Test Engineering Integrated Product Team (IPT) with Lockheed Martin Mission Systems.  She has worked as a test engineer with Loral/Lockheed Martin and CSC and has performed all aspects of software testing (planning, design, execution, reporting, development testing, maintenance testing) for a variety of systems from mainframe applications to worldwide deployable portable applications and distributed client/server networks, including proposal efforts and requirements engineering.  Ms. Bright earned a Masters degree in Computer Science with the thesis topic:  “An Approach to Automate Maintenance Testing” and a Doctorate degree in Computer Science with the dissertation topic:  “A Practical Worst-Case Methodology for Software Testing”.  The focus of the presentation, the free-form checklist, has been successfully used by Ms. Bright for over five years on five different projects and has been refined over the years to better accommodate leading edge technology.  The checklist was successfully employed proving Ms. Bright’s doctoral dissertation hypothesis and is now implemented in the ISC2 test engineering process. 

Back to Conference Schedule

Automation: Moving from Scripts to Tables
David Peters

Concepts: 

This presentation is meant as a case study of automation. It will describe how we moved from writing specific test case scripts to writing table-driven test cases supported by generic scripts. It will describe the success and the failures we encountered while automating test cases during the past four years.  

Presentation Outline: 

I.                    Automation History

II.                 Task At Hand

III.              Pilot Program

IV.               Implementation 

V.                  Current Status  

Biography: 

As the manager of Software Simulation, David Peters is responsible for developing tools to assist in the development and testing of our products.  Our products are medical devices that are subject to strict guidelines set forth by the U.S. Food and Drug Administration (FDA). While our products have hardware and software components, his group is primarily concerned with development and testing of software components with 4 years of experience automating test cases.  David was asked to find a way to increase our level of test automation. In fact, in order to meet their planned delivery dates, assumptions were already made that we would increase our level of automated testing by a factor of three within the course of a year. We needed to continue to support the existing test cases and automate an additional 600 test cases in one year without adding additional resources.  When I got involved, I researched several tools and techniques for automation. Following the research, I proposed a new table-driven method for developing and executing test cases. The table-driven approach presented several key points important to our environment—

• The table-driven method didn’t require everyone writing test cases to know how to write scripts
• The table-driven method enabled us to maintain generic scripts that were more easily maintained than script-specific test cases 

Shortly after the proposing this method, I managed a pilot to explore the method. The pilot lasted approximately two months. As a result of the pilot, we implemented the approach. We have been using this approach for approximately 6 months with tremendous success. In fact, we are expanding the method to include additional areas of testing, such as performance and integration testing.

 Back to Conference Schedule

Realistic Web Application Load Testing
Hugh Smith

Concepts: 

The necessity and value of automated load testing has become accepted within the IT industry.  There are a number of commercial tools available to assist with the load testing process.  Most of the tool vendors will provide classes on how to use their software products.   However, it is not an easy task to perform load tests that realistically represent the performance and capacity of production systems.  There are a number of important issues that must be considered to avoid the many pitfalls when constructing load tests for web applications.     A review of a set of load tests gone wrong provides some practical examples of how to build realistic load tests.  

Presentation Outline: 

A.  Introduction

B. Different types of tests

C. System Configuration Issues

D. Closed Models vs. Open Models

E.  How to choose the correct metrics

F. Workload Characterization

G. Load Test Design Issues

H. Understanding Bottlenecks

 I. Summary  

Biography: 

Hugh Smith is the founder and President of B.E.C. Consultants, Inc.   During more than 20 years as a consultant he has worked in the areas of Systems Programming, DBA, Performance Analysis and Capacity Planning.  He has been a speaker/presenter at conferences in the U.S, Europe and South America for the Computer Measurement Group and the Operations Research Society of America.  He has a BS degree in Industrial Engineering and Operations Research from Virginia Polytechnic Institute.

Back to Conference Schedule

PSQT Track Presentations (Tuesday 1:00 - 2:00 PM)

Managing at Internet Speed: Improving Time to Market Without Sacrificing Quality
Gregory Scott

Concepts: 

The presentation addresses the challenges of planning and executing project development using concurrent engineering.  Because concurrent engineering development phases overlap, there are no clear cut starting points for each phase making it very difficult to predict when a phase will be completed. Thus, accurate estimation of a project complete date is virtually impossible.  The start and end date ambiguity presents a challenge for functional management providing resources for multiple program. 

This paper presents a risk management strategy that minimizes the scheduling ambiguity enabling program management to predict the amount of overlap of the development phases and critical project dates. 

Presentation Outline: 

A.                              Assumption: The presentation length is 45 minutes including Q&A.

B.                              Overview of linear development models concentrating on the Waterfall Model

C.                              Overview of Concurrent Engineering

D.                              Detailed look at Development Phases

E.                               Another looks at Concurrent Engineer 

Biography: 

Gregory Scott has over 30 years of experience in software engineering and software engineering management.  For the last fourteen years Mr. Scott has developed and implemented management processes that ensure high quality while controlling development costs and improving time to market. During that time he has found Software Engineering Process Groups in four organizations and managed software development at the Initial, Repeatable and Defined levels of process maturity.   Mr. Scott currently supports approximately three 300 engineers using current engineering methodologies to improve time to market.   He chairs the group-level process improvement committee working with other quality professionals and program managers to identify and implement value-added “Internet Speed” multi-discipline concurrent engineering processes.  As a lead internal ISO 9001 assessor, Mr. Scott, is charged with monitoring compliance to development processes in other parts of the company.  This presentation is a result of numerous assessment non-conformance's and efforts to prevent non-conformance's in the group he supports.

Back to Conference Schedule

Issue and Risk Management - A Practitioner's Guide
Dr. Iqbal "Bob" Noor

Concepts: 

A number of key concepts relating to Issue and Risk Management are covered in the paper.  One key concept is that there is an evolution of issues into risks into problems that can be tracked according to the integrated project schedule. Another key concept is that Issue and Risk Management is not a one-time effort but is required continuously throughout the life of a project. These two concepts together with others will be explored in detail in the paper and accompanying presentation. 

Presentation Outline: 

This paper will focus on some of the challenges encountered in establishing a risk and issue management project culture and the steps that could be taken to overcome these challenges. A pragmatic approach using the standards established by the Software Engineering Institute combined with industry experience will be presented. Where applicable, examples will be provided to illustrate the concepts that are discussed. Details will be provided on the subtleties involved in ranking risks and in performing quantitative risk analysis. 

In addition, recommended features of mitigation plans including the use of threshold values and return on investment metrics will be highlighted. Tools that are required to support the risk and issue management processes including the supporting training initiatives will be discussed. The roles and responsibilities of team members in supporting the risk and issue management processes will be outlined.  

Biography: 

Dr. Iqbal “Bob” Noor is employed with PMOLink and is currently providing Program Management consultancy services for the Entergy Corporation. His Program Management experience spans engineering, construction and information systems projects.  His previous assignments include roles as the Risk Manager at the US Department of Defense, Space and Naval Warfare Information Technology Center, consultant for the Worldwide Engineering and Construction Group of AMOCO and project management consulting for public works and infrastructure projects.   He holds a B.Sc. (Honors) Degree in Civil Engineering from the University of the West Indies. He was a Canadian Commonwealth Scholar at McGill University (Canada) where he obtained his M.Eng Degree in Civil Engineering.  He received a Science and Engineering Research Council award, an Overseas Research Studentship and a Bursary at the University of Dundee (Scotland) where he obtained his Ph.D. Degree in Civil Engineering. The British Council awarded him a traveling fellowship to the University of Zagreb, Croatia. 

Dr. Noor is a licensed Professional Engineer in the States of Florida and Texas. He is a founding member of the Greater Miami Section of the Association for the Advancement of Cost Engineering (AACE) International and serves on the national committee for the review of the AACE International Certified Cost Engineer (CCE) certification examinations.  He is a member of both AACE International and the Project Management Institute (PMI) and serves as a reviewer of articles for the AACE International Cost Engineering magazine. He is a Certified Cost Engineer (CCE) with AACE International and a Project Management Professional (PMP) with PMI. He has published over 20 papers at national and international conferences. Dr. Noor is an Adjunct Professor in the Management Department at the University of New Orleans.

Back to Conference Schedule

How Fast is Fast? Using Non-Functional Requirements to Test Requirement Quality
Joseph D. Schulz

Concepts: 

This presentation discusses the difference between functional and non-functional requirements, defining each as they pertain to application software development. Further, it outlines a basic approach for using non-functional requirements to improve software quality. Suggested testing methods will also be discussed with an eye toward building validation plans using non-functional requirements for quantitative measurement. 

By attending this presentation, you should gain additional insight into the following areas:

·        The differences between functional and non-functional requirements

·        The importance of non-functional requirements during system analysis

·        The major categories of non-functional requirements

·        The primary characteristics of each non-functional requirement type

·        Suggested methods for integrating non-functional requirements into software testing practices 

Presentation Outline: 

I.  Overview

II.  The “Magic Five”

III.  Application Verification

IV. Non-Functional Categories

A.   Security Requirements

B.   Topographical Requirements

C.   Timing Requirements

D.   Frequency Requirements

E.   Performance Requirements

F.   Scalability Requirements

G.   Usability Requirements

V.  Requirements Traceability

VI.   Summary    

Biography: 

Joseph D. Schulz is a seasoned developer with more than 19 years of professional Information Systems experience spanning a wide range of vertical industries. Currently, Joe is the Senior Product Champion at Starbase Corporation.  In this position, he helps organizations around the globe improve their development processes by delivering methods/tool training and providing onsite mentoring.   He also supports Starbase's international distributors, ensuring their ability to support the Starbase family of products.   Joe has been actively involved in methodologies and development techniques since 1986. He has an extensive background in "non traditional" development that includes more than forty procedural, 4GL, CASE, and OOAD/OOP products on a wide variety of operating platforms.  A prolific speaker, Joe addresses a broad range of software quality topics at conferences in the United States and abroad. Previous speaking engagements have included the Practical Software Quality Techniques Conference (PSQT), the Pacific Northwest Software Quality Conference (PNSQC), the European Software Process Improvement Conference (EuroSPI), and the United Kingdom Software Metrics Association Conference (UKSMA), to name just a few.  Joe earned a Bachelor of Science in Mathematics at the University of Notre Dame and a Bachelor of Science in Computer Science from Saint Louis University.   He has also received numerous professional certifications in Information System methods, tools, and techniques as well as several accreditations for public speaking and technical instruction. 

Back to Conference Schedule

Defect Prevention Framework for Success
Neda L. Gutowski

Concepts:

This presentation defines a framework for organizations to use in building a comprehensive organization defect prevention program.  This comprehensive program is described so that an organization can understand their current defect prevention level. The organization would then chart a transitional plan, a road map, to evolve the existing program.   Any defect prevention improvement path or road map must be based on the current organization’s maturity level and current culture to increase the probability of success.

Learning Objectives:

-                      How to apply Defect prevention to various levels of maturity in organizations

-                      DP is not only a Level 5 practice

-                      DP should be integrated into the organizations current lifecycle of development and test

-                      These are various proven best practices to do defect prevention.

Biography:

Neda L. Gutowski is a 13-year Motorolan.   Graduated with a Bachelor’s Degree in Electrical Engineering from Marquette University, Milwaukee, WI.  For 4 years, held the position as Software Quality Engineering and Assurance Manager of the GSM Base Station development team.  Previously a Software Engineer in the same organization working on software development and new technology prototyping.  Published and presented papers at three Motorola Software Engineering Symposiums as well as customer conferences in the areas of Software Quality Audits, Level 5 topics, Knowledge Based Systems and Quantitative Process Management.     Published and presented at the 8^th International Conference on Software Quality in the area of Software audits and Continuous Improvement, and at the PSQT ’99 South.  Member of the Software Quality Council at Motorola.

Barbara Hirsh is the Metrics Manager and Analyst for Motorola's GSM System Division. She has 18 years of experience in the areas of software development, system testing, quality, process improvement and metrics.  She was instrumental  in  her organizational climbing from  level 1 to SEI SW CMM Level 5. The GSM BSS organization was the first Motorola organization to reach SEI SW CMM Level 5 within Motorola.  Barbara has been involved in metrics development at the corporate level at Motorola.  Barbara received the Motorola Best Practice award in 1994.  She has presented at numerous conferences including being a panelist at SEPG and ESEPG on Mark Paulk’s panel on SPC usage.

Back to Conference Schedule

PSTT Featured Presentation (Tuesday 1:00 - 2:00 p.m.)

How to Break Software
Dr. James Whittaker

Concepts:

How To Break Software is a collection of the most useful and effective techniques to quickly and effectively uncover software defects. The techniques are organized as "attacks" against software that target software’s weak points. The attacks walk testers through everything they

need to consider when designing and executing test cases. Attendees will learn the attacks and see them in action against real software. So be prepared to see some great bug demos and some cool tools in action.

Biography: 

James A. Whittaker is an associate professor of computer science and founder of the software testing program at Florida Tech. His research group focuses on the technical aspects of software testing and computer security. They build testing tools, perform basic research and also practice testing through contracts with industry and government.

Back to Conference Schedule

PSQT Featured Presentation (Tuesday 2:15 - 3:15 PM)

Ten Guaranteed Ways to Fail at Improving Software Quality and Ten Better Approaches
Tom Gilb

Concepts:

We are pretty famous for not delivering the software quality expected by our users and customers. Maybe it is time to systematically define exactly why we fail? Of course we also need to define some better ways to operate which have really resulted in delivered software quality. Key ideas are making all quality measurable, making it external at the user and stakeholder level, and delivering it evolutionarily (early, frequently and gradually)  - proving you know how to do it at all.

Biography: 

Tom Gilb is a freelance consultant, teacher and author serving clients in Europe and the US. He has written "Principles of Software Engineering Management" and is Principal author of "Software Inspection". He  specializes in software quality design and management. He lives in Norway, when he is not traveling. His work is available on  www.result-planning.com. He has taught and consulted with Medtronic, Cray Research, United Defense and other companies in Minneapolis. With Ericsson and Nokia in Dallas. Motorola in Austin. His methods are also in use at Hewlett Packard, Microsoft, Ericsson, Nokia, Nortel, DOD and many other companies.   

Back to Conference Schedule

PSTT Track Presentations (Tuesday 2:15 - 3:15 PM)

Software Test Process Assessments at Intel Corporation
Robert Topolski

Concepts: 

Software testing has been the earliest recognizable software quality activity since the early days of formal software development activity. Today’s fast cycles, distributed architectures, and development productivity tools make the need to improve our software test methods greater than ever before.    Why do most modern software process assessments literally ignore testing?  What process assessments are available?  How do you conduct an assessment?  How do you guard against incorrect information? 

But how do you know what to improve first?  How do you know your strengths from your areas requiring development?  And how can you make successful improvements without negatively impacting your work?   To answer all of these concerns, the software testing process can and should be assessed, base lined, and improved.  

Here is Intel’s experience.   At least two publicly available models will be shown (currently TPI and CMMI L3 KPA).  The method of a Test Process Assessment will be described, as used by Intel (which uses TPI).  The results of at least two assessments will be shown (scores, improvement suggestions, and adopted actions). 

Presentation Outline: 

I.                     Software Testing needs to improve

II.                  Software Process Assessments/Testing Assessments

III.                Intel’s Experience 

Biography: 

Robert Topolski has been a software test professional since 1993.  He has led numerous software test and SQA project teams within Intel.  He currently serves as an internal software quality methods consultant and leads the corporation’s Test Process Improvement efforts.

Back to Conference Schedule

Extreme Acceptance Testing
Jim Canter

Concepts: 

This presentation addresses techniques that can be used by the quality manager to raise critical questions to customers and engineers early on.   Posing such questions can demonstrate QA’s ability to identify very serious problems early on.  QA professionals can use this “juice” to push for low-overhead process that empowers development organizations to shorten time-to-market.  The same technique can dramatically shorten test design time, while generating confidence that tests will approach 100% functional coverage.  It’s been proven that this practice is far more effective than adding people to the test problem. 

Secondly, once effective tests have been defined, there are techniques that can allow the quality manager to add new automated tests to existing ones, BEFORE the software is ready for test.

Employment of these techniques and practices has a dramatic affect on interdisciplinary teamwork, cost avoidance, time-to-market, and customer satisfaction through quality 

Presentation Outline: 

I. Why quality managers find themselves “behind the curve”

A. Test environment inoperable or not similar to production environment

B. Immature deployment practices

C. No requirements or lack of access to requirements

D. Lack of access to customers and engineers

E. Participation begins after code delivery

F. Intuitive test design precludes near 100% functional coverage

G. Project slippage compresses QA, leaving insufficient time for comprehensive testing

H. Automation fails, requiring manual testing

I. Comprehensive manual testing, bringing in temps to complete tests with each code drop busts the budget

II. Where a manager can start – Initial Objectives

A. Put real test automation in place to free resources by automating what you have, eliminating manual test efforts as much as possible.  Tools selected must deliver the following capability.

B. Train test analysts in a scientific requirements analysis discipline, specifically Cause-Effect graphing of requirements.

III. Test Automation Truths

A. Record-Playback strategy is probably the biggest generator of “shelfware.” Why doesn’t it work?

B. So we’ll hire a professional script technologist, won’t that help?  Yes! But…

C. Tools exist in the marketplace that “partner” with the most popular test execution engines that help technologists and business analysts

IV. So how does this compress time-to-market, save money, and deliver low defect rates?

A. Using test design techniques that approach 100% functional coverage at the beginning of the life-cycle

B. Integrating new QA tests into the regression suite before code is delivered

Biography:

Jim Canter Graduated from the U.S. Naval Academy with a BS degree in Analytical Management (a math and computer oriented business course).  As a research project, Jim captured and articulated requirements for automation of sports administration at the Naval Academy.  His findings were used in automation of that system three years later.  He served as a Supply Corps officer for the balance of his career in the Naval Reserve, specializing in logistics, procurement, and associated software systems. During Desert Storm, he formed a development team that eliminated a $70 million billing overage within the Military Sealift Command.  Thereafter, his team developed the supply depot software.  The Navy Reserve’s Advanced Supply Depots depended upon the application during relief operations subsequent to hurricanes Hugo and Andrew.  Early in his civilian systems career, Jim managed over 36 major software and hardware development projects both as VP of Engineering, and IS Director in two startup companies.  About seven years ago, he transitioned into the Quality Assurance arena, pioneering advanced QA practices for Wells Fargo Bank’s CORBA application.  The CORBA application became the largest in the world, supporting over 4 million transactions/hour for Wells Fargo’s Internet Banking, Telephone Banking, and Online Customer Service applications.  Subsequent to establishing quality practices, this development group released defect free into production environments.

Thereafter, he served as QA Director at Egreetings.com, a top 50 website.   At Egreetings, Jim inaugurated the QA department, Requirements Group, and the Project Office.  He established innovative quality practices that allowed Egreetings to shorten time-to-market while accomplishing a near-zero defect rate.  In all, his efforts contributed to over $1 million/year in cost avoidance through efficient process.  As VP of Professional Services at SpiralTest, Inc., he’s been developing client capabilities to achieve the same quality, time-to-market, and cost avoidance capabilities that Egreetings enjoyed. 

Back to Conference Schedule

Making Automation Tools Work
Jamie L. Mitchell

Concepts: 

The presentation will start with a discussion of the limits of automation and why we need to plan, architect, and design our automation program with care.  We then will look at a number of models stressing how the environment, the system, and the tool actually work.  Then, using the models, we will discuss pragmatic decisions needed to create a successful automation suite.   The discussion will not center on any single tool; instead the general features of all of the tools will be featured. 

Learning Objectives: 

·         Understand the limits of automation

·         Models: how to understand what really happens in testing

o       R/P tool model

o       GUI environment model

o       Manual Tester model

·         Leveraging the people you have

·         Modeling the system under test

·         Constructing the system 

Biography: 

Jamie L. Mitchell is a Principal QA Consultant at BenchmarkQA. in Minneapolis, MN.  He is a contributing editor and columnist for “The Journal of Software Test Professionals.”  He previously was a Senior Consultant at CornerStone Consulting, and the Lead Automation Engineer for Distributed Integration Testing / Global for American Express Technologies Organization.  He has long been involved in test automation as automator, designer, architect, and mentor.  He has worked in test automation since the first automation tools were released in Windows 3.0, including stints with Prudential Insurance, IBM AS/400 division, and ShowCase Corporation.  He earned the Master of Computer Science degree in 1992 from Lehigh University and is a QAI Certified Software Test Engineer.  He resides in Farmington, MN, and is an active member of the Twin Cities Quality Assurance Association.  

Back to Conference Schedule

Testing Web Services for Scalability and Performance
Frank Cohen

Concepts: 

Building SOAP-based Web Services is unlike any other kind of software development. How can you ensure your Web Service is quickly responding with in an Internet or Intranet environment? Frank Cohen will outline a conceptual framework for building Web Services using the new SOAP protocols, show how to build a variety of intelligent test agents modeled from archetypal users, and introduces a free open-source set of tools and a scripting language called Load to help with your testing. 

Presentation Outline: 

1) A framework for building scaleable Web services

-          Real World Test Paradigms and Lifecycle Processes

-          Types of Testing (Click-Stream, User-Goal, Unit, System, Scalability, Performance)

-          Defining the Criteria for Good Web Performance

-          Methods for Measuring a Web Application

-          A Web Application Framework (Flapjacks)

2) Building intelligent test agents

-          The Outline of an Intelligent Test Agent

-          Modeling User Archetypes

-          Defining Test Steps into Modules

-          Concurrent Test Agent Issues

3) Test scenarios

-          Testing a Web-based Apply-for-a-job form

-          Testing Microsoft Passport and Hailstorm .NET Web services

-          Testing a shopping basket Web service 

Biography: 

Over the past 6 years Frank Cohen was principal architect for 3 large scale Internet systems: Sun Community Server, Inclusion.net and TuneUp.com. These are Internet messaging, collaboration and ecommerce systems. Each needed to be tested for performance and scalability. Frank developed Load, an open-source tool featuring test objects and an XML-based scripting language. Sun, Inclusion and TuneUp put Load to work testing for performance and scalability.

Back to Conference Schedule

PSQT Track Presentations (Tuesday 3:45 - 4:45 PM)

Release Management of Non-Zero Defect Software
Frank Tsui

Concepts: 

In spite of the many improvements made in software engineering, we are still faced with releasing software products which contain defects. Many times we release products with “known” defects, and this is not likely to change in the near future. The question is how do we manage such releases. We outline a simple approach with problem severity levels and a cut-off index that many commercial software enterprises utilize at project planning stage. We enhance that with a likelihood index that help us make decisions after testing but just prior to product release time. We further expand the discussion to include the increase in the failure severity levels and how that may help release management decisions. 

Presentation Outline: 

-          Introduction to Release Management and the Non-Zero Defect Software reality.

-          Software failure incidents and the definition of failure severity levels at the project planning stage.

-          The definition of cut-off index by failure severity level as the ratio of problem support cost and problem fix cost; this is an economic trade-off.

-          Making the software release decision with cutoff index.

-          The introduction of likelihood index to further enhance the release management decision process.

-          Increasing the failure severity levels as another possible enhancement to the release management decision process.

-          Tying the failure severity levels, cut-off index, and likelihood index to the software project management process and ,specifically, to software release management activities. 

Biography: 

Frank Tsui has 25 years of software development, release and support experiences with IBM Corporation. He was responsible for the development of and release of IBM EDI product, IBM CMAS product, and IBM MAPICS Release 2 product. He has also served as Director of Development for MARCAM Inc., Vice President of Technology for Metamor/PSInet, and Chief Technology Officer for RiverLogic Inc. He is currently an associate professor with Southern Polytechnic State University. 

Linda Brooks has over 20 years of experience in the IT industry ,7 of which was directly engaged in the QA process. Linda has worked at IBM for 12 years with release management, “critical” customer support management, and development management. She is currently the Vice President of Development at MAPICS Corporation, an ERP software vendor which was ISO9000 registered.

Back to Conference Schedule

People, Process, and Tools: Keys to Sucessful Software Projects
Gary Pollice 

Concepts: 

Successful software projects are those that have a balanced emphasis on people, process, and tools. Getting the right mix of these three key factors is not a simple job. There is no cookbook approach. The project leader, and the whole team, must make the right choices, based upon the goals of the current project and organizational environment.

This presentation provides practical advice about determining the right emphasis on people, process, and tools for your project. We present information gathered by practical experience over many years, on many different types and sizes of software development projects, such as small accounting systems, compiler development, and enterprise-level programs. While there is no substitute for common sense, this presentation provides guidelines to help a project leader or project team decide what is the proper approach for them. For example:

• How much process do you need when you have a small team of experienced engineers?
• Do you select a process and apply it to all of your projects or do you let the process follow the project?
• What is the real purpose of tools and how much should your process specify tool usage?

Examples of different mixes of people, process, and tools are presented that show healthy projects as well as challenged projects. We explore the root causes of how projects can become challenged from misapplication of these three factors and when it is appropriate to have extreme applications of them. 

Biography: