|
|
|
Marie Ackley |
|
ackley @ promail.com |
|
|
|
SASQAG panel discussion |
|
October 18, 2001 |
|
|
|
|
|
|
Requirements Management |
|
Configuration Management |
|
Test Planning and Test Case Management |
|
Test Evaluation - Coverage Analysis |
|
Automated Testing Systems - Front End testing |
|
Load Simulators and Performance Measurement
Tools |
|
Defect Tracking |
|
|
|
others: Cyclomatic Complexity; Bounds Checkers;
Comparators - Files, Screens, Reports; Data Generators - Combinatoric,
Cause / Effect Node Graphing |
|
|
|
|
|
|
|
|
|
|
is
Putting the CART before the HORSE |
|
|
|
Don't invest in Tools without having good QA
Methods and Practices. |
|
|
|
A good tool will compliment and augment good
processes but not even the greatest tool can overcome process chaos. |
|
|
|
|
|
|
Supports multi levels of linking and tracing:
Requirements to Specifications to Test Cases, including hierarchies within
those groups |
|
|
|
Provides traceability - both forward and
backward |
|
|
|
Provides coverage analysis |
|
|
|
Provides multi relationships: one-to-many,
many-to-one and coverage is tracked for partials when relationship is
one-to-many |
|
|
|
Provides Change Management of requirements along
with audit history and maybe security privileges for modifications |
|
|
|
Import and export features so stuff came go and
come between the tool and word processors or spreadsheets |
|
|
|
|
|
Links to tool comparisons |
|
|
|
the "Volere" Requirements Method - by
Atlantic Systems Guild |
|
http://www.systemsguild.com/GuildSite/Robs/Template.html |
|
|
|
Automating Requirements Management - by Karl E.
Wiegers |
|
Detailed feature comparisons of many
requirements management tools |
|
and guidance on selecting a requirements
management tool |
|
http://www.processimpact.com/articles/rm_tools.html |
|
|
|
|
"To change and to change for the better are
two different things" |
|
- German Proverb |
|
Provides Identifications (indexing of artifacts) |
|
|
|
Controls not only software code but
documentation, hardware, firmware, and system environment configuration |
|
|
|
Provides audit trail and history tracking |
|
|
|
Provides customizable reports |
|
|
|
High ease of use for View Only functions |
|
|
|
Sufficient security levels for the Promote
functions |
|
|
|
Accounting and reporting for different ”Status
Levels" - WIP, Waiting Room, Release/Production |
|
|
|
|
|
Links to tool comparisons |
|
|
|
Configuration Management Vendor Links - by UCM
Central |
|
http://www.snuffybear.com/ucmcentral_new_vendorlinks.htm |
|
|
|
config-mgmt Newsgroup |
|
http://www.faqs.org/faqs/by-newsgroup/comp/comp.software.config-mgmt.html |
|
|
|
|
Import test cases from other file formats such
as word processing or spreadsheet |
|
|
|
Customizable reporting |
|
|
|
Tracks history and status of test cases |
|
|
|
Allows file attachments |
|
|
|
Integration - can tool link to other tools such
as your Requirements tool and your Automated Testing tool |
|
|
|
Provide a Test Execution Results repository |
|
|
|
Maintains Master Set of test cases and then can
build “subsets” for project specific needs |
|
|
|
|
|
|
|
|
|
Links for tool comparisons |
|
|
|
Testing and Test Management Tools |
|
http://www.methods-tools.com/tools/frames_testing.html |
|
|
|
Information About Testing Tools - |
|
http://www.cs.queensu.ca/home/shepard/testing.dir/under.construction/tool_list.html |
|
|
|
|
|
|
|
|
|
|
Identifies what "unique" data
combinations are needed |
|
|
|
Uses an appropriate measurement technique |
|
Decision; Condition/decision; Path; Predicate |
|
|
|
Types of coverage included in the reporting |
|
Branch; Loop; Multi; Operator; Operand |
|
|
|
Integration - is it compatible with your other
tools such as Automated Test Execution tools |
|
|
|
Implementation - is the tool
"invasive" (must be compiled along with the source code) |
|
|
|
|
|
|
|
|
Links to tool comparisons |
|
|
|
Experience With the Cost of Different Coverage
Goals for Testing - by Brian Marick |
|
http://www.testing.com/writings/experience.pdf |
|
|
|
Coverage Analysis - by Steve Cornett |
|
http://www.bullseye.com/webCoverage.html |
|
|
|
|
|
|
|
|
Maintenance - object recognition when the
application changes |
|
|
|
Ease Of Use - complexity matches the programming
talent of QA persons |
|
|
|
Tool language and syntax matches what company
uses: Visual Basic or C |
|
|
|
Extensible Language - can use external
subroutines or DLLs |
|
|
|
Integration with other tools - Code Coverage,
Test Case Management, Defect Tracking |
|
|
|
Compatibilty - platforms (Client/Server, Web,
Mainframe) Operating Systems(Windows, Mac, Unix, OS/2, Mainframe) and GUI
Development (PowerBuilder, C, Visual Basic, Galaxy |
|
|
|
|
|
|
Supports Data Driven inputs from flat files,
spreadsheet files, and database connections |
|
|
|
For Evaluation/Proof of Concept - include
automating a Level One test |
|
|
|
Links to tool comparisons |
|
|
|
Making the Right Choice - by Elisabeth
Hendrickson |
|
http://www.qualitytree.com/feature/mtrc.pdf |
|
|
|
Short Sharp Advice about how to choose a testing
tool - Mark Fewster and Dorothy Graham |
|
http://www.grove.co.uk/Tool_Information/Choosing_Tools.html |
|
|
|
Tool Comparison between Rational, Mercury,
Segue, and Compuware - by Ray Robinson |
|
http://www.qacity.com/testtype/AUTOMATION_TOOLS.doc |
|
|
|
|
|
|
Links to articles |
|
|
|
Lessons in Test Automation - by Elfriede Dustin |
|
http://www.sqa-test.com/articles.html |
|
|
|
The Truth About Automated Test Tools - by Linda
Hayes |
|
http://www.worksoft.com/resources/4-97.htm |
|
http://www.worksoft.com/resources/5-97.htm |
|
|
|
Test Automation - by Bret Pettichord |
|
http://www.io.com/~wazmo/qa/#test_automation |
|
|
|
Testers' Network - VeriTest website |
|
http://www.veritest.com/testers'network |
|
|
|
Test Automation Frameworks - by Carl J. Nagle |
|
http://members.aol.com/sascanagl/FRAMESDataDrivenTestAutomationFrameworks.htm |
|
|
|
|
|
|
|
|
Reporting includes graphical charts |
|
|
|
Debugging features for script creation |
|
|
|
Reporting includes graphical charts and logs of
performance for server speed and responsiveness |
|
|
|
Quantity of simulated users |
|
|
|
Variations in session-specific items such as
cookies, usernames, passwords, proxies and ASP-session IDs |
|
|
|
Support for your middleware - SQLnet, DBLib or
CBLib, SQL Server, ODBC, Telnet |
|
|
|
Challenges include SSL sessions, cookies, forms,
proxies, password authentication, dynamic HTML, full HTTP, SSL, XML,
Javascript, ActiveX, WAP, Java |
|
|
|
Platforms and protocols |
|
|
|
|
|
|
Link to tool comparisons |
|
|
|
Load and Performance Test Tools - by Rick Hower |
|
http://www.softwareqatest.com/qatweb1.html |
|
|
|
Software Research - TestWorks |
|
http://www.methods-tools.com/tools/testing.html |
|
|
|
|
Determine how many current users you have and
consider and scaling for future users |
|
|
|
Web based or client-server |
|
|
|
Features: Customizable Fields; Customizable
Reports; File Attachments; Audit Trail; Email linking; Security Levels |
|
|
|
Provides customizable rules for the workflows,
security privileges and notifications |
|
|
|
Integration - does it link to your other tools |
|
|
|
|
|
|
|
|
|
|
Links to tool comparisons |
|
|
|
Tracking Down a Defect Management Tool - by Hung
Quoc Nguyen |
|
http://www.qacity.com/testgen/DefectTrackingNguyenSTQE701.pdf |
|
|
|
The Problem with Problem Tracking - by Linda
Hayes |
|
http://www.worksoft.com/resources/3-99.htm |
|
|
|
Notes