CONTENTS

  Title Page

  Copyright Page

  Preface

  1      Introduction to DEC/Test Manager

  1.1     Overview

  1.2     Entering Commands
    1.2.1      Getting Help
    1.2.2      Canceling Commands

  1.3     Getting Started

  2      Using DEC/Test Manager in DECwindows

  2.1     Overview
    2.1.1      Getting Help
    2.1.2      Displaying DEC/Test Manager Information in DECwindows
    2.1.3       DEC/Test Manager Command Correlation

  2.2     Sample DECwindows Session
    2.2.1      Creating a Library
    2.2.2      Creating a Test
    2.2.3      Recording a Test
    2.2.4      Creating a Collection
    2.2.5      Executing a Collection
    2.2.6      Displaying a Test Result
    2.2.7      Updating a Benchmark File
    2.2.8      Creating a Benchmark Mask

  3      Creating Tests

  3.1      DEC/Test Manager Libraries
    3.1.1      Creating a DEC/Test Manager Library
    3.1.2      Setting a Library
    3.1.3      Displaying DEC/Test Manager Library Information
    3.1.4      DEC/Test Manager History
      3.1.4.1      Adding a Remark to the History
      3.1.4.2      Deleting History Information

  3.2      DEC/Test Manager Tests

  3.3     Test Descriptions
    3.3.1      Creating Test Descriptions
    3.3.2      Displaying Test Descriptions
    3.3.3      Copying Test Descriptions
    3.3.4      Modifying Test Descriptions
    3.3.5      Deleting Test Descriptions

  3.4     Creating Noninteractive Tests
    3.4.1      Writing a Noninteractive Test
    3.4.2      Writing a Template File for a Noninteractive Test
    3.4.3      Creating a Noninteractive Test Description

  3.5     Creating Interactive Tests
    3.5.1      Recording Tests
      3.5.1.1      Recording Key Sequences
      3.5.1.2      Exiting from a Recording Session
      3.5.1.3      Redefining the Termination Character
      3.5.1.4      Redefining the Command Keysym Key
    3.5.2      Interactive Terminal Recording
    3.5.3      Interactive DECwindows Recording

  3.6     Creating an Input File from a Session File

  3.7     Playing Back an Interactive Test
    3.7.1      Playing an Interactive Terminal Session
    3.7.2      Playing an Interactive DECwindows Session

  3.8     Processing Considerations for Interactive Terminal Tests
    3.8.1      Time-Dependent Applications
    3.8.2      CTRL/C or CTRL/Y
    3.8.3      Type-Ahead
    3.8.4      Applications That Accept Unsolicited Input
    3.8.5      Device Type and Terminal Characteristics

  3.9     Processing Considerations for DECwindows Tests
    3.9.1      Playing DECwindows Tests
    3.9.2      Storing DECwindows Benchmark and Result Files
    3.9.3      Environment Initialization

  4      Organizing and Executing Test Collections

  4.1     Creating Collections

  4.2     Executing Collections
    4.2.1      Executing Collections in Batch
    4.2.2      Executing Collections Interactively
    4.2.3      Stopping the Execution of Collections

  4.3     Displaying a Collection Summary

  4.4     Deleting Collections

  4.5     Re-creating Collections

  4.6     Comparing Test Results

  4.7     Recomparing Partially Compared Collections

  5      Reviewing Test Results

  5.1     Review Concepts
    5.1.1      Using Result Descriptions
      5.1.1.1      Output Files
      5.1.1.2      Comparison Status
    5.1.2      Specifying Result Descriptions

  5.2     Examining Test Results
    5.2.1      Using the Review Subsystem
      5.2.1.1      Review Subsystem Overview
      5.2.1.2      Primary and Read-only Reviewers
      5.2.1.3      Canceling Review Subsystem Commands
      5.2.1.4      Locating Test Results in the Review Subsystem
      5.2.1.5      Using the Review Subsystem Keypads
    5.2.2      Displaying Test Results
    5.2.3      Printing Test Results

  5.3     Working with Test Results
    5.3.1      Updating an Existing Benchmark File
    5.3.2      Creating a Benchmark File for a New Test
    5.3.3      Reviewing Partially Run Collections

  6      Tailoring Your Test System

  6.1     Using Prologue and Epilogue Files
    6.1.1      Test Prologue and Epilogue Files
    6.1.2      Collection Prologues and Epilogues

  6.2     Grouping Tests
    6.2.1      Organizing Tests into Groups
    6.2.2      Displaying a Group Structure
    6.2.3      Removing Tests and Subgroups from Groups
    6.2.4      Deleting Groups

  6.3     Using Variables
    6.3.1      Modifying and Deleting Variables
    6.3.2      Overriding Variable Default Values
    6.3.3      Using Variables Defined by DEC/Test Manager
      6.3.3.1      DTM$COLLECTION_NAME Global Symbol
      6.3.3.2      DTM$TEST_NAME Local Symbol
      6.3.3.3      DTM$RESULT Logical Name
      6.3.3.4      DTM$DECW$DISPLAY Logical Name
      6.3.3.5      DTM$DELAY_TIMEOUT Logical Name
      6.3.3.6      DTM$OMIT_PRINTABLE_SCREENS Logical Name

  6.4     Using Filters
    6.4.1      Associating and Disabling Test Filters
    6.4.2      Applying File Filters

  6.5     Defining Keypad Keys

  6.6     Using Command Files
    6.6.1      Creating and Invoking a Command File
    6.6.2      Creating a DEC/Test Manager Initialization Command File

  6.7     Spawning or Attaching to Another Process

  7      Maintaining a DEC/Test Manager Library

  7.1     Correcting an Invalid DEC/Test Manager Library

  7.2     Storing Files Outside a DEC/Test Manager Library
    7.2.1      Setting Benchmark and Template Directories
    7.2.2      Storing Files in CMS Libraries

  7.3     Security Features
    7.3.1      Assigning UIC Protection
    7.3.2      Assigning ACL Protection
      7.3.2.1      Using ACLs on Library Directories
      7.3.2.2      Using ACLs on Library Files

  8      Working with Terminal Session Files

  8.1     Terminal Session Files
    8.1.1      Sample Session File
    8.1.2      Terminal Session File Structure
      8.1.2.1      Record Structure of Session Files
      8.1.2.2      Modifying Session Files Directly

  8.2     Input Files
    8.2.1      Sample Input File
    8.2.2      Special Strings
      8.2.2.1      Types of Special Strings Recognized by DEC/Test Manager
      8.2.2.2      Using Special Strings in Input Files

  8.3     Creating Input Files
    8.3.1      Extracting an Input File from a Session File
    8.3.2      Creating an Input File with a Text Editor

  8.4     Recording a Session File from an Input File
    8.4.1      Using the /INPUT Qualifier
    8.4.2      Using the INSERT Recording Function
    8.4.3      Terminal Characteristics
    8.4.4      Type-Ahead

  8.5     Translation Tables

  9      Working with DECwindows Session Files

  9.1     Creating a DECwindows Input File

  9.2     Creating a DECwindows Session File from a DECwindows Input File

  9.3     Editing a DECwindows Input File
    9.3.1      Commenting Input Files
    9.3.2      Synchronizing Play Back
    9.3.3      Repeating Tasks in an Input File
    9.3.4      Creating Informational Messages
    9.3.5      Changing the Times of Input Events

  10      DEC/Test Manager Callable Interface

  10.1    Calling Sequence for DTM$DTM
    10.1.1     Command Line (command_line)
    10.1.2     Message Routine (msg_routine)
    10.1.3     Prompt Routine (prompt_routine)
    10.1.4     Confirmation Routine (confirm_routine)
    10.1.5     Output Routine (output_routine)
    10.1.6     Output Width (width)
    10.1.7     Initialization Flag (init_flag)

  10.2    Rules for Writing DEC/Test Manager Callback Routines

  10.3    Handling Error Conditions

  10.4    Writing an Error Message Handler

  10.5    Linking with the DEC/Test Manager Image

  Command Dictionary

  1       Command Format
    1.1       Command Parameters
    1.2       Qualifiers
      1.2.1        Command Qualifiers
      1.2.2        Parameter Qualifiers
    1.3       Remark

  2       File Specification Format

  3       Command Descriptions

  @file-specification

  ATTACH

  COMPARE

  CONVERT LIBRARY

  COPY TEST_DESCRIPTION

  CREATE COLLECTION

  CREATE GROUP

  CREATE LIBRARY

  CREATE TEST_DESCRIPTION

  CREATE VARIABLE

  DEFINE/KEY

  DELETE COLLECTION

  DELETE GROUP

  DELETE HISTORY

  DELETE TEST_DESCRIPTION

  DELETE VARIABLE

  DISPLAY

  DTM

  EXIT

  EXTRACT

  FILTER

  HELP

  INSERT GROUP

  INSERT TEST_DESCRIPTION

  MODIFY GROUP

  MODIFY TEST_DESCRIPTION

  MODIFY VARIABLE

  PLAY

  RECORD

  RECREATE

  REMARK

  REMOVE GROUP

  REMOVE TEST_DESCRIPTION

  RESTORE

  REVIEW

  RUN

  SET BENCHMARK_DIRECTORY

  SET EPILOGUE

  SET LIBRARY

  SET PROLOGUE

  SET TEMPLATE_DIRECTORY

  SHOW ALL

  SHOW BENCHMARK_DIRECTORY

  SHOW COLLECTION

  SHOW EPILOGUE

  SHOW GROUP

  SHOW HISTORY

  SHOW LIBRARY

  SHOW PROLOGUE

  SHOW TEMPLATE_DIRECTORY

  SHOW TEST_DESCRIPTION

  SHOW VARIABLE

  SHOW VERSION

  SPAWN

  STOP

  SUBMIT

  VERIFY

  4       Review Subsystem Command Descriptions

  DTM_REVIEW> @file-specification

  DTM_REVIEW> ATTACH

  DTM_REVIEW> BACK

  DTM_REVIEW> DEFINE/KEY

  DTM_REVIEW> EXIT

  DTM_REVIEW> FIRST

  DTM_REVIEW> HELP

  DTM_REVIEW> INSERT

  DTM_REVIEW> LAST

  DTM_REVIEW> NEXT

  DTM_REVIEW> PCA

  DTM_REVIEW> PRINT

  DTM_REVIEW> SELECT

  DTM_REVIEW> SHOW

  DTM_REVIEW> SPAWN

  DTM_REVIEW> UPDATE

  A    DEC/Test Manager Messages

  A.1     Message Display
    A.1.1      Message Format
    A.1.2      Severity Codes

  A.2     DEC/Test Manager Messages

  Glossary

  EXAMPLES

  1-1        Sample Interactive Terminal Session

  3-1        Copying Test Descriptions

  3-2        Sample Noninteractive Test File

  3-3        Sample Noninteractive Test Template File

  5-1        Updating a Benchmark File

  5-2        Creating a Benchmark File

  6-1        Sample Test Prologue File

  6-2        Sample Test Epilogue File

  6-3        Sample Collection Prologue File

  6-4        Sample Collection Epilogue File

  6-5        Creating Groups

  6-6        Overriding Variables

  6-7        Using the DTM$TEST_NAME Local Symbol

  6-8        Using the DTM$RESULT Logical Name

  6-9        Sample Initialization Command File

  8-1        Sample Session File

  8-2        Inserting an Input File into a Recording Session

  9-1        DECwindows Input File

  9-2        Commented Input File

  9-3        Adding Synchronization Points

  9-4        Adding Loops to an Input File

  9-5        Creating Informational Messages

  FIGURES

  1-1        Regression Testing with DEC/Test Manager

  2-1 DEC/Test Manager DECwindows Title Bar and Main Menus

  2-2        Expanded Collection View

  2-3        Creating a DEC/Test Manager Library

  2-4        Creating a DECwindows Test

  2-5        Recording a DECwindows Test

  2-6        Ready to Record Dialog Box

  2-7        Sample DECwindows Recording Session

  2-8        Creating a Collection

  2-9        Executing a Collection

  2-10       Viewing Differences

  2-11       Updating a Benchmark Image

  2-12       Applying Masks to a Benchmark Image

  3-1        Overview of a Custom DEC/Test Manager Library

  5-1        Review Subsystem Default Keypad

  5-2        Review Subsystem SHOW/RESULT, SHOW/BENCHMARK, and DISPLAY/BENCHMARK Keypad

  5-3        Review Subsystem SHOW/DIFFERENCES Default Keypad

  5-4 DEC/Test Manager Benchmark File Screen 0

  5-5 DEC/Test Manager Difference File Screen 0

  6-1        Sample Group Hierarchy

  8-1        Format of the Terminal Characteristics Block

  TABLES

  1-1        Supported DEC/Test Manager Environments

  1-2 DEC/Test Manager Terms

  3-1 DEC/Test Manager Test Types

  3-2        Test Description Fields

  3-3        Recording Key Sequences

  4-1        Comparison Status Values

  5-1 DEC/Test Manager Output Files

  5-2        Result Descriptions and Comparison Status Qualifier Variations for the SHOW Command

  5-3        Locating Test Results

  5-4        Key Definitions for the Review Subsystem Keypad

  5-5        Key Definitions for the SHOW/RESULT, SHOW/BENCHMARK, and DISPLAY/BENCHMARK Keypad

  5-6        Key Definitions for the SHOW/DIFFERENCES Keypad

  6-1        Prologue and Epilogue Files

  6-2        Group Commands

  7-1 DEC/Test Manager Action in CMS Libraries

  7-2        Privileges Required for DEC/Test Manager Library Users

  7-3        Privileges Required for Individual DEC/Test Manager Commands

  8-1        Session File Record Types

  8-2        Translation of Nonprinting Characters and Control Codes When Extracting an Input File from a Session File

  8-3        Translation of Special Strings Representing Control and Nonprinting Characters When Recording a Session File from a

  8-4        Translation of Special Strings Representing 8-Bit Control Characters When Recording a Session File from an Input Fi

  8-5        Translation of Special Strings Representing the Function Key Codes When Recording a Session File from an Input File

  8-6        Translation of Special Strings Representing the Editing Key Codes When Recording a Session File from an Input File

  8-7        Translation of Special Strings Representing the Keypad Key Codes When Recording a Session File from an Input File

  8-8        Translation of Special Strings Representing the Arrow Key Codes When Recording a Session File from an Input File

  8-9        Translation of Special Strings Representing the Recording Functions When Recording a Session File from an Input Fil

  9-1        Input File Editing Operations

  10-1       Confirm_Routine Response String

  10-2       Confirm Routine Return Status

  CD-1       DEC/Test Manager Command Line Elements

  CD-2      Types of Collection Display Information

  A-1 DEC/Test Manager Message Fields

  A-2 DEC/Test Manager Message Severity Codes