CONTENTS Title Page Copyright Page Preface 1 Advantages of a VAXcluster Application Environment 1.1 VAXcluster System Hardware Advantages 1.1.1 Hardware Availability 1.1.1.1 Multiple-Access Paths to Disk and Disk Failover 1.1.1.2 Multiple CPUs 1.1.1.3 Multiple Printers 1.1.1.4 Terminals Connected with LAT Hardware and Software 1.1.1.5 CI Cables and the Star Coupler Connections 1.1.1.6 VAX Volume Shadowing 1.1.2 System and Data Availability 1.1.2.1 Common-Environment VAXcluster Configuration 1.1.2.2 Multiple-Environment VAXcluster Configuration 1.1.2.3 Combination of Common-Environment and Multiple-Environment VAXcluster Configuration 1.1.3 Mass Storage Device Availability 1.1.3.1 Disks Available Clusterwide Through an HSC Controller 1.1.3.2 Disks Available Clusterwide from a CPU 1.1.3.3 Disks Dual-Ported Between HSC Controllers 1.1.3.4 Disks Dual-Ported Between CPUs 1.1.3.5 DSSI-Connected Disks in Dual-Host LAVc Configurations 1.1.3.6 Disks that are Part of a Shadow Set 1.1.3.7 System Disks 1.1.3.8 Tape Configurations 1.1.4 Workload Balancing 1.1.4.1 CPU Power 1.1.4.2 Distributed Print and Batch Queues 1.1.4.3 Terminal Configurations 1.1.5 Supported VAXcluster Systems 1.2 VAXcluster System Software Advantages 1.2.1 Connection Manager 1.2.2 Distributed VMS Lock Manager 1.2.3 Distributed File System 1.2.4 Distributed Job Controller 2 Using the VAXcluster System for Application Development 2.1 Two Types of VAXcluster Application Activity 2.1.1 I/O-Intensive Application 2.1.2 CPU-Intensive Application 2.2 Characteristics of an Application Suitable for a VAXcluster System 2.3 Applications That May Not Be Suitable for a VAXcluster System 2.4 Is an Application a Candidate for Distribution by Replication Across VAXcluster CPUs? 2.5 Is an Application a Candidate for Distribution by Decomposition Across VAXcluster CPUs? 3 Programming Tools for VAXcluster Application Development 3.1 VMS Lock Manager 3.1.1 Lock Management System Services 3.1.2 Lock Modes 3.1.3 Locking Levels 3.1.4 Lock Queues 3.1.4.1 Preventing Deadlocks 3.1.5 Lock Value Block 3.1.6 Using ASTs and Blocking ASTs for Synchronization of Interprocess Events 3.1.6.1 ASTs 3.1.6.2 Blocking ASTs 3.2 VMS Record Management Services 3.2.1 VMS RMS and UIC-Based Protection 3.2.2 VMS RMS and Clusterwide Record Locking 3.2.3 VMS RMS Buffering and Global Buffering for a VAXcluster Application 3.2.4 VMS RMS and $QIO System Services 3.2.5 VMS RMS and XQP Operations 3.3 VMS Batch Facility 3.4 Clusterwide Process Services 3.4.1 Process Control System Services 3.4.2 Process Information System Services ($GETJPI and $PROCESS_SCAN) 3.4.2.1 $GETJPI System Service 3.4.2.2 $PROCESS_SCAN System Service 3.5 DECnet-VAX 3.5.1 DECnet Task-to-Task Operations 3.5.2 Remote File and Record Access 3.5.3 Transparent DECnet-VAX Task-to-Task Communication 3.5.4 Nontransparent DECnet-VAX Task-to-Task Communication 3.5.4.1 Comparison of Transparent and Nontransparent DECnet Communication 3.6 Single-Node Programming Tools Not Available Clusterwide 3.6.1 Permanent and Temporary Mailboxes 3.6.2 Common Event Flags 3.6.3 Logical Names 3.6.4 $CREPRC 3.6.5 Writeable Global Sections 4 Application Design Models for VAXcluster Software 4.1 File Sharing Model 4.1.1 Advantages of Using the File Sharing Model 4.1.2 Disadvantages of Using the File Sharing Model 4.1.3 Implementation Requirements for the File Sharing Model 4.1.4 Example of an Application Using the File Sharing Model 4.2 Client-Server Model 4.2.1 One-to-One Client-Server Model 4.2.1.1 Advantages of Using the One-to-One Client-Server Model 4.2.1.2 Disadvantages of Using the One-to-One Client-Server Model 4.2.1.3 Implementation Requirements for the One-to-One Client Server Model 4.2.1.4 Example of an Application Using the One-to-One Client-Server Model 4.2.2 Many-to-One Client-Server Model 4.2.2.1 Compute Server 4.2.2.2 Specialized Hardware Server 4.2.2.3 Database or File Server 4.2.2.4 Advantages of Using the Many-to-One Client-Server Model 4.2.2.5 Disadvantages of Using the Many-to-One Client-Server Model 4.2.2.6 Implementation Requirements for the Many-to-One Client-Server Model 4.2.2.7 Example of an Application Using the Many-to-One Client-Server Model 4.3 Parallelism Model 4.3.1 Advantages of Using the Parallelism Model 4.3.2 Disadvantages of Using the Parallelism Model 4.3.3 Implementation Requirements for the Parallelism Model 4.3.4 Example of an Application Using the Parallelism Model 5 Designing Distributed Applications for a VAXcluster System 5.1 Designing an Application for Increased Availability 5.2 Designing an Application for Faster Completion of a Task 5.3 Designing an Application for Maximum Throughput 5.4 Comparison of Application Design Models with Application Design Goals 5.5 Designing a VAXcluster Application Using Products Closely Associated with the VMS Operating System 5.5.1 VAX RMS Journaling 5.5.1.1 VAXcluster Features 5.5.2 VAX Volume Shadowing 5.5.2.1 VAXcluster Features 5.5.3 VMS DECwindows 5.5.3.1 VAXcluster Features 5.6 Designing a VAXcluster Application Using Layered Products Based on the VMS Operating System 5.6.1 DECintact 5.6.1.1 VAXcluster Features 5.6.2 VAX ACMS 5.6.2.1 VAXcluster Features 5.6.3 VAX DBMS 5.6.3.1 VAXcluster Features 5.6.4 VAX Rdb/VMS 5.6.4.1 VAXcluster Features 5.6.5 VAX DNS 5.6.5.1 VAXcluster Features 6 Programming Techniques for VAXcluster Applications 6.1 Remote Process Creation 6.1.1 Using Transparent DECnet-VAX Communications 6.1.1.1 Transparent DECnet-VAX-Programming Example 6.1.2 Using Nontransparent DECnet-VAX Communications 6.1.2.1 Nontransparent DECnet-VAX - Programming Example 6.1.3 Using the VMS Batch Facility 6.1.3.1 Executing Batch Jobs in Parallel - Programming Example 1 6.1.3.2 Using Checkpointing with a Batch Restart - Programming Example 2 6.2 Data Sharing 6.2.1 Using DECnet-VAX Communications 6.2.1.1 Data Sharing with Transparent DECnet-VAX Communications - Programming Example 1 6.2.1.2 Data Sharing with Nontransparent DECnet-VAX Communications - Programming Example 2 6.2.2 Using VMS RMS to Control Record Granularity for Multiple Access 6.2.2.1 Using VMS RMS to Control Record Granularity - Programming Example 6.2.3 Using Read-Only Global Sections 6.2.3.1 Disk File Read-Only Global Section - Programming Example 6.3 Process Synchronization 6.3.1 Using Clusterwide Process Services 6.3.1.1 Process Control System Services - Programming Example 1 6.3.1.2 Process Information System Services - Programming Example 2 6.3.2 Using Lock Management System Services 6.3.2.1 Using Simple Lock for Exclusive Access to a Shared Resource 6.3.2.2 Using Simple Lock Conversion for Event Notification 6.3.2.3 Using Completion ASTs and Blocking ASTs with Lock Management System Services to Synchronize Simultaneous Proc 6.3.2.4 Doorbell Scheme Using a Blocking AST - Programming Example 6.3.2.5 Using the Lock Value Block to Pass Information 6.3.2.6 Passing Information with the Lock Value Block - Programming Example 6.3.2.7 Using the Lock Value Block to Determine Who is First 6.3.3 Coordinating Processes Using DECnet-VAX Communications 6.3.3.1 Coordinating Processes Using Nontransparent DECnet-VAX Communications-Programming Example 6.4 Exception Conditions 6.4.1 Recovery from Interprocess Communication Failures 6.4.1.1 Using Nontransparent DECnet-VAX Communications to Recover from Interprocess Communication Failures - Programm 6.4.2 Recovery from Cluster State Transition Due to Node Failure 7 VAXcluster System Performance Considerations 7.1 Using VMS Utilities to Monitor the Cluster and Identify Bottlenecks 7.2 Potential Bottlenecks for an I/O-Bound Application Environment 7.2.1 CPU's QIO Processing 7.2.2 Processor's I/O Adapter 7.2.3 Medium Between the I/O Adapter and the Controller 7.2.4 Controller 7.2.5 Disk Drive 7.2.6 Summary of Performance Considerations for I/O in a VAXcluster System 7.2.7 Solving an I/O Bottleneck 7.3 Potential Bottlenecks for a Memory-Bound Application 7.3.1 Solving a Memory Bottleneck 7.4 Potential Bottlenecks for a CPU-Bound Application Environment 7.4.1 Solving a CPU Bottleneck 7.5 Layered Products Available for Monitoring Cluster Performance 7.5.1 VAX Software Performance Monitor 7.5.2 VAX Performance and Coverage Analyzer 7.5.3 VAX Performance Advisor 8 Sample Application for a VAXcluster System 8.1 Application Design 8.2 Application Implementation Glossary FIGURES 1-1 Conceptual Relationship of the Four Software Components in the VAXcluster System 2-1 Comparing the Direction of Information Flow for an I/O-Intensive Application Distributed by Replication and a CPU-D 2-2 I/O-Intensive Application (PROG_A1) Executing Multiple Copies on a VAXcluster System 2-3 Distributing Work for a CPU-Intensive Application Executing on a VAXcluster System 2-4 Sample Banking Application on a VAXcluster System 3-1 Functional Relationship of VAXcluster System Programming Tools 3-2 VAXcluster Programming Tool: VMS Lock Manager 3-3 Resource Granularity Locking 3-4 Lock Queues of the VMS Lock Manager 3-5 VAXcluster Programming Tool: VMS RMS 3-6 Application Software Levels Interfacing with the VMS Lock Manager 3-7 VAXcluster Programming Tool: VMS Batch Facility 3-8 VAXcluster Programming Tool: Process Control and Process Information System Services 3-9 VAXcluster Programming Tool: DECnet-VAX 3-10 Transmitting DECnet-VAX Task-to-Task Data 3-11 One-to-Many or Many-to-One Nontransparent Communications 4-1 File Sharing Model Using Distribution by Replication 4-2 Log In to the Application 4-3 Search and Display All Available Courses According to a Keyword 4-4 Select a Course and View a Course Description 4-5 Register or Withdraw from a Selected Course 4-6 One-to-One Client-Server Model 4-7 Using a One-to-One Client-Server Model for Parallel Processing 4-8 Many-to-One Client-Server Model as a File Server 4-9 Many-to-One Client-Server Model as a File Server for a Corporate Database 4-10 Parallelism Model 4-11 Parallelism Model Using Self-Scheduling and Queueing 4-12 Car Crash Simulation 5-1 Running VMS DECwindows in a VAXcluster System 5-2 The Components of DECintact 5-3 Running VAX ACMS on Two VAXcluster CPUs 6-1 Flow Diagram for Programming Example 1 6-2 Coordinating the Use of Process Control System Services with the Process Information System Services 6-3 Deadman Lock Scheme 6-4 Doorbell Lock Scheme 6-5 Format of Lock Status Block 6-6 Determining Who is First with a Lock Value Block Scheme 6-7 A Process Designed for Failover in a VAXcluster System 7-1 I/O Pathways in a Mixed-Interconnect VAXcluster System 8-1 Diagram of a Demonstration Application 8-2 Function of Demonstration Application 8-3 SERVER.EXE Set-Up Activities 8-4 USER_IFACE.EXE Message Sending Activities 8-5 RR_AST Routine Broadcast Protocol Activities 8-6 Modules for Demonstration Application TABLES 1-1 Comparing the CI-Based, Local Area VAXcluster, and Mixed-Interconnect VAXcluster Systems 2-1 CI Adapter Throughput Capacity (Fastest to Slowest) 2-2 Ethernet Adapter Throughput Capacity (Fastest to Slowest) 2-3 Disk Type I/O Rates (Fastest to Slowest) 2-4 Goals for Distributing an Application Across VAXcluster CPUs 3-1 Lock Management System Services 3-2 Lock Modes 3-3 Lock Mode Compatibility 3-4 Multiple Lock Requests Creating a Deadlock 3-5 Supported VMS System Services for Process Control 3-6 Process Control System Services Status Codes 3-7 Using Nontransparent DECnet-VAX Communication 3-8 Transparent and Nontransparent DECnet Communication 5-1 Comparison of Application Design Models with Application Design Goals 6-1 Required Modules for Two-Way, Transparent, Task-to-Task Communications 6-2 Parameters for Lock Management System Services 7-1 VMS Utilities and Commands for Monitoring a VAXcluster System 7-2 Summary of VMS Documentation Resources for Identifying Bottlenecks 7-3 Layered Products for Monitoring a Cluster