CONTENTS Title Page Copyright Page Preface 1 Introduction to Performance Management 1.1 Knowing Your Workload 1.1.1 Workload Management 1.1.2 Workload Distribution 1.1.3 Code Sharing 1.2 Evaluating User Complaints 1.2.1 Recognizing Hardware and MWAIT Problems 1.2.2 Recognizing Unrealistic Expectations 1.3 Preparing for a Performance Investigation 1.3.1 Understanding What Tuning Is 1.3.2 Predicting When Tuning Is Required 1.4 Evaluating Tuning Success 1.5 Deciding When To Stop Tuning 1.6 Performance Options for the System Manager 2 Review of VMS Resource Management 2.1 A VMS Memory Management Analogy 2.2 Advanced Memory Management Mechanisms and Policies 2.2.1 VMS Automatic Working Set Adjustment 2.2.1.1 Working Set Sizes 2.2.1.2 Initial Working Set Limits and Characteristics 2.2.1.3 Adjustments to Working Set Sizes 2.2.1.4 Adjustments to Automatic Working Set Adjustment Parameters 2.2.1.5 Performance Management Strategies and Automatic Working Set Adjustment 2.2.2 VMS Swapper Trimming 2.2.3 Proactive Reclamation of Memory from Idle Processes 2.2.3.1 How Is This Policy Enabled? 2.2.3.2 Reclaiming Memory from Long-Waiting Processes 2.2.3.3 Reclaiming Memory from Periodically Waking Processes 2.2.3.3.1 Setting the FREEGOAL Parameter 2.2.3.3.2 Sizing Page and Swap Files 2.2.4 VMS Memory Sharing 2.2.5 VMS Scheduling 3 Managing System Resources 3.1 Evaluation Ground Rules 3.2 Collecting and Interpreting Image-Level Accounting Data 3.3 Maintaining and Interpreting MONITOR Summaries 3.4 Understanding System Responsiveness 3.4.1 Evaluating and Improving Responsiveness of System Resources 3.5 Understanding the CPU Resource 3.5.1 Evaluating CPU Responsiveness 3.5.1.1 The Compute Queue 3.5.1.2 Estimating Available CPU Capacity 3.5.1.2.1 Voluntary Wait States 3.5.1.2.2 Involuntary Wait States 3.5.2 Improving CPU Responsiveness 3.5.2.1 Equitable CPU Sharing 3.5.2.2 Reduction of CPU Consumption by the System 3.5.2.2.1 Interpreting MONITOR MODES Data 3.5.2.3 CPU Offloading 3.5.2.4 CPU Load Balancing in a VAXcluster 3.6 Understanding the Memory Resource 3.6.1 Evaluating Memory Responsiveness 3.6.1.1 Page Faulting 3.6.1.1.1 The Secondary Page Cache 3.6.1.2 Swapping and Swapper Trimming 3.6.2 Improving Memory Responsiveness 3.6.2.1 Equitable Memory Sharing 3.6.2.2 Reduction of Memory Consumption by the System 3.6.2.3 Memory Offloading 3.6.2.4 Memory Load Balancing 3.7 Understanding the Disk I/O Resource 3.7.1 Components of a Disk Transfer 3.7.2 Disk Capacity and Demand 3.7.2.1 Seek Capacity 3.7.2.2 Data Transfer Capacity 3.7.2.3 Demand 3.7.3 Evaluating Disk I/O Responsiveness 3.7.4 Improving Disk I/O Responsiveness 3.7.4.1 Equitable Disk I/O Sharing 3.7.4.2 Reduction of Disk I/O Consumption by the System 3.7.4.3 Disk I/O Offloading 3.7.4.4 Disk I/O Load Balancing 3.8 Summary of Important MONITOR Data Items 4 Diagnosing Resource Limitations 4.1 Diagnostic Strategy 4.2 Isolating Memory Limitations 4.2.1 Analyzing the Excessive Page Faulting Symptom 4.2.1.1 Image Activations Are Excessive 4.2.1.2 Characterizing Hard Versus Soft Faults 4.2.1.3 Total Working Set Size Is Too Small-Overview of the Problem 4.2.1.4 WSDEFAULT, WSQUOTA, and WSEXTENT Values Are Inappropriate 4.2.1.5 Borrowing Is Ineffective 4.2.1.6 AWSA Might Be Disabled 4.2.1.7 AWSA Is Ineffective-Overview 4.2.1.8 Swapper Trimming Is Too Vigorous 4.2.2 Analyzing the Swapping Symptom 4.2.3 Detecting Harmful Swapping 4.2.4 Investigating Why Processes Consume Unreasonable Amounts of Memory 4.2.4.1 Large, Compute-Bound Process Gains Inordinate Control of Memory 4.2.4.2 Large Waiting Process Is Never Outswapped 4.2.4.3 Too Many Processes Compete for Available Memory 4.2.4.4 Borrowing Is Too Generous 4.2.4.5 Swapper Trimming Is Ineffective 4.2.4.6 Many Working Sets Are Too Large 4.2.4.7 Disk Thrashing Occurs 4.2.4.8 System Swaps Rather Than Pages 4.2.4.9 Demand Exceeds Available Memory 4.2.5 Analyzing The Limited Free Memory Symptom 4.2.6 Special VAX-11/782 Tuning Considerations 4.3 Isolating I/O Limitations 4.3.1 Disk or Tape Operation Problems (Direct I/O) 4.3.2 Determining I/O Rates 4.3.2.1 Device I/O Rate Is Below Capacity 4.3.2.2 Direct I/O Rate Is Abnormally High 4.3.2.3 Disk Activity Is Due to Paging or Swapping 4.3.3 Reduce I/O Demand or Add Capacity 4.3.4 Terminal Operation Problems (Buffered I/O) 4.3.4.1 Interrupt Stack Time Is Excessive 4.3.4.2 Kernel Mode Time Is Excessive 4.4 Isolating CPU Limitations 4.4.1 Processes Are Blocked by a Higher-Priority Process 4.4.2 Time-Slicing Occurs Between Processes 4.4.3 Interrupt Stack Activity Is Excessive 4.4.3.1 Memory Limitation Is Disguised 4.4.4 Operating System Incurs Overhead 4.4.5 VMS RMS Is Misused 4.4.6 CPU Is Operating at Full Capacity 4.5 Conclusion 5 Compensating for Resource Limitations 5.1 Changing System Parameters 5.2 Compensating for Memory-Limited Behavior 5.2.1 Reduce Number of Image Activations 5.2.2 Increase Page Cache Size 5.2.3 Decrease Page Cache Size 5.2.4 Adjust Working Set Characteristics (for Quota and Extent) 5.2.4.1 Establish Values for Ancillary Control Processes (ODS-1 Only) 5.2.4.2 Establish Values for Other Processes 5.2.4.3 Establish Values for Detached Processes or Subprocesses 5.2.4.4 Establish Values for Batch Jobs 5.2.5 Tune to Make Borrowing More Effective 5.2.6 Tune AWSA to Respond Quickly to Increased Demand 5.2.7 Disable Voluntary Decrementing 5.2.8 Tune Voluntary Decrementing 5.2.9 Turn on Voluntary Decrementing 5.2.10 Enable AWSA 5.2.11 Adjust Swapper Trimming 5.2.12 Convert to a System That Rarely Swaps 5.2.13 Adjust BALSETCNT 5.2.13.1 Reduce BALSETCNT to Reduce Paging 5.2.13.2 Increase BALSETCNT to Decrease Swapping Problems 5.2.14 Reduce Large Page Caches 5.2.15 Curtail Large, Compute-Bound Process 5.2.16 Suspend Large, Compute-Bound Process 5.2.17 Control Growth of Large, Compute-Bound Processes 5.2.18 Enable Swapping for Disk ACPs (ODS-1 Only) 5.2.19 Enable Swapping for Other Processes 5.2.20 Reduce Number of Concurrent Processes 5.2.21 Discourage Working Set Loans When Memory Is Scarce 5.2.22 Increase Swapper Trimming Memory Reclamation 5.2.23 Reduce Rate of Inswapping 5.2.24 Induce Paging to Reduce Swapping 5.2.25 Add Page Files 5.2.26 Reduce Demand or Add Memory 5.2.26.1 Reduce Demand 5.2.26.2 Add Memory 5.3 Compensating for I/O-Limited Behavior 5.3.1 Remove Blockage Due to ACP 5.3.1.1 Blockage Due to a Device, Controller, or Bus (ODS-1 Only) 5.3.1.2 Enlarge Hardware Capacity 5.3.2 Improve VMS RMS Caching 5.3.3 Adjust File System Caches 5.3.4 Reduce Interrupts for Terminal I/O 5.3.4.1 Choose the Appropriate Environment for DMF32s 5.3.4.2 Consider Application Design 5.3.4.3 Lower the Baud Rate if Terminals Smooth Scroll 5.3.4.4 Reduce Demand or Increase CPU Capacity 5.4 Compensating for CPU-Limited Behavior 5.4.1 Adjust Priorities 5.4.2 Reduce Interrupts 5.4.3 Increase QUANTUM 5.4.4 Reduce Demand or Add CPU Capacity EXAMPLES 3-1 Sample Image-Level Accounting Report 3-2 Sample WORKSET.COM Display 3-3 Sample Procedure to Obtain Working Set Information 3-4 Sample Prime Time VAXcluster Multifile Summary Report FIGURES 1-1 Verifying the Validity of a Performance Complaint 1-2 Time Spent Tuning Versus Performance Improvements 2-1 Workshop Layout 2-2 VMS Memory Configuration 2-3 The Working Set Regions for a Process 2-4 Effect of Working Set Size on Page Fault Rate-Graph 1 2-5 Effect of Working Set Size on Page Fault Rate-Graph 2 2-6 Effect of Working Set Size on Page Fault Rate-Graph 3 2-7 An Example of Working Set Adjustment at Work 2-8 Example Without Shared Code 2-9 Example With Shared Code 4-1 Steps in the Preliminary Investigation Process 4-2 Investigating Excessive Paging-Phase I 4-3 Investigating Excessive Paging-Phase II 4-4 Investigating Excessive Paging-Phase III 4-5 Investigating Excessive Paging-Phase IV 4-6 Investigating Excessive Paging-Phase V 4-7 Investigating Swapping-Phase I 4-8 Investigating Swapping-Phase II 4-9 Investigating Swapping-Phase III 4-10 Investigating Limited Free Memory-Phase I 4-11 Investigating Disk I/O Limitations-Phase I 4-12 Investigating Disk I/O Limitations-Phase II 4-13 Investigating Terminal I/O Limitations-Phase I 4-14 Investigating Terminal I/O Limitations-Phase II 4-15 Investigating Specific CPU Limitations-Phase I 4-16 Investigating Specific CPU Limitations-Phase II TABLES 2-1 Corresponding Terms in the Workshop and VMS Analogy 3-1 Components of a Typical Disk Transfer on a VAX-11/780 (Four to Eight Block Transfer Size) 3-2 Summary of Important MONITOR Data Items