CS147 2015-06-15 CS 147: Computer Systems Performance Analysis Selecting Techniques CS 147: Computer Systems Performance Analysis Selecting Techniques 1 / 37
Overview CS147 Overview 2015-06-15 Making Decisions Techniques Metrics Response Time Processing Rate Resource Consumption Error Metrics Financial Measures Making Decisions Overview Types of Metrics Choosing Metrics Criteria Classes of Metrics Requirements Techniques Metrics Response Time Processing Rate Resource Consumption Error Metrics Financial Measures Types of Metrics Choosing Metrics Criteria Classes of Metrics Requirements 2 / 37
Making Decisions Decisions to Be Made CS147 Decisions to Be Made 2015-06-15 Making Decisions ◮ Evaluation technique ◮ Performance metrics ◮ Performance requirements Decisions to Be Made ◮ Evaluation technique ◮ Performance metrics ◮ Performance requirements 3 / 37
Techniques Evaluation Techniques CS147 Evaluation Techniques 2015-06-15 Techniques Experimentation isn’t always the answer. Alternatives: ◮ Analytic modeling (queueing theory) ◮ Simulation ◮ Experimental measurement Evaluation Techniques But always verify your conclusions! Experimentation isn’t always the answer. Alternatives: ◮ Analytic modeling (queueing theory) ◮ Simulation ◮ Experimental measurement But always verify your conclusions! 4 / 37
Techniques Analytic Modeling CS147 Analytic Modeling 2015-06-15 Techniques ◮ Cheap and quick ◮ Don’t need working system ◮ Usually must simplify and make assumptions Analytic Modeling ◮ Cheap and quick ◮ Don’t need working system ◮ Usually must simplify and make assumptions 5 / 37
Techniques Simulation CS147 Simulation 2015-06-15 Techniques ◮ Arbitrary level of detail ◮ Intermediate in cost, effort, accuracy ◮ Can get bogged down in model-building Simulation ◮ Arbitrary level of detail ◮ Intermediate in cost, effort, accuracy ◮ Can get bogged down in model-building 6 / 37
Techniques Measurement CS147 Measurement 2015-06-15 Techniques ◮ Expensive ◮ Time-consuming ◮ Difficult to get detail ◮ But accurate Measurement ◮ Expensive ◮ Time-consuming ◮ Difficult to get detail ◮ But accurate 7 / 37
Metrics Selecting Performance Metrics CS147 Selecting Performance Metrics 2015-06-15 Metrics ◮ Three major perfomance metrics: ◮ Time (responsiveness) ◮ Processing rate (productivity) ◮ Resource consumption (utilization) ◮ Error (reliability) metrics: ◮ Availability (% time up) ◮ Mean Time to Failure (MTTF/MTBF) Selecting Performance Metrics ◮ Same as mean uptime ◮ Mean Time to Repair (MTTR) ◮ Cost/performance ◮ Three major perfomance metrics: ◮ Time (responsiveness) ◮ Processing rate (productivity) ◮ Resource consumption (utilization) ◮ Error (reliability) metrics: ◮ Availability (% time up) ◮ Mean Time to Failure (MTTF/MTBF) ◮ Same as mean uptime ◮ Mean Time to Repair (MTTR) ◮ Cost/performance 8 / 37
Metrics Response Time Response Time CS147 Response Time 2015-06-15 Metrics ◮ How quickly does system produce results? Response Time ◮ Critical for applications such as: ◮ Time sharing/interactive systems ◮ Real-time systems ◮ Parallel computing Response Time ◮ How quickly does system produce results? ◮ Critical for applications such as: ◮ Time sharing/interactive systems ◮ Real-time systems ◮ Parallel computing 9 / 37
Metrics Response Time Examples of Response Time CS147 Examples of Response Time 2015-06-15 Metrics ◮ Time from keystroke to echo on screen Response Time Examples of Response Time ◮ Time from keystroke to echo on screen 10 / 37
Metrics Response Time Examples of Response Time CS147 Examples of Response Time 2015-06-15 Metrics ◮ Time from keystroke to echo on screen Response Time ◮ End-to-end packet delay in networks Examples of Response Time ◮ Time from keystroke to echo on screen ◮ End-to-end packet delay in networks 10 / 37
Metrics Response Time Examples of Response Time CS147 Examples of Response Time 2015-06-15 Metrics ◮ Time from keystroke to echo on screen Response Time ◮ End-to-end packet delay in networks ◮ OS bootstrap time Examples of Response Time ◮ Time from keystroke to echo on screen ◮ End-to-end packet delay in networks ◮ OS bootstrap time 10 / 37
Metrics Response Time Examples of Response Time CS147 Examples of Response Time 2015-06-15 Metrics ◮ Time from keystroke to echo on screen Response Time ◮ End-to-end packet delay in networks ◮ OS bootstrap time ◮ Leaving Galileo to getting food in Hoch-Shanahan Examples of Response Time ◮ Time from keystroke to echo on screen ◮ End-to-end packet delay in networks ◮ OS bootstrap time ◮ Leaving Galileo to getting food in Hoch-Shanahan 10 / 37
Metrics Response Time Examples of Response Time CS147 Examples of Response Time 2015-06-15 Metrics ◮ Time from keystroke to echo on screen Response Time ◮ End-to-end packet delay in networks ◮ OS bootstrap time ◮ Leaving Galileo to getting food in Hoch-Shanahan Examples of Response Time ◮ Edibility not a factor ◮ Time from keystroke to echo on screen ◮ End-to-end packet delay in networks ◮ OS bootstrap time ◮ Leaving Galileo to getting food in Hoch-Shanahan ◮ Edibility not a factor 10 / 37
Metrics Response Time Measures of Response Time CS147 Measures of Response Time 2015-06-15 Metrics ◮ Response time: request-response interval Response Time ◮ Measured from end of request ◮ Ambiguous: beginning or end of response? ◮ Reaction time: end of request to start of processing Measures of Response Time ◮ Turnaround time: end of request to end of response ◮ Response time: request-response interval ◮ Measured from end of request ◮ Ambiguous: beginning or end of response? ◮ Reaction time: end of request to start of processing ◮ Turnaround time: end of request to end of response 11 / 37
Metrics Response Time The Stretch Factor CS147 The Stretch Factor 2015-06-15 Metrics ◮ Response time usually goes up with load ◮ Stretch Factor measures this: Response Time High stretch The Stretch Factor Response Time Low stretch Load ◮ Response time usually goes up with load ◮ Stretch Factor measures this: High stretch Response Time Low stretch Load 12 / 37
Metrics Processing Rate Processing Rate CS147 Processing Rate 2015-06-15 Metrics ◮ How much work is done per unit time? Processing Rate ◮ Important for: ◮ Sizing multi-user systems ◮ Comparing alternative configurations ◮ Multimedia Processing Rate ◮ How much work is done per unit time? ◮ Important for: ◮ Sizing multi-user systems ◮ Comparing alternative configurations ◮ Multimedia 13 / 37
Metrics Processing Rate Examples of Processing Rate CS147 Examples of Processing Rate 2015-06-15 Metrics ◮ Bank transactions per hour Processing Rate Examples of Processing Rate ◮ Bank transactions per hour 14 / 37
Metrics Processing Rate Examples of Processing Rate CS147 Examples of Processing Rate 2015-06-15 Metrics ◮ Bank transactions per hour Processing Rate ◮ File-transfer bandwidth Examples of Processing Rate ◮ Bank transactions per hour ◮ File-transfer bandwidth 14 / 37
Metrics Processing Rate Examples of Processing Rate CS147 Examples of Processing Rate 2015-06-15 Metrics ◮ Bank transactions per hour Processing Rate ◮ File-transfer bandwidth ◮ Aircraft control updates per second Examples of Processing Rate ◮ Bank transactions per hour ◮ File-transfer bandwidth ◮ Aircraft control updates per second 14 / 37
Metrics Processing Rate Examples of Processing Rate CS147 Examples of Processing Rate 2015-06-15 Metrics ◮ Bank transactions per hour Processing Rate ◮ File-transfer bandwidth ◮ Aircraft control updates per second ◮ Jurassic Park customers per day Examples of Processing Rate ◮ Bank transactions per hour ◮ File-transfer bandwidth ◮ Aircraft control updates per second ◮ Jurassic Park customers per day 14 / 37
Metrics Processing Rate Measures of Processing Rate CS147 Measures of Processing Rate 2015-06-15 Metrics ◮ Throughput : requests per unit time: MIPS, MFLOPS, Mb/s, TPS Processing Rate ◮ Nominal capacity : theoretical maximum: bandwidth ◮ Knee capacity : where things go bad ◮ Usable capacity : where response time hits a specified limit Measures of Processing Rate ◮ Efficiency : ratio of usable to nominal capacity ◮ Throughput : requests per unit time: MIPS, MFLOPS, Mb/s, TPS ◮ Nominal capacity : theoretical maximum: bandwidth ◮ Knee capacity : where things go bad ◮ Usable capacity : where response time hits a specified limit ◮ Efficiency : ratio of usable to nominal capacity 15 / 37
Metrics Processing Rate Nominal, Knee, and Usable Capacities CS147 Nominal, Knee, and Usable Capacities 2015-06-15 Metrics Processing Rate Nominal Response-Time Limit Capacity Nominal, Knee, and Usable Capacities Knee Usable Capacity Knee Capacity Nominal Capacity Response-Time Limit Knee Usable Capacity Knee Capacity 16 / 37
Recommend
More recommend
