The response-time curve is fundamental to our work and is simple to understand. Take a look at the classic response-time curve graph shown in Figure 1-3. The vertical axis is the response time. For simplicity, think of how long it takes a query to complete. Keep in mind that response time is the sum of the service time and the queue time. The horizontal axis is the arrival rate. As more transactions enter the system, we shift to the right. Notice that when the arrival rate is small, the response time is equal to the service time; that is, there is no queuing. But as more transactions enter the system per unit of time (that is, the arrival rate increases), queuing will eventually occur. Notice that just a little queuing occurs near the beginning. But as the arrival rate continues to increase, at some point (usually around 75% utilization), the queue time becomes significant and eventually will skyrocket. When this occurs, the response time also skyrockets, performance slows, and users get extremely upset.

Figure 1-3. Graph of the classic response time curve. This example shows at an arrival rate (the workload) of 1.55 transactions per millisecond (trx/ms). The response time is 3 ms/trx, the service time is 2 ms/trx, and the queue time is 1 ms/trx.

While at a university, I had a job answering the computer operations room telephone. The phone could handle multiple lines, and since I could talk with only one person at a time, sometimes I had to put someone on hold. When things were calm and someone called, I would listen to the request and handle the call, hang up, and then wait for the next call. That's when the job was relaxing. However, if the call arrival rate increased enough, someone would call while I was already talking to someone on the other line. As a result, someone had to wait his turn, or in forecasting terms, queued. I noticed once the rate of calls caused people to queue, it took only a slight increase in the call arrival rate before there were many people in the queue and they were waiting a long time. It was like everything was calm, and then-wham!-everyone was queuing.