In Chapter 4, we covered how CPU and IO subsystems are fundamentally different from a queuing perspective. The central difference is there is only one CPU queue, but each IO device has its own queue, so transactions have no choice but to read or write to a given IO device, regardless of its queue size. This can result in a busy device with a massive queue, while another device has little or no queue time. As a result, CPU subsystems with multiple cores exhibit little queue time until they are utilized starting around 70%, whereas IO subsystems immediately exhibit queue time. As I detailed in Chapter 4, this is true even for perfectly balanced IO subsystems.

Figure 9-8 contrasts an eight-device IO subsystem (solid line) and an eight-CPU core subsystem (dotted line) having the exact same service time. We know their service times are the same because, at a minimal arrival rate when no queuing occurs, their response time is exactly the same. With the understanding that service time does not change, regardless of the arrival rate, we know that any increase in the response time is due to queue time.

Figure 9-8 shows that a CPU subsystem can maintain a fairly consistent response time until it reaches near capacity.2 This means little or no queuing exists until the arrival rate significantly increases. In contrast, IO subsystems start queuing immediately, as reflected in the upward-sloping response-time curve.