Surely, the total service time is increasing with each arrival, but the service time per arrival (called simply the service time) remains roughly the same. It is easy to get the two terms confused. We know that when the workload increases, the total CPU consumption increases. But along with the CPU consumption increase comes a workload increase. The two offset each other, keeping the service time the same while the utilization increases.

When CPU-intensive Oracle algorithms begin to break down, you will notice that service time starts to increase as the arrival rate increases. If you look ahead to Figure 9-9, you will see a slight service time upward slope. As discussed back to Chapter 3, the CPU-intensive Oracle latching acquisition algorithm, with its combination of spinning and sleeping, does a tremendous job of limiting the increase in service time as the workload increases. What you feel and what users feel when performance begins to degrade is probably queue time increasing, rather than service time increasing.

In Chapter 4, we covered how CPU and IO subsystems are fundamentally different from a queuing perspective. The central difference is there is only one CPU queue, but each IO device has its own queue, so transactions have no choice but to read or write to a given IO device, regardless of its queue size. This can result in a busy device with a massive queue, while another device has little or no queue time. As a result, CPU subsystems with multiple cores exhibit little queue time until they are utilized starting around 70%, whereas IO subsystems immediately exhibit queue time. As I detailed in Chapter 4, this is true even for perfectly balanced IO subsystems.