While these details are not shown, when queuing took a second dramatic increase at around an arrival rate of 7E6 pio/ms, it wasn't because the physical devices became busy. They averaged at around only 3% utilized. Queue time took this big increase because CPU utilization reached around 80%. Because IO requests were being satisfied primarily with CPU resources, from an Oracle performance perspective, IO response time was based on CPU utilization! Since we can see that Figure 9-10 does exhibit response-time curve characteristics, in this particular situation, we can use a CPU queuing theory model to anticipate Oracle IO read times.

However, this is a very specific situation (though not as unusual as most people think), in which the CPU is used heavily to satisfy IO requests. The majority of Oracle systems have their IO requests satisfied from a dynamic mix of physical disk IO and caching. As a result, our queuing theory mathematics will not come close enough to reality when used for evaluating performance firefighting options.

Another challenge when anticipating IO response time is visually demonstrated by the initial response time jump around an arrival rate of 4.5E6 pio/ms. This occurred not because of the IO subsystem, or even the CPU subsystem, reaching capacity. It occurred because of concurrency issues! This jump in queuing time occurred when the server processes started asking for the same database blocks to be brought into the cache at nearly the same time. Eventually, this concurrency issue accounted for about 30% of the total queue time. This is an Oracle Database 10g Release 1 system, and the currency wait event is read by other session. In earlier Oracle versions, the wait event would have been buffer busy wait.