However, this is a very specific situation (though not as unusual as most people think), in which the CPU is used heavily to satisfy IO requests. The majority of Oracle systems have their IO requests satisfied from a dynamic mix of physical disk IO and caching. As a result, our queuing theory mathematics will not come close enough to reality when used for evaluating performance firefighting options.

Another challenge when anticipating IO response time is visually demonstrated by the initial response time jump around an arrival rate of 4.5E6 pio/ms. This occurred not because of the IO subsystem, or even the CPU subsystem, reaching capacity. It occurred because of concurrency issues! This jump in queuing time occurred when the server processes started asking for the same database blocks to be brought into the cache at nearly the same time. Eventually, this concurrency issue accounted for about 30% of the total queue time. This is an Oracle Database 10g Release 1 system, and the currency wait event is read by other session. In earlier Oracle versions, the wait event would have been buffer busy wait.

Did you notice the initial drop (not increase) in the response time? This occurs in Oracle systems because cache efficiencies (Oracle and the operating system) increase as the workload begins to increase. For IO subsystems, I have seen response-time curves (based on real data) that look like a smiling face because of the significant cache efficiency effect. This is what we want to see! Eventually, however, as the workload increases, some component in the system will reach its capacity limit (in Figure 9-10, it was the CPU subsystem and concurrency issues), and the classic response-time curve elbow will appear.