Figure 4-20. An AWR report's Top 5 Timed Events table. Notice IO-related wait time accounts for about half of the response time, and multiblock reads account for about 28% of the wait time, with an average response time of 20 ms.

When you find that IO response times are only a couple of milliseconds or less, remember that it takes CPU resources to satisfy these IO requests. It is very common for a seemingly IO-bottlenecked system to actually be suffering from a CPU bottleneck. Performing an OraPub 3-circle analysis and the associated ORTA will clearly show this, so there should be no surprises.

Suppose your analysis clearly shows the IO subsystem takes on average 783 ms to respond to Oracle multiblock read requests. You show this to your IO subsystem vendor, who laughs and then says something like, "First of all, that's what Oracle is telling you, which is false. And second, if you had the tools we have, you would see that our IO subsystem is performing within normal parameters." If you've been working with large Oracle systems, you've probably run into this situation. It can be extremely frustrating.