Figure 4-13 shows a number of 30-second vmstat samples from a very active and highly volatile Oracle workload. As a review, the average CPU utilization is around 73% percent, or said another way, the average CPU idle time is around 27%. Based on queuing theory, a four-CPU core system running at 73% will have an average run queue of 5.3, and queue time will be around 31% of the response time. So while it may seem surprising, the response time is already significantly degrading. The vmstat report shows the average run queues to be between 3 and 7, with an average of 5.2 (I did the math). Notice the queuing theory prediction of 5.3 was very close. So based both on utilization and run queue length, this Oracle system is running out of CPU resources, and there is a very high likelihood the users are not pleased with performance.

Figure 4-13. A vmstat report on a very active four-CPU core database server. This particular system has a very dynamic workload. This is why even with around 30% idle time, the run queue frequently exceeds the number of CPU cores.

Figure 4-14 is a sample sar -q report. The runq-sz column shows average CPU run queue, and the information is from the same system shown in Figure 4-13. Key to making this report useful is knowing how many CPU cores exist on the server. This Oracle database server has four CPU cores, so processes are frequently waiting for CPU resources. This is not what we want for an OLTP-centric system.