When does queuing become a problem? The answer begins with, "Well, it depends." And what it depends on, in large part, is the number of CPU cores. A general answer is queuing becomes significant when CPU cores are busier more than 75% of the time. This is truly a general statement and does not imply that you want your CPUs running below 75%.

Figures 4-3 and 4-4 are based on the same 12 CPU core model. It's easy to think that when queuing represents 10% of the response time, this is a very serious situation. However, based on queuing theory, which CPU subsystems follow very closely, at around 75% busy, queue time accounts for around 10% of the response time. Now, 10% may seem very significant, but if you look at the situation graphically in Figure 4-4, 75% busy doesn't look so bad. This demonstrates why it can be a good idea to show data both numerically and graphically.

Figure 4-3. Numerically, on a 12-CPU core system, queuing accounts for 10% of response time at around 75% busy. However, once queuing sets in, it quickly becomes significant. When busyness exceeds 100%, queue time becomes infinite (grows and grows) and therefore so does response time. Excel shows this as #N/A.