One way to reduce the impact of this oscillation-like situation is to inject into the mix some sleep time randomness. Interestingly, Oracle lets the operating system do this! Figure 3-8 is the partial output of an Oracle server process operating system trace. The Oracle system is experiencing severe CBC contention, and the CPUs are pegged at 100% busy. When the server process is traced, Oracle Database 10g Release 1 on Linux creatively uses and issues a select system call to enact the sleep effect. It's unfortunate for Oracle DBAs that the select call is issued, because it creates confusion with the Select SQL statement. But rest assured, select is a system call kernel developers put into their code, and you can even get a Linux manual page by submitting the man select command at the operating system prompt. As the manual page states, the select call provides a fairly portable way to sleep with subsecond precision. However, while the sleep parameter is in microseconds, the operating system does not guarantee precision to the microsecond.

Figure 3-8. Oracle uses the system call select to put a process to sleep during heavy latch contention. The Linux strace -r option shows the call time on the next line before the name of the next call. For example, the first select shown in this figure took 0.099707(s.

During the Figure 3-8 trace, you can see the select system call and its key time parameter, which was always 10240, or 10,240 ms-not exactly 10 ms, but close enough to the pseudocode. What I find more interesting is the actual sleep time is close to 10 ms, but never exactly 10 ms. The -r option of the Linux strace option, used in Figure 3-8, shows the call time at the beginning of the next line. If you look at the actual sleep time in microseconds in Figure 3-8, you will see sleep times of 99.707 ms, 99.352 ms, 10.602 ms, and 10.504 ms, and the next value is 10.562 ms. I have observed that the more intense the CPU bottleneck, the more variance in the sleep times. This is actually good news for Oracle, because in a very real way, the operating system is providing the desired randomness without Oracle needing to randomize the sleep time itself!