In some cases, the users may not even be aware of the workload shift. For example, during a consulting engagement, I noticed a messaging process woke every 30 seconds to check for messages to transfer and then performed the transfer. I discovered even with only a few messages, there was a tremendous amount of overhead. I asked the application administrator (not the end users) if the message process could wake up every 5 minutes just during the peak processing times. To my surprise, he willingly embraced this rather elegant, zero downtime, and zero capital investment performance-improving solution.

From a queuing theory perspective, when the workload is better balanced, the arrival rate is reduced. Figure 9-20 graphically shows when the arrival rate is decreased, we moved from point A to point B. When the arrival rate is decreased, system requirements decrease, resulting in a utilization decrease, as well as a response time decrease. Unlike with the tune and buy options, there is no response-time curve shift. What has shifted is the workload; that is, the system has traveled along the response-time curve. This is usually difficult to initially understand. But consider that when the workload has decreased, there is no change in the service time, as it takes a transaction processor just as long to process a single transaction as before. Therefore, the response-time curve does not shift down. The response-time curve does not shift to the right because no additional capacity has been implemented. What changed is the arrival rate, so we simply move along the existing response-time curve as the response time decreases. As we move to the left, while service time remains the same, the queue time will decrease, resulting in an improved response time.

Figure 9-20. Shown is the effect of workload balancing on response time. During peak processing time, the response time is at point A. By shifting some of the workload to another time, the arrival rate is reduced from 24 trx/ms to 18 trx/ms, resulting in a significant response time reduction. Notice the response-time curve does not shift, but rather the system activity has shifted.