In reality, with every additional parallel resource (for example, CPU core), a fraction of the power effectively becomes unavailable or lost. As mentioned, if a batch process is split, there may be the need to merge the results. The merge process is the direct result of the increased parallelism, and this constitutes a piece of perceived processing gain we effectively lose. It's true that overall we can reduce elapsed time, but the scalability effect is real, and it grows as the number of parallel streams increases.

There are a number of ways to determine the scalability effect. The simplest way is to run tests and get to know your application. If that is not practical, then be conservative. There are also a number of ways we can numerically represent the scalability effect. For example, let's suppose with every additional parallel stream, 10% is lost to scalability. This results in a more realistic elapsed time expectation. There are many methods of account for scalability. For this example, I chose to use a simple yet robust geometric scaling model. The elapsed time formula now becomes as follows:

Applying scalability to our example of splitting the batch process into three streams and an optimistic parallelization factor of 98%, the elapsed time is calculated as follows: