Scalability

On mid-range systems with x64 multicore-CPUs and DDR2 ECC RAM helper shows this performance results:

4 cores - 180000 requests/second

8 cores - 245000 requests/second

This reqults got with default internal queue size (1024) with max latency 4 ms. Each request size is 1 Kb.

If it is not enough, you can increase internal queue size up to 8192 slots.

Due to helper uses MPMC-bounded queue, and each worker thread has own queue (with size from 1024 up to 8192), it is non-reasonable make internal queue too long. For better scalability it is recommended to increase hardware processing cores and/or it's clock speed. As you can see, helper scales almost linear (In accordance with the Amdahl's law).

Workers use work-distribution and job-stealing strategies. Job stealing implemented using round robin algorithm. This effectively prevents internal queues overflow.

Also, for highly uneven proxy load, you can plays with proxy external queue to compensate for load spikes.