Help to scale/sizing an application in prod

[giulianopenido]

I have an ASGI application that is a WebSocket server with Django Channels that receives audio streaming from multiple users simultaneously. It's running in a Docker container inside a VPC with 8 vCPU and 32GB of memory. For these configurations, I set up 17 Granian workers (2 * CPUs + 1) and 17 threads.

Even during peak usage, the machine's CPU usage doesn't reach 20%, and memory usage is also low. My bandwidth usage is well below what the provider guarantees, but there's still significant lag in communication. To rule out network/machine/database issues, I manually set up a replica that only I have access to and tested the connection directly with it during peak usage hours (while the other replica was lagging), and there was no lag. Everything indicates that the workers are undersized. Could you give me any suggestions?

Other important info. All my open connections open another connection to other server and act like a gateway to some messages. So this application handles very heavy network I/O work.

Thank you for reading

[gi0baro]

@giulianopenido IMO both those numbers for workers and threads are just too much.
Since you're on ASGI, and given you're not blocking the Python event loop, you won't gain any benefit from having workers > vCPU. Also, the threads you specify are rust runtime threads and are per-worker, which means you have 289 threads fighting each other for scheduling time.

I would rather use a number of workers equal to your vCPU count, and 1 thread. Once you have fully load-tested such configuration, it might be worth checking wether 2 threads help or not.