Decouple code for server startup, lifecycle hooks, FastAPI app, and routes

[stbaione]

From an application standpoint, the code that we have to startup our server, add routes, add lifecycle hooks, etc. is very tightly-coupled and doesn't provide room for scale. It also makes it a bit difficult to debug and write unit tests for the individual components.

This separates the logic for starting the server, creating the FastAPI application, registering lifecycle hooks, and adding new routes.

It should make it pretty quick to expand upon each component individually, and will allow us to add more features to the server cleanly.

As a manual validation step (along with our CI), I ran my script that sends 30 requests concurrently, results for that here

[renxida]

I:

• agree we need to break up server.py
• disagree that it has to be so many small chunks. People have difficulty reading large files and remembering everything, but navigating through multiple small files have a nonnegligible overhead also. Splitting by relevance feels better.

Suggestion:

• split server.py in 2 pieces. server_app, and serving_manager.
  • app responsibilities:
    • parse args and split between shortfin args and fastapi server args
    • map routes
    • configure http related arguments
    • start the app
  • serving_manager responsibilities
    • take args from args and initialize a GenerateService
    • allow server_app to import a set of functions / a single context manager to initialize, use, and deinitialize this.

[renxida]

(maybe in a different PR)