Support in-place update of pod resources

[skonto]

/area autoscale
/area API

Describe the feature

• Related KEP is going to be beta in 1.33.
• We might want to think how this could be used with KPA. In the past we had a design document for VPA but the integration never landed. We probably want to revise that.
• Some older example of the feature here.

cc @dprotaso @evankanderson @dsimansk

[evankanderson]

We may want to also track the workload resize design sketch in the KEP: https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/1287-in-place-update-pod-resources/README.md#design-sketch-workload-resource-resize

Without workload resize, we may need something like a custom VPA or running a VPA for each Pod.

[dprotaso]

Container concurrency with tuned resource request/limits are really tied together and work well for horizontal scaling.

In place updates only really work when concurrency is very high (essentially you don't care to optimize the horizontal scaling case)

[skonto]

@dprotaso

In place updates only really work when concurrency is very high (essentially you don't care to optimize the horizontal scaling case)

I am thinking of scenarios where you may want to grow the pod instead of the number of instances to avoid for example cold starts e.g. AI cases. You may also not to do so from the beginning so you don't request a lot of resources depending on the traffic pattern e.g. bursty.