OCPBUGS-43745: Add IdleCloseOnResponse field to IngressControllerTuni…

…ngOptions

Introduce an optional IdleCloseOnResponse field to control whether
HAProxy keeps idle frontend connections open during a soft stop (router
reload). This allows users to prevent errors in clients or load
balancers that do not properly handle connection resets and to retain
the pre-2.4 HAProxy behaviour if that is desirable.

operator/v1/types_ingress.go

@@ -1950,6 +1950,76 @@ type IngressControllerTuningOptions struct {
 	// +kubebuilder:validation:Type:=string
 	// +optional
 	ReloadInterval metav1.Duration `json:"reloadInterval,omitempty"`
+
+	// IdleCloseOnResponse maps directly to HAProxy's
+	// idle-close-on-response option and controls whether HAProxy
+	// keeps idle frontend connections open during a soft stop
+	// (router reload).
+	//
+	// When set to true, HAProxy will maintain idle connections
+	// during a soft reload instead of closing them immediately.
+	// These connections remain open until any of the following
+	// occurs:
+	//
+	//   - A new request is received on the connection, in which
+	//     case HAProxy handles it in the old process and closes
+	//     the connection after sending the response. New client
+	//     connections will use the updated HAProxy configuration.
+	//
+	//   - HAProxy's `timeout http-keep-alive` duration expires
+	//     (300 seconds in OpenShift's configuration, not
+	//     configurable).
+	//
+	//   - The client's keep-alive timeout expires, causing the
+	//     client to close the connection.
+	//
+	// Enabling this option can help prevent errors in clients or
+	// load balancers that do not properly handle connection
+	// resets. Additionally, this option allows you to retain the
+	// pre-2.4 HAProxy behaviour: in HAProxy version 2.2
+	// (OpenShift versions < 4.14), maintaining idle connections
+	// during a soft reload was the default behaviour, but
+	// starting with HAProxy 2.4, the default changed to closing
+	// idle connections immediately.
+	//
+	// Important Consideration:
+	//
+	// Enabling this option may result in a single request per
+	// persistent connection being routed using the old
+	// configuration after a route update. When a route is
+	// modified and the router performs a soft reload, any
+	// requests sent over existing idle connections will be
+	// handled by the old HAProxy process with the previous
+	// configuration. After handling the request, HAProxy closes
+	// the connection, ensuring that subsequent requests from the
+	// client will come via a new connection.
+	//
+	// This behaviour could lead to temporary inconsistencies,
+	// which might affect applications requiring immediate
+	// propagation of route changes.
+	//
+	// Operational Considerations:
+	//
+	//   - Keeping idle connections open during reloads may lead
+	//     to an accumulation of old HAProxy processes if
+	//     connections remain idle for extended periods,
+	//     especially in environments where frequent reloads
+	//     occur.
+	//
+	//   - Consider monitoring the number of HAProxy processes in
+	//     the router pods when this option is enabled.
+	//
+	//   - You may need to enable or adjust the
+	//     `ingress.operator.openshift.io/hard-stop-after`
+	//     duration (configured via an annotation on the
+	//     IngressController resource) in environments with
+	//     frequent reloads to prevent resource exhaustion.
+	//
+	// If unset or set to false (default), idle connections are
+	// closed immediately during a soft stop.
+	//
+	// +optional
+	IdleCloseOnResponse *bool `json:"idleCloseOnResponse,omitempty"`
 }
 
 // HTTPEmptyRequestsPolicy indicates how HTTP connections for which no request