dpif-netdev: Add per PMD sleep config.

Extend 'pmd-sleep-max' so that individual PMD thread cores may have a
specified max sleep request value.

Existing behaviour is maintained.

Any PMD thread core without a value will use the global value if set
or default no sleep.

To set PMD thread cores 8 and 9 to never request a load based sleep
and all other PMD thread cores to be able to request a max sleep of
50 usecs:

 $ ovs-vsctl set open_vswitch . other_config:pmd-sleep-max=50,8:0,9:0

To set PMD thread cores 10 and 11 to request a max sleep of 100 usecs
and all other PMD thread cores to never request a sleep:

 $ ovs-vsctl set open_vswitch . other_config:pmd-sleep-max=10:100,11:100

'pmd-sleep-show' is updated to show the max sleep value for each PMD
thread.

Signed-off-by: Kevin Traynor <ktraynor@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>

Documentation/topics/dpdk/pmd.rst

@@ -353,10 +353,6 @@ and can differ significantly depending on system configuration. The actual
 time not processing packets will be determined by the sleep and processor
 wake-up times and should be tested with each system configuration.
 
-The current configuration of the PMD load based sleeping can be shown with::
-
-    $ ovs-appctl dpif-netdev/pmd-sleep-show
-
 Sleep time statistics for 10 secs can be seen with::
 
     $ ovs-appctl dpif-netdev/pmd-stats-clear \
@@ -379,5 +375,35 @@ system configuration (e.g. enabling processor C-states) and workloads.
     extra latency before the PMD thread returns to processing packets at full
     rate.
 
+Maximum sleep values can also be set for individual PMD threads using
+key:value pairs in the form of core:max_sleep. Any PMD thread that has been
+assigned a specified value will use that. Any PMD thread that does not have
+a specified value will use the current global value.
+
+Specified values for individual PMD threads can be added or removed at
+any time.
+
+For example, to set PMD threads on cores 8 and 9 to never request a load based
+sleep and all others PMD threads to be able to request a max sleep of
+50 microseconds (us)::
+
+    $ ovs-vsctl set open_vswitch . other_config:pmd-sleep-max=50,8:0,9:0
+
+The max sleep value for each PMD thread can be checked in the logs or with::
+
+    $ ovs-appctl dpif-netdev/pmd-sleep-show
+    pmd thread numa_id 0 core_id 8:
+      max sleep:    0 us
+    pmd thread numa_id 1 core_id 9:
+      max sleep:    0 us
+    pmd thread numa_id 0 core_id 10:
+      max sleep:   50 us
+    pmd thread numa_id 1 core_id 11:
+      max sleep:   50 us
+    pmd thread numa_id 0 core_id 12:
+      max sleep:   50 us
+    pmd thread numa_id 1 core_id 13:
+      max sleep:   50 us
+
 .. _ovs-vswitchd(8):
     http://openvswitch.org/support/dist-docs/ovs-vswitchd.8.html

NEWS

@@ -32,6 +32,10 @@ Post-v3.2.0
      * Added support for Generic Segmentation Offloading for the cases where
        TSO is enabled but not supported by an egress interface (except for
        tunnel interfaces).
+     * 'pmd-sleep-max' is updated to also accept pmd-thread-core:sleep-max.
+       The existing behaviour is maintained and a non key:value pair value
+       will be applied to all other PMD thread cores.'pmd-sleep-show' is
+       updated to show the maximum sleep for each PMD thread core.
 
 
 v3.2.0 - 17 Aug 2023

lib/dpif-netdev-private-thread.h

@@ -180,6 +180,9 @@ struct dp_netdev_pmd_thread {
     int numa_id;                    /* numa node id of this pmd thread. */
     bool isolated;
 
+    /* Max sleep request in microseconds. */
+    atomic_uint64_t max_sleep;
+
     /* Queue id used by this pmd thread to send packets on all netdevs if
      * XPS disabled for this netdev. All static_tx_qid's are unique and less
      * than 'cmap_count(dp->poll_threads)'. */