Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

dpdk: collection of user-facing improvements v1.4 #12654

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open

dpdk: collection of user-facing improvements v1.4 #12654

wants to merge 6 commits into from

Conversation

lukashino
Copy link
Contributor

@lukashino lukashino commented Feb 22, 2025 •
edited
Loading

Follow-up of: #12571

Tickets:
https://redmine.openinfosecfoundation.org/issues/7373
https://redmine.openinfosecfoundation.org/issues/7382
https://redmine.openinfosecfoundation.org/issues/7381
https://redmine.openinfosecfoundation.org/issues/7380
https://redmine.openinfosecfoundation.org/issues/7374
https://redmine.openinfosecfoundation.org/issues/7373
https://redmine.openinfosecfoundation.org/issues/5838

v1.4:

  • rebased

v1.3:

  • rebased

Describe changes:

  • document vlan stripping offload
  • check for link-up before full startup
  • use the default iface-copy value if not specified
  • replace global with per-thread mempools
  • auto-configure Rx/Tx descriptors and mempool size

Lukas Sismis added 6 commits February 22, 2025 10:06
@lukashino
dpdk: auto configure Rx/Tx descriptors and mempool size
1c64da9 
Ticket: 7380
Ticket: 7373
@lukashino
dpdk: remove mempool pointer from the thread structure
4e214b0
@lukashino
dpdk: replace global with per-thread mempools
9744aac 
It turned out that having global (interface-specific) mempool
that is shared by the threads of the interface is slower than
having individual mempools per queue for each interface.

The commit brings this change and should be user-invisible,
the config setting remains still as a number of objects of
all mempools summed (of that interface).

Ticket: 7382
@lukashino
dpdk: use default iface-copy value if not specified
15804c5 
Ticket: 7374
@lukashino
dpdk: check for link up before full startup
6dce61a 
ICE card (Intel E810) was not receiving packets immediatelly
after startup, Suricata workers would act as processing while
it was not. This eliminates the problem by only continuing
in the initialization if the link is already up.

The setting can be turned off manually from the configuraiton
file.

Ticket: 7381
@lukashino
dpdk: document vlan stripping offload
3695929 
Ticket: 5838
@lukashino lukashino mentioned this pull request Feb 22, 2025
Closed
@codecov Codecov
Copy link

codecov bot commented Feb 22, 2025

Codecov Report

Attention: Patch coverage is 49.53704% with 109 lines in your changes missing coverage. Please review.

Project coverage is 80.74%. Comparing base (3bc2a14) to head (3695929).

Additional details and impacted files 
@@            Coverage Diff             @@
##           master   #12654      +/-   ##
==========================================
- Coverage   80.77%   80.74%   -0.03%     
==========================================
  Files         932      933       +1     
  Lines      259517   259654     +137     
==========================================
+ Hits       209629   209668      +39     
- Misses      49888    49986      +98
Flag Coverage Δ
fuzzcorpus 56.99% <ø> (ø)
livemode 19.38% <49.53%> (+<0.01%) ⬆️
pcap 44.19% <ø> (+0.03%) ⬆️
suricata-verify 63.51% <ø> (+<0.01%) ⬆️
unittests 58.33% <ø> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

@suricata-qa
Copy link

Information: QA ran without warnings.

Pipeline 24860

@victorjulien
Copy link
Member

Default config with just the interface set and affinity enabled:

Starting program: /home/victor/dev/suricata/src/suricata -c suricata.yaml --disable-detection -v --dpdk
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Notice: suricata: This is Suricata version 8.0.0-dev (3695929f3e 2025-02-22) running in SYSTEM mode [LogVersion:suricata.c:1154]
Info: cpu: CPUs/cores online: 8 [UtilCpuPrintSummary:util-cpu.c:149]
Info: suricata: Setting engine mode to IDS mode by default [PostConfLoadedSetup:suricata.c:2686]
Info: exception-policy: master exception-policy set to: auto [ExceptionPolicyMasterParse:util-exception-policy.c:201]
Info: logopenfile: fast output device (regular) initialized: fast.log [SCConfLogOpenGeneric:util-logopenfile.c:619]
Info: logopenfile: eve-log output device (regular) initialized: eve.json [SCConfLogOpenGeneric:util-logopenfile.c:619]
Info: logopenfile: stats output device (regular) initialized: stats.log [SCConfLogOpenGeneric:util-logopenfile.c:619]
[New Thread 0x7fffee7336c0 (LWP 20240)]
[New Thread 0x7fffedf326c0 (LWP 20241)]
[New Thread 0x7fffed7316c0 (LWP 20242)]
[New Thread 0x7fffecf306c0 (LWP 20243)]
[New Thread 0x7fffec72f6c0 (LWP 20244)]
[New Thread 0x7fffebf2e6c0 (LWP 20245)]
[New Thread 0x7fffeb72d6c0 (LWP 20246)]
[New Thread 0x7fffeaf2c6c0 (LWP 20247)]
[New Thread 0x7fffea72b6c0 (LWP 20248)]
mlx5_net: No available register for sampler.
mlx5_net: No available register for sampler.
[New Thread 0x7fffe9f2a6c0 (LWP 20249)]
TELEMETRY: No legacy callbacks, legacy socket not created
Warning: dpdk: "all" specified in worker CPU cores affinity, excluding management threads [ConfigSetThreads:runmode-dpdk.c:400]
Info: dpdk: 0000:01:00.0: creating 7 packet mempools of size 65535, cache size 257, mbuf size 2176 [DeviceConfigureQueues:runmode-dpdk.c:1413]
Error: dpdk: 0000:01:00.0: rte_pktmbuf_pool_create failed with code 12 (mempool: mp_1_0000:01:00.0) - Cannot allocate memory [DeviceConfigureQueues:runmode-dpdk.c:1422]

Thread 1 "Suricata-Main" received signal SIGSEGV, Segmentation fault.
DPDKDeviceResourcesDeinit (dpdk_vars=0x555556ebae20) at util-dpdk-common.c:55
55                      if (dpdk_vars->pkt_mp[j] != NULL) {
(gdb) bt
#0  DPDKDeviceResourcesDeinit (dpdk_vars=0x555556ebae20) at util-dpdk-common.c:55
#1  0x00005555557ee240 in DPDKDerefConfig (conf=0x5555570f1860) at runmode-dpdk.c:339
#2  DPDKDerefConfig (conf=0x5555570f1860) at runmode-dpdk.c:333
#3  0x00005555557f22d6 in ParseDpdkConfigAndConfigureDevice (iface=0x555555fb2fc0 "0000:01:00.0") at runmode-dpdk.c:1760
#4  0x00005555557f5cdd in RunModeSetLiveCaptureWorkers (ConfigParser=ConfigParser@entry=0x5555557f2230 <ParseDpdkConfigAndConfigureDevice>, ModThreadsCount=ModThreadsCount@entry=0x5555557ede50 <DPDKConfigGetThreadsCount>, 
    recv_mod_name=recv_mod_name@entry=0x555555d05773 "ReceiveDPDK", decode_mod_name=decode_mod_name@entry=0x555555d0577f "DecodeDPDK", thread_name=0x555555d04a64 "W", live_dev=live_dev@entry=0x0) at util-runmodes.c:336
#5  0x00005555557ee5dc in RunModeIdsDpdkWorkers () at runmode-dpdk.c:1910
#6  0x000055555576a754 in RunModeDispatch (runmode=11, custom_mode=0x555555d04854 "workers", capture_plugin_name=<optimized out>, capture_plugin_args=<optimized out>) at runmodes.c:423
#7  0x000055555569d1be in SuricataInit () at suricata.c:2978
#8  0x000055555569882a in main (argc=6, argv=0x7fffffffe398) at main.c:54

@victorjulien
Copy link
Member

If I increase the number of hugepages, it fails somewhere else

sudo ./src/suricata -c suricata.yaml --disable-detection -v --dpdk
Notice: suricata: This is Suricata version 8.0.0-dev (3695929f3e 2025-02-22) running in SYSTEM mode [LogVersion:suricata.c:1154]
Info: cpu: CPUs/cores online: 8 [UtilCpuPrintSummary:util-cpu.c:149]
Info: suricata: Setting engine mode to IDS mode by default [PostConfLoadedSetup:suricata.c:2686]
Info: exception-policy: master exception-policy set to: auto [ExceptionPolicyMasterParse:util-exception-policy.c:201]
Info: logopenfile: fast output device (regular) initialized: fast.log [SCConfLogOpenGeneric:util-logopenfile.c:619]
Info: logopenfile: eve-log output device (regular) initialized: eve.json [SCConfLogOpenGeneric:util-logopenfile.c:619]
Info: logopenfile: stats output device (regular) initialized: stats.log [SCConfLogOpenGeneric:util-logopenfile.c:619]
mlx5_net: No available register for sampler.
mlx5_net: No available register for sampler.
TELEMETRY: No legacy callbacks, legacy socket not created
Warning: dpdk: "all" specified in worker CPU cores affinity, excluding management threads [ConfigSetThreads:runmode-dpdk.c:400]
Info: dpdk: 0000:01:00.0: creating 7 packet mempools of size 65535, cache size 257, mbuf size 2176 [DeviceConfigureQueues:runmode-dpdk.c:1413]
mlx5_net: minimal data inline requirements (18) are not satisfied (12) on port 0, try the smaller Tx queue size (32768)
mlx5_net: port 0 unable to allocate queue index 0
Error: dpdk: 0000:01:00.0: failed to setup TX queue 0: Cannot allocate memory [DeviceConfigureQueues:runmode-dpdk.c:1465]
Segmentation fault

victorjulien
victorjulien requested changes Feb 26, 2025
View reviewed changes
Copy link
Member

@victorjulien victorjulien left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see crash reports inline

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@victorjulien victorjulien victorjulien requested changes

@jufajardini jufajardini Awaiting requested review from jufajardini jufajardini is a code owner

Requested changes must be addressed to merge this pull request.

Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@lukashino @suricata-qa @victorjulien