Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dns rust keywords 7529 v4 #12605

Draft
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

catenacyber
Copy link
Contributor

Link to ticket: https://redmine.openinfosecfoundation.org/issues/
https://redmine.openinfosecfoundation.org/issues/7529
https://redmine.openinfosecfoundation.org/issues/3725

Describe changes:

  • dns: move keywords to rust
  • on the way, move unit tests to SV for dns.query keyword
  • extend some dns keywords so that they can use string enumerations
  • improves UTHBuildPacketReal to produce more real packets

#11932 next protocol
#12535 with new commits to

  • rename SCSigTableElmt to SCSigTableAppLiteElmt
  • introduce a pure rust helper to register sticky buffer cc @jasonish

SV_BRANCH=OISF/suricata-verify#2297

Ticket: 7529
Ticket: 3725

Adds url for dns.opcode on the way

Move dns.query unit tests to SV on the way
So that its contents can be reused when translating unit tests
to SV tests
Copy link

codecov bot commented Feb 17, 2025

Codecov Report

Attention: Patch coverage is 97.14286% with 13 lines in your changes missing coverage. Please review.

Project coverage is 80.75%. Comparing base (10ede91) to head (59c8406).
Report is 41 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff            @@
##           master   #12605    +/-   ##
========================================
  Coverage   80.74%   80.75%            
========================================
  Files         931      925     -6     
  Lines      259144   258691   -453     
========================================
- Hits       209242   208894   -348     
+ Misses      49902    49797   -105     
Flag Coverage Δ
fuzzcorpus 56.99% <77.83%> (+0.02%) ⬆️
livemode 19.40% <46.30%> (+0.02%) ⬆️
pcap 44.19% <61.82%> (+0.03%) ⬆️
suricata-verify 63.46% <94.08%> (+0.02%) ⬆️
unittests 58.28% <56.70%> (-0.07%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

@suricata-qa
Copy link

Information: QA ran without warnings.

Pipeline 24762

@victorjulien
Copy link
Member

Where is AppLite explained? It's a new concept.

@catenacyber
Copy link
Contributor Author

Where is AppLite explained? It's a new concept.

Please give me a better name :-)

It comes from your comment #12535 (comment)

It's currently a app-layer specific "lite" version

Comment on lines +370 to +372
name: static_cstring_from!("dns.answer.name"),
desc: static_cstring_from!("DNS answer name sticky buffer"),
url: static_cstring_from!("/rules/dns-keywords.html#dns-answer-name"),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really think this is what I had in mind. The Rust developer still has to concern themselves that this is a C struct, which is what I'd like to abstract away from.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the feedback.

I did not manage to have a rust string "dns.answer.name", because then I need to allocate "dns.answer.name\0" and I leak memory cf fixup commit and red CI before that...

Do you see another way ?
I also tried Cstring::from(c"dns.answer.name") but it looks like a big project because it is breaking other tools like rustfmt which ignores Cargo.toml showing this is edition 2021

Copy link
Member

@jasonish jasonish Feb 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The real problem is that our module system of registering keywords is "unsafe" (in the rust sense). We pass it a pointer to a C string and it requires that C string to remain rather than taking ownership, or a copy of it.

I don't have any examples doing this in Suricata, but I've created a "shadow" registry before. Basically a static mutable hasmap that maps Rust strings to C strings so they can live until the program dies. Not ideal but might be easier than modifying the keyword registration to take ownership of passed in strings?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The real problem is that our module system of registering keywords is "unsafe" (in the rust sense). We pass it a pointer to a C string and it requires that C string to remain rather than taking ownership, or a copy of it.

Indeed, the C registering system expects only static strings that it does not free

How do you free your "shadow" registry ?

What do you think of c"dns.answer.name" ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do you free your "shadow" registry ?

I don't. Because its held onto until program exit, it is not reported as a leak. It was never orphaned. Rust actually has Box::leak which is a common way to forget about pointer that is passed to C that will live til program exit, but ASAN reports that as a leak.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think of c"dns.answer.name" ?

Need Rust 1.77. But I like it in cases where you'd never want to pass a formatted string at least.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will try this shadow registry approach then...

@catenacyber catenacyber marked this pull request as draft February 21, 2025 13:35
@catenacyber catenacyber added the needs rebase Needs rebase to master label Feb 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs rebase Needs rebase to master
Development

Successfully merging this pull request may close these issues.

4 participants