Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

flow/output: log triggered exception policy - v1 #12683

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

jufajardini
Copy link
Contributor

Make sure these boxes are checked accordingly before submitting your Pull Request -- thank you.

Changes (if applicable):

  • I have updated the User Guide (in doc/userguide/) to reflect the changes made
  • I have updated the JSON schema (in etc/schema.json) to reflect all logging changes
    (including schema descriptions)

Link to ticket: https://redmine.openinfosecfoundation.org/issues/
https://redmine.openinfosecfoundation.org/issues/6215

Describe changes:

behavioral:

  • add exception_policy_triggered object to flow eve logs, which is logged when an exception policy is triggered

implementation-related:

  • to keep track of the triggered exception policy, added a flow flag for this. Not sure if there is an issue in the exception policy definitions being an unsigned 16 while the flow flag is an unsigned 32. Defined the exception policies as uint16 as we already plan on adding more targets, so a uint8 soon wouldn't be enough

sample output:

"flow": {
  "pkts_toserver": 4,
  "pkts_toclient": 5,
  "bytes_toserver": 495,
  "bytes_toclient": 351,
  "start": "2016-07-13T22:42:07.199672+0000",
  "end": "2016-07-13T22:42:07.573174+0000",
  "age": 0,
  "state": "new",
  "reason": "shutdown",
  "alerted": false,
  "action": "pass",
  "exception_policy_triggered": {
    "target": "stream_midstream",
    "policy": "pass_flow"
  }
},

Minor:

  • clarify that exception policy stats are only logged when exception policies are enabled

Provide values to any of the below to override the defaults.

SV_BRANCH=OISF/suricata-verify#2325

To accompany the Exception Policy stats, also add information about any
Exception Policy triggered and for which target to the flow log event.

Task OISF#6215
The stats for exception policies are only logged/ present when any of
the exception policies are enabled (which means any value other than
"auto" or "ignore" in IDS mode, or "ignore" in IPS mode).

This wasn't clearly stated in the docs.
@jufajardini jufajardini requested review from victorjulien and a team as code owners February 26, 2025 21:14
Copy link

codecov bot commented Feb 26, 2025

Codecov Report

Attention: Patch coverage is 65.71429% with 24 lines in your changes missing coverage. Please review.

Project coverage is 80.71%. Comparing base (80dbaac) to head (6ff5138).
Report is 8 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #12683      +/-   ##
==========================================
- Coverage   80.75%   80.71%   -0.05%     
==========================================
  Files         934      935       +1     
  Lines      259594   259475     -119     
==========================================
- Hits       209634   209431     -203     
- Misses      49960    50044      +84     
Flag Coverage Δ
fuzzcorpus 56.97% <64.28%> (-0.03%) ⬇️
livemode 19.41% <2.85%> (+0.04%) ⬆️
pcap 44.16% <65.71%> (+<0.01%) ⬆️
suricata-verify 63.52% <64.28%> (-0.01%) ⬇️
unittests 58.19% <12.85%> (-0.13%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

@suricata-qa
Copy link

WARNING:

field baseline test %
SURI_TLPR1_stats_chk
.uptime 637 613 96.23%

Pipeline 24919

Copy link
Member

@victorjulien victorjulien left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good for the most part. My main concern is with the growth of the Flow structure.

@@ -40,6 +40,15 @@ enum ExceptionPolicy {
* "tcp.reassembly_exception_policy.drop_packet" + 1 */
#define EXCEPTION_POLICY_COUNTER_MAX_LEN 45

/** exception policy flags */
#define EXCEPTION_DEFRAG_MEMCAP BIT_U16(1)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add a EXCEPTION_NONE BIT_U16(0) here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually just 0, the other bit should start at offset 0
BIT_U16(0) is just 1 << 0 == 1

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙇🏽‍♀️

case PKT_DROP_REASON_APPLAYER_ERROR:
return EXCEPTION_APPLAYER_ERROR;
default:
return BIT_U16(0);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

then use that EXCEPTION_NONE here?

void ExceptionPolicyApply(Packet *p, enum ExceptionPolicy policy, enum PacketDropReason drop_reason)
{
SCLogDebug("start: pcap_cnt %" PRIu64 ", policy %u", p->pcap_cnt, policy);
if (p->flow) {
p->flow->flags |= FLOW_TRIGGERED_EXCEPTION_POLICY;
p->flow->applied_exception_policy |= ExceptionPolicyFlag(drop_reason);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when would we have more than one? Or should this be a simple assignment instead?

In fact, if it is possible that this is already set, what do that mean?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when would we have more than one? Or should this be a simple assignment instead?

Should it maybe be a counter, then?
And then if we have it >1, we check all flags that match?

In fact, if it is possible that this is already set, what do that mean?

This did cross my mind, but I wasn't sure this was a reasonable concern, as I didn't recall seeing such a scenario. (although probably because I didn't look for such situations when creating most tests/checks)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could applied_exception_policy itself work as the counter, maybe?

Copy link
Contributor Author

@jufajardini jufajardini Feb 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for now, going with a different approach (inspired by the exception policy counters struct)

@@ -480,6 +481,9 @@ typedef struct Flow_
* has been set. */
const struct SigGroupHead_ *sgh_toserver;

/** which exception policy was applied, if any */
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

by putting this between 2 pointers we're essentially use 8 bytes for this, which is a lot for something from which we just use 6 bits

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I didn't realize that. >__<'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants