Uniquely identify Windows events for automated data collection

[b1t-hunter]

Dear OSSEM-DM Team,

thanks for your great work! I have worked with your detection model relationships to extract Windows events that would be required for comprehensive monitoring in a Windows domain. In a past version of your generated relationships (_all_ossem_relationships.yml), you always specified the channel (log_channel), provider (log_source) and event_id for each referenced Windows-based security event. This allowed to generate corresponding log policies and Windows event subscriptions to retrieve those events.

In a newer version of your specification, those fields have a changed semantics (log_source) or have been removed (log_channel). This makes the automated derivation of required events much more difficult or even impossible. Is it planned that this information is reintroduced, maybe under a different key name? If not, do you see any other way to retrieve the channel and provider from the relationship data, maybe with some mapping table?

While the simple cases of Microsoft-Windows-Security-Auditing and sysmon could be mapped to the channel/provider-pairs Security:Microsoft-Windows-Security-Auditing and Microsoft-Windows-Sysmon/Operational:Microsoft-Windows-Sysmon, this becomes much more difficult for event providers that log into multiple channels, such as Microsoft-Windows-Eventlog. So in order to derive the corresponding WEF subscription or generally event filter, I would have to know which event_id belongs to which channel/provider pair. On the other hand, there are some log sources that go into two different providers (and channels), like Powershell. Powershell seems to be covered now by the two log_sources powershell and Microsoft-Windows-PowerShell. powershell seems to be used for the two providers PowerShell (channel: Windows PowerShell) and Microsoft-Windows-PowerShell (channel: Microsoft-Windows-PowerShell/Operational). Hence, it cannot be determined automatically (or manually without additional knowledge of existing events) whether events should be collected from either Windows PowerShell or Microsoft-Windows-PowerShell/Operational channel.

[Cyb3rPandaH]

Hey @b1t-hunter , I hope you are doing well 😃

We are happy to hear you are using the project in your environment. Also, thank you for taking the time to submit this issue.

Regarding renaming of fields (provider --> log_source): we decided to use a more general term when identifying the source of telemetry collected. This is helping us to continue documenting the project with telemetry from different platforms in a more standard way. So the equivalent to provider would be log_source now.

Regarding the log_channel field: we decided to remove the field to align the schemas of OSSEM Data Dictionaries (DD) and Detection Modeling (DM). The log_channel field was Windows focused and we did not consider it as part of the key (log_source / id / version / platform) to identify a data dictionary uniquely within DD. However, the use case that you describe here is very interesting. We are already using optional / conditional fields within DM such as audit_category and audit_sub_category that are related to the Windows platform, but let me get some feedback from the team first.

Then we can make a final decision about adding the "channel" field to the yaml schema.

Thank you again 🍻

[Cyb3rPandaH]

Some notes to consider:

• If we add the channel field, we would also need to change/review the log_source names used for Sysmon and PowerShell events in DD and DM.

[b1t-hunter]

Hi @Cyb3rPandaH,

that sounds great! 😃 I like what you are doing in this project, because it is more free and beyond what the ATT&CK datamodels are providing at the moment. Especially when it comes to the standardization of event specs. And it seems you are contributing the experience you get from this project into ATT&CK datamodels.

I saw that you have introduced audit_category and audit_sub_category, which greatly helps to derive an audit policy automatically. At the moment, I do this derivation based on the event code, if it is part of the Microsoft-Windows-Security-Auditing provider. So that could help to skip that extra lookup step.

[Cyb3rPandaH]

Hey @b1t-hunter , hope you are doing well 😃

We have decided to add the channel field to our YAML schema. I have updated the README file of the repo (Developer branch) with an initial description of the YAML schema (Including channel). I will really appreciate it if you can take a look at it and let me know if you have any comment or feedback: https://github.com/OTRF/OSSEM-DM/blob/developer/README.md

I will update all theYAML files so they include the channel field over this week and push all the changes to the main branch by the end of this weekend.

Thank you 🍻

[b1t-hunter]

Thanks for your support, that's great news! 😃 That should make the matching much easier again.

I think the current schema looks nice. I saw that you already fixed the sysmon parts to Microsoft-Windows-Sysmon. That means you would also switch powershell to the corresponding Microsoft-Windows-Powershell and PowerShell providers?

[Cyb3rPandaH]

Thank you @b1t-hunter

and yes, I will update Security Auditing, System and PowerShell logs too

[Cyb3rPandaH]

Thank you for the feedback and suggestions @b1t-hunter 🍻
I just merged a PR to the main branch that closes this issue.
We have added the channel field for the following providers: Security Auditing, Sysmon, PowerShell, Windows Firewall, Service Control Manager and Event Log.
Please let me know in case you have any comments or feedback 🙏

[b1t-hunter]

Thanks for implementing those changes and for fixing all the channels 😃 . Automation can go on now 🦾

[b1t-hunter]

Hi @Cyb3rPandaH,

I have checked the current config with my old scripts, and works nicely for the majority of events, but there are still a few that are missing channel information:

File	event_id	channel	log_source
sensor_health_changed	1100	Security	✔️
sensor_health_changed	1101	Security	✔️
sensor_health_changed	1102	Security	✔️
sensor_health_changed	1104	Security	✔️
user_created_wmi_object	5857	Microsoft-Windows-WMI-Activity/Operational	✔️
user_created_wmi_object	5858	Microsoft-Windows-WMI-Activity/Operational	✔️
user_created_wmi_object	5859	Microsoft-Windows-WMI-Activity/Operational	✔️
user_created_wmi_object	5860	Microsoft-Windows-WMI-Activity/Operational	✔️
user_created_wmi_object	5861	Microsoft-Windows-WMI-Activity/Operational	✔️
user_created_wmi_object	5861	Microsoft-Windows-WMI-Activity/Operational	✔️
wmi_object_created	5861	Microsoft-Windows-WMI-Activity/Operational	✔️
process_searched_ldap	30	Microsoft-Windows-LDAP-Client/Debug	Microsoft-Windows-LDAP-Client

[Cyb3rPandaH]

Hey @b1t-hunter

Sorry for the late reply. Thank you for letting me know about this.
I will check those relationships over the weekend and update them 👍