Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce LoggingMarkers marker for logging sensitive data #2051

Closed
wants to merge 6 commits into from

Conversation

kpavlov
Copy link

@kpavlov kpavlov commented Jan 7, 2025

Introduce a utility class LoggingMarkers providing an SLF4J marker for tagging log entries with different data security classification, such as:

  • SENSITIVE_DATA_MARKER
  • RESTRICTED_DATA_MARKER
  • REGULATED_DATA_MARKER
  • PUBLIC_DATA_MARKER

Update BeanOutputConverter to use the SENSITIVE_DATA_MARKER in error logs for invalid JSON conversions. Enhance tests to verify SENSITIVE_DATA_MARKER marker usage in logging.

Updated Maven configurations to dynamically reference the Java version using ${java.version}. Added Kotlin compiler settings, including jvmTarget alignment with Java version and enabling javaParameters to ensure consistency and better compatibility across builds.

@kpavlov kpavlov marked this pull request as ready for review January 7, 2025 09:29
@ilayaperumalg ilayaperumalg self-assigned this Jan 17, 2025
@ilayaperumalg ilayaperumalg added this to the 1.0.0-M6 milestone Jan 17, 2025
@ilayaperumalg
Copy link
Member

@kpavlov Could you check the test failure on this PR? Thanks

@kpavlov kpavlov changed the title Introduce PII marker for logging sensitive data Introduce LoggingMarkers marker for logging sensitive data Jan 21, 2025
Konstantin Pavlov added 6 commits January 21, 2025 11:04
Introduce a utility class `LoggingMarkers` providing an SLF4J marker for tagging log entries with Personally Identifiable Information (PII). Update `BeanOutputConverter` to use the `PII_MARKER` in error logs for invalid JSON conversions. Enhance tests to verify PII marker usage in logging.

Signed-off-by: Konstantin Pavlov <{ID}+{username}@users.noreply.github.com>
Updated Maven configurations to dynamically reference the Java version using `${java.version}`. Added Kotlin compiler settings, including `jvmTarget` alignment with Java version and enabling `javaParameters`. This ensures consistency and better compatibility across builds.

Signed-off-by: Konstantin Pavlov <{ID}+{username}@users.noreply.github.com>
Updated the test to assert log size explicitly before accessing the first log entry. This ensures the test is more robust and avoids potential issues with accessing logs unexpectedly.

Signed-off-by: Konstantin Pavlov <{ID}+{username}@users.noreply.github.com>
Replaced string concatenation with a placeholder in the logger.error call to improve performance and maintain consistency with logging best practices. This helps avoid unnecessary overhead when logging is disabled.

Signed-off-by: Konstantin Pavlov <{ID}+{username}@users.noreply.github.com>
Replaced `PII_MARKER` with `SENSITIVE_DATA_MARKER`. Introduced `RESTRICTED_DATA_MARKER`, `REGULATED_DATA_MARKER` and `PUBLIC_DATA_MARKER`

Updated associated logging logic and tests to reflect these changes.

Signed-off-by: Konstantin Pavlov <{ID}+{username}@users.noreply.github.com>
Added missing periods to improve consistency and clarity in the Javadoc comments. This change ensures proper formatting and adheres to standard writing conventions.

Signed-off-by: Konstantin Pavlov <{ID}+{username}@users.noreply.github.com>
@kpavlov
Copy link
Author

kpavlov commented Jan 21, 2025

@ilayaperumalg I have updated the PR and fixed incompatibility with JDK 17

@markpollack
Copy link
Member

The core of Spring AI does not depend on Spring Boot, which is usually a green flag for using slf4j. Spring projects that do not depend on spring boot use commons logging, typically via the LogAccessor in Spring Framework. We just merged a PR that changes us fom slf4j to LogAccessor. Not sure what is the best practice in this case. I'll inquire.

@kpavlov
Copy link
Author

kpavlov commented Jan 29, 2025

LogAccessor wouldn't allow adding additional metadata to log messages. If so, my proposal for log message classification would not work. Please feel free to close this PR.

@markpollack
Copy link
Member

@kpavlov Yea, lots of convenience in sl4j. I'm reconsidering the switch... what happens in the logging impls that make use of this feature - sl4j will do the masking so we can be sure all impls will behave correctly?

@@ -0,0 +1,69 @@
package org.springframework.ai.util;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will need to add copywrite statement. will add it when merging.

@ilayaperumalg
Copy link
Member

@kpavlov Thanks for your contribution! Squashed, rebased and merged as b525309

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants