Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(observability/logs): add logging support to grafana #149

Open
wants to merge 19 commits into
base: main
Choose a base branch
from

Conversation

edobry
Copy link
Contributor

@edobry edobry commented Jan 29, 2025

Description

This PR builds on the previously-added Grafana support to add log collection by:

  • adding promtail_params and loki_params sections to the top-level observability parameter section
  • adding an enable_k8s_features parameter to the observability section
  • conditionally deploying a promtail instance and Loki server

The major caveat to this work is that due to Kurtosis limitations, log collection is only supported when running on the Kubernetes Kurtosis backend, as to allow promtail to collect logs from all enclave services, it must be deployed with Helm. To this end, the observability.enable_k8s_features parameter must be explicitly set by the user, as Kurtosis does not provide a method to detect which backend the Starlark program will be executed against.

Note that in order to work around Kurtosis limitations (Kurtosis runs pods using the namespace's default ServiceAccount, which is not typically able to modify cluster-level resources, such as ClusterRoles, as the promtail Helm chart requires), this PR also creates the ns-authz Helm chart, which must be installed to the Kubernetes cluster serving as the Kurtosis backend using the following command:

just install-ns-authz

This PR also updates the package README to document the observability stack, and adds a justfile to enable easy access to Grafana:

just open-grafana <enclave name>

This PR has been tested and successfully configures log collection, exposing logs in Grafana:
image

@edobry edobry force-pushed the logs branch 2 times, most recently from e91a7e0 to 3bf7f63 Compare February 14, 2025 18:56
@edobry edobry marked this pull request as ready for review February 14, 2025 19:03
@edobry edobry force-pushed the logs branch 2 times, most recently from 5de34db to 49ae104 Compare February 25, 2025 18:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants