Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support clickhouse exporter #38633

Closed
Cairry opened this issue Mar 14, 2025 · 5 comments
Closed

support clickhouse exporter #38633

Cairry opened this issue Mar 14, 2025 · 5 comments
Labels
enhancement New feature or request exporter/clickhouse needs triage New item requiring triage

Comments

@Cairry
Copy link

Cairry commented Mar 14, 2025

Component(s)

No response

Is your feature request related to a problem? Please describe.

Is it supported to export data to ClickHouse for storage, or which version supports this operation now?

Describe the solution you'd like

support ClickHouse export


Version:v0.100.0

Error: failed to get config: cannot unmarshal the configuration: 1 error(s) decoding:

* error decoding 'exporters': error reading configuration for "clickhouse": 1 error(s) decoding:

* '' has invalid keys: create_schema
2025/03/14 08:45:53 collector server run finished with error: failed to get config: cannot unmarshal the configuration: 1 error(s) decoding:

* error decoding 'exporters': error reading configuration for "clickhouse": 1 error(s) decoding:

* '' has invalid keys: create_schema
@Cairry Cairry added enhancement New feature or request needs triage New item requiring triage labels Mar 14, 2025
Copy link
Contributor

Pinging code owners for exporter/clickhouse: @hanjm @dmitryax @Frapschen @SpencerTorres. See Adding Labels via Comments if you do not have permissions to add labels yourself. For example, comment '/label priority:p2 -needs-triaged' to set the priority and remove the needs-triaged label.

@crobert-1
Copy link
Member

Hello @Cairry, thanks for filing this issue! Can you please share the full collector configuration you're using that's hitting this issue? This will help us give better guidance.

@SpencerTorres
Copy link
Member

Please provide the full configuration file.

Also confirm that you are using the correct distribution. Only opentelemetry-collector-contrib contains the ClickHouse exporter.

Thanks!

@Cairry
Copy link
Author

Cairry commented Mar 15, 2025

Please provide the full configuration file.

Also confirm that you are using the correct distribution. Only opentelemetry-collector-contrib contains the ClickHouse exporter.

Thanks!

receivers:
  filelog:
    include:
      - /opt/data/logs/access-structured.log
    start_at: beginning
    operators:
      - type: json_parser
        timestamp:
          parse_from: attributes.time_local
          layout: '%Y-%m-%d %H:%M:%S'
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
processors:
  batch:
    timeout: 5s
    send_batch_size: 5000
exporters:
  clickhouse:
    endpoint: tcp://localhost:9000?dial_timeout=10s&compress=lz4&async_insert=1
    # ttl: 72h
    traces_table_name: otel_traces
    logs_table_name: otel_logs
    create_schema: true
    timeout: 5s
    database: default
    sending_queue:
      queue_size: 1000
    retry_on_failure:
      enabled: true
      initial_interval: 5s
      max_interval: 30s
      max_elapsed_time: 300s


service:
  pipelines:
    logs:
      receivers: [filelog]
      processors: [batch]
      exporters: [clickhouse]
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [clickhouse]

Below is the helm version of the operator

apiVersion: v2
appVersion: 0.74.0
description: OpenTelemetry Operator Helm chart for Kubernetes
home: https://opentelemetry.io/
icon: https://raw.githubusercontent.com/cncf/artwork/a718fa97fffec1b9fd14147682e9e3ac0c8817cb/projects/opentelemetry/icon/color/opentelemetry-icon-color.png
maintainers:
- name: Allex1
- name: dmitryax
- name: TylerHelmuth
name: opentelemetry-operator
sources:
- https://github.com/open-telemetry/opentelemetry-operator
type: application
version: 0.26.2

@SpencerTorres
Copy link
Member

I may be misunderstanding the appVersion but it looks like 0.74.0 is an older version.

The create_schema option was added to the ClickHouse exporter in #32282 and released in v0.102.0.

Can you try updating your operator?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request exporter/clickhouse needs triage New item requiring triage
Projects
None yet
Development

No branches or pull requests

3 participants