Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use upstream filereceiver and fileexporter with updated docs #278

Merged
merged 7 commits into from
Jan 17, 2025
Merged

Use upstream filereceiver and fileexporter with updated docs #278

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
ece1201
Removed local filereceiver and fileexporter in favor of upstream impl…
drewrelmas Jan 16, 2025
83ffac5
Properly modify go.mod and go.sum
drewrelmas Jan 16, 2025
2d84723
Merge branch 'main' into drewrelmas/housekeeping
drewrelmas Jan 17, 2025
6e1b024
Remove filereceiver and fileexporter from versions.yaml
drewrelmas Jan 17, 2025
c74ea99
Add changelog entry
drewrelmas Jan 17, 2025
8fe890d
Point to proper 0.117.0 otlpjsonfilereceiver
drewrelmas Jan 17, 2025
d99c2f9
Re-run to get updated go.mod for 117 otlpjsonfilereceiver
drewrelmas Jan 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
## Unreleased

- Update to v0.117.0 collector dependencies. [#277](https://github.com/open-telemetry/otel-arrow/pull/277), [#279](https://github.com/open-telemetry/otel-arrow/pull/279)
- Deprecate local versions of `fileexporter` and `filereceiver` in favor of upstream components with related README changes. [#232](https://github.com/open-telemetry/otel-arrow/issues/232)

## [0.31.0](https://github.com/open-telemetry/otel-arrow/releases/tag/v0.31.0) - 2024-11-19

Expand Down
36 changes: 16 additions & 20 deletions collector/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,17 @@ This directory contains the primary collector components for using
OpenTelemetry Protocol with Apache Arrow as well as a number of useful
accessory components that were developed to assist the project.

The primary components are:
The primary components (now included in the upstream [Collector-Contrib repository][COLLECTORCONTRIB]) are:

- [Exporter][EXPORTER]: for sending OpenTelemetry Protocol with Apache Arrow data
- [Receiver][RECEIVER]: for receiving OpenTelemetry Protocol with Apache Arrow data
- [OTel Arrow Exporter][ARROWEXPORTER]: for sending OpenTelemetry Protocol with Apache Arrow data
- [OTel Arrow Receiver][ARROWRECEIVER]: for receiving OpenTelemetry Protocol with Apache Arrow data

Note that the exporter and receiver components have been included in the official
OpenTelemetry Collector-Contrib release images since v0.105.0.

## Building and distributing these components

The exporter and receiver components are included in the official
OpenTelemetry Collector-Contrib release images since v0.105.0.
See [Building][BUILDING].

## Components included in this repository

Expand All @@ -30,14 +32,6 @@ maintainers. These components are:

### For research and validation

- [`exporter/fileexporter`][ARROWFILEEXPORTER]: Derived from the
upstream [fileexporter][UPSTREAMFILEEXPORTER], this component
supports writing files that can be read by the corresponding
`filereceiver` in this package (which the upstream cannot do).
- [`receiver/filereceiver`][ARROWFILERECEIVER]: Derived from the
upstream [filereceiver][UPSTREAMFILERECEIVER], this component
supports reading files written by the corresponding `fileexporter`
in this package (unlike the upstream).
- [`processor/obfuscationprocessor`][OBFUSCATIONPROCESSOR]: Supports
obfuscation of OpenTelemetry data using a [Feistel
cipher](https://en.wikipedia.org/wiki/Feistel_cipher).
Expand All @@ -51,23 +45,25 @@ Several Collector-Contrib extensions are included in the build:
- [headersetter][HEADERSETTEREXT]: Allows propagating headers through
a pipeline
- [pprof][PPROFEXT]: Allows use of Golang profiling tools.
- [fileexporter][FILEEXPORTER]: Writes telemetry data to files on disk.
- [otlpjsonfilereceiver][FILERECEIVER]: Reads telemetry data from JSON files.

Note that while previously this repository had its own local versions of a basic fileexporter and filereceiver derived from upstream, it is now recommended to use the upstream [fileexporter][FILEEXPORTER] and [otlpjsonfilereceiver][FILERECEIVER] instead.

From the core collector repository:

- [otelhttpexporter][UPSTREAMHTTPOTLP]: Useful for debugging, sends standard OTLP over HTTP
- [debugexporter][UPSTREAMDEBUG]: Useful for debugging, prints OTLP data to the console

[BUILDING]: ./BUILDING.md
[EXPORTER]: ./exporter/otelarrowexporter/README.md
[RECEIVER]: ./receiver/otelarrowreceiver/README.md
[CONTRIBUTION]: https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/26491
[COLLECTORCONTRIB]: https://github.com/open-telemetry/opentelemetry-collector-contrib
[ARROWEXPORTER]: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/exporter/otelarrowexporter/README.md
[ARROWRECEIVER]: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/otelarrowreceiver/README.md
[UPSTREAMBATCHPROCESSOR]: https://github.com/open-telemetry/opentelemetry-collector/blob/main/processor/batchprocessor/README.md
[CONCURRENTBATCHPROCESSOR]: ./processor/concurrentbatchprocessor/README.md
[ARROWFILEEXPORTER]: ./exporter/fileexporter/README.md
[ARROWFILERECEIVER]: ./receiver/filereceiver/README.md
[UPSTREAMFILEEXPORTER]: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/fileexporter/README.md
[UPSTREAMFILERECEIVER]: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/filereceiver/README.md
[OBFUSCATIONPROCESSOR]: ./processor/obfuscationprocessor/README.md
[FILEEXPORTER]: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/exporter/fileexporter/README.md
[FILERECEIVER]: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/otlpjsonfilereceiver/README.md
[BASICAUTHEXT]: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/extension/basicauthextension/README.md
[HEADERSETTEREXT]: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/extension/headerssetterextension/README.md
[PPROFEXT]: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/extension/pprofextension/README.md
Expand Down
8 changes: 4 additions & 4 deletions collector/cmd/otelarrowcol/components.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

27 changes: 17 additions & 10 deletions collector/cmd/otelarrowcol/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,15 @@ go 1.22.0
toolchain go1.22.2

require (
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/fileexporter v0.117.0
github.com/open-telemetry/opentelemetry-collector-contrib/exporter/otelarrowexporter v0.117.0
github.com/open-telemetry/opentelemetry-collector-contrib/extension/basicauthextension v0.117.0
github.com/open-telemetry/opentelemetry-collector-contrib/extension/headerssetterextension v0.117.0
github.com/open-telemetry/opentelemetry-collector-contrib/extension/pprofextension v0.117.0
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/otelarrowreceiver v0.117.0
github.com/open-telemetry/otel-arrow/collector/exporter/fileexporter v0.31.0
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/otlpjsonfilereceiver v0.117.0
github.com/open-telemetry/otel-arrow/collector/processor/concurrentbatchprocessor v0.31.0
github.com/open-telemetry/otel-arrow/collector/processor/obfuscationprocessor v0.31.0
github.com/open-telemetry/otel-arrow/collector/receiver/filereceiver v0.31.0
go.opentelemetry.io/collector/component v0.117.0
go.opentelemetry.io/collector/confmap v1.23.0
go.opentelemetry.io/collector/confmap/provider/envprovider v1.23.0
Expand All @@ -42,16 +42,19 @@ require (
github.com/apache/arrow/go/v17 v17.0.0 // indirect
github.com/axiomhq/hyperloglog v0.0.0-20230201085229-3ddf4bad03dc // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/bmatcuk/doublestar/v4 v4.7.1 // indirect
github.com/btcsuite/btcd/btcec/v2 v2.3.2 // indirect
github.com/cenkalti/backoff/v4 v4.3.0 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/cyrildever/feistel v1.5.5 // indirect
github.com/cyrildever/go-utls v1.9.7 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.0.1 // indirect
github.com/dgryski/go-metro v0.0.0-20180109044635-280f6062b5bc // indirect
github.com/ebitengine/purego v0.8.1 // indirect
github.com/elastic/lunes v0.1.0 // indirect
github.com/ethereum/go-ethereum v1.13.1 // indirect
github.com/expr-lang/expr v1.16.9 // indirect
github.com/fatih/color v1.15.0 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/fsnotify/fsnotify v1.8.0 // indirect
Expand All @@ -69,15 +72,20 @@ require (
github.com/google/uuid v1.6.0 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.23.0 // indirect
github.com/hashicorp/go-version v1.7.0 // indirect
github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect
github.com/holiman/uint256 v1.2.3 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/jonboulle/clockwork v0.4.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/compress v1.17.11 // indirect
github.com/klauspost/cpuid/v2 v2.2.8 // indirect
github.com/knadh/koanf/maps v0.1.1 // indirect
github.com/knadh/koanf/providers/confmap v0.1.0 // indirect
github.com/knadh/koanf/v2 v2.1.2 // indirect
github.com/leodido/go-syslog/v4 v4.2.0 // indirect
github.com/leodido/ragel-machinery v0.0.0-20190525184631-5f46317e436b // indirect
github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0 // indirect
github.com/magefile/mage v1.15.0 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.19 // indirect
github.com/mitchellh/copystructure v1.2.0 // indirect
Expand All @@ -86,12 +94,14 @@ require (
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/mostynb/go-grpc-compression v1.2.3 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/open-telemetry/opentelemetry-collector-contrib/internal/coreinternal v0.117.0 // indirect
github.com/open-telemetry/opentelemetry-collector-contrib/internal/grpcutil v0.117.0 // indirect
github.com/open-telemetry/opentelemetry-collector-contrib/internal/otelarrow v0.117.0 // indirect
github.com/open-telemetry/opentelemetry-collector-contrib/internal/sharedcomponent v0.117.0 // indirect
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza v0.117.0 // indirect
github.com/open-telemetry/otel-arrow v0.31.0 // indirect
github.com/pierrec/lz4/v4 v4.1.22 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/power-devops/perfstat v0.0.0-20210106213030-5aafc221ea8c // indirect
github.com/prometheus/client_golang v1.20.5 // indirect
github.com/prometheus/client_model v0.6.1 // indirect
Expand All @@ -105,6 +115,7 @@ require (
github.com/tg123/go-htpasswd v1.2.3 // indirect
github.com/tklauser/go-sysconf v0.3.12 // indirect
github.com/tklauser/numcpus v0.6.1 // indirect
github.com/valyala/fastjson v1.6.4 // indirect
github.com/vmihailenco/msgpack/v5 v5.3.5 // indirect
github.com/vmihailenco/tagparser/v2 v2.0.0 // indirect
github.com/x448/float16 v0.8.4 // indirect
Expand Down Expand Up @@ -180,11 +191,11 @@ require (
go.uber.org/zap v1.27.0 // indirect
golang.org/x/crypto v0.32.0 // indirect
golang.org/x/exp v0.0.0-20240506185415-9bf2ced13842 // indirect
golang.org/x/mod v0.18.0 // indirect
golang.org/x/mod v0.19.0 // indirect
golang.org/x/net v0.34.0 // indirect
golang.org/x/sync v0.10.0 // indirect
golang.org/x/text v0.21.0 // indirect
golang.org/x/tools v0.22.0 // indirect
golang.org/x/tools v0.23.0 // indirect
golang.org/x/xerrors v0.0.0-20231012003039-104605ab7028 // indirect
gonum.org/v1/gonum v0.15.1 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20241104194629-dd2ea8efbc28 // indirect
Expand All @@ -197,10 +208,6 @@ require (

replace github.com/open-telemetry/otel-arrow => ../../../

replace github.com/open-telemetry/otel-arrow/collector/exporter/fileexporter => ../../exporter/fileexporter

replace github.com/open-telemetry/otel-arrow/collector/receiver/filereceiver => ../../receiver/filereceiver

replace github.com/open-telemetry/otel-arrow/collector/processor/concurrentbatchprocessor => ../../processor/concurrentbatchprocessor

replace github.com/open-telemetry/otel-arrow/collector/processor/obfuscationprocessor => ../../processor/obfuscationprocessor
Loading
Loading