Merge connector repositories #242
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jjaakola-aiven
force-pushed
the
jjaakola-aiven-merge-connector-repositories-2024-06-17
branch
3 times, most recently
from
b588ff2 to
4030046
Compare
|
Does it make sense to move the connectors to connectors folders like so in future all connectors could be managed under that dir ? |
|
Am trying to build locally on mac, but fails with the below. Could it be unrelated ? |
as they are individual modules, we can keep as is, but could we end up in more connectors in root |
Added 6 new properties to control back off policies: - kafka.retry.backoff.ms - gcs.retry.backoff.initial.delay.ms - gcs.retry.backoff.max.delay.ms - gcs.retry.backoff.delay.multiplier - gcs.retry.backoff.total.timeout.ms - gcs.retry.backoff.max.attempts
It's incorrect to say `Aiven Kafka GCS Connector` for the trademark reason.
partiiton => partition enbale => enable
Bumps `testcontainersVersion` from 1.19.4 to 1.19.6. Updates `org.testcontainers:junit-jupiter` from 1.19.4 to 1.19.6 - [Release notes](https://github.com/testcontainers/testcontainers-java/releases) - [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md) - [Commits](testcontainers/testcontainers-java@1.19.4...1.19.6) Updates `org.testcontainers:kafka` from 1.19.4 to 1.19.6 - [Release notes](https://github.com/testcontainers/testcontainers-java/releases) - [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md) - [Commits](testcontainers/testcontainers-java@1.19.4...1.19.6) --- updated-dependencies: - dependency-name: org.testcontainers:junit-jupiter dependency-type: direct:production update-type: version-update:semver-patch - dependency-name: org.testcontainers:kafka dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [org.assertj:assertj-core](https://github.com/assertj/assertj) from 3.25.2 to 3.25.3. - [Release notes](https://github.com/assertj/assertj/releases) - [Commits](assertj/assertj@assertj-build-3.25.2...assertj-build-3.25.3) --- updated-dependencies: - dependency-name: org.assertj:assertj-core dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [com.google.cloud:google-cloud-nio](https://github.com/googleapis/java-storage-nio) from 0.127.7 to 0.127.13. - [Release notes](https://github.com/googleapis/java-storage-nio/releases) - [Changelog](https://github.com/googleapis/java-storage-nio/blob/main/CHANGELOG.md) - [Commits](googleapis/java-storage-nio@v0.127.7...v0.127.13) --- updated-dependencies: - dependency-name: com.google.cloud:google-cloud-nio dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [com.google.cloud:google-cloud-storage](https://github.com/googleapis/java-storage) from 2.32.1 to 2.36.1. - [Release notes](https://github.com/googleapis/java-storage/releases) - [Changelog](https://github.com/googleapis/java-storage/blob/main/CHANGELOG.md) - [Commits](googleapis/java-storage@v2.32.1...v2.36.1) --- updated-dependencies: - dependency-name: com.google.cloud:google-cloud-storage dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [com.google.cloud:google-cloud-nio](https://github.com/googleapis/java-storage-nio) from 0.127.13 to 0.127.15. - [Release notes](https://github.com/googleapis/java-storage-nio/releases) - [Changelog](https://github.com/googleapis/java-storage-nio/blob/main/CHANGELOG.md) - [Commits](googleapis/java-storage-nio@v0.127.13...v0.127.15) --- updated-dependencies: - dependency-name: com.google.cloud:google-cloud-nio dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps com.github.spotbugs from 6.0.7 to 6.0.9. --- updated-dependencies: - dependency-name: com.github.spotbugs dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [net.jqwik:jqwik](https://github.com/jqwik-team/jqwik) from 1.8.2 to 1.8.4. - [Release notes](https://github.com/jqwik-team/jqwik/releases) - [Commits](jqwik-team/jqwik@1.8.2...1.8.4) --- updated-dependencies: - dependency-name: net.jqwik:jqwik dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps com.github.spotbugs from 6.0.9 to 6.0.13. --- updated-dependencies: - dependency-name: com.github.spotbugs dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [com.github.spotbugs:spotbugs-annotations](https://github.com/spotbugs/spotbugs) from 4.8.3 to 4.8.4. - [Release notes](https://github.com/spotbugs/spotbugs/releases) - [Changelog](https://github.com/spotbugs/spotbugs/blob/master/CHANGELOG.md) - [Commits](spotbugs/spotbugs@4.8.3...4.8.4) --- updated-dependencies: - dependency-name: com.github.spotbugs:spotbugs-annotations dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [com.google.cloud:google-cloud-nio](https://github.com/googleapis/java-storage-nio) from 0.127.15 to 0.127.16. - [Release notes](https://github.com/googleapis/java-storage-nio/releases) - [Changelog](https://github.com/googleapis/java-storage-nio/blob/main/CHANGELOG.md) - [Commits](googleapis/java-storage-nio@v0.127.15...v0.127.16) --- updated-dependencies: - dependency-name: com.google.cloud:google-cloud-nio dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [com.google.cloud:google-cloud-storage](https://github.com/googleapis/java-storage) from 2.36.1 to 2.37.0. - [Release notes](https://github.com/googleapis/java-storage/releases) - [Changelog](https://github.com/googleapis/java-storage/blob/main/CHANGELOG.md) - [Commits](googleapis/java-storage@v2.36.1...v2.37.0) --- updated-dependencies: - dependency-name: com.google.cloud:google-cloud-storage dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [org.junit.jupiter:junit-jupiter](https://github.com/junit-team/junit5) from 5.10.1 to 5.10.2. - [Release notes](https://github.com/junit-team/junit5/releases) - [Commits](junit-team/junit5@r5.10.1...r5.10.2) --- updated-dependencies: - dependency-name: org.junit.jupiter:junit-jupiter dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>
Users willing to leverage GCS capability to decompress gzip objects on server-side when accessing them through the Storage API requested the fixed-metadata `Content-Encoding` (default: null) to become configurable so that its value can be set (ie. to `gzip`) when the connector uploads a new file to the bucket. https://cloud.google.com/storage/docs/metadata#content-encoding
Changes necessary for S3 connector to build.
Changes necessary for GCS connector to build. Fails on checkstyle, which will be removed as tooling and replaced with tooling from the GCS connector already uses (spotbugs, spotless and pmd)
Removed unnecesary files from submodules.
nosahama
force-pushed
the
jjaakola-aiven-merge-connector-repositories-2024-06-17
branch
from
4030046 to
95ca809
Compare
nosahama
force-pushed
the
jjaakola-aiven-merge-connector-repositories-2024-06-17
branch
from
95ca809 to
4acce53
Compare
Performing the Diffs on the various JARs using the Jarcomp toolThe strategy for both connectors was to build the connector from the existing repository, naming it as ( We then perform the diff on the JARs using this command: GCS Connector Diff
S3 Connector Diff
|
|
After running the i.e.
|
nosahama
approved these changes
Jul 5, 2024
nosahama
deleted the
jjaakola-aiven-merge-connector-repositories-2024-06-17
branch
|
🥇 thank you 👍 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Supersedes #220
The change and release process for the dependent repositories is hard to manage. This PR proposes to move GCS and S3 connector development to this repository.
Repository merge is done with --allow-unrelated-histories and the history of GCS and S3 repositories is available: $ git log --follow
Items not done:
[ ] Cleaning of dependencies.
[ ] Versioning of components.
[ ] Release process and GH release workflow files.
[ ] Multi-module Jacoco report.
[ ] Integration test convention, some duplication in GCS and S3 build files.
Items to do after merging:
[ ] Update GCS repository with link to this repository.
[ ] Transfer issue from GCS repository to this repository.
[ ] Set GCS repository read-only.
[ ] Update S3 repository with link to this repository.
[ ] Transfer issue from S3 repository to this repository.
[ ] Set S3 repository read-only.
[ ] Update connector urls in files.
[ ] Update connector build sources in
aiven-coredeps, this requires us to agree on a release strategy for the consolidated connectors.[ ] Rename this repository.