Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for predefined consumers of CDC topics #119

Merged

Conversation

Eistern
Copy link
Contributor

@Eistern Eistern commented Jan 20, 2025

In this PR, we are adding an option to specify a list of consumers that will be created alongside the CDC topic during the initial bootstrap of the application.

A minor consideration is that TopicService/AlterTopic is not idempotent and will throw an exception if we specify already existing consumers in the addConsumer block. To address this behavior, we will perform a describe operation each time a table is attempted to be created.

These changes have been tested against local YDB (ydbplatform/local-ydb:latest version).

@Eistern Eistern changed the title Support for predefined consumers for CDC topics Support for predefined consumers of CDC topics Jan 20, 2025
@Eistern
Copy link
Contributor Author

Eistern commented Jan 20, 2025

This update replicates terraform resource behavior (https://ydb.tech/docs/en/dev/terraform#ydb-table-changefeed) and allows the user to implement non-trivial CDC reading logic without worrying about creation of the consumers

@nvamelichev nvamelichev merged commit 6aaa1a3 into ydb-platform:main Jan 21, 2025
1 check passed
@Eistern Eistern deleted the feature/predefined-consumers-for-cdc branch January 22, 2025 10:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants