Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AGENT-11268] Add a performance tuning note about increasing records per poll #44

Merged
merged 3 commits into from
Apr 16, 2024
+10 −3
Merged

[AGENT-11268] Add a performance tuning note about increasing records per poll #44

Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c7f8bc7
[AGENT-11268] Add a performance tuning note about increasing records …
jszwedko Apr 12, 2024
36ac51f
Update README.md
jszwedko Apr 14, 2024
30d08db
Update README.md
jszwedko Apr 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 10 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,9 +111,16 @@ A REST call can be executed against one of the cluster instances, and the config

To improve performance of the connector, you can try the following options:

* Increase the number of parallel tasks by adjusting the `tasks.max` parameter. Only do this if the hardware is
underutilized, such as low CPU, low memory usage and low data injection throughput. Do not set more tasks than partitions.
* Increase hardware resources on cluster nodes in case of resource exhaustion, such as high CPU, or high memory usage.
* Update the number of records fetched per poll by setting
`consumer.override.max.poll.records` in the plugin configuration. This plugin
sends batches of records synchronously with each poll so a low number of records
per poll will reduce throughput. Consider setting this to 500 or 1000.
Comment on lines +114 to +117
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking, why are sentences broken up like this? Is it a formatting thing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, yeah, this was just word wrapping. When it is rendered as markdown, it'll all be one line that overflows.

* Increase the number of parallel tasks by adjusting the `tasks.max` parameter.
Only do this if the hardware is underutilized, such as low CPU, low memory
usage and low data injection throughput. Do not set more tasks than
jszwedko marked this conversation as resolved.
Show resolved Hide resolved
partitions.
* Increase hardware resources on cluster nodes in case of resource exhaustion,
such as high CPU, or high memory usage.
* Increase the number of Kafka Connect nodes.

## Single Message Transforms
Expand Down