Optimization of PushRowPage for high number of cpu cores #11182
Conversation
Here is the speed-up in comparison with the current master branch. As for unit-tests, it is not easy to cover such a case. I will try to figure it out. |
|
One particular caution about similar optimizations, thread local buffers can cost significant amount of memory usage when the core count is high. For large server CPUs with hundreds of CPUs this can be severe. We partially reverted a similar optimization in the CPU predictor which uses thread local blocks due to this reason. |
|
Ref #6659 |
…om/razdoburdin/xgboost into dev/cpu/push_row_page_optimisation
|
Sorry for a long delay. I have reorganized the optimization to reduce memory overhead (see the PR description for some details). I also added some tests to verify the changes. |
|
Thank you for continuing the optimization work. Could you please share the latest profiling result including performance and memory usage on the datasets that you are targeting? |
Here is the benchmarking for the last version of the PR. The numbers in the table >1 mean that this PR is faster than master branch, and consuming more memory. |
| size_t n_entries = entries_per_columns[column_idx]; | ||
|
|
||
| if (n_entries > 0) { | ||
| size_t n_splits = std::min(nthreads * n_entries / total_entries, n_entries); |
There was a problem hiding this comment.
Why is n_splits calculated in this way?
There was a problem hiding this comment.
The goal is to assign for processing of the columns amount of threads being proportional to number of entries in the column.
|
|
||
| std::vector<ThreadWorkLoad> baskets; | ||
| std::vector<bool> is_column_splited; | ||
| bool has_splitted = false; |
| typename WQSketch::SummaryContainer main_summary; | ||
| main_summary.Reserve(sketches_[column_idx].limit_size); | ||
| typename WQSketch::SummaryContainer split_summary; | ||
| split_summary.Reserve(2 * sketches_[column_idx].limit_size); | ||
| typename WQSketch::SummaryContainer comb_summary; | ||
| comb_summary.Reserve(3 * sketches_[column_idx].limit_size); |
There was a problem hiding this comment.
How are these 2 * and 3 * sizes calculated?
There was a problem hiding this comment.
The size of split_summary size guaranties absence of intermediate pruning while calculating summary. While the size of comb_summary is the sum of sizes of main_summary and split_summary
Co-authored-by: Jiaming Yuan <jm.yuan@outlook.com>
I suggest we postpone this PR and revisit it once other hotspots are resolved—at that point, the importance of this fix will become more significant. |
|
Thank you for sharing. Yes, please take your time and feel free to open refactoring PRs before optimization PRs. This way we can (hopefully) build some abstractions and utilities/helpers before optimization. |
This PR adds optimal row-wise parallelization for
PushRowPage.The current realization is ineffective for multithread CPUs in case of a dataset has large number of rows and small number of columns. Current PR fixes this bottleneck. I observed up to 15% perf improvements for large datasets like airline on system with 2x56 cores CPU.
upg:
Some illustrations of changes in parallel processing:
We have a sparse input matrix. Each cell corresponds to a feature, and we process each cell individually, placing the results into the output array.
The original implementation processes data column by column. This approach is suboptimal when there are many CPU cores but only a few columns.
In the initial optimization, each thread processed a subset of rows but included all columns. This approach required significant memory overhead because buffers for intermediate results had to be used.
In the current version, I have reorganized the logic. Now, the input matrix is analyzed by the
LoadBalancefunction. If a column is too large, it is split across multiple threads. This reduces buffer sizes since each thread now processes only a single feature.