Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unify use of OpenMP for HNSW threading models #724

Open
wants to merge 6 commits into
base: branch-25.04
Choose a base branch
from
Open

Unify use of OpenMP for HNSW threading models #724

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
1e4d2e7
synchronize threading
divyegala Feb 25, 2025
620b7ed
unused var
divyegala Feb 25, 2025
431ece0
remove usage of thread pool in hnsw wrapper
divyegala Feb 26, 2025
80d0699
Merge remote-tracking branch 'upstream/branch-25.04' into hnsw-threading
divyegala Feb 26, 2025
b62983f
add missing header
divyegala Feb 27, 2025
cfb9732
Merge remote-tracking branch 'upstream/branch-25.04' into hnsw-threading
divyegala Feb 27, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 6 additions & 8 deletions cpp/bench/ann/src/hnswlib/hnswlib_wrapper.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,13 +66,13 @@ class hnsw_lib : public algo<T> {
struct build_param {
int m;
int ef_construction;
int num_threads = omp_get_num_procs();
int num_threads = omp_get_max_threads();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are the consequences of switching from omp_get_num_procs() to omp_get_max_threads() with respect to the SMT/hyperthreading?
Does hyperthreading make HNSW slower or faster?
Also, does OpenMP take into account whether the number of cores is limited by numactl?

Copy link
Member Author

@divyegala divyegala Feb 26, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Answering your questions one by one:

I don't see any changes to CMakeLists. Is the benchmark executable already linked with openmp?

All benchmark executables link to OpenMP already.

Can we now remove cpp/bench/ann/src/common/thread_pool.hpp completely?

No, unfortunately the FAISS wrappers still use it.

What are the consequences of switching from omp_get_num_procs() to omp_get_max_threads() with respect to the SMT/hyperthreading?

omp_get_num_procs() only returns number of physical cores. The difference in search times are very visibly apparent when accounting for thread usage to be all available hyperthreads vs just using physical cores. It makes HNSW faster.

Also, does OpenMP take into account whether the number of cores is limited by numactl?

No, it does not. We would either have to write a custom implementation or use some thread-pool library that can account for this. I think it would fall out of the scope of this PR and would be a general design philosophy discussion that we would have to undertake with the team, as we use OpenMP in quite a lot of places outside of HNSW.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the answers!

The difference in search times are very visibly apparent when accounting for thread usage to be all available hyperthreads vs just using physical cores. It makes HNSW faster.

Could you please add this to the comment above where we set this default?

Can we now remove cpp/bench/ann/src/common/thread_pool.hpp completely?

No, unfortunately the FAISS wrappers still use it

Would it be worth the effort to adjust FAISS wrappers to use OpenMP in this PR as well?

};

using search_param_base = typename algo<T>::search_param;
struct search_param : public search_param_base {
int ef;
int num_threads = 1;
int num_threads = omp_get_max_threads();
};

hnsw_lib(Metric metric, int dim, const build_param& param);
Expand Down Expand Up @@ -175,12 +175,7 @@ void hnsw_lib<T>::set_search_param(const search_param_base& param_, const void*
auto param = dynamic_cast<const search_param&>(param_);
appr_alg_->ef_ = param.ef;
num_threads_ = param.num_threads;
// bench_mode_ = param.metric_objective;
bench_mode_ = Mode::kLatency; // TODO(achirkin): pass the benchmark mode in the algo parameters

// Create a pool if multiple query threads have been set and the pool hasn't been created already
bool create_pool = (bench_mode_ == Mode::kLatency && num_threads_ > 1 && !thread_pool_);
if (create_pool) { thread_pool_ = std::make_shared<fixed_thread_pool>(num_threads_); }
}

template <typename T>
Expand All @@ -192,7 +187,10 @@ void hnsw_lib<T>::search(
get_search_knn_results(query + i * dim_, k, indices + i * k, distances + i * k);
};
if (bench_mode_ == Mode::kLatency && num_threads_ > 1) {
thread_pool_->submit(f, batch_size);
#pragma omp parallel for num_threads(num_threads_)
for (int i = 0; i < batch_size; i++) {
f(i);
}
} else {
for (int i = 0; i < batch_size; i++) {
f(i);
Expand Down
27 changes: 8 additions & 19 deletions cpp/src/neighbors/detail/hnsw.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -489,26 +489,15 @@ void search(raft::resources const& res,
auto const* hnswlib_index =
reinterpret_cast<hnswlib::HierarchicalNSW<typename hnsw_dist_t<T>::type> const*>(
idx.get_index());
auto num_threads = params.num_threads == 0 ? omp_get_max_threads() : params.num_threads;

// when num_threads == 0, automatically maximize parallelism
if (params.num_threads) {
#pragma omp parallel for num_threads(params.num_threads)
for (int64_t i = 0; i < queries.extent(0); ++i) {
get_search_knn_results(hnswlib_index,
queries.data_handle() + i * queries.extent(1),
neighbors.extent(1),
neighbors.data_handle() + i * neighbors.extent(1),
distances.data_handle() + i * distances.extent(1));
}
} else {
#pragma omp parallel for
for (int64_t i = 0; i < queries.extent(0); ++i) {
get_search_knn_results(hnswlib_index,
queries.data_handle() + i * queries.extent(1),
neighbors.extent(1),
neighbors.data_handle() + i * neighbors.extent(1),
distances.data_handle() + i * distances.extent(1));
}
#pragma omp parallel for num_threads(num_threads)
for (int64_t i = 0; i < queries.extent(0); ++i) {
get_search_knn_results(hnswlib_index,
queries.data_handle() + i * queries.extent(1),
neighbors.extent(1),
neighbors.data_handle() + i * neighbors.extent(1),
distances.data_handle() + i * distances.extent(1));
}
}

Expand Down
2 changes: 1 addition & 1 deletion python/cuvs_bench/cuvs_bench/config/algos/cuvs_cagra_hnswlib.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ groups:
graph_degree: [32, 64, 96, 128]
intermediate_graph_degree: [32, 64, 96, 128]
graph_build_algo: ["NN_DESCENT"]
hierarchy: ["none", "cpu"]
hierarchy: ["none", "cpu", "gpu"]
ef_construction: [64, 128, 256, 512]
search:
ef: [10, 20, 40, 60, 80, 120, 200, 400, 600, 800]
Expand Down
Loading