Add a FAISS_OPT_LEVEL=avx512_spr build mode. #2404

mulugetam · 2025-01-17T20:50:46Z

Description

FAISS recently added a new build mode, avx512_spr, which enables the use of advanced AVX512 instructions available since Intel® Sapphire Rapids. These instructions include avx512_fp16, avx512_bf16, avx512_vpopcntdq, and many others. Some optimizations leveraging these instructions, such as speeding up Hamming distance evaluation for binary vectors, have already been implemented and merged. Additional optimizations are also in the pipeline. This PR enables K-NN to utilize this new avx512_spr build mode.

The code changes are implemented such that, by default, if the underlying system is an Intel® Sapphire Rapids or a newer-generation processor, avx512_spr binaries are generated. Otherwise, depending on the features available in the system and consistent with the existing implementation, avx512, avx2, or generic binaries may be generated.

Check List

New functionality includes testing.
New functionality has been documented.
API changes companion pull request created.
Commits are signed per the DCO using --signoff.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

- avx512_spr enables advanced avx512 instructions available since Intel(R) Sapphire Rapids. Signed-off-by: Mulugeta Mammo <mulugeta.mammo@intel.com>

Signed-off-by: Assane Diop <assane.diop@intel.com>

naveentatikonda · 2025-01-17T21:51:14Z

DEVELOPER_GUIDE.md

+# else if (system_supports_avx512) generate_avx512_binaries()
+# else if (system_supports_ avx2) generate_avx2_binaries()
+# else() generate_generic_binaries()
+./gradlew build -Davx2.enabled=true


I think we can remove this line

You mean lines 294-298?

no, line 298

scripts/build.sh

naveentatikonda · 2025-01-17T22:01:51Z

@mulugetam Thanks for making these changes. Can you fix the failing CI and add CHANGELOG

mulugetam · 2025-01-17T23:03:07Z

@naveentatikonda Looks like the CI is using gcc 10.5.0 (we need 12.1+). Will proceed with fixing the other issues (which are mainly formatting).

Co-authored-by: Naveen Tatikonda <navtat@amazon.com> Signed-off-by: mulugetam <mulugeta.mammo@intel.com>

Signed-off-by: Mulugeta Mammo <mulugeta.mammo@intel.com>

.github/workflows/CI.yml

naveentatikonda · 2025-01-21T17:58:33Z

@mulugetam also pls fix the failing unit test related to avx512_spr. Thanks!

scripts/build.sh

Signed-off-by: Mulugeta Mammo <mulugeta.mammo@intel.com>

mulerm · 2025-01-21T22:53:16Z

Thanks! Could you re-run CI?

Signed-off-by: Naveen Tatikonda <navtat@amazon.com>

naveentatikonda · 2025-01-22T20:01:27Z

I tried to cherry-pick these changes and built them on Peter's custom image with GCC 12.4 without any compile-time or runtime issues when tested with some sample curl commands.

naveentatikonda · 2025-01-22T20:08:02Z

BWC framework is flaky and the failing BWC tests not related to these changes

peterzhuamazon · 2025-01-22T20:23:16Z

Pending opensearch-project/opensearch-build#5241 to be merged and images to be pushed before new tests would take effect.

naveentatikonda · 2025-01-22T22:01:13Z

@mulugetam can you pls share some performance benchmarking results when tested from opensearch comparing avx512_spr results with avx512 and avx2 atleast for a couple of datasets ?

mulugetam · 2025-01-22T22:18:46Z

@naveentatikonda We're currently working on that and will share it when its ready. For a quick check, you may run hamming bench with avx512_spr.

peterzhuamazon · 2025-01-24T18:36:22Z

New image is online we can retry the test on this PR.

Thanks.

* Add avx512_spr option to FAISS_OPT_LEVEL. - avx512_spr enables advanced avx512 instructions available since Intel(R) Sapphire Rapids. Signed-off-by: Mulugeta Mammo <mulugeta.mammo@intel.com> * Fix documentation. Signed-off-by: Assane Diop <assane.diop@intel.com> * Update scripts/build.sh Co-authored-by: Naveen Tatikonda <navtat@amazon.com> Signed-off-by: mulugetam <mulugeta.mammo@intel.com> * Update scripts/build.sh Co-authored-by: Naveen Tatikonda <navtat@amazon.com> Signed-off-by: mulugetam <mulugeta.mammo@intel.com> * Fix formatting issues. Signed-off-by: Mulugeta Mammo <mulugeta.mammo@intel.com> * Update CHANGELOG.md for avx512_spr build mode. Signed-off-by: Mulugeta Mammo <mulugeta.mammo@intel.com> * Fix bugs in build options and avx512_spr flag testing. Signed-off-by: Mulugeta Mammo <mulugeta.mammo@intel.com> * Upgrade gcc version to 12.4. Signed-off-by: Mulugeta Mammo <mulugeta.mammo@intel.com> --------- Signed-off-by: Mulugeta Mammo <mulugeta.mammo@intel.com> Signed-off-by: Assane Diop <assane.diop@intel.com> Signed-off-by: mulugetam <mulugeta.mammo@intel.com> Signed-off-by: Naveen Tatikonda <navtat@amazon.com> Co-authored-by: Assane Diop <assane.diop@intel.com> Co-authored-by: Naveen Tatikonda <navtat@amazon.com> (cherry picked from commit 90fdad4)

* Add a FAISS_OPT_LEVEL=avx512_spr build mode. (#2404) * Add avx512_spr option to FAISS_OPT_LEVEL. - avx512_spr enables advanced avx512 instructions available since Intel(R) Sapphire Rapids. Signed-off-by: Mulugeta Mammo <mulugeta.mammo@intel.com> * Fix documentation. Signed-off-by: Assane Diop <assane.diop@intel.com> * Update scripts/build.sh Co-authored-by: Naveen Tatikonda <navtat@amazon.com> Signed-off-by: mulugetam <mulugeta.mammo@intel.com> * Update scripts/build.sh Co-authored-by: Naveen Tatikonda <navtat@amazon.com> Signed-off-by: mulugetam <mulugeta.mammo@intel.com> * Fix formatting issues. Signed-off-by: Mulugeta Mammo <mulugeta.mammo@intel.com> * Update CHANGELOG.md for avx512_spr build mode. Signed-off-by: Mulugeta Mammo <mulugeta.mammo@intel.com> * Fix bugs in build options and avx512_spr flag testing. Signed-off-by: Mulugeta Mammo <mulugeta.mammo@intel.com> * Upgrade gcc version to 12.4. Signed-off-by: Mulugeta Mammo <mulugeta.mammo@intel.com> --------- Signed-off-by: Mulugeta Mammo <mulugeta.mammo@intel.com> Signed-off-by: Assane Diop <assane.diop@intel.com> Signed-off-by: mulugetam <mulugeta.mammo@intel.com> Signed-off-by: Naveen Tatikonda <navtat@amazon.com> Co-authored-by: Assane Diop <assane.diop@intel.com> Co-authored-by: Naveen Tatikonda <navtat@amazon.com> (cherry picked from commit 90fdad4) * Fix failing rolling upgrade workflow Signed-off-by: Naveen Tatikonda <navtat@amazon.com> * Update init-faiss.cmake Signed-off-by: Naveen Tatikonda <navtat@amazon.com> --------- Signed-off-by: Naveen Tatikonda <navtat@amazon.com> Co-authored-by: mulugetam <mulugeta.mammo@intel.com> Co-authored-by: Naveen Tatikonda <navtat@amazon.com>

* Add avx512_spr option to FAISS_OPT_LEVEL. - avx512_spr enables advanced avx512 instructions available since Intel(R) Sapphire Rapids. Signed-off-by: Mulugeta Mammo <mulugeta.mammo@intel.com> * Fix documentation. Signed-off-by: Assane Diop <assane.diop@intel.com> * Update scripts/build.sh Co-authored-by: Naveen Tatikonda <navtat@amazon.com> Signed-off-by: mulugetam <mulugeta.mammo@intel.com> * Update scripts/build.sh Co-authored-by: Naveen Tatikonda <navtat@amazon.com> Signed-off-by: mulugetam <mulugeta.mammo@intel.com> * Fix formatting issues. Signed-off-by: Mulugeta Mammo <mulugeta.mammo@intel.com> * Update CHANGELOG.md for avx512_spr build mode. Signed-off-by: Mulugeta Mammo <mulugeta.mammo@intel.com> * Fix bugs in build options and avx512_spr flag testing. Signed-off-by: Mulugeta Mammo <mulugeta.mammo@intel.com> * Upgrade gcc version to 12.4. Signed-off-by: Mulugeta Mammo <mulugeta.mammo@intel.com> --------- Signed-off-by: Mulugeta Mammo <mulugeta.mammo@intel.com> Signed-off-by: Assane Diop <assane.diop@intel.com> Signed-off-by: mulugetam <mulugeta.mammo@intel.com> Signed-off-by: Naveen Tatikonda <navtat@amazon.com> Co-authored-by: Assane Diop <assane.diop@intel.com> Co-authored-by: Naveen Tatikonda <navtat@amazon.com>

naveentatikonda · 2025-02-12T17:59:55Z

Benchmarking Results Comparing Advanced AVX512(avx512_spr) and AVX512

mulugetam and others added 3 commits January 17, 2025 07:20

Add avx512_spr option to FAISS_OPT_LEVEL.

effdaa7

- avx512_spr enables advanced avx512 instructions available since Intel(R) Sapphire Rapids. Signed-off-by: Mulugeta Mammo <mulugeta.mammo@intel.com>

Merge branch 'opensearch-project:main' into avx512_spr

91e3588

Fix documentation.

21b4eff

Signed-off-by: Assane Diop <assane.diop@intel.com>

mulugetam requested review from heemin32, navneet1v, VijayanB, vamshin, jmazanec15, naveentatikonda, junqiu-lei, martin-gaievski, ryanbogan, luyuncheng, shatejas and 0ctopus13prime as code owners January 17, 2025 20:50

naveentatikonda reviewed Jan 17, 2025

View reviewed changes

naveentatikonda added backport 2.x Features Introduces a new unit of functionality that satisfies a requirement labels Jan 17, 2025

mulugetam and others added 4 commits January 17, 2025 16:03

Update scripts/build.sh

df235bd

Co-authored-by: Naveen Tatikonda <navtat@amazon.com> Signed-off-by: mulugetam <mulugeta.mammo@intel.com>

Update scripts/build.sh

fca92cb

Co-authored-by: Naveen Tatikonda <navtat@amazon.com> Signed-off-by: mulugetam <mulugeta.mammo@intel.com>

Fix formatting issues.

d1d69a8

Signed-off-by: Mulugeta Mammo <mulugeta.mammo@intel.com>

Update CHANGELOG.md for avx512_spr build mode.

fe86175

Signed-off-by: Mulugeta Mammo <mulugeta.mammo@intel.com>

naveentatikonda reviewed Jan 21, 2025

View reviewed changes

.github/workflows/CI.yml Show resolved Hide resolved

naveentatikonda reviewed Jan 21, 2025

View reviewed changes

scripts/build.sh Outdated Show resolved Hide resolved

naveentatikonda reviewed Jan 21, 2025

View reviewed changes

scripts/build.sh Show resolved Hide resolved

mulugetam added 2 commits January 21, 2025 15:47

Fix bugs in build options and avx512_spr flag testing.

19e6864

Signed-off-by: Mulugeta Mammo <mulugeta.mammo@intel.com>

Upgrade gcc version to 12.4.

04a2d74

Signed-off-by: Mulugeta Mammo <mulugeta.mammo@intel.com>

Merge branch 'main' into avx512_spr

ce7bf16

Signed-off-by: Naveen Tatikonda <navtat@amazon.com>

naveentatikonda approved these changes Jan 22, 2025

View reviewed changes

naveentatikonda mentioned this pull request Jan 22, 2025

Bump up GCC version to 12.x or later to support SIMD AVX512 fp16 instructions opensearch-project/opensearch-build#5226

Closed

naveentatikonda added the v2.19.0 label Jan 23, 2025

naveentatikonda mentioned this pull request Jan 23, 2025

[DOC] Update k-NN plugin documentation to reflect AVX512_spr SIMD optimization update opensearch-project/documentation-website#9108

Closed

4 tasks

navneet1v approved these changes Jan 24, 2025

View reviewed changes

naveentatikonda merged commit 90fdad4 into opensearch-project:main Jan 24, 2025
33 of 34 checks passed

opensearch-trigger-bot bot mentioned this pull request Jan 24, 2025

[Backport 2.x] Add a FAISS_OPT_LEVEL=avx512_spr build mode. #2437

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a FAISS_OPT_LEVEL=avx512_spr build mode. #2404

Add a FAISS_OPT_LEVEL=avx512_spr build mode. #2404

mulugetam commented Jan 17, 2025

naveentatikonda Jan 17, 2025

mulugetam Jan 17, 2025

naveentatikonda Jan 17, 2025

mulugetam Jan 17, 2025

naveentatikonda commented Jan 17, 2025

mulugetam commented Jan 17, 2025

naveentatikonda commented Jan 21, 2025

mulerm commented Jan 21, 2025

naveentatikonda commented Jan 22, 2025

naveentatikonda commented Jan 22, 2025

peterzhuamazon commented Jan 22, 2025

naveentatikonda commented Jan 22, 2025

mulugetam commented Jan 22, 2025

peterzhuamazon commented Jan 24, 2025

naveentatikonda commented Feb 12, 2025

Add a FAISS_OPT_LEVEL=avx512_spr build mode. #2404

Add a FAISS_OPT_LEVEL=avx512_spr build mode. #2404

Conversation

mulugetam commented Jan 17, 2025

Description

Check List

naveentatikonda Jan 17, 2025

Choose a reason for hiding this comment

mulugetam Jan 17, 2025

Choose a reason for hiding this comment

naveentatikonda Jan 17, 2025

Choose a reason for hiding this comment

mulugetam Jan 17, 2025

Choose a reason for hiding this comment

naveentatikonda commented Jan 17, 2025

mulugetam commented Jan 17, 2025

naveentatikonda commented Jan 21, 2025

mulerm commented Jan 21, 2025

naveentatikonda commented Jan 22, 2025

naveentatikonda commented Jan 22, 2025

peterzhuamazon commented Jan 22, 2025

naveentatikonda commented Jan 22, 2025

mulugetam commented Jan 22, 2025

peterzhuamazon commented Jan 24, 2025

naveentatikonda commented Feb 12, 2025

Benchmarking Results Comparing Advanced AVX512(avx512_spr) and AVX512