Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add vector search documentation #9135

Open
wants to merge 36 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 33 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
4331ce1
Add vector database section
kolchfa-aws Jan 15, 2025
db9e95e
More restructuring
kolchfa-aws Jan 16, 2025
3d93fc8
Layout update
kolchfa-aws Jan 16, 2025
47b235f
Add cards to search topics
kolchfa-aws Jan 21, 2025
2ac67d9
More restructuring
kolchfa-aws Jan 23, 2025
99ac17a
Add images and more rewrites
kolchfa-aws Jan 27, 2025
27107c8
Update settings
kolchfa-aws Jan 28, 2025
f009e68
Formatting update
kolchfa-aws Jan 28, 2025
8779a7c
Resolve merge conflicts
kolchfa-aws Jan 29, 2025
6cdb7b0
Add more explanations to getting started
kolchfa-aws Feb 10, 2025
f922fae
Change about page and remove k-NN terminology
kolchfa-aws Feb 11, 2025
1900e75
unify terminology
kolchfa-aws Feb 11, 2025
a5e8b8d
Review comments
kolchfa-aws Feb 11, 2025
3f5301a
More review comments
kolchfa-aws Feb 12, 2025
18b40af
Resolve merge conflicts
kolchfa-aws Feb 12, 2025
8b8d831
Resolve merge conflicts
kolchfa-aws Feb 12, 2025
d977108
More restructuring
kolchfa-aws Feb 14, 2025
f73a161
Remove table from exact search
kolchfa-aws Feb 14, 2025
8fde05d
Fix links
kolchfa-aws Feb 14, 2025
7f549c4
Merge branch 'main' into vector-restructure
kolchfa-aws Feb 14, 2025
73f4fd7
Remove text from top page
kolchfa-aws Feb 14, 2025
e2a1cf9
Merge branch 'vector-restructure' of https://github.com/opensearch-pr…
kolchfa-aws Feb 14, 2025
0be522f
More updates
kolchfa-aws Feb 17, 2025
63c0d97
Update _query-dsl/specialized/kNN.md
kolchfa-aws Feb 17, 2025
79eeff1
Review comments
kolchfa-aws Feb 17, 2025
9d42335
Fix links
kolchfa-aws Feb 17, 2025
a6a1cfb
Fix links
kolchfa-aws Feb 17, 2025
ad6fb3c
Rename query file
kolchfa-aws Feb 17, 2025
18e10df
Fix links
kolchfa-aws Feb 17, 2025
c007e46
Add sparse vector option
kolchfa-aws Feb 17, 2025
e2a7427
Compress requests
kolchfa-aws Feb 17, 2025
8d4ffe4
Apply suggestions from code review
kolchfa-aws Feb 19, 2025
e19af36
Apply suggestions from code review
kolchfa-aws Feb 19, 2025
8c49459
Apply suggestions from code review
kolchfa-aws Feb 19, 2025
0ca38f0
Editorial comments
kolchfa-aws Feb 20, 2025
e8595c3
Reformat concepts page
kolchfa-aws Feb 20, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 49 additions & 54 deletions _about/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,51 @@ redirect_from:
- /docs/opensearch/
- /opensearch/
- /opensearch/index/
why_use:
- heading: "Vector database"
description: "Use OpenSearch as a vector database to combine the power of traditional search, analytics, and vector search."
link: "/vector-search/"
- heading: "Fast, scalable full-text search"
description: "Help users find the right information in your application, website, or data lake catalog."
link: "/search-plugins/"
- heading: "Application and infrastructure monitoring"
description: "Use observability logs, metrics, and traces to monitor your applications in real time."
link: "/observing-your-data/"
- heading: "Security and event information management"
description: "Centralize logs to enable real-time security monitoring and forensic analysis."
link: "/security/"
features:
- heading: "Vector search"
description: "Build AI/ML-powered vector search applications using semantic, hybrid, or conversational search and more."
link: "/vector-search/"
- heading: "Machine learning"
description: "Integrate machine learning models into your workloads."
link: "/ml-commons-plugin/"
- heading: "Customizing your search"
description: "From optimizing performance to improving relevance, customize your search experience."
link: "/search-plugins/"
- heading: "Workflow automation"
description: "Automate complex OpenSearch setup and preprocessing tasks."
link: "/automating-configurations/"
- heading: "Anomaly detection"
description: "Identify atypical data and receive automatic notifications."
link: "/monitoring-plugins/ad/"
- heading: "Building visualizations"
description: "Visualize your data in OpenSearch Dashboards."
link: "/dashboards/"
getting_started:
- heading: "Get started with OpenSearch"
description: "Learn about OpenSearch and start ingesting and searching data."
link: "/getting-started/"
- heading: "Get started with OpenSearch Dashboards"
description: "Learn about OpenSearch Dashboards applications and tools used to visualize data."
link: "/dashboards/quickstart/"
- heading: "Get started with vector search"
description: "Learn about vector search options and build your first vector search application."
link: "/search-plugins/"
- heading: "Get started with OpenSearch security"
description: "Learn about security in OpenSearch."
link: "/getting-started/security/"
---

{%- comment -%}The `/docs/opensearch/` redirect is specifically to support the UI links in OpenSearch Dashboards 1.0.0.{%- endcomment -%}
Expand All @@ -22,70 +67,20 @@ This section contains documentation for OpenSearch and OpenSearch Dashboards.

## Getting started

To get started, explore the following documentation:

- [Getting started guide]({{site.url}}{{site.baseurl}}/getting-started/):
- [Intro to OpenSearch]({{site.url}}{{site.baseurl}}/getting-started/intro/)
- [Installation quickstart]({{site.url}}{{site.baseurl}}/getting-started/quickstart/)
- [Communicate with OpenSearch]({{site.url}}{{site.baseurl}}/getting-started/communicate/)
- [Ingest data]({{site.url}}{{site.baseurl}}/getting-started/ingest-data/)
- [Search data]({{site.url}}{{site.baseurl}}/getting-started/search-data/)
- [Getting started with OpenSearch security]({{site.url}}{{site.baseurl}}/getting-started/security/)
- [Install OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/install-opensearch/index/)
- [Install OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/install-and-configure/install-dashboards/index/)
- [FAQ](https://opensearch.org/faq)
{% include cards.html cards=page.getting_started %}

## Why use OpenSearch?

<table style="table-layout: auto ; width: 100%;">
<tbody>
<tr style="text-align: center; vertical-align:center;">
<td><img src="{{site.url}}{{site.baseurl}}/images/1_search.png" class="no-border" alt="Fast, scalable full-text search" height="100"/></td>
<td><img src="{{site.url}}{{site.baseurl}}/images/2_monitoring.png" class="no-border" alt="Application and infrastructure monitoring" height="100"/></td>
<td><img src="{{site.url}}{{site.baseurl}}/images/3_security.png" class="no-border" alt="Security and event information management" height="100"/></td>
<td><img src="{{site.url}}{{site.baseurl}}/images/4_tracking.png" class="no-border" alt="Operational health tracking" height="100"/></td>
</tr>
<tr style="text-align: left; vertical-align:top; font-weight: bold; color: rgb(0,59,92)">
<td>Fast, scalable full-text search</td>
<td>Application and infrastructure monitoring</td>
<td>Security and event information management</td>
<td>Operational health tracking</td>
</tr>
<tr style="text-align: left; vertical-align:top;">
<td>Help users find the right information within your application, website, or data lake catalog. </td>
<td>Easily store and analyze log data, and set automated alerts for performance issues.</td>
<td>Centralize logs to enable real-time security monitoring and forensic analysis.</td>
<td>Use observability logs, metrics, and traces to monitor your applications in real time.</td>
</tr>
</tbody>
</table>
{% include cards.html cards=page.why_use documentation_link=true %}

## Key features

OpenSearch provides several features to help index, secure, monitor, and analyze your data:

- [Anomaly detection]({{site.url}}{{site.baseurl}}/monitoring-plugins/ad/) -- Identify atypical data and receive automatic notifications.
- [SQL]({{site.url}}{{site.baseurl}}/search-plugins/sql/index/) -- Use SQL or a Piped Processing Language (PPL) to query your data.
- [Index State Management]({{site.url}}{{site.baseurl}}/im-plugin/) -- Automate index operations.
- [Search methods]({{site.url}}{{site.baseurl}}/search-plugins/knn/) -- From traditional lexical search to advanced vector and hybrid search, discover the optimal search method for your use case.
- [Machine learning]({{site.url}}{{site.baseurl}}/ml-commons-plugin/index/) -- Integrate machine learning models into your workloads.
- [Workflow automation]({{site.url}}{{site.baseurl}}/automating-configurations/index/) -- Automate complex OpenSearch setup and preprocessing tasks.
- [Performance evaluation]({{site.url}}{{site.baseurl}}/monitoring-plugins/pa/) -- Monitor and optimize your cluster.
- [Asynchronous search]({{site.url}}{{site.baseurl}}/search-plugins/async/) -- Run search requests in the background.
- [Cross-cluster replication]({{site.url}}{{site.baseurl}}/replication-plugin/index/) -- Replicate your data across multiple OpenSearch clusters.


## The secure path forward

OpenSearch includes a demo configuration so that you can get up and running quickly, but before using OpenSearch in a production environment, you must [configure the Security plugin manually]({{site.url}}{{site.baseurl}}/security/configuration/index/) with your own certificates, authentication method, users, and passwords. To get started, see [Getting started with OpenSearch security]({{site.url}}{{site.baseurl}}/getting-started/security/).

## Looking for the Javadoc?
{% include cards.html cards=page.features%}

See [opensearch.org/javadocs/](https://opensearch.org/javadocs/).

## Get involved

[OpenSearch](https://opensearch.org) is supported by Amazon Web Services. All components are available under the [Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0.html) on [GitHub](https://github.com/opensearch-project/).
[OpenSearch](https://opensearch.org) is supported by the OpenSearch Software Foundation. All components are available under the [Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0.html) on [GitHub](https://github.com/opensearch-project/).
The project welcomes GitHub issues, bug fixes, features, plugins, documentation---anything at all. To get involved, see [Contributing](https://opensearch.org/source.html) on the OpenSearch website.

---
Expand Down
6 changes: 6 additions & 0 deletions _config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,9 @@ collections:
workspace:
permalink: /:collection/:path/
output: true
vector-search:
permalink: /:collection/:path/
output: true

opensearch_collection:
# Define the collections used in the theme
Expand Down Expand Up @@ -173,6 +176,9 @@ opensearch_collection:
search-plugins:
name: Search features
nav_fold: true
vector-search:
name: Vector search
nav_fold: true
ml-commons-plugin:
name: Machine learning
nav_fold: true
Expand Down
Loading