Merge branch 'main' into refresh_ubi_docs_part_deux

.github/PULL_REQUEST_TEMPLATE.md

@@ -2,7 +2,7 @@
 _Describe what this change achieves._
 
 ### Issues Resolved
-Closes #[_insert issue number_]
+Closes #[_delete this text, including the brackets, and replace with the issue number_]
 
 ### Version
 _List the OpenSearch version to which this PR applies, e.g. 2.14, 2.12--2.14, or all._

.github/vale/styles/OpenSearch/LinksExplicit.yml

@@ -0,0 +1,8 @@
+extends: existence
+message: "In links, use '{{site.url}}{{site.baseurl}}' instead of 'https://www.opensearch.org/docs/latest'."
+level: error
+nonword: true
+scope: raw
+tokens:
+  - '\]\(https:\/\/www.opensearch.org\/docs\/latest'
+

.github/vale/styles/OpenSearch/SubstitutionErroCaseSensitive.yml

@@ -0,0 +1,8 @@
+extends: substitution
+message: "Use '%s' instead of '%s'. Note the correct capitalization."
+ignorecase: false
+level: error
+action:
+  name: replace
+swap:
+  'Retrieval-Augmented Generation': retrieval-augmented generation

.github/vale/styles/OpenSearch/SubstitutionsError.yml

@@ -37,6 +37,7 @@ swap:
   'pre-trained': pretrained
   'premigration': pre-migration
   're-enable': reenable
+  'retrieval augmented generation': retrieval-augmented generation
   'screen shot': screenshot
   'sample request': example request
   'sample response': example response

.github/vale/styles/Vocab/OpenSearch/Products/accept.txt

@@ -68,6 +68,7 @@ OpenSearch Assistant
 OpenSearch Assistant Toolkit
 OpenSearch Benchmark
 OpenSearch Dashboards
+OpenSearch Flow
 OpenSearch Playground
 OpenSearch Project
 OpenSearch Service
@@ -99,4 +100,5 @@ Tribuo
 VisBuilder
 Winlogbeat
 XGBoost
+Zipf
 Zstandard

.github/vale/styles/Vocab/OpenSearch/Words/reject.txt

@@ -1 +1 @@
-
+[Aa]ss

.github/vale/tests/test-style-pos.md

@@ -38,6 +38,8 @@ This sentence tests [links end slash]({{site.url}}{{site.baseurl}}/opensearch).
 
 This sentence tests [links mid slash]({{site.url}}{{site.baseurl}}opensearch).
 
+This sentence tests [links explicit](https://www.opensearch.org/docs/latest/double-slash/).
+
 This sentence tests log-in as a noun. To login, we test this as a verb.
 
 To test merge conflicts, remove tick marks in `<<<<<<< HEAD`.
@@ -74,6 +76,8 @@ This sentence tests splling.
 
 This sentence tests substitution error by using the word indices.
 
+This sentence tests substitution case-sensitive error by using the word Retrieval-Augmented Generation.
+
 This sentence tests substitution suggestion due to its nature.
 
 This Table | tests capitalization

.github/workflows/update-api-components.yml

@@ -24,7 +24,7 @@ jobs:
       - run: bundle install
 
       - name: Download spec and insert into documentation
-        run: bundle exec jekyll spec-insert
+        run: bundle exec jekyll spec-insert -F -R
 
       - name: Get current date
         id: date

.vale.ini

@@ -38,6 +38,7 @@ OpenSearch.LatinismsSubstitution = YES
 OpenSearch.LinksDoubleParentheses = YES
 OpenSearch.LinksDoubleSlash = YES
 OpenSearch.LinksEndSlash = YES
+OpenSearch.LinksExplicit = YES
 OpenSearch.LinksMidSlash = YES
 OpenSearch.LoginNoun = YES
 OpenSearch.LoginVerb = YES
@@ -60,6 +61,7 @@ OpenSearch.SpacingSlash = YES
 OpenSearch.SpacingWords = YES
 OpenSearch.Spelling = YES
 OpenSearch.StackedHeadings = YES
+OpenSearch.SubstitutionsErrorCaseSensitive = YES
 OpenSearch.SubstitutionsError = YES
 OpenSearch.SubstitutionsSuggestion = YES
 OpenSearch.TableHeadings = YES

DEVELOPER_GUIDE.md

@@ -7,7 +7,7 @@
 - [Spec insert components](#spec-insert-components)
   - [Query parameters](#query-parameters)
   - [Path parameters](#path-parameters)
-  - [Paths and HTTP methods](#paths-and-http-methods)
+  - [Endpoints](#endpoints)
 
 ## Introduction
 
@@ -49,7 +49,13 @@ bundle exec jekyll spec-insert
 If you are working on multiple Markdown files and do not want to keep running the `jekyll spec-insert` command, you can add the `--watch` (or `-W`) flag to the command to watch for changes in the Markdown files and automatically render the API components:
 
 ```shell
-bundle exec jekyll spec-insert --watch
+bundle exec jekyll spec-insert -W
+```
+
+By default, when the plugin encounters an error when processing a file, the plugin prints out the error than moves on to the next file. If you want it to short-circuit when an error occurs, add the `--fail-on-error` (or `-F`) flag to the command:
+
+```shell
+bundle exec jekyll spec-insert -F
 ```
 
 Depending on the text editor you are using, you may need to manually reload the file from disk to see the changes applied by the plugin if the editor does not automatically reload the file periodically.
@@ -63,21 +69,26 @@ bundle exec jekyll spec-insert --refresh-spec
 ### Ignoring files and folders
 The `spec-insert` plugin ignores all files and folders listed in the [./_config.yml#exclude](./_config.yml) list, which is also the list of files and folders that Jekyll ignores.
 
+### Configuration
+You can update the configuration settings for this plugin through the [config.yml](./spec-insert/config.yml) file. 
+
+_Note that tests for this plugin use a mock configuration [file](./spec-insert/spec/mock_config.yml) to assure that the tests still pass when the config file is altered. The expected output for the tests is based on the mock configuration file and will look different from the actual output when the plugin is run._
+
 ## CI/CD
 The `spec-insert` plugin is run as part of the CI/CD pipeline to ensure that the API components are up to date in the documentation. This is performed through the [update-api-components.yml](.github/workflows/update-api-components.yml) GitHub Actions workflow, which creates a pull request containing the updated API components every Sunday.
 
 ## Spec insert components
 All spec insert components accept the following arguments:
 - `api` (String; required): The name of the API to render the component from. This is equivalent to the `x-operation-group` field in the OpenSearch OpenAPI Spec.
-- `component` (String; required): The name of the component to render,  such as `query_parameters`, `path_parameters`, or `paths_and_http_methods`.
+- `component` (String; required): The name of the component to render, such as `query_parameters`, `path_parameters`, or `endpoints`.
 - `omit_header` (Boolean; Default is `false`): If set to `true`, the markdown header of the component will not be rendered.
 
-### Paths and HTTP methods
-To insert paths and HTTP methods for the `search` API, use the following snippet:
+### Endpoints
+To insert endpoints for the `search` API, use the following snippet:
 ```markdown
 <!-- spec_insert_start
 api: search
-component: paths_and_http_methods
+component: endpoints
 -->
 <!-- spec_insert_end -->
 ```

FORMATTING_GUIDE.md

@@ -399,6 +399,12 @@ Some Markdown paragraph. Here's a formula:
 And back to Markdown.
 ```
 
+Alternatively, you can use double dollar signs (`$$`) for both display and inline math directly in Markdown:
+
+```
+The probability of selecting pair $$i$$ is proportional to $$1 \over i^\alpha$$.
+```
+
 ## Tables
 
 Markdown table columns are automatically sized, and there is no need to specify a different number of dashes in the formatting. 

STYLE_GUIDE.md

@@ -23,11 +23,11 @@ The following naming conventions should be observed in OpenSearch Project conten
 
 #### Product names
 
-Capitalize product names. The OpenSearch Project has three products: OpenSearch, OpenSearch Dashboards, and Data Prepper. For example:
+Capitalize product names. The OpenSearch Project has three products: OpenSearch, OpenSearch Dashboards, and OpenSearch Data Prepper. For example:
 
 * "To install *OpenSearch*, download the Docker image."
 * "To access *OpenSearch Dashboards*, open your browser and navigate to http://localhost:5601/app/home."
-* "*Data Prepper* contains the following components:"
+* "*OpenSearch Data Prepper* contains the following components:"
 
 Capitalize the names of clients and tools. For example:
 
@@ -128,6 +128,7 @@ The following table lists acronyms that you don't need to spell out.
 | CSV | comma-separated values |
 | DNS | Domain Name System |
 | DOS | disk operating system |
+| Faiss | Facebook AI Similarity Search |
 | FAQ | frequently asked questions |
 | FTP | File Transfer Protocol |
 | GIF | Graphics Interchange Format |
@@ -141,8 +142,10 @@ The following table lists acronyms that you don't need to spell out.
 | JPEG | Joint Photographic Experts Group |
 | JSON | JavaScript Object Notation |
 | k-NN | k-nearest neighbors |
+| MS MARCO | Microsoft Machine Reading Comprehension |
 | NAT | network address translation |
 | NGINX | engine x |
+| NMSLIB | Non-Metric Space Library |
 | PDF | Portable Document Format |
 | RAM | random access memory |
 | REST | Representational State Transfer |

TERMS.md

@@ -46,7 +46,7 @@ Use to describe a list of items that are allowed (not blocked). Do not use as a
 
 **Amazon OpenSearch Service**
 
-Amazon OpenSearch Service is a managed service that makes it easy to deploy, operate, and scale OpenSearch clusters in the AWS Cloud. Amazon OpenSearch Service is the successor to Amazon Elasticsearch Service (Amazon ES) and supports OpenSearch and legacy Elasticsearch OSS (up to 7.10, the final open-source version of the software).
+Use "Amazon OpenSearch Service" on first appearance; "OpenSearch Service" is acceptable for subsequent appearances. Amazon OpenSearch Service is a managed service that makes it easy to deploy, operate, and scale OpenSearch clusters in the AWS Cloud. Amazon OpenSearch Service is the successor to Amazon Elasticsearch Service (Amazon ES) and supports OpenSearch and legacy Elasticsearch OSS (up to 7.10, the final open-source version of the software).
 
 **Anomaly Detection**
 
@@ -196,6 +196,10 @@ Use data is, not data are. Don't use datas. Use pieces of data or equivalent to
 
 **data center**
 
+**OpenSearch Data Prepper**
+
+Use "OpenSearch Data Prepper" on first appearance; "Data Prepper" is acceptable for subsequent appearances. OpenSearch Data Prepper is a server-side data collector capable of filtering, enriching, transforming, normalizing, and aggregating data for downstream analytics and visualization. Data Prepper also lets users build custom pipelines to improve the operational view of applications.
+
 **dataset**
 
 **data source**
@@ -299,6 +303,8 @@ Exception: *Execution* is unavoidable for third-party terms for which no alterna
 
 **Faiss**
 
+Facebook AI Similarity Search. Do not define on first appearance. Faiss is a library that allows developers to quickly search for embeddings of multimedia documents that are similar to each other.
+
 **file name**
 
 **frontend (n., adj.)**
@@ -501,6 +507,10 @@ Do not use. Use *management account* instead.
 
 Avoid. Use _can_ or _might_ instead.
 
+**MS MARCO**
+
+Microsoft Machine Reading Comprehension. Do not define on first appearance. MS MARCO is a collection of datasets focused on deep learning in search.
+
 **multilayer, multilayered**
 
 **must, shall, should**
@@ -521,6 +531,10 @@ Use _near real time_ as a noun; use near real-time as an adjective. Don't add a
 
 Spell out _near real time_ on first mention; _NRT_ can be used on subsequent mentions.
 
+**NMSLIB**
+
+Non-Metric Space Library. Do not define on first appearance. NMSLIB is an efficient similarity search library and a toolkit for evaluation of k-NN methods for generic non-metric spaces. 
+
 **node**
 
 A server that stores your data and processes search requests with OpenSearch, usually as part of a cluster. Do not use _master node_ and avoid using _worker node_.
@@ -650,6 +664,8 @@ Copy of a primary shard. Helps improve performance when using indexes across mul
 
 Use as a synonym for repository, on second and subsequent use.
 
+**retrieval-augmented generation (RAG)**
+
 **RPM Package Manager (RPM)**
 
 Formerly known as RedHat Package Manager. An open-source package management system for use with Linux distributions.
@@ -698,7 +714,7 @@ Do not use. Use *replica*, *secondary*, or *standby* instead.
 
 **Snapshot Management (SM)**
 
-**solid state drive (SSD)**
+**solid-state drive (SSD)**
 
 **standalone**
 

_about/version-history.md

@@ -9,6 +9,7 @@ permalink: /version-history/
 
 OpenSearch version | Release highlights | Release date  
 :--- | :--- | :--- 
+[2.19.0](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-2.19.0.md) | Adds workload management, additional query insights, and template queries. Introduces a query insights page to OpenSearch Dashboards. Includes improvements and bug fixes to snapshots, search statistics, star-tree search, and index management. For a full list of release highlights, see the Release Notes. | 11 February 2025
 [2.18.0](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-2.18.0.md) | Adds a redesigned home page, updated Discover interface, and collaborative workspaces to OpenSearch Dashboards. Includes improvements to ML inference processor and query grouping. Introduces reranking by field and paginated CAT APIs. Includes experimental OpenSearch Dashboards Assistant capabilities. For a full list of release highlights, see the Release Notes. | 05 November 2024
 [2.17.1](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-2.17.1.md) | Includes bug fixes for ML Commons, anomaly detection, k-NN, and security analytics. Adds various infrastructure and maintenance updates. For a full list of release highlights, see the Release Notes. | 1 October 2024
 [2.17.0](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-2.17.0.md) | Includes disk-optimized vector search, binary quantization, and byte vector encoding in k-NN. Adds asynchronous batch ingestion for ML tasks. Provides search and query performance enhancements and a new custom trace source in trace analytics. Includes application-based configuration templates. For a full list of release highlights, see the Release Notes. | 17 September 2024

_aggregations/bucket/terms.md

@@ -112,7 +112,7 @@ While the `doc_count` field provides a representation of the number of individua
 * The field does not support nested arrays; only positive integers can be used.
 * If a document does not contain the `_doc_count` field, aggregation uses the document to increase the count by 1.
 
-OpenSearch features that rely on an accurate document count illustrate the importance of using the `_doc_count` field. To see how this field can be used to support other search tools, refer to [Index rollups](https://opensearch.org/docs/latest/im-plugin/index-rollups/index/), an OpenSearch feature for the Index Management (IM) plugin that stores documents with pre-aggregated data in rollup indexes.
+OpenSearch features that rely on an accurate document count illustrate the importance of using the `_doc_count` field. To see how this field can be used to support other search tools, refer to [Index rollups]({{site.url}}{{site.baseurl}}/im-plugin/index-rollups/index/), an OpenSearch feature for the Index Management (IM) plugin that stores documents with pre-aggregated data in rollup indexes.
 {: .tip}
 
 #### Example request