Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fetch default options for ML processor config maps #354

Merged
merged 4 commits into from
Sep 9, 2024

Conversation

ohltyler
Copy link
Member

@ohltyler ohltyler commented Sep 9, 2024

Description

This PR adds functionality to dynamically fetch a list of options when configuring input / output maps for ML processors. Specifically, adds support for 3 scenarios:

  1. List of document fields as values for input maps when the document form field is populated (ingest)
  2. List of query fields as values for input maps when the query form is populated (search)
  3. List of document fields as values for input maps when configuring ML search response processors. References the selected index's mappings to populate options, including the field mapping type.

Note the dropdown still allows custom options if users need (e.g., pulling out a single value from an array, aggregation via JSONPath, etc.).

More details

  • adds docFields/queryFields/indexMappingFields as state vars in MLProcessorInputs to handle the options as mentioned above
  • adds MIT-licensed flattie to flatten nested JSON objs. Used for getting all options (keys) of a document or query JSON which can be deeply nested.
  • more type safety on the options props passed to the form components
  • index fetching on SearchInputs base component so any index created from ingest, has all details available on the search form without requiring refresh. This is useful to fetch the index mappings to populate options for ML search response processors.
  • replacing the default query path logic on ML search request processors to derive from the query itself instead of hardcoding

Demo video, showing the options visible in the 3 different contexts of ML processors (ingest, search req, search resp):

screen-capture.15.webm

Check List

  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>
Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>
Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>
@ohltyler ohltyler marked this pull request as ready for review September 9, 2024 21:32
Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>
@saimedhi saimedhi merged commit dbe5689 into opensearch-project:main Sep 9, 2024
6 checks passed
opensearch-trigger-bot bot pushed a commit that referenced this pull request Sep 9, 2024
* fetch and set presets for input / output maps where applicable

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>

* Switch to search-specified index name

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>

* Propagate values to input modal; tune wording

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>

* Set defaults for mock store in tests

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>

---------

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>
(cherry picked from commit dbe5689)
@ohltyler ohltyler deleted the doc-presets branch September 9, 2024 22:18
ohltyler added a commit that referenced this pull request Sep 9, 2024
Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>
(cherry picked from commit dbe5689)

Co-authored-by: Tyler Ohlsen <ohltyler@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants