-
-
Notifications
You must be signed in to change notification settings - Fork 743
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
4 changed files
with
27 additions
and
14 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,10 +1,13 @@ | ||
/* | ||
Package outputparser provides a set of output parsers to process structured or unstructured data from language models (LLMs). | ||
Package outputparser provides a set of output parsers to process structured or | ||
unstructured data from language models (LLMs). | ||
The outputparser package includes the following parsers: | ||
- Simple: a basic parser that returns the raw text as-is without any processing. | ||
- Structured: a parser that expects a JSON-formatted response and returns it as a map[string]string while validating against a provided schema. | ||
- CommaSeparatedList: a parser that takes a string with comma-separated values and returns them as a string slice. | ||
- Simple: a basic parser that returns the raw text as-is without any processing. | ||
- Structured: a parser that expects a JSON-formatted response and returns it as | ||
a map[string]string while validating against a provided schema. | ||
- CommaSeparatedList: a parser that takes a string with comma-separated values | ||
and returns them as a string slice. | ||
*/ | ||
package outputparser |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,12 +1,17 @@ | ||
/* | ||
Package textsplitter provides tools for splitting long texts into smaller chunks based on configurable rules and parameters. It aims to help in processing these chunks more efficiently when interacting with language models or other text-processing tools. | ||
Package textsplitter provides tools for splitting long texts into smaller chunks | ||
based on configurable rules and parameters. | ||
It aims to help in processing these chunks more efficiently | ||
when interacting with language models or other text-processing tools. | ||
The main components of this package are: | ||
- TextSplitter interface: a common interface for splitting texts into smaller chunks. | ||
- RecursiveCharacter: a text splitter that recursively splits texts by different characters (separators) combined with chunk size and overlap settings. | ||
- RecursiveCharacter: a text splitter that recursively splits texts by different characters (separators) | ||
combined with chunk size and overlap settings. | ||
- Helper functions: utility functions for creating documents out of split texts and rejoining them if necessary. | ||
Using the TextSplitter interface, developers can implement custom splitting strategies for their specific use cases and requirements. | ||
Using the TextSplitter interface, developers can implement custom | ||
splitting strategies for their specific use cases and requirements. | ||
*/ | ||
package textsplitter | ||
package textsplitter |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,12 +1,15 @@ | ||
/* | ||
Package vectorstores contains the implementation of VectorStore, an interface for saving and querying documents as vector embeddings. | ||
Package vectorstores contains the implementation of VectorStore, an interface for saving | ||
and querying documents as vector embeddings. | ||
The main components of this package are: | ||
- VectorStore interface: a common interface for saving and querying vector embeddings of documents. | ||
- Options: a set of options for similarity search and document addition. | ||
- Retriever: a retriever for vector stores that implements the schema.Retriever interface. | ||
The package provides a flexible way to handle different types of vector stores by using the VectorStore interface as an abstraction. It supports customization of the search and storage operation via the Options mechanism. | ||
The package provides a flexible way to handle different types of vector stores | ||
by using the VectorStore interface as an abstraction. | ||
It supports customization of the search and storage operation via the Options mechanism. | ||
*/ | ||
package vectorstores | ||
package vectorstores |