EnterpriseDB · josh-heyer · Jan 9, 2025 · Dec 5, 2024 · Dec 5, 2024 · Dec 12, 2024
@@ -18,18 +18,18 @@ This storage location can be an S3 bucket or a local file system.
 
 The storage locations can be used to create a volume, suitable for a retriever to use to access the data it contains.
 
-### A model is registered
+### A model is created
 
-A [model](models) is registered with the Pipelines system. This model can be a machine learning model, a deep learning model, or any other type of model that can be used for AI tasks.
+A [model](models) is created with the Pipelines system. This model can be a machine learning model, a deep learning model, or any other type of model that can be used for AI tasks.
 
-### A retriever is registered
+### A retriever is created
 
-A retriever is registered with the Pipelines system. A retriever is a function that retrieves data from a table or volume and returns it in a format that can be used by the model.
+A retriever is created with the Pipelines system. A retriever is a function that retrieves data from a table or volume and returns it in a format that can be used by the model.
 
 By default, a retriever only needs:
 
 * a name
-* the name of a registered model to use
+* the name of a model to use
 
 If the retriever is for a table, it also needs:
 
@@ -42,10 +42,10 @@ If, on the other hand, the retriever is for a volume, it needs:
 * the name of the volume
 * the name of the column in the volume that contains the data
 
-When a retriever is registered, by default it will create a vector table to store the embeddings of the data that is retrieved.
+When a retriever is created, by default it will create a vector table to store the embeddings of the data that is retrieved.
 This table will have a column to store the embeddings and a column to store the key of the data.
 
-The name of the vector table and the name of the vector column and the key column can be specified when the retriever is registered; this is useful if you are migrating to aidb and want to use an existing vector table.
+The name of the vector table and the name of the vector column and the key column can be specified when the retriever is created; this is useful if you are migrating to aidb and want to use an existing vector table.
 
 ### Embeddings are created
 
@@ -65,4 +65,4 @@ While auto-embedding is enabled, the embeddings are always up-to-date and applic
 
 ### Cleanup
 
-If the embeddings are no longer required, the retriever can be unregistered, the vector table can be dropped and the model can be unregistered too.
+If the embeddings are no longer required, the retriever can be deleted, the vector table can be dropped and the model can be deleted.
@@ -58,22 +58,22 @@ INSERT 0 9
 
 So now we have a table with some data in it, food products and some very personal opinions about them.
 
-## Registering a Retriever
+## Creating a Retriever
 
-The first step to using Pipelines with this data is to register a retriever. A retriever is a way to access the data in the table and use it in AI workflows.
+The first step to using Pipelines with this data is to create a retriever. A retriever is a way to access the data in the table and use it in AI workflows.
 
 ```sql
-select aidb.register_retriever_for_table('products_retriever', 't5', 'products', 'description', 'Text');
+select aidb.create_retriever_for_table('products_retriever', 't5', 'products', 'description', 'Text');
 __OUTPUT__
- register_retriever_for_table
+ create_retriever_for_table
 ------------------------------
  products_retriever
 (1 row)
 ```
 
 ## Querying the retriever
 
-Now that we have a retriever registered, we can query it to get similar results based on the data in the table.
+Now that we have created a retriever, we can query it to get similar results based on the data in the table.
 
 ```sql
 select * from aidb.retrieve_key('products_retriever','I like it',5);
@@ -189,6 +189,6 @@ __OUTPUT__
 
 ## Further reading
 
-In the [Models](../models) section, you can learn how to register more models with Pipelines, including external models from OpenAI API compatible services.
+In the [Models](../models) section, you can learn how to create more models with Pipelines, including external models from OpenAI API compatible services.
 
 In the [Retrievers](../retrievers) section, you can learn more about how to use retrievers with external data sources, local files or S3 storage, and how to use the retriever functions to get the data you need.
@@ -4,7 +4,7 @@ navTitle: "AI Accelerator"
 directoryDefaults:
   product: "EDB Postgres AI"
   iconName: BrainCircuit
-indexCards: simple
+indexCards: extra
 description: "All about the EDB Postgres AI - AI Accelerator suite of tools including Pipelines and PGvector."
 navigation:
 - overview
@@ -15,7 +15,7 @@ navigation:
 - limitations
 - compatibility
 - installing
-- "#Piplelines components"
+- "#Pipelines components"
 - models
 - retrievers
 - pgfs

@@ -6,13 +6,15 @@ navigation:
 - using-models
 - primitives
 - supported-models
+- openai-api-compatibility
 ---
 
-Pipelines has a model registry that manages configured instances of models. Any Pipelines functions that use models, such as embedding and retrieving, must reference a registered model.
+Pipelines has a model registry that manages configured instances of models. Any Pipelines functions that use models, such as embedding and retrieving, must reference a created model.
 
-* Learn how to [register models](./using-models) in Pipelines.
+* Learn how to [create models](./using-models) in Pipelines.
 * Discover the [primitives](./primitives) that can be used to interact with models.
 * See the [supported models](./supported-models) that come with Pipelines.
+* Learn how to use [OpenAI API compatible services](./openai-api-compatibility) with Pipelines.
 
 ## Next Steps
 

@@ -0,0 +1,69 @@
+---
+title: "Using an OpenAI compatible API with Pipelines"
+navTitle: "OpenAI Compatible Models"
+description: "Using an OpenAI compatible API with Pipelines by setting options and credentials."
+---
+
+To make use of an OpenAI compliant API, you can use the openai_embeddings or openai_completions model providers. Note that a retriever will need to encode first so you can only use the embeddings model provider with a retriever.
+
+## Why use an OpenAI compatible API?
+
+Some examples of why you might want to use an OpenAI compatible API include:
+
+* If you have a local system running [Ollama](https://ollama.com) and, assuming you have configured [Ollama as a server](https://github.com/ollama/ollama/blob/main/docs/faq.md#how-do-i-configure-ollama-server), then you may want that local system to handle embeddings.
+
+* If you have access to a service which provides different or specifically tuned models, you can use it instead of other models.
+
+## Creating the model
+
+The starting point for this process is creating a model. When you create a model, you can pass `options` and `credentials` to the registration. The defaults will point to the OpenAI service endpoint, so by overriding that, you can point to any service. Here is an example that will create a model that uses a local Ollama server:
+
+```sql
+select aidb.create_model(
+'my_local_ollama',
+'openai_embeddings',
+'{"model":"llama3.3", "url":"http://llama.local:11434/v1/embeddings", "dimensions":8192}'::JSONB,
+'{"api_key":""}'::JSONB);
+```
+
+### Model name and model provider
+
+The model name is the first parameter and set to “my_local_ollama” which we will use later.
+
+We specify the model provider as “openai_embeddings” which is the provider that defaults to using OpenAI servers, but can be overridden by the configuration (the next parameter), to talk to any compliant server.
+
+### Configuration
+
+The next parameter is the configuration. This is a JSON string, which when expanded has three parameters, the model, the url and the dimensions.
+
+```json
+'{"model":"llama3.3", "url":"http://llama.local:11434/v1/embeddings", "dimensions":8192}'::JSONB
+```
+
+In this case, we are setting the model to [“llama3.3”](https://ollama.com/library/llama3.3), a relatively new and powerful model. Remember to run `ollama run llama3.3` to pull and start the model on the server.
+
+The next json setting is the important one, overriding the endpoint that the aidb model will use. 
+
+* Our server is running on a machine called `llama.local`.
+* It has port 11434 (the default port for Ollama) open to service requests over HTTP (not HTTPS in this case).
+* The path to the endpoint on the server `/v1/embeddings`; the same as OpenAI.
+
+Putting those components together we get `[`http://llama.local:11434/v1/embeddings`](http://art.local:11434/v1/embeddings","api_key":"","dimensions":8192}'::JSONB)` as our end point.
+
+The last JSON parameter in this example is “dimensions” which is a hint to the system about how many vector values to expect from the model. If we [look up llama3.3’s properties](https://ollama.com/library/llama3.3/blobs/4824460d29f2) we can see the `llama.embedding_length` value is 8192\. The provider defaults to 1536 (with some hard-wired exceptions depending on model) but it doesn’t know about llama3.3, so we have to pass the dimension value of 8192 in the configuration.
+
+That completes the configuration parameter.
+
+### Credentials
+
+The last parameter is the credentials parameter, which is another JSON string. It’s usually used for carrying the `api_key` for the OpenAI service and any other necessary credential information. It is not part of the configuration and by being separate, it can be securely hidden from users with lesser permissions. For our ollama connection, we don’t need an api\_key, but the model provider currently requires that one is specified. We can specify an empty string for the api\_key to satisfy this requirement.
+
+## Using the model
+
+Use the model name you created earlier to use the model just like any other Pipelines model. Here is an example of how to use the model to get an embedding:
+
+```sql
+select aidb.encode_text('my_local_ollama','I like it');
+```
+
+Pipelines will take care of all the connection management leaving you to focus on your data and the model results.
@@ -31,17 +31,18 @@ Read more about [BERT on Wikipedia](https://en.wikipedia.org/wiki/BERT_(language
 * sentence-transformers/paraphrase-multilingual-mpnet-base-v2
 * sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
 
-## Register the default implementation
-
+## Creating the default model
 
 ```sql
-SELECT aidb.register_model('my_bert_model', 'bert_local');
+SELECT aidb.create_model('my_bert_model', 'bert_local');
 ```
 
-## Register another model
+## Creating a specific model
+
+You can specify a model and revision in the options JSONB object. In this example, we are creating a `sentence-transformers/all-distilroberta-v1` model with the name `another_bert_model`:
 
 ```sql
-select aidb.register_model(
+select aidb.create_model(
   'another_bert_model',
   'bert_local',
   '{"model": "sentence-transformers/all-distilroberta-v1", "revision": "main"}'::JSONB 

@@ -24,15 +24,15 @@ Read more about [CLIP on OpenAI's website](https://openai.com/research/clip/).
 
 * openai/clip-vit-base-patch32 (default)
 
-## Register the default implementation
+## Creating the default model
 
 ```sql
-SELECT aidb.register_model('my_clip_model', 'clip_local');
+SELECT aidb.create_model('my_clip_model', 'clip_local');
 ```
 
 There is only one model, the default `openai/clip-vit-base-patch32`, so we do not need to specify the model in the configuration. No credentials are required for the CLIP model.
 
-## Register another model
+## Creating a specific model
 
 There are no other model configurations available for the CLIP model.
 

@@ -21,18 +21,18 @@ See a list of supported OpenAI models [here](https://platform.openai.com/docs/mo
 
 * Any text generation model that is supported by OpenAI. This includes models such as GPT-4o, GPT-4o mini, GPT-4 and GPT-3.5.
 
-## Registering the default model
+## Creating the default model
 
-There is no default model for OpenAI Completions. You can register any supported OpenAI model using the `aidb.register_model` function. See [Registering a model](#registering-a-model).
+There is no default model for OpenAI Completions. You can create any supported OpenAI model using the `aidb.create_model` function. See [Creating a model](#creating-a-specific-model).
 
-## Registering a model
+## Creating a specific model
 
-You can register any supported OpenAI model using the `aidb.register_model` function.
+You can create any supported OpenAI model using the `aidb.create_model` function.
 
-In this example, we are registering a GPT-4o model with the name `my_openai_model`:
+In this example, we are creating a GPT-4o model with the name `my_openai_model`:
 
 ```sql
-SELECT aidb.register_model(
+SELECT aidb.create_model(
   'my_openai_model',
   'openai_completions',
   '{"model": "gpt-4o"}::JSONB,

@@ -22,22 +22,22 @@ See a list of supported OpenAI models [here](https://platform.openai.com/docs/gu
 * Any text embedding model that is supported by OpenAI. This includes `text-embedding-3-small`, `text-embedding-3-large`, and `text-embedding-ada-002`.
 * Defaults to `text-embedding-3-small`.
 
-## Registering the default model
+## Creating the default model
 
 ```sql
-SELECT aidb.register_model('my_openai_embeddings', 
+SELECT aidb.create_model('my_openai_embeddings', 
                             'openai_embeddings',
                             credentials=>'{"api_key": "sk-abc123xyz456def789ghi012jkl345mn"'::JSONB);
 ```
 
 As we are defaulting the model to `text-embedding-3-small`, we do not need to specify the model in the configuration. But we do need to pass an OpenAI API key in the credentials, and for that we have to pass credentials as a named parameter.
 
-## Registering a model
+## Creating a specific model
 
-You can register any supported OpenAI embedding model using the `aidb.register_model` function. In this example, we are registering a `text-embedding-3-large` model with the name `my_openai_model`:
+You can create any supported OpenAI embedding model using the `aidb.create_model` function. In this example, we are creating a `text-embedding-3-large` model with the name `my_openai_model`:
 
 ```sql
-SELECT aidb.register_model(
+SELECT aidb.create_model(
   'my_openai_model',
   'openai_embeddings',
   '{"model": "text-embedding-3-large"}'::JSONB,
@@ -55,6 +55,20 @@ The following configuration settings are available for OpenAI models:
 * `url` - The URL of the OpenAI model to use. This is optional and can be used to specify a custom model URL. Defaults to `https://api.openai.com/v1/chat/completions`.
 * `max_concurrent_requests` - The maximum number of concurrent requests to make to the OpenAI model. Defaults to `25`.
 
+## Available OpenAI Embeddings models
+
+* sentence-transformers/all-MiniLM-L6-v2 (default)
+* sentence-transformers/all-MiniLM-L6-v1
+* sentence-transformers/all-MiniLM-L12-v1
+* sentence-transformers/msmarco-bert-base-dot-v5
+* sentence-transformers/multi-qa-MiniLM-L6-dot-v1
+* sentence-transformers/paraphrase-TinyBERT-L6-v2
+* sentence-transformers/all-distilroberta-v1
+* sentence-transformers/all-MiniLM-L6-v2
+* sentence-transformers/multi-qa-MiniLM-L6-cos-v1
+* sentence-transformers/paraphrase-multilingual-mpnet-base-v2
+* sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
+
 ## Model credentials
 
 The following credentials are required for OpenAI models:

@@ -27,16 +27,16 @@ Read more about [T5 on Wikipedia](https://en.wikipedia.org/wiki/T5_(language_mod
 * t5-3b
 * t5-11b
 
-## Registering the default model
+## Creating the default model
 
 ```sql
-SELECT aidb.register_model('my_t5_model', 't5_local');
+SELECT aidb.create_model('my_t5_model', 't5_local');
 ```
 
-## Registering a specific model
+## Creating a specific model
 
 ```sql
-SELECT aidb.register_model(
+SELECT aidb.create_model(
   'another_t5_model',
   't5_local',
   '{"model": "t5-large", "revision": "main"}'::JSONB