Skip to content

Commit

Permalink
docs: add missing step to local model provider page (#366)
Browse files Browse the repository at this point in the history
**Commit Message**

Tell user to download `localmodel.yaml` first before kubectl applying
it. Make some minor Markdown formatting improvements.

---------

Signed-off-by: David Xia <david@davidxia.com>
Co-authored-by: Erica Hughberg <erica.sundberg.90@gmail.com>
  • Loading branch information
davidxia and missBerg authored Feb 18, 2025
1 parent 81a1557 commit 42fb1d4
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 7 deletions.
8 changes: 4 additions & 4 deletions site/docs/getting-started/connect-providers/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,11 @@ Before configuring any provider:
--for=delete
```

3. Download configuration template
3. Download configuration template

```shell
curl -O https://raw.githubusercontent.com/envoyproxy/ai-gateway/main/examples/basic/basic.yaml
```
```shell
curl -O https://raw.githubusercontent.com/envoyproxy/ai-gateway/main/examples/basic/basic.yaml
```

## Security Best Practices

Expand Down
8 changes: 5 additions & 3 deletions site/docs/getting-started/connect-providers/localmodel.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ sidebar_position: 3

# Connect Local Model

This guide will help you configure Envoy AI Gateway to work with locally hosted model such as [DeepSeek R1](https://github.com/deepseek-ai/DeepSeek-R1).
This guide will help you configure Envoy AI Gateway to work with a locally hosted model such as [DeepSeek R1](https://github.com/deepseek-ai/DeepSeek-R1).

## Prerequisites

Expand All @@ -15,7 +15,7 @@ Before you begin, you'll need:
- [Ollama](https://ollama.com/) installed on local machine considering for self-hosted model
- Serve DeepSeek R1 or similar model on your local machine

```
```shell
ollama pull deepseek-r1:7b
OLLAMA_HOST=0.0.0.0 ollama serve
```
Expand All @@ -32,9 +32,11 @@ Ensure you have followed the steps in [Connect Providers](../connect-providers/)
### Apply Configuration

Apply the updated configuration and wait for the Gateway pod to be ready. If you already have a Gateway running,
then the secret credential update will be picked up automatically in a few seconds.
the secret credential update will take effect automatically in a few seconds.

```shell
curl -O https://raw.githubusercontent.com/envoyproxy/ai-gateway/main/examples/basic/localmodel.yaml

kubectl apply -f localmodel.yaml

kubectl wait pods --timeout=2m \
Expand Down

0 comments on commit 42fb1d4

Please sign in to comment.