Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
rushmash91 authored Feb 11, 2025
1 parent 189bcc8 commit 62a8184
Showing 1 changed file with 7 additions and 7 deletions.
14 changes: 7 additions & 7 deletions examples/aws/llm/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,15 +43,15 @@ Create an instance YAML file with your desired configuration:
apiVersion: kro.run/v1alpha1
kind: OllamaDeployment
metadata:
name: deepseek
name: phi
spec:
name: deepseek
name: phi
namespace: default
values:
storage: 50Gi # Storage for model files
model:
name: deepseek-r1 # Model name
size: 1.5b # Model size variant
name: phi # Model name
size: 2.7b # Model size variant
resources:
requests:
cpu: "2"
Expand Down Expand Up @@ -87,7 +87,7 @@ Access the model within the cluster using:
curl http://<name>.<namespace>.svc.cluster.local/v1/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "deepseek-r1:1.5b",
"model": "phi:2.7b",
"prompt": "Hello, who are you?"
}'
```
Expand All @@ -98,7 +98,7 @@ The service will be available at:
curl http://<ALB-ADDRESS>/<name>/v1/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "deepseek-r1:1.5b",
"model": "phi:2.7b",
"prompt": "Hello, who are you?"
}'
```
Expand All @@ -113,4 +113,4 @@ curl http://<ALB-ADDRESS>/<name>/v1/completions \
### Scaling Considerations
- Increase storage for larger models
- Adjust CPU/Memory based on load
- Add GPUs for better inference performance
- Add GPUs for better inference performance

0 comments on commit 62a8184

Please sign in to comment.