Skip to content

Commit 7ff03b5

Browse files
lianhaoleslieluyu
lianhao
authored and
leslieluyu
committed
helm: Update chatqna to latest
- Update chart to latest tgi-gaudi version - Add helm test Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
1 parent d75d8f2 commit 7ff03b5

File tree

5 files changed

+71
-44
lines changed

5 files changed

+71
-44
lines changed

‎helm-charts/chatqna/README.md

+37-9
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,16 @@
11
# ChatQnA
22

3-
Helm chart for deploying ChatQnA service.
3+
Helm chart for deploying ChatQnA service. ChatQnA depends on the following services:
4+
5+
- [data-prep](../common/data-prep)
6+
- [embedding-usvc](../common/embedding-usvc)
7+
- [tei](../common/tei)
8+
- [retriever-usvc](../common/retriever-usvc)
9+
- [redis-vector-db](../common/redis-vector-db)
10+
- [reranking-usvc](../common/reranking-usvc)
11+
- [teirerank](../common/teirerank)
12+
- [llm-uservice](../common/llm-uservice)
13+
- [tgi](../common/tgi)
414

515
## Installing the Chart
616

@@ -15,15 +25,33 @@ export MODELDIR="/mnt/opea-models"
1525
export MODELNAME="Intel/neural-chat-7b-v3-3"
1626
helm install chatqna chatqna --set global.HUGGINGFACEHUB_API_TOKEN=${HFTOKEN} --set global.modelUseHostPath=${MODELDIR} --set tgi.LLM_MODEL_ID=${MODELNAME}
1727
# To use Gaudi device
18-
# helm install chatqna chatqna --set global.HUGGINGFACEHUB_API_TOKEN=${HFTOKEN} --values chatqna/gaudi-values.yaml
28+
#helm install chatqna chatqna --set global.HUGGINGFACEHUB_API_TOKEN=${HFTOKEN} --set global.modelUseHostPath=${MODELDIR} --set tgi.LLM_MODEL_ID=${MODELNAME} -f chatqna/gaudi-values.yaml
29+
```
30+
31+
### IMPORTANT NOTE
32+
33+
1. Make sure your `MODELDIR` exists on the node where your workload is schedueled so you can cache the downloaded model for next time use. Otherwise, set `global.modelUseHostPath` to 'null' if you don't want to cache the model.
34+
35+
## Verify
36+
37+
To verify the installation, run the command `kubectl get pod` to make sure all pods are running.
38+
39+
Then run the command `kubectl port-forward svc/chatqna 8888:8888` to expose the service for access.
40+
41+
Open another terminal and run the following command to verify the service if working:
42+
43+
```console
44+
curl http://localhost:8888/v1/chatqna \
45+
-H "Content-Type: application/json" \
46+
-d '{"messages": "What is the revenue of Nike in 2023?"}'
1947
```
2048

2149
## Values
2250

23-
| Key | Type | Default | Description |
24-
| ------------------------------- | ------ | ----------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
25-
| image.repository | string | `"opea/chatqna:latest"` | |
26-
| service.port | string | `"8888"` | |
27-
| global.HUGGINGFACEHUB_API_TOKEN | string | `""` | Your own Hugging Face API token |
28-
| global.modelUseHostPath | string | `"/mnt/opea-models"` | Cached models directory, tgi will not download if the model is cached here. The host path "volume" will be mounted to container as /data directory |
29-
| tgi.LLM_MODEL_ID | string | `"Intel/neural-chat-7b-v3-3"` | Models id from https://huggingface.co/, or predownloaded model directory |
51+
| Key | Type | Default | Description |
52+
| ------------------------------- | ------ | ----------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
53+
| image.repository | string | `"opea/chatqna:latest"` | |
54+
| service.port | string | `"8888"` | |
55+
| global.HUGGINGFACEHUB_API_TOKEN | string | `""` | Your own Hugging Face API token |
56+
| global.modelUseHostPath | string | `"/mnt/opea-models"` | Cached models directory, tgi will not download if the model is cached here. The host path "modelUseHostPath" will be mounted to container as /data directory. Set this to empty/null will force it to download models every time. |
57+
| tgi.LLM_MODEL_ID | string | `"Intel/neural-chat-7b-v3-3"` | Models id from https://huggingface.co/, or predownloaded model directory |

‎helm-charts/chatqna/gaudi-values.yaml

+10-1
Original file line numberDiff line numberDiff line change
@@ -29,13 +29,22 @@ securityContext:
2929
seccompProfile:
3030
type: RuntimeDefault
3131

32+
# wait for issue https://github.com/opea-project/GenAIExamples/issues/426 to be resolved
33+
#tei:
34+
# image:
35+
# repository: opea/tei-gaudi
36+
# tag: "latest"
37+
# resources:
38+
# limits:
39+
# habana.ai/gaudi: 1
40+
3241
# To override values in subchart tgi
3342
tgi:
3443
LLM_MODEL_ID: Intel/neural-chat-7b-v3-3
3544
# LLM_MODEL_ID: /data/OpenCodeInterpreter-DS-6.7B
3645
image:
3746
repository: ghcr.io/huggingface/tgi-gaudi
38-
tag: "1.2.1"
47+
tag: "2.0.1"
3948
resources:
4049
limits:
4150
habana.ai/gaudi: 1

‎helm-charts/chatqna/templates/NOTES.txt

-16
This file was deleted.

‎helm-charts/chatqna/templates/tests/test-connection.yaml

-18
This file was deleted.

‎helm-charts/chatqna/templates/tests/test-pod.yaml

+24
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# Copyright (C) 2024 Intel Corporation
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
apiVersion: v1
5+
kind: Pod
6+
metadata:
7+
name: "{{ include "chatqna.fullname" . }}-testpod"
8+
labels:
9+
{{- include "chatqna.labels" . | nindent 4 }}
10+
annotations:
11+
"helm.sh/hook": test
12+
#"helm.sh/hook-delete-policy": "hook-succeeded, hook-failure"
13+
spec:
14+
containers:
15+
- name: curl
16+
image: alpine/curl
17+
#image: python:3.10.14
18+
command: ['sh', '-c']
19+
args:
20+
- |
21+
curl http://{{ include "chatqna.fullname" . }}:{{ .Values.service.port }}/v1/chatqna -sS --fail-with-body \
22+
-d '{"messages": "What is the revenue of Nike in 2023?"}' \
23+
-H 'Content-Type: application/json'
24+
restartPolicy: Never

0 commit comments

Comments
 (0)