Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tgi: revert xeon version to 2.2.0 #328

Merged
merged 2 commits into from
Aug 21, 2024
Merged

tgi: revert xeon version to 2.2.0 #328

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1312570
tgi: revert xeon version to 2.2.0
lianhao Aug 21, 2024
2f0166c
Retriever: Fix missing HF_TOKEN issue of v0.9 retriever-redis image
lianhao Aug 21, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .github/workflows/scripts/e2e/gmc_xeon_test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,7 @@ function validate_chatqna() {
kubectl create ns $CHATQNA_NAMESPACE
sed -i "s|namespace: chatqa|namespace: $CHATQNA_NAMESPACE|g" $(pwd)/config/samples/chatQnA_xeon.yaml
# workaround for issue #268
yq -i '(.spec.nodes.root.steps[] | select ( .name == "Tgi")).internalService.config.MODEL_ID = "bigscience/bloom-560m"' $(pwd)/config/samples/chatQnA_xeon.yaml
#yq -i '(.spec.nodes.root.steps[] | select ( .name == "Tgi")).internalService.config.MODEL_ID = "bigscience/bloom-560m"' $(pwd)/config/samples/chatQnA_xeon.yaml
kubectl apply -f $(pwd)/config/samples/chatQnA_xeon.yaml

# Wait until the router service is ready
Expand Down Expand Up @@ -238,7 +238,7 @@ function validate_chatqna_with_dataprep() {
kubectl create ns $CHATQNA_DATAPREP_NAMESPACE
sed -i "s|namespace: chatqa|namespace: $CHATQNA_DATAPREP_NAMESPACE|g" $(pwd)/config/samples/chatQnA_dataprep_xeon.yaml
# workaround for issue #268
yq -i '(.spec.nodes.root.steps[] | select ( .name == "Tgi")).internalService.config.MODEL_ID = "bigscience/bloom-560m"' $(pwd)/config/samples/chatQnA_dataprep_xeon.yaml
#yq -i '(.spec.nodes.root.steps[] | select ( .name == "Tgi")).internalService.config.MODEL_ID = "bigscience/bloom-560m"' $(pwd)/config/samples/chatQnA_dataprep_xeon.yaml
kubectl apply -f $(pwd)/config/samples/chatQnA_dataprep_xeon.yaml

# Wait until the router service is ready
Expand Down Expand Up @@ -331,7 +331,7 @@ function validate_chatqna_in_switch() {
kubectl create ns $CHATQNA_SWITCH_NAMESPACE
sed -i "s|namespace: switch|namespace: $CHATQNA_SWITCH_NAMESPACE|g" $(pwd)/config/samples/chatQnA_switch_xeon.yaml
# workaround for issue #268
yq -i '(.spec.nodes.root.steps[] | select ( .name == "Tgi")).internalService.config.MODEL_ID = "bigscience/bloom-560m"' $(pwd)/config/samples/chatQnA_switch_xeon.yaml
#yq -i '(.spec.nodes.root.steps[] | select ( .name == "Tgi")).internalService.config.MODEL_ID = "bigscience/bloom-560m"' $(pwd)/config/samples/chatQnA_switch_xeon.yaml
kubectl apply -f $(pwd)/config/samples/chatQnA_switch_xeon.yaml

# Wait until the router service is ready
Expand Down Expand Up @@ -452,7 +452,7 @@ function validate_modify_config() {
fi

#change the model id of the step named "Tgi" in the codegen_xeon_mod.yaml
yq -i '(.spec.nodes.root.steps[] | select ( .name == "Tgi")).internalService.config.MODEL_ID = "bigscience/bloom-560m"' $(pwd)/config/samples/codegen_xeon_mod.yaml
yq -i '(.spec.nodes.root.steps[] | select ( .name == "Tgi")).internalService.config.MODEL_ID = "HuggingFaceH4/mistral-7b-grok"' $(pwd)/config/samples/codegen_xeon_mod.yaml
kubectl apply -f $(pwd)/config/samples/codegen_xeon_mod.yaml

pods_count=$(kubectl get pods -n $MODIFY_STEP_NAMESPACE -o jsonpath='{.items[*].metadata.name}' | wc -w)
Expand Down
1 change: 1 addition & 0 deletions helm-charts/common/retriever-usvc/templates/configmap.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,3 +32,4 @@ data:
LANGCHAIN_API_KEY: {{ .Values.global.LANGCHAIN_API_KEY | quote }}
LANGCHAIN_PROJECT: "opea-retriever-service"
HF_HOME: "/tmp/.cache/huggingface"
HUGGINGFACEHUB_API_TOKEN: {{ .Values.global.HUGGINGFACEHUB_API_TOKEN | quote}}
1 change: 1 addition & 0 deletions helm-charts/common/retriever-usvc/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,3 +92,4 @@ global:
no_proxy: ""
LANGCHAIN_TRACING_V2: false
LANGCHAIN_API_KEY: "insert-your-langchain-key-here"
HUGGINGFACEHUB_API_TOKEN: "insert-your-huggingface-token-here"
2 changes: 1 addition & 1 deletion helm-charts/common/tgi/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ image:
repository: ghcr.io/huggingface/text-generation-inference
pullPolicy: IfNotPresent
# Overrides the image tag whose default is the chart appVersion.
tag: "latest-intel-cpu"
tag: "2.2.0"

imagePullSecrets: []
nameOverride: ""
Expand Down
1 change: 1 addition & 0 deletions microservices-connector/config/manifests/retriever-usvc.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ data:
LANGCHAIN_API_KEY: "insert-your-langchain-key-here"
LANGCHAIN_PROJECT: "opea-retriever-service"
HF_HOME: "/tmp/.cache/huggingface"
HUGGINGFACEHUB_API_TOKEN: "insert-your-huggingface-token-here"
---
# Source: retriever-usvc/templates/service.yaml
# Copyright (C) 2024 Intel Corporation
Expand Down
2 changes: 1 addition & 1 deletion microservices-connector/config/manifests/tgi.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ spec:
optional: true
securityContext:
{}
image: "ghcr.io/huggingface/text-generation-inference:latest-intel-cpu"
image: "ghcr.io/huggingface/text-generation-inference:2.2.0"
imagePullPolicy: IfNotPresent
volumeMounts:
- mountPath: /data
Expand Down
2 changes: 1 addition & 1 deletion microservices-connector/config/samples/chatQnA_switch_xeon.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,5 +120,5 @@ spec:
serviceName: tgi-service-llama
config:
endpoint: /generate
MODEL_ID: bigscience/bloom-560m
MODEL_ID: HuggingFaceH4/mistral-7b-grok
isDownstreamService: true
Loading