Skip to content

Commit

Permalink
llm-uservice: Adapt to API change
Browse files Browse the repository at this point in the history
Adapt to API change for the following service:

- llm-docsum
- llm-faqgen

Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
  • Loading branch information
lianhao committed Jan 17, 2025
1 parent 378b984 commit 11d3bd9
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions helm-charts/common/llm-uservice/templates/tests/test-pod.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,10 @@ spec:
- |
{{- if contains "llm-docsum" .Values.image.repository }}
url="http://{{ include "llm-uservice.fullname" . }}:{{ .Values.service.port }}/v1/docsum";
body='{"query":"Text Embeddings Inference (TEI) is a toolkit for deploying and serving open source text embeddings and sequence classification models. TEI enables high-performance extraction for the most popular models, including FlagEmbedding, Ember, GTE and E5.","max_tokens":17}';
body='{"messages":"Text Embeddings Inference (TEI) is a toolkit for deploying and serving open source text embeddings and sequence classification models. TEI enables high-performance extraction for the most popular models, including FlagEmbedding, Ember, GTE and E5.","max_tokens":17}';
{{- else if contains "llm-faqgen" .Values.image.repository }}
url="http://{{ include "llm-uservice.fullname" . }}:{{ .Values.service.port }}/v1/faqgen";
body='{"query":"Text Embeddings Inference (TEI) is a toolkit for deploying and serving open source text embeddings and sequence classification models. TEI enables high-performance extraction for the most popular models, including FlagEmbedding, Ember, GTE and E5.","max_tokens":17}';
body='{"messages":"Text Embeddings Inference (TEI) is a toolkit for deploying and serving open source text embeddings and sequence classification models. TEI enables high-performance extraction for the most popular models, including FlagEmbedding, Ember, GTE and E5.","max_tokens":17}';
{{- else }}
url="http://{{ include "llm-uservice.fullname" . }}:{{ .Values.service.port }}/v1/chat/completions";
body='{"model": "{{ .Values.LLM_MODEL_ID }}", "messages": [{"role": "user", "content": "What is Deep Learning?"}], "max_tokens":17}';
Expand Down

0 comments on commit 11d3bd9

Please sign in to comment.