-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
9 changed files
with
139 additions
and
2 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
name: dbt-transform | ||
kind: dbt | ||
metadata: | ||
description: 'A simple DBT transformation operation example that converts the | ||
input file (employee table dataitem) to another table using a SQL-based | ||
transformation. Run it with e.g. inputs={"employees": di.key}, | ||
and outputs={"output_table": "department-60"})' | ||
version: 91da740dccfc42ee8acf82c1f9165439 | ||
labels: | ||
- dbt | ||
- etl | ||
spec: | ||
source: | ||
base64: V0lUSCB0YWIgQVMgKAogICAgU0VMRUNUICAqCiAgICBGUk9NICAgIHt7IHJlZignZW1wbG95ZWVzJykgfX0KKQpTRUxFQ1QgICoKRlJPTSAgICB0YWIKV0hFUkUgICB0YWIuIkRFUEFSVE1FTlRfSUQiID0gJzYwJwo= | ||
lang: sql | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
name: echoservice | ||
kind: python | ||
metadata: | ||
description: A simple python-based API responding to a POST message with the | ||
"text" field. The function exposes an API using "serve" task. | ||
version: 5cdc18d933df4efb9d157f0c460db724 | ||
labels: | ||
- python | ||
- api | ||
spec: | ||
python_version: PYTHON3_10 | ||
source: | ||
source: src/serve.py | ||
handler: serve | ||
base64: aW1wb3J0IGpzb24KCmRlZiBpbml0KGNvbnRleHQpOgogICAgcHJpbnQoInNvbWUgaW5pdGlhbGl6YXRpb24gZnVuY3Rpb24iKQogICAgc2V0YXR0cihjb250ZXh0LCAidmFsdWUiLCAic29tZSB2YWx1ZSIpCgpkZWYgc2VydmUoY29udGV4dCwgZXZlbnQpOgoKICAgIGlmIGlzaW5zdGFuY2UoZXZlbnQuYm9keSwgYnl0ZXMpOgogICAgICAgIGJvZHkgPSBqc29uLmxvYWRzKGV2ZW50LmJvZHkpCiAgICBlbHNlOgogICAgICAgIGJvZHkgPSBldmVudC5ib2R5CiAgICBjb250ZXh0LmxvZ2dlci5pbmZvKGYiUmVjZWl2ZWQgZXZlbnQ6IHtib2R5fSIpCiAgICB0ZXh0ID0gYm9keVsidGV4dCJdCgogICAgcmV0dXJuIHsicmVzdWx0IjogZiJoZWxsbyB7dGV4dH0gZnJvbSAne2NvbnRleHQudmFsdWV9JyJ9 | ||
lang: python | ||
init_function: init | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
name: llm-serving | ||
kind: huggingfaceserve | ||
metadata: | ||
description: Expose OpenAI - compatible API based on a model from HugginFace. | ||
The model should be one of the models/derivatives supported by vLLM or | ||
compatible with the Huggingface transformer library. The examples deploys | ||
Qwen/Qwen2.5-0.5B model as an example. | ||
version: 2184afc12d9e40e593da77c16ddaf969 | ||
labels: | ||
- openai | ||
- huggingface | ||
- llm | ||
spec: | ||
model_name: model | ||
path: huggingface://Qwen/Qwen2.5-0.5B |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
name: mlflow-server | ||
kind: mlflowserve | ||
metadata: | ||
name: mlflow-server | ||
description: An Open Inference-compatible API that exposes trained ML Model in | ||
the MLFlow format. The model is specified as a reference to the Model entity | ||
already present in the project. | ||
version: 12a6081daaf640f792ea726786ad255f | ||
labels: | ||
- open inference | ||
- mlflow | ||
spec: | ||
model_name: model | ||
path: store://test1/model/mlflow/model-mlflow |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
name: mlflow_train_model | ||
kind: python | ||
metadata: | ||
description: "Train a cancer classifier ML model using MLFlow framework to | ||
structure and represent the model data. The trained model then can be | ||
deployed with the MLFlowServe framework or used for batch classifications | ||
loading it using the MLFlow framework. " | ||
version: 1cf74099137f462bae03d73f1bbfceb1 | ||
labels: | ||
- machine learning | ||
- mlflow | ||
spec: | ||
requirements: | ||
- scikit-learn==1.5.0 | ||
- mlflow==2.15.1 | ||
source: | ||
lang: python | ||
source: src/train.py | ||
base64: ZnJvbSBkaWdpdGFsaHViX3J1bnRpbWVfcHl0aG9uIGltcG9ydCBoYW5kbGVyCgpmcm9tIGRpZ2l0YWxodWIgaW1wb3J0IGZyb21fbWxmbG93X3J1bgppbXBvcnQgbWxmbG93Cgpmcm9tIHNrbGVhcm4gaW1wb3J0IGRhdGFzZXRzLCBzdm0KZnJvbSBza2xlYXJuLm1vZGVsX3NlbGVjdGlvbiBpbXBvcnQgR3JpZFNlYXJjaENWCgpAaGFuZGxlcigpCmRlZiB0cmFpbihwcm9qZWN0KToKICAgIG1sZmxvdy5za2xlYXJuLmF1dG9sb2cobG9nX2RhdGFzZXRzPVRydWUpCgogICAgaXJpcyA9IGRhdGFzZXRzLmxvYWRfaXJpcygpCiAgICBwYXJhbWV0ZXJzID0geyJrZXJuZWwiOiAoImxpbmVhciIsICJyYmYiKSwgIkMiOiBbMSwgMTBdfQogICAgc3ZjID0gc3ZtLlNWQygpCiAgICBjbGYgPSBHcmlkU2VhcmNoQ1Yoc3ZjLCBwYXJhbWV0ZXJzKQoKICAgIGNsZi5maXQoaXJpcy5kYXRhLCBpcmlzLnRhcmdldCkKICAgIHJ1bl9pZCA9IG1sZmxvdy5sYXN0X2FjdGl2ZV9ydW4oKS5pbmZvLnJ1bl9pZAoKICAgICMgdXRpbGl0eSB0byBtYXAgbWxmbG93IHJ1biBhcnRpZmFjdHMgdG8gbW9kZWwgbWV0YWRhdGEKICAgIG1vZGVsX3BhcmFtcyA9IGZyb21fbWxmbG93X3J1bihydW5faWQpCgogICAgcHJvamVjdC5sb2dfbW9kZWwoCiAgICAgICAgbmFtZT0ibW9kZWwtbWxmbG93IiwKICAgICAgICBraW5kPSJtbGZsb3ciLAogICAgICAgICoqbW9kZWxfcGFyYW1zCik= | ||
handler: train | ||
python_version: PYTHON3_10 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
name: python-etl | ||
kind: python | ||
metadata: | ||
description: A simple ETL procedure in Python that downloads a csv file and then | ||
returns the result as a dataframe that is stored in the project repository | ||
as table. | ||
version: 8ee36148df634014adf6ab64039477fd | ||
labels: | ||
- python | ||
- etl | ||
spec: | ||
python_version: PYTHON3_10 | ||
source: | ||
source: etl.py | ||
handler: download_and_process | ||
base64: aW1wb3J0IHBhbmRhcyBhcyBwZApmcm9tIGRpZ2l0YWxodWJfcnVudGltZV9weXRob24gaW1wb3J0IGhhbmRsZXIKClVSTD0iaHR0cHM6Ly9yYXcuZ2l0aHVidXNlcmNvbnRlbnQuY29tL2RhdGFzZXRzL3dvcmxkLWNpdGllcy9yZWZzL2hlYWRzL21haW4vZGF0YS93b3JsZC1jaXRpZXMuY3N2IgoKQGhhbmRsZXIob3V0cHV0cz1bIndvcmxkLWNpdGllcyJdKQpkZWYgZG93bmxvYWRfYW5kX3Byb2Nlc3MoKToKICAgIGRmID0gcGQucmVhZF9jc3YoVVJMKQogICAgZGYucmVzZXRfaW5kZXgoaW5wbGFjZT1UcnVlKQogICAgcmV0dXJuIGRm | ||
lang: python |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
name: sklearn-server | ||
kind: sklearnserve | ||
metadata: | ||
description: "A Open Inference-compatible API exposing a trained scikit-learn ML | ||
model. The model should already exist and is expected to be found in the | ||
project with the specified key. " | ||
version: 83cb0f2727204b99905f22c7167369de | ||
labels: | ||
- scikit-learn | ||
- service | ||
- open inference | ||
spec: | ||
model_name: model | ||
path: store://test1/model/sklearn/cancer_classifier |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
name: train_cancer_model | ||
kind: python | ||
metadata: | ||
description: A simple example of training a scikit-learn ML model using a | ||
standard dataset available. The function represents a training job that logs | ||
the resulting ML model artifact as Model in the project under | ||
'cancer_classifier' name. | ||
version: 5e064afe3b464b11a3e5c83baa51197e | ||
labels: | ||
- scikit-learn | ||
- machine learning | ||
spec: | ||
requirements: | ||
- scikit-learn==1.2.2 | ||
python_version: PYTHON3_10 | ||
source: | ||
source: src/train.py | ||
handler: breast_cancer_generator | ||
base64: aW1wb3J0IHBhbmRhcyBhcyBwZApmcm9tIHNrbGVhcm4uZGF0YXNldHMgaW1wb3J0IGxvYWRfYnJlYXN0X2NhbmNlcgoKZnJvbSBkaWdpdGFsaHViX3J1bnRpbWVfcHl0aG9uIGltcG9ydCBoYW5kbGVyCgpmcm9tIHNrbGVhcm4ubW9kZWxfc2VsZWN0aW9uIGltcG9ydCB0cmFpbl90ZXN0X3NwbGl0Cgpmcm9tIHNrbGVhcm4uc3ZtIGltcG9ydCBTVkMKZnJvbSBwaWNrbGUgaW1wb3J0IGR1bXAKaW1wb3J0IHNrbGVhcm4ubWV0cmljcwppbXBvcnQgb3MKCkBoYW5kbGVyKCkKZGVmIGJyZWFzdF9jYW5jZXJfZ2VuZXJhdG9yKHByb2plY3QpOgogICAgIiIiCiAgICBBIGZ1bmN0aW9uIHdoaWNoIGdlbmVyYXRlcyB0aGUgYnJlYXN0IGNhbmNlciBkYXRhc2V0CiAgICAiIiIKICAgIGJyZWFzdF9jYW5jZXIgPSBsb2FkX2JyZWFzdF9jYW5jZXIoKQogICAgZGZfY2FuY2VyID0gcGQuRGF0YUZyYW1lKAogICAgICAgIGRhdGE9YnJlYXN0X2NhbmNlci5kYXRhLCBjb2x1bW5zPWJyZWFzdF9jYW5jZXIuZmVhdHVyZV9uYW1lcwogICAgKQogICAgYnJlYXN0X2NhbmNlcl9sYWJlbHMgPSBwZC5EYXRhRnJhbWUoZGF0YT1icmVhc3RfY2FuY2VyLnRhcmdldCwgY29sdW1ucz1bInRhcmdldCJdKQogICAgZGZfY2FuY2VyID0gcGQuY29uY2F0KAogICAgICAgIFtkZl9jYW5jZXIsIGJyZWFzdF9jYW5jZXJfbGFiZWxzXSwgYXhpcz0xCiAgICApCgogICAgWCA9IGRmX2NhbmNlci5kcm9wKFsndGFyZ2V0J10sYXhpcz0xKQogICAgeSA9IGRmX2NhbmNlclsndGFyZ2V0J10KICAgIFhfdHJhaW4sIFhfdGVzdCwgeV90cmFpbiwgeV90ZXN0ID0gdHJhaW5fdGVzdF9zcGxpdChYLCB5LCB0ZXN0X3NpemUgPSAwLjIwLCByYW5kb21fc3RhdGU9NSkKICAgIHN2Y19tb2RlbCA9IFNWQygpCiAgICBzdmNfbW9kZWwuZml0KFhfdHJhaW4sIHlfdHJhaW4pCiAgICB5X3ByZWRpY3QgPSBzdmNfbW9kZWwucHJlZGljdChYX3Rlc3QpCgogICAgaWYgbm90IG9zLnBhdGguZXhpc3RzKCJtb2RlbCIpOgogICAgICAgIG9zLm1ha2VkaXJzKCJtb2RlbCIpCgogICAgd2l0aCBvcGVuKCJtb2RlbC9jYW5jZXJfY2xhc3NpZmllci5wa2wiLCAid2IiKSBhcyBmOgogICAgICAgIGR1bXAoc3ZjX21vZGVsLCBmLCBwcm90b2NvbD01KQoKICAgIG1ldHJpY3MgPSB7CiAgICAgICAgImYxX3Njb3JlIjogc2tsZWFybi5tZXRyaWNzLmYxX3Njb3JlKHlfdGVzdCwgeV9wcmVkaWN0KSwKICAgICAgICAiYWNjdXJhY3kiOiBza2xlYXJuLm1ldHJpY3MuYWNjdXJhY3lfc2NvcmUoeV90ZXN0LCB5X3ByZWRpY3QpLAogICAgICAgICJwcmVjaXNpb24iOiBza2xlYXJuLm1ldHJpY3MucHJlY2lzaW9uX3Njb3JlKHlfdGVzdCwgeV9wcmVkaWN0KSwKICAgICAgICAicmVjYWxsIjogc2tsZWFybi5tZXRyaWNzLnJlY2FsbF9zY29yZSh5X3Rlc3QsIHlfcHJlZGljdCksCiAgICB9CiAgICBwcm9qZWN0LmxvZ19tb2RlbCgKICAgICAgICAgICAgbmFtZT0iY2FuY2VyX2NsYXNzaWZpZXIiLAogICAgICAgICAgICBraW5kPSJza2xlYXJuIiwKICAgICAgICAgICAgc291cmNlPSIuL21vZGVsLyIsCiAgICAgICAgICAgIG1ldHJpY3M9bWV0cmljcwogICAgKQ== | ||
lang: python | ||
|