azureml model development-deployment without mlflow.ipynb

{
  "cells": [
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      },
      "source": [
        "# ML model development and Deployment using azureML (without mlflow)\n",
        "\n",
        "When you run a script as an Azure Machine Learning job, you need to define the execution context for the job run. One key configuration is the compute target on which the script will be run. This could be the local workstation (in this case the compute instance), or a remote compute target such as the Azure Machine Learning managed compute cluster that is provisioned on-demand.\n",
        "\n",
        "In this notebook, you'll create a compute cluster and explore compute targets for jobs.\n",
        "\n",
        "## Before you start\n",
        "\n",
        "You'll need the latest version of the  **azureml-ai-ml** package to run the code in this notebook. Run the cell below to verify that it is installed.\n",
        "\n",
        "> **Note**:\n",
        "> If the **azure-ai-ml** package is not installed, run `pip install azure-ai-ml` to install it."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 33,
      "metadata": {
        "gather": {
          "logged": 1665745893251
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Name: azure-ai-ml\n",
            "Version: 1.6.0\n",
            "Summary: Microsoft Azure Machine Learning Client Library for Python\n",
            "Home-page: https://github.com/Azure/azure-sdk-for-python\n",
            "Author: Microsoft Corporation\n",
            "Author-email: azuresdkengsysadmins@microsoft.com\n",
            "License: MIT License\n",
            "Location: c:\\users\\anshu\\appdata\\roaming\\python\\python310\\site-packages\n",
            "Requires: azure-common, azure-core, azure-mgmt-core, azure-storage-blob, azure-storage-file-datalake, azure-storage-file-share, colorama, isodate, jsonschema, marshmallow, msrest, opencensus-ext-azure, pydash, pyjwt, pyyaml, strictyaml, tqdm, typing-extensions\n",
            "Required-by: \n",
            "Note: you may need to restart the kernel to use updated packages.\n"
          ]
        }
      ],
      "source": [
        "pip show azure-ai-ml"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      },
      "source": [
        "## Connect to your workspace\n",
        "\n",
        "With the required SDK packages installed, now you're ready to connect to your workspace.\n",
        "\n",
        "To connect to a workspace, we need identifier parameters - a subscription ID, resource group name, and workspace name. The resource group name and workspace name are already filled in for you. You only need the subscription ID to complete the command.\n",
        "\n",
        "To find the necessary parameters, click on the subscription and workspace name at the top right of the Studio. A pane will open on the right.\n",
        "\n",
        "<p style=\"color:red;font-size:120%;background-color:yellow;font-weight:bold\"> Copy the subscription ID and replace **YOUR-SUBSCRIPTION-ID** with the value you copied. </p>"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "gather": {
          "logged": 1665745927409
        },
        "jupyter": {
          "outputs_hidden": false,
          "source_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      },
      "outputs": [],
      "source": [
        "# enter details of your AML workspace\n",
        "subscription_id = \"4980acc2-d47d-47e6-8498-a6b0744658d8\"\n",
        "resource_group = \"EYMAY\"\n",
        "workspace = \"aml-workspace\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {},
      "outputs": [],
      "source": [
        "from azure.identity import DefaultAzureCredential,InteractiveBrowserCredential\n",
        "credential = InteractiveBrowserCredential(tenant_id=\"13a86542-2185-4187-8e07-7512f5525c55\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {},
      "outputs": [],
      "source": [
        "from azure.ai.ml import MLClient\n",
        "from azure.identity import DefaultAzureCredential\n",
        "\n",
        "# get a handle to the workspace\n",
        "ml_client = MLClient(\n",
        "    credential, subscription_id, resource_group, workspace\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {},
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Class ManagedNetwork: This is an experimental class, and may change at any time. Please see https://aka.ms/azuremlexperimental for more information.\n",
            "The deployment request aml-workspace-276305 was accepted. ARM deployment URI for reference: \n",
            "https://portal.azure.com//#blade/HubsExtension/DeploymentDetailsBlade/overview/id/%2Fsubscriptions%2F4980acc2-d47d-47e6-8498-a6b0744658d8%2FresourceGroups%2FEYMAY%2Fproviders%2FMicrosoft.Resources%2Fdeployments%2Faml-workspace-276305\n"
          ]
        }
      ],
      "source": [
        "from azure.ai.ml.entities import Workspace\n",
        "try:\n",
        "    ws = ml_client.workspace.get(workspace)\n",
        "    print(\"workspace already exists\")\n",
        "\n",
        "except:\n",
        "    ws = Workspace(name=workspace,location=\"centralindia\")\n",
        "    ws = ml_client.workspaces.begin_create(ws)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      },
      "source": [
        "## Create a compute cluster\n",
        "\n",
        "In many cases, your local compute resources may not be sufficient to process a complex or long-running experiment that needs to process a large volume of data; and you may want to take advantage of the ability to dynamically create and use compute resources in the cloud. Azure Machine Learning supports a range of compute targets, which you can define in your workpace and use to run jobs; paying for the resources only when using them.\n",
        "\n",
        "You can create a compute cluster in [Azure Machine Learning studio](https://ml.azure.com), by using the Python SDK, or the Azure CLI. The following code cell checks your workspace for the existence of a compute cluster names `aml-cluster`, and if it doesn't exist, creates it."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "gather": {
          "logged": 1665746211525
        },
        "jupyter": {
          "outputs_hidden": false,
          "source_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Creating a new cpu compute target...\n"
          ]
        }
      ],
      "source": [
        "from azure.ai.ml.entities import AmlCompute\n",
        "\n",
        "# Name assigned to the compute cluster\n",
        "cpu_compute_target = \"cpu-cluster\"\n",
        "\n",
        "try:\n",
        "    # let's see if the compute target already exists\n",
        "    cpu_cluster = ml_client.compute.get(cpu_compute_target)\n",
        "    print(\n",
        "        f\"You already have a cluster named {cpu_compute_target}, we'll reuse it as is.\"\n",
        "    )\n",
        "\n",
        "except Exception:\n",
        "    print(\"Creating a new cpu compute target...\")\n",
        "\n",
        "    # Let's create the Azure ML compute object with the intended parameters\n",
        "    cpu_cluster = AmlCompute(\n",
        "        name=cpu_compute_target,\n",
        "        # Azure ML Compute is the on-demand VM service\n",
        "        type=\"amlcompute\",\n",
        "        # VM Family\n",
        "        size=\"STANDARD_DS11_V2\",\n",
        "        # Minimum running nodes when there is no job running\n",
        "        min_instances=0,\n",
        "        # Nodes in cluster\n",
        "        max_instances=1,\n",
        "        # How many seconds will the node running after the job termination\n",
        "        idle_time_before_scale_down=120,\n",
        "        # Dedicated or LowPriority. The latter is cheaper but there is a chance of job termination\n",
        "        tier=\"Dedicated\",\n",
        "    )\n",
        "\n",
        "    # Now, we pass the object to MLClient's create_or_update method\n",
        "    cpu_cluster = ml_client.compute.begin_create_or_update(cpu_cluster)\n"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      },
      "source": [
        "After you've created a compute cluster, you can only change the configuration for:\n",
        "\n",
        "- `min_instances`: Minimum number of nodes\n",
        "- `max_instances`: Maximum number of nodes\n",
        "- `idle_time_before_scale_down`: Idle time before scale down\n",
        "\n",
        "Currently, your compute cluster `aml-cluster` can only scale do a maximum of one node. Let's change that to two, to allow for parallel compute."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "gather": {
          "logged": 1665746829045
        },
        "jupyter": {
          "outputs_hidden": false,
          "source_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<azure.core.polling._poller.LROPoller at 0x28afa8b5030>"
            ]
          },
          "execution_count": 24,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "from azure.ai.ml.entities import AmlCompute\n",
        "\n",
        "cluster_scale = AmlCompute(\n",
        "    name=\"cpu-cluster\",\n",
        "    max_instances=2\n",
        ")\n",
        "ml_client.begin_create_or_update(cluster_scale)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      },
      "source": [
        "When the compute cluster is updated, you can verify its configuration by printing its attributes."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "metadata": {
        "gather": {
          "logged": 1665747135475
        },
        "jupyter": {
          "outputs_hidden": false,
          "source_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "AMLCompute with name cpu-cluster has a maximum of 2 nodes\n"
          ]
        }
      ],
      "source": [
        "cpu_cluster = ml_client.compute.get(\"cpu-cluster\")\n",
        "\n",
        "print (\n",
        "        f\"AMLCompute with name {cpu_cluster.name} has a maximum of {cpu_cluster.max_instances} nodes\"\n",
        "    )"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Create a script to train a model\n",
        "\n",
        "To train a model, you'll first create the **diabetes_training.py** script in the **src** folder. The script uses the **diabetes.csv** file in the same folder as the training data."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 26,
      "metadata": {},
      "outputs": [],
      "source": [
        "import os\n",
        "os.makedirs(\"src\",exist_ok=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 45,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Overwriting src/diabetes-training.py\n"
          ]
        }
      ],
      "source": [
        "%%writefile src/diabetes-training.py\n",
        "# import libraries\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "from sklearn.metrics import roc_auc_score\n",
        "from sklearn.metrics import roc_curve\n",
        "\n",
        "# load the diabetes dataset\n",
        "print(\"Loading Data...\")\n",
        "diabetes = pd.read_csv('diabetes.csv')\n",
        "\n",
        "# separate features and labels\n",
        "X, y = diabetes[['Pregnancies','PlasmaGlucose','DiastolicBloodPressure','TricepsThickness','SerumInsulin','BMI','DiabetesPedigree','Age']].values, diabetes['Diabetic'].values\n",
        "\n",
        "# split data into training set and test set\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=0)\n",
        "\n",
        "\n",
        "\n",
        "# set regularization hyperparameter\n",
        "reg = 0.01\n",
        "\n",
        "# train a logistic regression model\n",
        "print('Training a logistic regression model with regularization rate of', reg)\n",
        "model = LogisticRegression(C=1/reg, solver=\"liblinear\").fit(X_train, y_train)\n",
        "\n",
        "# calculate accuracy\n",
        "y_hat = model.predict(X_test)\n",
        "acc = np.average(y_hat == y_test)\n",
        "print('Accuracy:', acc)\n",
        "\n",
        "# calculate AUC\n",
        "y_scores = model.predict_proba(X_test)\n",
        "auc = roc_auc_score(y_test,y_scores[:,1])\n",
        "print('AUC: ' + str(auc))\n",
        "\n",
        "import os\n",
        "import joblib\n",
        "\n",
        "os.makedirs('outputs', exist_ok=True)\n",
        "#file = open(\"outputs/version.txt\",\"w\")\n",
        "#file.write(str(joblib.__version__))\n",
        "#file.close()\n",
        "joblib.dump(model, open(\"outputs/model.pkl\",\"wb\"))\n"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      },
      "source": [
        "## Run a job on a compute cluster\n",
        "\n",
        "Now, you're ready to run the job on the compute cluster you created.\n",
        "\n",
        "> **Note**:\n",
        "> The job will take some time to start as the compute cluster will need to scale from zero to one node. Once the compute cluster is ready, the script will be run. When the job has finished, the compute cluster will scale back down to zero nodes. You can review the compute cluster's status in the **Compute** page."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 46,
      "metadata": {
        "gather": {
          "logged": 1665750791983
        },
        "jupyter": {
          "outputs_hidden": false,
          "source_hidden": false
        },
        "nteract": {
          "transient": {
            "deleting": false
          }
        }
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\u001b[32mUploading src (0.53 MBs): 100%|##########| 530144/530144 [00:00<00:00, 1220313.17it/s]\n",
            "\u001b[39m\n",
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Monitor your job at https://ml.azure.com/runs/gifted_drop_6kqs1qj856?wsid=/subscriptions/4980acc2-d47d-47e6-8498-a6b0744658d8/resourcegroups/EYMAY/workspaces/aml-workspace&tid=13a86542-2185-4187-8e07-7512f5525c55\n"
          ]
        }
      ],
      "source": [
        "from azure.ai.ml import command\n",
        "\n",
        "# configure job\n",
        "job = command(\n",
        "    code=\"./src\",\n",
        "    command=\"python diabetes-training.py\",\n",
        "    environment=\"AzureML-sklearn-0.24-ubuntu18.04-py37-cpu@latest\",\n",
        "    compute=\"cpu-cluster\",\n",
        "    display_name=\"diabetes-train-cluster\",\n",
        "    experiment_name=\"diabetes-training\"\n",
        "    )\n",
        "\n",
        "# submit job\n",
        "returned_job = ml_client.create_or_update(job)\n",
        "aml_url = returned_job.studio_url\n",
        "print(\"Monitor your job at\", aml_url)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "<h4 style=\"background-color:Tomato;\">wait till the job runs successfully, </h4>"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 47,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "diabetes-train-cluster type: command\n",
            "outputs:\n",
            "  default:\n",
            "    mode: rw_mount\n",
            "    type: uri_folder\n",
            "    path: azureml://datastores/workspaceartifactstore/ExperimentRun/dcid.gifted_drop_6kqs1qj856\n",
            "environment: azureml:AzureML-sklearn-0.24-ubuntu18.04-py37-cpu@latest\n",
            "resources:\n",
            "  instance_count: 1\n",
            "  shm_size: 2g\n",
            "properties:\n",
            "  mlflow.source.git.repoURL: https://github.com/anshupandey/MLOps-with-Azure\n",
            "  mlflow.source.git.branch: main\n",
            "  mlflow.source.git.commit: 74a2b02c1726d10446088b0b76cc6223498d5708\n",
            "  azureml.git.dirty: 'True'\n",
            "  _azureml.ComputeTargetType: amlctrain\n",
            "  ContentSnapshotId: 1c6f2afd-ad63-4153-8aac-1902b34a27b3\n",
            "compute: azureml:cpu-cluster\n",
            "component:\n",
            "  name: gifted_drop_6kqs1qj856\n",
            "  display_name: diabetes-train-cluster\n",
            "  type: command\n",
            "  outputs:\n",
            "    default:\n",
            "      type: uri_folder\n",
            "      mode: rw_mount\n",
            "  command: python diabetes-training.py\n",
            "  environment: azureml:AzureML-sklearn-0.24-ubuntu18.04-py37-cpu@latest\n",
            "  code: azureml:/subscriptions/4980acc2-d47d-47e6-8498-a6b0744658d8/resourceGroups/EYMAY/providers/Microsoft.MachineLearningServices/workspaces/aml-workspace/codes/04f28144-947a-4fad-8955-d49e5120c3d3/versions/1\n",
            "  creation_context:\n",
            "    created_at: '2023-05-17T18:22:22.845478+00:00'\n",
            "    created_by: Anshu Pandey\n",
            "    created_by_type: User\n",
            "  is_deterministic: true\n",
            "services:\n",
            "  Tracking:\n",
            "    endpoint: azureml://centralindia.api.azureml.ms/mlflow/v1.0/subscriptions/4980acc2-d47d-47e6-8498-a6b0744658d8/resourceGroups/EYMAY/providers/Microsoft.MachineLearningServices/workspaces/aml-workspace?\n",
            "    type: Tracking\n",
            "  Studio:\n",
            "    endpoint: https://ml.azure.com/runs/gifted_drop_6kqs1qj856?wsid=/subscriptions/4980acc2-d47d-47e6-8498-a6b0744658d8/resourcegroups/EYMAY/workspaces/aml-workspace&tid=13a86542-2185-4187-8e07-7512f5525c55\n",
            "    type: Studio\n",
            " gifted_drop_6kqs1qj856\n"
          ]
        }
      ],
      "source": [
        "print(returned_job.display_name,returned_job,returned_job.name)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "\n",
        "    A path on your local computer\t                    mlflow-model/model.pkl\n",
        "    A path on an Azure Machine Learning Datastore\t    azureml://datastores/<datastore-name>/paths/<path_on_datastore>\n",
        "    A path from an Azure Machine Learning job\t        azureml://jobs/<job-name>/outputs/<output-name>/paths/<path-to-model-relative-to-the-named-output-location>\n",
        "    A path from an MLflow job\t                        runs:/<run-id>/<path-to-model-relative-to-the-root-of-the-artifact-location>\n",
        "    A path from a Model Asset in Azure Machine Learning Workspace\tazureml:<model-name>:<version>\n",
        "    A path from a Model Asset in Azure Machine Learning Registry\tazureml://registries/<registry-name>/models/<model-name>/versions/<version>"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 48,
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/plain": [
              "'azureml://subscriptions/4980acc2-d47d-47e6-8498-a6b0744658d8/resourceGroups/EYMAY/workspaces/aml-workspace/datastores/workspaceartifactstore/paths/ExperimentRun/dcid.gifted_drop_6kqs1qj856/outputs'"
            ]
          },
          "execution_count": 48,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# register model\n",
        "from azure.ai.ml.entities import Model\n",
        "from azure.ai.ml.constants import AssetTypes\n",
        "path = f\"azureml://jobs/{returned_job.name}/outputs/artifacts/paths/outputs\"\n",
        "\n",
        "file_model = Model(path=path,type=AssetTypes.CUSTOM_MODEL,\n",
        "                   name=\"diabetes_model\")\n",
        "\n",
        "out = ml_client.models.create_or_update(file_model)\n",
        "out.path"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 49,
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/plain": [
              "'diabetes_model'"
            ]
          },
          "execution_count": 49,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "out.name"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 50,
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/plain": [
              "'2'"
            ]
          },
          "execution_count": 50,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "out.version"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Deployment: Real time online deployment"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {},
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": 36,
      "metadata": {},
      "outputs": [],
      "source": [
        "from azure.ai.ml.entities import ManagedOnlineEndpoint\n",
        "import datetime\n",
        "\n",
        "ep_name = \"endpoint-diabetes\" \n",
        "ep = ManagedOnlineEndpoint(name = ep_name,\n",
        "                           auth_mode=\"key\")\n",
        "ep_result = ml_client.online_endpoints.begin_create_or_update(ep).result()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 37,
      "metadata": {},
      "outputs": [],
      "source": [
        "ep = ml_client.online_endpoints.get(name=\"endpoint-diabetes\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 38,
      "metadata": {},
      "outputs": [],
      "source": [
        "import os\n",
        "os.makedirs(\"score\",exist_ok=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 51,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Overwriting score/score.py\n"
          ]
        }
      ],
      "source": [
        "%%writefile score/score.py\n",
        "\n",
        "import os\n",
        "import joblib\n",
        "import json \n",
        "import logging\n",
        "import numpy\n",
        "def init():\n",
        "    global model\n",
        "    path = os.path.join(os.getenv(\"AZUREML_MODEL_DIR\"),\"outputs/model.pkl\") # AZUREML_MODEL_DIR = ./azureml-models/MODEL_NAME/VERSION\n",
        "    #logging.info(path)\n",
        "    model = joblib.load(path)\n",
        "    logging.info(\"initialization completed\")\n",
        "\n",
        "def run(raw_data):\n",
        "    logging.info(\"model: request received\")\n",
        "    data = json.loads(raw_data)[\"data\"]\n",
        "    data = numpy.array(data)\n",
        "    result = model.predict(data)\n",
        "    logging.info(\"request procsesed\")\n",
        "    return result.tolist()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 52,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Overwriting score/conda.yml\n"
          ]
        }
      ],
      "source": [
        "%%writefile score/conda.yml\n",
        "name: model-env\n",
        "channels:\n",
        "  - conda-forge\n",
        "dependencies:\n",
        "  - python=3.7\n",
        "  - numpy=1.21.2\n",
        "  - pip=21.2.4\n",
        "  - scikit-learn=0.24.2\n",
        "  - scipy=1.7.1\n",
        "  - numpy\n",
        "  - pandas\n",
        "  - pip:\n",
        "    - azureml-defaults==1.47.0\n",
        "    - applicationinsights\n",
        "    - inference-schema[numpy-support]==1.5\n",
        "    - joblib==1.2.0"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 54,
      "metadata": {},
      "outputs": [],
      "source": [
        "# deployment configuration\n",
        "from azure.ai.ml.entities import Model,ManagedOnlineDeployment,CodeConfiguration\n",
        "from azure.ai.ml.constants import AssetTypes\n",
        "\n",
        "# configure an environment\n",
        "from azure.ai.ml.entities import Environment\n",
        "env = Environment(\n",
        "    conda_file=\"score/conda.yml\",\n",
        "    image=\"mcr.microsoft.com/azureml/openmpi3.1.2-ubuntu18.04:20220708.v1\",\n",
        ")\n",
        "\n",
        "# configure an inference configuration with a scoring script\n",
        "from azure.ai.ml.entities import CodeConfiguration\n",
        "code_config = CodeConfiguration(\n",
        "        code=\"score\", scoring_script=\"score.py\"\n",
        "    )\n",
        "\n",
        "model = ml_client.models.get(name=out.name, version=out.version)\n",
        "\n",
        "# define an online deployment\n",
        "blue_deployment = ManagedOnlineDeployment(\n",
        "    name=\"blue\",\n",
        "    endpoint_name=ep_name,\n",
        "    model=model,\n",
        "    instance_type=\"Standard_F4s_v2\",\n",
        "    instance_count=1,\n",
        "    environment=env,\n",
        "    code_configuration=code_config\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 55,
      "metadata": {},
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Check: endpoint endpoint-diabetes exists\n",
            "\u001b[32mUploading score (0.0 MBs): 100%|##########| 887/887 [00:00<00:00, 3584.09it/s]\n",
            "\u001b[39m\n",
            "\n",
            "data_collector is not a known attribute of class <class 'azure.ai.ml._restclient.v2022_02_01_preview.models._models_py3.ManagedOnlineDeployment'> and will be ignored\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "..................................................................................................."
          ]
        }
      ],
      "source": [
        "blue_deploy_result = ml_client.online_deployments.begin_create_or_update(blue_deployment).result()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 56,
      "metadata": {},
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Readonly attribute principal_id will be ignored in class <class 'azure.ai.ml._restclient.v2022_05_01.models._models_py3.ManagedServiceIdentity'>\n",
            "Readonly attribute tenant_id will be ignored in class <class 'azure.ai.ml._restclient.v2022_05_01.models._models_py3.ManagedServiceIdentity'>\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "<azure.core.polling._poller.LROPoller at 0x28a8a58f580>"
            ]
          },
          "execution_count": 56,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# blue deployment takes 100 traffic\n",
        "ep.traffic = {\"blue\": 100}\n",
        "ml_client.online_endpoints.begin_create_or_update(ep)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "#### Check the status of the endpoint\n",
        "You can check the status of the endpoint to see whether the model was deployed without error:\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 57,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Name: endpoint-diabetes\n",
            "Status: Updating\n",
            "Description: None\n"
          ]
        }
      ],
      "source": [
        "\n",
        "# return an object that contains metadata for the endpoint\n",
        "endpoint = ml_client.online_endpoints.get(name=ep_name)\n",
        "\n",
        "# print a selection of the endpoint's metadata\n",
        "print(\n",
        "    f\"Name: {endpoint.name}\\nStatus: {endpoint.provisioning_state}\\nDescription: {endpoint.description}\"\n",
        ")\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 58,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "{'blue': 0}\n"
          ]
        }
      ],
      "source": [
        "# existing traffic details\n",
        "print(endpoint.traffic)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Inference"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 56,
      "metadata": {},
      "outputs": [],
      "source": [
        "os.makedirs(\"testing\",exist_ok=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 59,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Overwriting testing/sample-request.json\n"
          ]
        }
      ],
      "source": [
        "%%writefile testing/sample-request.json\n",
        "{\"data\": [\n",
        " [9,103,78,25,304,29.58219193,1.282869847,43], \n",
        " [7,115,47,52,35,41.51152348,0.079018568,23]\n",
        "]}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 60,
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/plain": [
              "'[1, 0]'"
            ]
          },
          "execution_count": 60,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# using json file to send a request using ml client\n",
        "# test the blue deployment with some sample data\n",
        "ml_client.online_endpoints.invoke(\n",
        "    endpoint_name=ep_name,\n",
        "    deployment_name=\"blue\",\n",
        "    request_file=\"testing/sample-request.json\",\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 61,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Scoring URI: https://endpoint-diabetes.centralindia.inference.ml.azure.com/score\n"
          ]
        }
      ],
      "source": [
        "print(\"Scoring URI:\", ep.scoring_uri)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 70,
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/plain": [
              "'2dnsCMdcS4f5YrPLXu982f2go0b61DZP'"
            ]
          },
          "execution_count": 70,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "endpoint_cred = ml_client.online_endpoints.get_keys(name=ep_name).primary_key\n",
        "endpoint_cred"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 71,
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/plain": [
              "b'[1, 0]'"
            ]
          },
          "execution_count": 71,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# sample testing via rest API\n",
        "import json\n",
        "import requests\n",
        "\n",
        "url = ep.scoring_uri\n",
        "api_key = endpoint_cred\n",
        "headers = {'Content-Type':'application/json', 'Authorization':('Bearer '+ api_key), 'azureml-model-deployment': 'blue' }\n",
        "data = {\"data\": [\n",
        " [9,103,78,25,304,29.58219193,1.282869847,43], \n",
        " [7,115,47,52,35,41.51152348,0.079018568,23]\n",
        "]}\n",
        "\n",
        "response = requests.post(url,data=json.dumps(data),headers=headers)\n",
        "response.content"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 72,
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<azure.core.polling._poller.LROPoller at 0x28a8a537df0>"
            ]
          },
          "execution_count": 72,
          "metadata": {},
          "output_type": "execute_result"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "................................................................."
          ]
        }
      ],
      "source": [
        "# delete the endpoint and deployment\n",
        "ml_client.online_endpoints.begin_delete(name=ep_name)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Batch Deployment"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 73,
      "metadata": {},
      "outputs": [],
      "source": [
        "from azure.ai.ml import MLClient, Input\n",
        "from azure.ai.ml.entities import BatchEndpoint, BatchDeployment, Model, AmlCompute, Data\n",
        "\n",
        "import random\n",
        "import string\n",
        "\n",
        "# Creating a unique endpoint name by including a random suffix\n",
        "allowed_chars = string.ascii_lowercase + string.digits\n",
        "endpoint_suffix = \"\".join(random.choice(allowed_chars) for x in range(5))\n",
        "endpoint_name = \"diabetes-\" + endpoint_suffix"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 74,
      "metadata": {},
      "outputs": [],
      "source": [
        "endpoint = BatchEndpoint(\n",
        "    name=endpoint_name,\n",
        "    description=\"A classifier for batch inference\",\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 75,
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<azure.ai.ml._restclient.v2022_05_01.models._models_py3.BatchEndpointData at 0x28a8a58e860>"
            ]
          },
          "execution_count": 75,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "ml_client.batch_endpoints.begin_create_or_update(endpoint).result()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 76,
      "metadata": {},
      "outputs": [],
      "source": [
        "compute_name = \"cpu-cluster\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 77,
      "metadata": {},
      "outputs": [],
      "source": [
        "os.makedirs(\"batch_scoring_script\",exist_ok=True)"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Best practices for batch deployment : https://learn.microsoft.com/en-us/azure/machine-learning/how-to-batch-scoring-script?view=azureml-api-2&tabs=cli"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 78,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Writing batch_scoring_script/batch_driver.py\n"
          ]
        }
      ],
      "source": [
        "%%writefile batch_scoring_script/batch_driver.py\n",
        "import os\n",
        "import pandas as pd\n",
        "import joblib\n",
        "import logging\n",
        "import numpy as np\n",
        "\n",
        "\n",
        "def init():\n",
        "    global model\n",
        "    # AZUREML_MODEL_DIR is an environment variable created during deployment\n",
        "    # It is the path to the model folder\n",
        "    path = os.path.join(os.getenv(\"AZUREML_MODEL_DIR\"),\"outputs/model.pkl\") # AZUREML_MODEL_DIR = ./azureml-models/MODEL_NAME/VERSION\n",
        "    model = joblib.load(path)\n",
        "    logging.info(\"initialization completed\")\n",
        "\n",
        "def run(mini_batch) -> pd.DataFrame:\n",
        "    print(f\"Executing run method over batch of {len(mini_batch)} files.\")\n",
        "\n",
        "    results = []\n",
        "    for data_batch in mini_batch:\n",
        "        logging.info(data_batch)\n",
        "        print(data_path)\n",
        "        # Read comma-delimited data into an array\n",
        "        data = np.genfromtxt(data_batch, delimiter=',')\n",
        "        logging.info(data)\n",
        "        # Reshape into a 2-dimensional array for model input\n",
        "        prediction = model.predict(data.reshape(1, -1))\n",
        "        # Append prediction to results\n",
        "        resultList.append({\"predictions\":prediction})\n",
        "\n",
        "    return pd.DataFrame(results)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 79,
      "metadata": {},
      "outputs": [],
      "source": [
        "# configure an inference configuration with a scoring script\n",
        "from azure.ai.ml.entities import CodeConfiguration\n",
        "code_config = CodeConfiguration(\n",
        "        code=\"batch_scoring_script\", scoring_script=\"batch_driver.py\"\n",
        "    )"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 80,
      "metadata": {},
      "outputs": [],
      "source": [
        "from azure.ai.ml.constants import BatchDeploymentOutputAction\n",
        "from azure.ai.ml.entities import BatchRetrySettings\n",
        "\n",
        "deployment = BatchDeployment(\n",
        "    name=\"diabetes-prediction\",\n",
        "    description=\"A diabetes classifier\",\n",
        "    endpoint_name=endpoint.name,\n",
        "    model=model,\n",
        "    compute=compute_name,\n",
        "    instance_count=1,\n",
        "    max_concurrency_per_instance=2,\n",
        "    mini_batch_size=10,\n",
        "    output_action=BatchDeploymentOutputAction.APPEND_ROW,\n",
        "    output_file_name=\"predictions.csv\",\n",
        "    retry_settings=BatchRetrySettings(max_retries=3, timeout=300),\n",
        "    logging_level=\"info\",\n",
        "    environment=env,\n",
        "    code_configuration=code_config\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 81,
      "metadata": {},
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\u001b[32mUploading batch_scoring_script (0.0 MBs): 100%|##########| 1040/1040 [00:00<00:00, 18162.23it/s]\n",
            "\u001b[39m\n",
            "\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "BatchDeployment({'deployment_type': 'Model', 'job_definition': None, 'provisioning_state': 'Succeeded', 'endpoint_name': 'diabetes-78co6', 'type': None, 'name': 'diabetes-prediction', 'description': 'A diabetes classifier', 'tags': {}, 'properties': {}, 'print_as_yaml': True, 'id': '/subscriptions/4980acc2-d47d-47e6-8498-a6b0744658d8/resourceGroups/EYMAY/providers/Microsoft.MachineLearningServices/workspaces/aml-workspace/batchEndpoints/diabetes-78co6/deployments/diabetes-prediction', 'Resource__source_path': None, 'base_path': 'd:\\\\AI\\\\MLOps\\\\EYMAY23\\\\MLOps-with-Azure', 'creation_context': <azure.ai.ml.entities._system_data.SystemData object at 0x0000028A8A5AB0A0>, 'serialize': <msrest.serialization.Serializer object at 0x0000028A8A5ABD00>, 'model': '/subscriptions/4980acc2-d47d-47e6-8498-a6b0744658d8/resourceGroups/EYMAY/providers/Microsoft.MachineLearningServices/workspaces/aml-workspace/models/diabetes_model/versions/2', 'code_configuration': {'code': '/subscriptions/4980acc2-d47d-47e6-8498-a6b0744658d8/resourceGroups/EYMAY/providers/Microsoft.MachineLearningServices/workspaces/aml-workspace/codes/181b1779-2be6-4e32-9e28-dc6e66560e5c/versions/1'}, 'environment': '/subscriptions/4980acc2-d47d-47e6-8498-a6b0744658d8/resourceGroups/EYMAY/providers/Microsoft.MachineLearningServices/workspaces/aml-workspace/environments/CliV2AnonymousEnvironment/versions/c3c9ec467c08b408c9d321e22a49ab9a', 'environment_variables': {}, 'compute': '/subscriptions/4980acc2-d47d-47e6-8498-a6b0744658d8/resourceGroups/EYMAY/providers/Microsoft.MachineLearningServices/workspaces/aml-workspace/computes/cpu-cluster', 'resources': {'instance_count': 1, 'properties': {}}, 'output_action': 'append_row', 'output_file_name': 'predictions.csv', 'error_threshold': -1, 'retry_settings': <azure.ai.ml.entities._deployment.deployment_settings.BatchRetrySettings object at 0x0000028A8A5A8550>, 'logging_level': 'Info', 'mini_batch_size': 10, 'max_concurrency_per_instance': 2})"
            ]
          },
          "execution_count": 81,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "ml_client.batch_deployments.begin_create_or_update(deployment).result()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 82,
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<azure.ai.ml._restclient.v2022_05_01.models._models_py3.BatchEndpointData at 0x28a8a5abd90>"
            ]
          },
          "execution_count": 82,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "endpoint = ml_client.batch_endpoints.get(endpoint.name)\n",
        "endpoint.defaults.deployment_name = deployment.name\n",
        "ml_client.batch_endpoints.begin_create_or_update(endpoint).result()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 83,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "The default deployment is diabetes-prediction\n"
          ]
        }
      ],
      "source": [
        "print(f\"The default deployment is {endpoint.defaults.deployment_name}\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 84,
      "metadata": {},
      "outputs": [],
      "source": [
        "data_path = \"diabetes-unlabeled.csv\"\n",
        "dataset_name = \"unlabeled-diabetes\"\n",
        "\n",
        "dataset_unlabeled = Data(\n",
        "    path=data_path,\n",
        "    type=AssetTypes.URI_FILE,\n",
        "    description=\"An unlabeled dataset for diabetes\",\n",
        "    name=dataset_name,\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 85,
      "metadata": {},
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\u001b[32mUploading diabetes-unlabeled.csv\u001b[32m (< 1 MB): 100%|##########| 86.0/86.0 [00:00<00:00, 1.20kB/s]\n",
            "\u001b[39m\n",
            "\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "Data({'skip_validation': False, 'mltable_schema_url': None, 'referenced_uris': None, 'type': 'uri_file', 'is_anonymous': False, 'auto_increment_version': False, 'name': 'unlabeled-diabetes', 'description': 'An unlabeled dataset for diabetes', 'tags': {}, 'properties': {}, 'print_as_yaml': True, 'id': '/subscriptions/4980acc2-d47d-47e6-8498-a6b0744658d8/resourceGroups/EYMAY/providers/Microsoft.MachineLearningServices/workspaces/aml-workspace/data/unlabeled-diabetes/versions/1', 'Resource__source_path': None, 'base_path': 'd:\\\\AI\\\\MLOps\\\\EYMAY23\\\\MLOps-with-Azure', 'creation_context': <azure.ai.ml.entities._system_data.SystemData object at 0x0000028A8A537C40>, 'serialize': <msrest.serialization.Serializer object at 0x0000028A8A534A90>, 'version': '1', 'latest_version': None, 'path': 'azureml://subscriptions/4980acc2-d47d-47e6-8498-a6b0744658d8/resourcegroups/EYMAY/workspaces/aml-workspace/datastores/workspaceblobstore/paths/LocalUpload/b431d23b18a17faae11c9366640b8c7f/diabetes-unlabeled.csv', 'datastore': None})"
            ]
          },
          "execution_count": 85,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "ml_client.data.create_or_update(dataset_unlabeled)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 86,
      "metadata": {},
      "outputs": [],
      "source": [
        "dataset_unlabeled = ml_client.data.get(name=dataset_name, label=\"latest\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 87,
      "metadata": {},
      "outputs": [],
      "source": [
        "input = Input(type=AssetTypes.URI_FILE, path=dataset_unlabeled.id)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 88,
      "metadata": {},
      "outputs": [
        {
          "ename": "Exception",
          "evalue": "BY_POLICY",
          "output_type": "error",
          "traceback": [
            "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[1;31mJSONDecodeError\u001b[0m                           Traceback (most recent call last)",
            "File \u001b[1;32m~\\AppData\\Roaming\\Python\\Python310\\site-packages\\azure\\ai\\ml\\_utils\\_endpoint_utils.py:116\u001b[0m, in \u001b[0;36mvalidate_response\u001b[1;34m(response)\u001b[0m\n\u001b[0;32m    115\u001b[0m \u001b[39mtry\u001b[39;00m:\n\u001b[1;32m--> 116\u001b[0m     r_json \u001b[39m=\u001b[39m response\u001b[39m.\u001b[39;49mjson()\n\u001b[0;32m    117\u001b[0m \u001b[39mexcept\u001b[39;00m \u001b[39mValueError\u001b[39;00m:\n\u001b[0;32m    118\u001b[0m     \u001b[39m# exception is not in the json format\u001b[39;00m\n",
            "File \u001b[1;32m~\\AppData\\Roaming\\Python\\Python310\\site-packages\\azure\\core\\rest\\_http_response_impl.py:293\u001b[0m, in \u001b[0;36m_HttpResponseBaseImpl.json\u001b[1;34m(self)\u001b[0m\n\u001b[0;32m    292\u001b[0m \u001b[39mif\u001b[39;00m \u001b[39mnot\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_json:\n\u001b[1;32m--> 293\u001b[0m     \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_json \u001b[39m=\u001b[39m loads(\u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49mtext())\n\u001b[0;32m    294\u001b[0m \u001b[39mreturn\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_json\n",
            "File \u001b[1;32mc:\\ProgramData\\anaconda3\\lib\\json\\__init__.py:346\u001b[0m, in \u001b[0;36mloads\u001b[1;34m(s, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)\u001b[0m\n\u001b[0;32m    343\u001b[0m \u001b[39mif\u001b[39;00m (\u001b[39mcls\u001b[39m \u001b[39mis\u001b[39;00m \u001b[39mNone\u001b[39;00m \u001b[39mand\u001b[39;00m object_hook \u001b[39mis\u001b[39;00m \u001b[39mNone\u001b[39;00m \u001b[39mand\u001b[39;00m\n\u001b[0;32m    344\u001b[0m         parse_int \u001b[39mis\u001b[39;00m \u001b[39mNone\u001b[39;00m \u001b[39mand\u001b[39;00m parse_float \u001b[39mis\u001b[39;00m \u001b[39mNone\u001b[39;00m \u001b[39mand\u001b[39;00m\n\u001b[0;32m    345\u001b[0m         parse_constant \u001b[39mis\u001b[39;00m \u001b[39mNone\u001b[39;00m \u001b[39mand\u001b[39;00m object_pairs_hook \u001b[39mis\u001b[39;00m \u001b[39mNone\u001b[39;00m \u001b[39mand\u001b[39;00m \u001b[39mnot\u001b[39;00m kw):\n\u001b[1;32m--> 346\u001b[0m     \u001b[39mreturn\u001b[39;00m _default_decoder\u001b[39m.\u001b[39;49mdecode(s)\n\u001b[0;32m    347\u001b[0m \u001b[39mif\u001b[39;00m \u001b[39mcls\u001b[39m \u001b[39mis\u001b[39;00m \u001b[39mNone\u001b[39;00m:\n",
            "File \u001b[1;32mc:\\ProgramData\\anaconda3\\lib\\json\\decoder.py:337\u001b[0m, in \u001b[0;36mJSONDecoder.decode\u001b[1;34m(self, s, _w)\u001b[0m\n\u001b[0;32m    333\u001b[0m \u001b[39m\"\"\"Return the Python representation of ``s`` (a ``str`` instance\u001b[39;00m\n\u001b[0;32m    334\u001b[0m \u001b[39mcontaining a JSON document).\u001b[39;00m\n\u001b[0;32m    335\u001b[0m \n\u001b[0;32m    336\u001b[0m \u001b[39m\"\"\"\u001b[39;00m\n\u001b[1;32m--> 337\u001b[0m obj, end \u001b[39m=\u001b[39m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49mraw_decode(s, idx\u001b[39m=\u001b[39;49m_w(s, \u001b[39m0\u001b[39;49m)\u001b[39m.\u001b[39;49mend())\n\u001b[0;32m    338\u001b[0m end \u001b[39m=\u001b[39m _w(s, end)\u001b[39m.\u001b[39mend()\n",
            "File \u001b[1;32mc:\\ProgramData\\anaconda3\\lib\\json\\decoder.py:355\u001b[0m, in \u001b[0;36mJSONDecoder.raw_decode\u001b[1;34m(self, s, idx)\u001b[0m\n\u001b[0;32m    354\u001b[0m \u001b[39mexcept\u001b[39;00m \u001b[39mStopIteration\u001b[39;00m \u001b[39mas\u001b[39;00m err:\n\u001b[1;32m--> 355\u001b[0m     \u001b[39mraise\u001b[39;00m JSONDecodeError(\u001b[39m\"\u001b[39m\u001b[39mExpecting value\u001b[39m\u001b[39m\"\u001b[39m, s, err\u001b[39m.\u001b[39mvalue) \u001b[39mfrom\u001b[39;00m \u001b[39mNone\u001b[39m\n\u001b[0;32m    356\u001b[0m \u001b[39mreturn\u001b[39;00m obj, end\n",
            "\u001b[1;31mJSONDecodeError\u001b[0m: Expecting value: line 1 column 1 (char 0)",
            "\nDuring handling of the above exception, another exception occurred:\n",
            "\u001b[1;31mException\u001b[0m                                 Traceback (most recent call last)",
            "Cell \u001b[1;32mIn[88], line 1\u001b[0m\n\u001b[1;32m----> 1\u001b[0m job \u001b[39m=\u001b[39m ml_client\u001b[39m.\u001b[39;49mbatch_endpoints\u001b[39m.\u001b[39;49minvoke(endpoint_name\u001b[39m=\u001b[39;49mendpoint\u001b[39m.\u001b[39;49mname, \u001b[39minput\u001b[39;49m\u001b[39m=\u001b[39;49m\u001b[39minput\u001b[39;49m)\n",
            "File \u001b[1;32m~\\AppData\\Roaming\\Python\\Python310\\site-packages\\azure\\core\\tracing\\decorator.py:76\u001b[0m, in \u001b[0;36mdistributed_trace.<locals>.decorator.<locals>.wrapper_use_tracer\u001b[1;34m(*args, **kwargs)\u001b[0m\n\u001b[0;32m     74\u001b[0m span_impl_type \u001b[39m=\u001b[39m settings\u001b[39m.\u001b[39mtracing_implementation()\n\u001b[0;32m     75\u001b[0m \u001b[39mif\u001b[39;00m span_impl_type \u001b[39mis\u001b[39;00m \u001b[39mNone\u001b[39;00m:\n\u001b[1;32m---> 76\u001b[0m     \u001b[39mreturn\u001b[39;00m func(\u001b[39m*\u001b[39margs, \u001b[39m*\u001b[39m\u001b[39m*\u001b[39mkwargs)\n\u001b[0;32m     78\u001b[0m \u001b[39m# Merge span is parameter is set, but only if no explicit parent are passed\u001b[39;00m\n\u001b[0;32m     79\u001b[0m \u001b[39mif\u001b[39;00m merge_span \u001b[39mand\u001b[39;00m \u001b[39mnot\u001b[39;00m passed_in_parent:\n",
            "File \u001b[1;32m~\\AppData\\Roaming\\Python\\Python310\\site-packages\\azure\\ai\\ml\\_telemetry\\activity.py:263\u001b[0m, in \u001b[0;36mmonitor_with_activity.<locals>.monitor.<locals>.wrapper\u001b[1;34m(*args, **kwargs)\u001b[0m\n\u001b[0;32m    260\u001b[0m \u001b[39m@functools\u001b[39m\u001b[39m.\u001b[39mwraps(f)\n\u001b[0;32m    261\u001b[0m \u001b[39mdef\u001b[39;00m \u001b[39mwrapper\u001b[39m(\u001b[39m*\u001b[39margs, \u001b[39m*\u001b[39m\u001b[39m*\u001b[39mkwargs):\n\u001b[0;32m    262\u001b[0m     \u001b[39mwith\u001b[39;00m log_activity(logger, activity_name \u001b[39mor\u001b[39;00m f\u001b[39m.\u001b[39m\u001b[39m__name__\u001b[39m, activity_type, custom_dimensions):\n\u001b[1;32m--> 263\u001b[0m         \u001b[39mreturn\u001b[39;00m f(\u001b[39m*\u001b[39margs, \u001b[39m*\u001b[39m\u001b[39m*\u001b[39mkwargs)\n",
            "File \u001b[1;32m~\\AppData\\Roaming\\Python\\Python310\\site-packages\\azure\\ai\\ml\\operations\\_batch_endpoint_operations.py:316\u001b[0m, in \u001b[0;36mBatchEndpointOperations.invoke\u001b[1;34m(self, endpoint_name, deployment_name, inputs, **kwargs)\u001b[0m\n\u001b[0;32m    309\u001b[0m     headers[EndpointInvokeFields\u001b[39m.\u001b[39mMODEL_DEPLOYMENT] \u001b[39m=\u001b[39m deployment_name\n\u001b[0;32m    311\u001b[0m response \u001b[39m=\u001b[39m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_requests_pipeline\u001b[39m.\u001b[39mpost(\n\u001b[0;32m    312\u001b[0m     endpoint\u001b[39m.\u001b[39mproperties\u001b[39m.\u001b[39mscoring_uri,\n\u001b[0;32m    313\u001b[0m     json\u001b[39m=\u001b[39mBatchJobResource(properties\u001b[39m=\u001b[39mbatch_job)\u001b[39m.\u001b[39mserialize(),\n\u001b[0;32m    314\u001b[0m     headers\u001b[39m=\u001b[39mheaders,\n\u001b[0;32m    315\u001b[0m )\n\u001b[1;32m--> 316\u001b[0m validate_response(response)\n\u001b[0;32m    317\u001b[0m batch_job \u001b[39m=\u001b[39m json\u001b[39m.\u001b[39mloads(response\u001b[39m.\u001b[39mtext())\n\u001b[0;32m    318\u001b[0m \u001b[39mreturn\u001b[39;00m BatchJobResource\u001b[39m.\u001b[39mdeserialize(batch_job)\n",
            "File \u001b[1;32m~\\AppData\\Roaming\\Python\\Python310\\site-packages\\azure\\ai\\ml\\_utils\\_endpoint_utils.py:119\u001b[0m, in \u001b[0;36mvalidate_response\u001b[1;34m(response)\u001b[0m\n\u001b[0;32m    116\u001b[0m         r_json \u001b[39m=\u001b[39m response\u001b[39m.\u001b[39mjson()\n\u001b[0;32m    117\u001b[0m     \u001b[39mexcept\u001b[39;00m \u001b[39mValueError\u001b[39;00m:\n\u001b[0;32m    118\u001b[0m         \u001b[39m# exception is not in the json format\u001b[39;00m\n\u001b[1;32m--> 119\u001b[0m         \u001b[39mraise\u001b[39;00m \u001b[39mException\u001b[39;00m(response\u001b[39m.\u001b[39mcontent\u001b[39m.\u001b[39mdecode(\u001b[39m\"\u001b[39m\u001b[39mutf-8\u001b[39m\u001b[39m\"\u001b[39m))\n\u001b[0;32m    120\u001b[0m failure_msg \u001b[39m=\u001b[39m r_json\u001b[39m.\u001b[39mget(\u001b[39m\"\u001b[39m\u001b[39merror\u001b[39m\u001b[39m\"\u001b[39m, {})\u001b[39m.\u001b[39mget(\u001b[39m\"\u001b[39m\u001b[39mmessage\u001b[39m\u001b[39m\"\u001b[39m, response)\n\u001b[0;32m    121\u001b[0m error_map \u001b[39m=\u001b[39m {\n\u001b[0;32m    122\u001b[0m     \u001b[39m401\u001b[39m: ClientAuthenticationError,\n\u001b[0;32m    123\u001b[0m     \u001b[39m404\u001b[39m: ResourceNotFoundError,\n\u001b[0;32m    124\u001b[0m     \u001b[39m409\u001b[39m: ResourceExistsError,\n\u001b[0;32m    125\u001b[0m }\n",
            "\u001b[1;31mException\u001b[0m: BY_POLICY"
          ]
        }
      ],
      "source": [
        "job = ml_client.batch_endpoints.invoke(endpoint_name=endpoint.name, input=input)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": []
    }
  ],
  "metadata": {
    "kernel_info": {
      "name": "python38-azureml"
    },
    "kernelspec": {
      "display_name": "base",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.10.9"
    },
    "microsoft": {
      "host": {
        "AzureML": {
          "notebookHasBeenCompleted": true
        }
      }
    },
    "nteract": {
      "version": "nteract-front-end@1.0.0"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 2
}