Deploy ML model with flask app

This is a simple Flask app that demonstrates how to deploy a trained ML model to API endpoint. The ML model is trained and saved as a pickle file. Showing how we trained the model is out of the scope of this exercise.

File tree

• app/

  • data/
    • col_dtypes.json
    • dataset.csv
  • model/
    • lg_pipeline_v1_0.pkl
    • lg_pipeline_v1_1.pkl
  • app.py
  • gunicorn.conf.py
  • swagger.yml

• deploy/

  • azure_setup_serviceprincipal.txt
  • config_file_format.md
  • deploy.sh
  • deploy_multi_container_app.sh
  • login.sh

• nginx/

  • Dockerfile
  • scoringapp.conf

• tests/

  • Azure/
    • Metrics.png
  • local_deployment/
    • Docker.png
    • Postman.png

• .dockerignore

• .gitignore

• Dockerfile

• README.md

• docker-compose-azure.yml

• docker-compose.yml

• requirements.txt

app directory

This directory includes data, the model (trained with scikit-learn and saved as the pipeline as a pickle file), and app.py file that defines the flask application. swagger.yml defines the outline of Swagger UI which can be reached after the model deployment through this link: http://your_app_main_url/apidocs. gunicorn.conf.py file defines the settings for guinicorn.

app.py : Environment variables

You can store environment variables in .env file at dev/test time. These variables are loaded with dotenv library as follows:

from dotenv import load_dotenv
load_dotenv()

.env file content can look like this:

# .env
JWT_SECRET_KEY=some_random_token #Random token
JWT_ACCESS_TOKEN_EXPIRES=600  # Set your desired expiration time in seconds

Note

Do not include .env file in the public repository such as GitHub. Add .env in .gitignore file to make sure that it will not be accidentally pushed to a public repository.

Run model locally with Docker

docker-compose -f docker-compose.yml up

Set up

To begin, go to Azure portal and create a resource group. Then create Service Principal and Azure Key Vault. Next, assign Key Vault Secrets Officer role using Azure RBAC and login with service principal credentials. Store your secrets in Azure Key Vault. You will need to log in with your Service Principal and retrieve the secrets to further use them in your app's environment configuration. This is one of secure ways of working with secrets (e.g., username, password, token etc.).

Important

For CLI commands, refer to ./deploy/azure_setup_serviceprincipal.txt

Create config.env file and store these parameters in it:

AZURE_SUBSCRIPTION_ID=<value>
AZURE_TENANT_ID=<value>
AZURE_CLIENT_ID=<value>
AZURE_CLIENT_SECRET=<value>

You can obtain these values when you create Service Principal. Do not store config.env file in a public repository. Once you have completed setting up resources and configurations you are ready to deploy the app.

Deploy single container app on Azure

Use ./deploy/deploy.sh script to deploy on Azure. In bash, run the script by typing: bash ./deploy/deploy.sh.
The script does the followings:

• Logs in to your Azure account by using ./deploy/login.sh script
• Retrieves secrets from Azure Key Vault
• Creates Azure Container Registry (ACR) resource
• Builds a container image in ACR by using Dockerfile in the root directory
• Creates App Service Plan
• Creates App Service, pulls the image from ACR and runs in the App Service
• Adds secrets as environment variables in App Service.

Note that it might take some time to finish the deployment after running the script. You can monitor the deployment process by navigating to Azure App Service > Deployment center > Logs.

Deploy a multi-container group using Docker Compose

Use ./deploy/deploy_multi_container_app.sh script to deploy on Azure. The script does the followings:

• Logs in to your Azure account by using ./deploy/login.sh script
• Retrieves secrets from Azure Key Vault
• Creates Azure Container Registry (ACR) resource
• Container images:
  • For Flask app, builds a container image in ACR by using Dockerfile in the root directory
  • For nginx, builds a container image in ACR by using Dockerfile in ./nginx/Dockerfile
• Creates App Service Plan
• Creates App Service, instructs the App Service to build multicontainer app by using docker-compose-azure.yml file. This pulls the images from ACR and runs them in the App Service.
• Adds secrets as environment variables in App Service.

Logging

If needed, configure customized logging for the app. The logs for all containers (Linux/custom containers) can be downloaded through the following link:
https://<app-name>.scm.azurewebsites.net/api/logs/docker/zip. For this specific exercise, I have used loguru library.

Security

Azure has various security services for Azure App Service. For more info navigate to this page: https://learn.microsoft.com/en-us/azure/app-service/overview-security