Deploying Machine Learning Models with Flask, Docker, Jenkins and Kubernetes

Posted on 标签 ,

This article introduces an automated approach to deploying machine learning models using Flask, Docker, Jenkins and Kubernetes. The basic principle: Flask provides RESTful API to receive client prediction requests; Docker packages the service into a docker image for easy deployment and migration; Jenkins triggers automatic builds when code or models are updated; Kubernetes manages containers for scalability and reliability. This article is based on Deploy a machine learning model in 10 minutes with Flask, Docker, and Jenkins with improvements and extensions, such as a simple shell script to trigger Jenkins and Kubernetes deployment instructions. All code is available at DeployMachineLearningModel.

Follow the steps below. To avoid issues, don’t configure on Windows—while these tools have Windows versions, they often have various problems. Also don’t use Windows 10 WSL, because Docker involves Linux’s underlying cgroup, which can’t be installed directly in WSL. The experiments in this article were initially attempted in those two environments with much wasted time before finally succeeding in Ubuntu 16.04 on VirtualBox.

The figure below (from the aforementioned article) clearly shows the entire deployment and access process:

deploy model

Flask Provides RESTful API

Flask’s main role is providing RESTful API for client predictions. AI services from Google, Microsoft, Face++ (face recognition, emotion recognition, etc.) are basically provided through RESTful API. The basic principle: clients send samples for prediction via POST request; the server extracts samples, predicts, and returns results. Usually identity verification via ID is needed for billing, but we’ll ignore that for simplicity.

Flask makes it very easy to set up an HTTP Server listening on a specified port. If a POST request is received, it calls the model to predict and returns results. First, we need to train the model and load it into memory. For simplicity, we’ll use sklearn’s built-in iris classification.

Train and Save Model

Code for training and persisting the model (train_model.py):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
# coding: utf-8
import pickle
from sklearn import datasets
from sklearn.model_selection import train_test_split
from sklearn import tree

# simple demo for training and saving model
iris=datasets.load_iris()
x=iris.data
y=iris.target

#labels for iris dataset
labels ={
  0: "setosa",
  1: "versicolor",
  2: "virginica"
}