DRUM, The Open Source Tool MLOps Has Been Waiting For
Part 1: The Basics
Introduction:
What is MLOps? A technical definition that can be found on ml-ops.org states the following:
“The extension of the DevOps methodology to include Machine Learning and Data Science assets as first-class citizens within the DevOps ecology” — SIG MLOps
To understand this we first need a little bit of history. Deploying machine learning into production environments has been a historically tricky problem. For reasons, we will discuss in a minute, many of the tried and true practices of DevOps don’t have a natural extension to the ML world. The article, “Why do 87% of data science projects never make it into production?” by Venture Beat describes the world of AI implementation in industry and concludes that only 13% of companies' data science projects make it into production. The article explains many of the reasons for the gap in deployment but here’s a quick summary of the largest problems.
- Data is messy — Models are created in “digital labs” and extending them to the messy world of real data often breaks data science pipelines or seriously underperforms.
- Data is hard to get — This is true internally as well as externally. Using data science for next-level business insights requires that various departments within a company can come together and aggregate their data in a prescribed way. In general, this isn’t how things work. Check out my old blog post on some of the many reasons why this is.
- The data science app lifecycle is fundamentally different from the software development lifecycle.
For the rest of this blog, we are going to focus on the third item, and how DRUM — DataRobot User Models — can help solve this problem.
DRUM
DRUM is an open-source MLOps package that seeks to streamline and simplify the process of deploying machine learning artifacts into a production environment. The clash between the data science and software lifecycles has been well known for some time. Recently, the popular solution to fix this ingrained issue is to separate the deployment process for each side and have the software, make requests to the data science side for information/predictions. DRUM implements this idea through the creation of an API from which to call our model artifact. The unbelievably awesome part, is the API comes pre-configured ready to go and has built-in support for the majority of popular machine learning frameworks. The rest of this article will focus on the features which can be found in the DRUM tutorial which was presented at ODSC East 2021. The GitHub repo can be found here. The tutorial comes in a jupyter notebook that I highly recommend running in colab for the best experience. This tutorial is meant to demo the various features of DRUM in an intractable way. DRUM will ultimately run on a server environment without a graphical interface when pushed into production.
What Does DRUM Have To Offer?
DRUM is a simple pip install for basic functionality. The DRUM repo quickstart will suggest installing various other requirements for the demo. See the README for details on the requirements.txt file.
pip install datarobot-drumWith the installation, we get a command-line interface (CLI) that’s available in the scope in which DRUM was installed.
Model Directories
Because DRUM comes configured out of the box, there are some formatting requirements that all users need to be aware of. I’ll take you through the general requirements for the directory which will hold your model artifact and will be the subject of the majority of DRUM’s commands.
Let’s assume we have a trained sk-learn SVMClassifier model that we want to deploy using DRUM. The first thing I’m going to do is take the model artifact and store it inside a directory. For this example, we’ll say we have a pickled model.
svm-classifier
--svm-classifier.pkl
--custom.pyThe only other required file that MUST be included is a python file named custom.py. Now, this can’t be any old python file. It must implement several functions to act as hooks that DRUM can use to interact with the model artifact. So how do we do this exactly? In this case, there’s no need to reinvent the wheel because DRUM comes with preconfigured custom.py files for several of the most popular model types. You can either head over to the DRUM repo, or use the “drum new” command like this…
%%shell
drum new model-dir-name \
--code-dir ./path_to_new_dir \
--language python_or_rThe new command will create the custom.py file with all hook functions pre-configured but commented out. You’ll also find a short README file. I highly recommend going through the preconfigured custom.py file and inspecting the various hooks. A future blog with cover some use cases as the hooks are what will allow us to customize our workflow to be a true full-stack ML deployment pipeline.
With a directory that satisfies the DRUM configuration requirements, let’s go ahead and inspect some of DRUM’s core functionality.
The CLI
This interface is one of the shining pieces of the DRUM package. We won’t cover everything, but it’s worth looking at some of the main commands.
validation
As you might expect, this command is a health check that demonstrates if a model is ready for deployment. Building the validation command into the deployment pipeline is a must for anyone who wants to remain sane. Validating a model is easy as pie.
%%shelldrum validation --code-dir ./path_to_model_dir \
--input ./path_to_data \
--target-type binary\ #binary is an example
--positive-class-label True \ #use your actual label
--negative-class-label False #use your actual label
We call “drum validation” and point DRUM to the directory that contains our model. We then specify our testing file (readmissions_short.csv in this case), the type of model, and labels. This command will test the model with the specified data file and notify you if there are any failures.
pref-test
This is the performance test command. A slick feature included in DRUM, the performance test command will run the model through various size portions of a provided data file and report the results to STDOUT. Here’s an example of it’s use.
%%shell
drum perf-test --code-dir ./path_to_model_dir \
--input ./path_to_data \
--target-type binary \ #binary is an example
--positive-class-label True \ #use your actual label
--negative-class-label False \ #use your actual label# optional verbose command but you should know it exists
--verbose
score
Perhaps the command every reader is waiting for, the score command generates batch predictions.
%%shell
drum score --code-dir --code-dir ./path_to_model_dir \
--input ./path_to_data \
--output ./path_to_created_file \
--target-type binary \ #binary is an example
--positive-class-label True \ #use your actual label
--negative-class-label False \ #use your actual labelUsing the same workflow as the other commands, we add the additional “output” flag to shunt our predictions into a file that will be populated with the prediction data.
server
The true gem of DRUM! This is why we’re here. DRUM has a built-in server command that will use a flask server by default but can be configured to be used with nginx servers if they are installed on the host machine. Let’s run through what such a command might look like.
%%shelldrum server \
--code-dir ./path_to_model_dir \
--address 0.0.0.0:6789 \
--show-pref \
--target-type binary \ #binary is an example
--positive-class-label True \ #use your actual label
--negative-class-label False \ #use your actual label
--logging-level info \
--verbose \
# --production \
# --max-workers 5 \
By now you’ve noticed the similar flow that each command has. This template will be incredibly useful when containerizing our API but we’ll cover that in a subsequent blog post as well. Looking at the shell command the only mandatory flag that we must include here is the “address” flag which tells DRUM which port to open and serve our model on. For dev purposes, we can test this functionality out on our localhost, either in colab or in jupyter. There are several predetermined routes in the API that allow up to utilize our model and monitor various server stats. We can go ahead and ping the model server with the following request.
requests.request("GET", "http://0.0.0.0:6789/").content# returns {"message": "OK"}\n
Getting predictions is slightly more complicated given that we need to send a POST request with our data encoded as a BytesIO class. The following example assumes you have your data in a pandas DataFrame.
def score(data, port = "6789"):
b_buf = BytesIO()
b_buf.write(data.to_csv(index=False).encode("utf-8"))
b_buf.seek(0)
url = "http://localhost:{}/predict/".format(port)
files = [
('X', b_buf)
]
response = requests.request(
"POST",
url,
files = files,
timeout=None,
verify=False
) return response
If you notice the URL for the request you’ll see that the endpoint which will process our data and send back our predictions is …/predict/ endpoint. Take some time to google the BytesIO class in the score function if anything’s unclear. Sending a bytes class is one of the standard ways to transmit info in a POST request, and the knowledge will serve you well in the future!
If this were a production environment we might start worrying about utilizing resources for a server at all times. For this reason, there is also a ./shutdown endpoint. This one is remarkably easy. Sending an empty POST request to the shutdown endpoint will terminate the server.
requests.request("POST", "http://0.0.0.0:6789/shutdown").content# the .content method is only to confirm the server has shutdown
# on successful shutdown it will return the following response:
# b'Server shutting down...'
There are many more routes built-in and that can be configured by digging into the source code. Check out the docs if you want to know more.
But what about the production side? Well, to be honest, there isn’t much difference. If you spin up a VM with python in the cloud you can pip install all the requirements and set up the API just as shown here. The main difference will be the URL for any request will live on the internet instead of your local machine. Now, that being said, doing this by hand every time you want to have a client get predictions would be a massive pain. That’s where containers will come in handy, but more on that at a later date.
In The End
DRUM is pretty slick and is worth getting excited about. Picture it, you work for a company and the finance department wants some ML forecasting for their next year's budget considerations. Because this is a real company, their data is likely in excel or some data warehouse that isn’t at all user-friendly. That’s ok because you know about DRUM! You go into the digital lab and whip up an LSTM on keras and throw that sucka into a DRUM model directory with some custom hooks to boot. You or a college design a simple form-like front end for the finance folks to use that uploads their data into the browser and sends a request to the API for predictions. You route the response back to the front-end for download and they walk away happy. If there are problems and the front-end breaks, it doesn’t affect your model so you’re happy (unless you’re also the front-end person, in which case good luck)!
Other departments start hearing the stories told by finance and how easy it was to get something other than a, “we’ll look into it” response from IT. Soon they start coming to you wanting their own versions of the finance app. Look at that, you brought utilizing machine learning from the textbook into reality.
For Next Time
The dream of working with and managing a working MLOps show isn’t quite as simple as all this, but, it’s not as far off as you think. Next time we’ll go over some other features in DRUM that deal with one of the other problems or production machine learning models: Processing and monitoring. The hooks that I mentioned earlier are going to come back in a big way as they will define how we are going to preprocess our data before sending it to the model. Don’t worry, DRUM also has tools that allow for serialized processing pipelines to plug right into the ML stack, just as it would with code executions in jupyter. We can also write humility rules that will aid our pipeline in handling new data that may not look like that which the model was trained on. I look forward to seeing you next time!
References
- Bruce, Ryan. “Drumsticks Resting On Snare Photo.” Burst. https://burst.shopify.com/photos/drumsticks-resting-on-snare
