Launch! How to Create A Quick and Dirty Interactable Cloud-Based Dash App for a CNN Demo
Image classification is a hot topic in the medical industry and beyond. At the heart of a CNN model is the spatial distribution of information via pixels. Before diving into the model I created let’s take a quick detour back to get some motivation for how a CNN works. Feel free to skip if their old hat at this point.
How does a CNN work (High Level)
As humans, we don’t often take the time to appreciate the true complexity of being able to see something, process it, and turn that into understanding. I recommend to everyone reading this post to take 5 minutes and google what an image looks like in terms of data. Congratulations! Understanding that massive series of numbers is exactly what your eyes are doing more than a thousand times a second! — keep in mind that you see much high resolution than most digital images so the actual data would be much larger.
So how do we get from pixels to something that a computer will understand? It would be a lot of help if we had complete knowledge of how the human brain gets the job done! Good thing biologists haven’t been slacking off over the past 30 years because they had the answer: Locally connected pieces of information.
What does this mean?
It’s really a matter of perspective. When you go to the art museum (probably non-covid times) and look at a painting where do you stand? Do you stand right next to the painting? How about the middle distance away? Or are you more of a big picture kind of person? CNNs take the same approach. They begin by looking at a patch of an image and recording “what they see” before moving on to the next patch. This is done row by row until they’ve seen the entire image.
Just like people, CNNs can take on many forms. The patch a CNN looks at is called the window. The window can be any size but it’s usually best to keep them small. After all, it's hard to see all the important details from the back of the room.
CNNs also have variable speed at which they read over an image. Some models shift their window only a single pixel at a time, while others can skip entire sections depending on how much attention they want to give to small, medium, or large sections.
The last major difference is the number of times a CNN looks at an image. This is called the number of filters. Just like a good Monet, CNNs can and almost always do, pick up different details every time they look at an image.
So a CNN scans and reads images just like people do when trying to understand them. Reading for a computer, though, is a little different than what you might be used to. In general, most models are just large pipelines of matrix multiplication. For the case of a CNN, the model records what it sees by multiplying the pixel information by its starting matrix — this is usually randomly created for diversity in what the model picks upon. The information is then stored in the new matrix as this product. A CNN keeps track of all matrix products and then hands them off to the next layer in the network for further “learning.”
If you’re interested in the math behind how all this works, follow the link which gives a great visual representation of what the math is doing.
https://cs231n.github.io/convolutional-networks/
My App
This brings us back to seeing this in action. I recently completed a project in which I created a neural network to classify whether or not a patient has pneumonia based on chest x-rays. The model was built on top of the Keras Functional API and ended up highly capable of classifying patients — 93% accuracy overall with 97% recall of the pneumonia case. While I’m passionate about neural networks and the cool intricacies of how they work, it was important to me that the model be available for use. While not the fanciest app in the world, it does allow for users to upload a photo(s) of chest x-rays for immediate prediction results by the model. For anyone else interested in making this functionality (even if it’s not the cleanest interface) I’ll run through the steps to get you there.
If you want to check this out and upload an x-ray, you can find the application at
https://mod4-dash.herokuapp.com/
Create a Dash App
This tutorial will only be using the dash framework to create a python application. There are others out there but I find Dash to be the easiest, while also having a decent amount of flexibility. I’m going to go ahead and link the Dash docs because they will be your friend in all things dash.
The cloud hosting I will be demoing is Heroku, an awesome and easy-to-use platform for hosting and managing applications. There’s a free tier, it’s lackluster but then again, it’s FREE.
step 1: create a project repo
This one is easy. Go to whatever part of your machine you want to house the files for the project and create a folder. For this blog, we are going to call it ~/project_folder/
Source tracking is a must-have if you want your life to not be an endless cycle of perpetual and avoidable misery. You have git bash installed right? RIGHT. You have a Github account right? RIGHT. Okay, let’s move on.
In a terminal window, cd into the project repo (I highly recommend VS code for all things code). Initialize a repo in the root of the directory and create 2 files: app.py and Procfile. Run the following commands to do this.
git init \
touch app.py \
touch Procfile \App.py is where the code for your dash app and Keras model is going to go. Procfile is a special file that Heroku is going to use to run your app, we’ll come back to it at the end. At this time both files are empty.
step 2: create your virtual environment
Technically, everything I’m about to do can also be done in Anaconda environments. Personally, I like creating the virtual environment for any project in the project repo, but ultimately, there’s no harm in using a conda environment if that’s how you roll. I’ll admit that conda is a lot more storage-friendly so it does have its benefits. If neither of these things sound familiar, go ahead and stick with my way and look up conda environments in your free time.
Creating a virtual environment is easy!
python -m venv app-envThis command will create a virtual environment called “app-env” in the project folder. Poke around the environment folder if you’re curious but don’t be alarmed if you don’t understand everything that’s in there. The app-env folder will house the python commands and submodules that will be importing when creating your Keras CNN.
I highly recommend you add app-env to your .gitignore file. This way you can keep the tracked portion of your project folder small. If you’re working out of the terminal, run the following command to create and add this line.
echo "app-env/*" >> .gitignoreThis way, the next time you add all files to stage changes, all the files in the app-env file will be selectively ignored.
Now let's activate your environment. From the root folder run the command.
source app-env/Scripts/activateYou should see the environment activated underneath your terminal cursor. At this point, go ahead and pip install all of the packages you need in order to create your Keras model. If you’ve used a jupyter notebook then the packages you should install are all the imports you would need to run in a notebook to get your model working. It’s best practice not to install more than you need.
In addition, add the following packages.
pip install dash
pip install gunicornWe’re ready to rock and roll!
step 3: get the dash application working
There is a complete template available in the dash docs at https://dash.plotly.com/layout. I’ll post their sample page here for ease. Take this code and insert it into your app.py file.
#Run this app with `python app.py` and
# visit http://127.0.0.1:8050/ in your web browser.import dash
import dash_core_components as dcc
import dash_html_components as html
import plotly.express as px
import pandas as pd external_stylesheets = ['https://codepen.io/chriddyp/pen/bWLwgP.css'] app = dash.Dash(__name__, external_stylesheets=external_stylesheets) # assume you have a "long-form" data frame
# see https://plotly.com/python/px-arguments/ for more options df = pd.DataFrame({
"Fruit": [
"Apples",
"Oranges",
"Bananas",
"Apples",
"Oranges",
"Bananas"
],
"Amount": [4, 1, 2, 2, 4, 5],
"City": ["SF", "SF", "SF", "Montreal", "Montreal", "Montreal"]
})
fig = px.bar(df, x="Fruit", y="Amount", color="City", barmode="group")app.layout = html.Div(
children=[
html.H1(children='Hello Dash'),
html.Div(
children='''
Dash: A web application framework for Python.
'''
),
dcc.Graph(
id='example-graph',
figure=fig
)
]
)
if __name__ == '__main__':
app.run_server(debug=True)
At this point, you can go ahead and check your progress up until this point. In your terminal run the following command.
python app.pyThis should create a server and which will host your newly created app at localhost 8050. There should be a link in the terminal printout that you can click to get to the app page. The page should look like this.
Another milestone! At this point I would feel free to add “app developer” to your resume!
Joking aside, this really is everything when it comes to running a dash app. At this point, I’m going to point you to the dash docs to learn what kinds of objects you can add to the page.
step 4: create the code to allow a user to upload and get predictions from your model.
Feel free to head over to the GitHub repo for my project and take my app.py file for reference. It has some dash features that go beyond this blog post that you might find useful. Here’s a trimmed-down version that you can use for this walkthrough.
import dash
import dash_core_components as dcc
import dash_html_components as html
import dash.dependencies as dd
import plotly.express as pximport tensorflow.keras as keras
from tensorflow.keras.preprocessing.image import ImageDataGenerator, img_to_array, array_to_img, load_imgfrom PIL import Image as pilImage
import iofrom base64 import decodebytesimport datetimeimport pandas as pdimport numpy as npimport osfrom flask import Flaskexternal_stylesheets = ['https://codepen.io/chriddyp/pen/bWLwgP.css', './assests/app.css']server = Flask('mod4-dash')app = dash.Dash(__name__, external_stylesheets=external_stylesheets, server=server)#load in the classification model from the source files
modelPath = './final-model' #os.Path('final-model')
model = keras.models.load_model(modelPath)app.layout = html.Div(
children=[
html.Div(
children = [
html.H1(
children = [
'''
Classify an Image
'''],
style={
'width': '60%',
'lineHeight': 'auto',
'textAlign': 'center',
'margin': '2.5% auto',
'fontSize' : '3em',
},
),
html.P(
children = [
'''
This is some template text. Go ahead and give whatever instructions you would like!
'''],
style={
'width': '60%',
'lineHeight': 'auto',
'textAlign': 'center',
'margin': 'auto auto 2.5% auto',
'fontSize' : '1.2em',
},
), # This is the upload widget that productionized the model and auto predicts the class of the image uploaded.
dcc.Upload(
id='upload-image',
children=[
html.Div([
'Drag and Drop or ',
html.A('Select Files')
]),
html.Br()
],
style={
'width': '20%',
'height': '60px',
'lineHeight': '60px',
'borderWidth': '1px',
'borderStyle': 'dashed',
'borderRadius': '5px',
'textAlign': 'center',
'margin': 'auto',
'font-size': '20px'
},
# Allow multiple files to be uploaded
multiple=True
),
]
),
html.Div(id = 'prediction-output'),
html.Div(id='output-image-upload'),
dcc.Store(
id = 'user-session',
)
],
className='app')def parse_contents(content, filename):
try:
imageBytes = decodebytes(content.split(',')[1].encode('utf-8'))
image = pilImage.open(io.BytesIO(imageBytes))
image = image.convert('RGB')
image = imageToDisplay = image.resize((256, 256), pilImage.NEAREST)
image = img_to_array(image).reshape((1,256,256,3)) print('fail 2') generator = ImageDataGenerator(
rescale = 1./255) print('fail 5')
pred = model.predict(image)
label = np.where(model.predict(image) > .5, 'Pneumonia','Normal')
print(pred) print('fail 6')
except:
print('The file image uploaded is not supported')
preds = 'The file type you have uploaded is not supported for this model. Plese use: jpeg, png' return html.Div(
children = [
html.H4('File Name: '+filename),
html.H5('The prediction for this image is: '+ str(label).replace('[', '').replace(']', '').replace("'", '')),
html.H6('The calculated probability of having Pneunonia was: '+ str(pred).replace('[', '').replace(']', '').replace("'", '')),
html.Hr(),
html.Br(), # HTML images accept base64 encoded strings in the same format
# that is supplied by the upload
html.Img(src=imageToDisplay, id = filename),
html.Hr(),],
style={
'width': '60%',
'textAlign': 'center',
'margin': 'auto'
})# callback to save the users image into the session as JSON
@app.callback(dd.Output('user-session', 'data'),
dd.Output('output-image-upload', 'children'),
dd.Input('upload-image', 'contents'),
dd.State('upload-image', 'filename'))
def update_user_session(list_of_contents, list_of_names):
# create an empty list to contin our dictonaries
children = [] # loop through the uploaded images and save the image to the users session in a dcc.Store
children = []
data = []
if list_of_contents is not None:
for content,name in zip(list_of_contents, list_of_names): # save each of the uploaded images and their file names into a dictonary (JSON)
data.append({'content':content, 'name':name})
children.append(parse_contents(content, name)) return data, children
else:
return data, childrenif __name__ == '__main__':
app.run_server(debug=True)
If you want to continue with the manual walkthrough there’s only one more thing to change on the app.py file. In app.py there is a line near the top of the file that reads…
app = dash.Dash(__name__, external_stylesheets=external_stylesheets)Right above this line add…
server = Flask('app')and then change the previously mentioned line to this…
app = dash.Dash(__name__, external_stylesheets=external_stylesheets, server=server)What happened here? When you’ve been viewing your app on your local machine, it’s been working on the development server that dash has built-in. When switching to a production host, it’s better to make explicit what server the app should be using, so there isn’t the possibility that the host uses the production server, or more likely, says it can’t find the server.
Flask, the server we’ll be using, is a flexible Python web server that is used by many application frameworks. If you want to know more about how this works under the hood go check out the docs at https://flask.palletsprojects.com/en/1.1.x/
With that done we’re finally going to mess with that weird Procfile thing. Heroku will be able to tell you everything about what this file is and why you need it. I’ll skip all that and give you the one line of code that needs to go in the file.
web: gunicorn app:server --log-file=-That’s it! The point of this file is to describe to Heroku what kind of app you’re trying to create and what heroku services you want it to use. In essence, this is a configuration file specifically for Heroku services. It can get complicated, and if you continue to publish apps you’ll likely want to explore more options but for this example, we are going to keep it as bare-bones as possible.
Last file change! With the app working in the development server, navigate back to the root of the project folder and run this in the terminal.
pip freeze >> requirements.txtThis step will create a file named requirements.txt in the root and populate it with all the packages you have installed in your virtual environment. This step is VERY IMPORTANT. Without this file, you won’t be able to deploy to heroku.
Because you definitely didn’t lie when you said you had a Github account, go create a remote repository with the same name as your repo. go ahead and add all the changes you’ve made and commit them. On your Github account, after making the remote repo, Github will give you the exact command to connect your local repo to the remote and push up your local changes. Do that.
step 6: get to the cloud!
Ok last part, are you ready!
Set up an account at Heroku. Upon landing at the home page, click the “new” button near the top right and select app. Give you app a name and connect it to the region closest to you or your clients. That should land you here.
If this doesn’t look familiar, it might be because Heroku has asked you to do some basic tutorials. These are a good thing to go through now because they will give a general familiarity with how to navigate the site.
The first place we are going to go is the settings tab. Clicking that you should land on a page that looks like this.
Click the add buildpack button and select python from the icons.
We’re almost there. You’ve hung in for so long, don’t quit now! Go ahead and connect your Github account if prompted. You’ll want to go to the deploy tab near the top of the application dashboard and select Github as the deployment method.
Are you ready!!!
Run through the Heroku checklist for Github — I recommend enabling automatic deployments. At the end click the button
DRUM ROLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLL
It probably didn’t work.
Welcome to the wonderful world of development. Now it’s time to solve requirements errors. In Heroku, click the build logs on the deployment page and read the output near the end. Here’s an image of what a failed build looks like.
On the third line from the bottom you see “Error: no matching distribution found for pandas==1.2.2”
This means the version that Heroku has stored on it’s servers isn’t the one you build the app with. In complicated application this might be a big problem. Fortunately for you, this has an easy fix for us! Go into the requirements.txt file and erase the version from the file that is causing the issue. This will default the build to whatever most recent version Heroku carries. Keep doing this with packages until the build succeeds.
I want to make a note that there is a much better way of doing this that you should begin looking into if this is something you want to keep doing. For places like Heroku, they tell you exactly what versions of popular packages they carry. As a developer, if you are going to be cloud hosting your applications, it’s a good idea to look into these details and have a general idea about what they support. If there’s something your application absolutely cannot live without and heroku doesn’t support it that means you have to start manually changing setup files. Such things are beyond the scope of this simple blog.
If the page tells you build succeeded go ahead and click the open app button in the top right corner of the page. Enjoy it you earned it!
and there you have it. your first web-hosted app. it may be somewhat basic, but in the end, you’re much farther than most other folks. happy hacking
