Take a tour: Keras functional API
Keras is my favorite tool in the neural network world. Yeah, yeah it’s not the fully customizable, maximum optimized super dee duper neural network thingy. It doesn’t have to be either, but more on that in a future blog…
Keras is by far the easiest and, in my very humble opinion, the most useful neural network framework (that I’m aware of). It makes it easy to build damn near state-of-the-art models without having to tune a million and a half hyperparameters or configuring the network architecture to a level that I believe to be both tedious and pointless.
Regardless of your feelings towards Keras, it is still an important and widely used tool in the neural network pedigree. As one of the tools of choice for new members of the neural network community, Keras is a critical juncture for hooking those interested in modeling into the neural network world. But where to go from there? Tensorflow and PyTorch are not user-friendly in comparison to Keras. If you would disagree, I would remind you that everything seems easy in hindsight.
But ease of use is also the root of the main complaint I hear about Keras. Oversimplification.
I would like to introduce you to my little friend, the Keras functional API.
I’m unsure why I wasn’t initially introduced to the functional API when learning Keras initially. A quick google search for Keras tutorials reveals several examples using the Sequential API. It’s the same API I, and every other student at flatiron school was trained on.
The functional API is everything the Sequential API is, but you know, better in every way. Let’s begin with the basics. We don’t declare the model object first, rather, we create our architecture and plug it into the model at the end.
Similar to the Sequential API, the functional API has an expressive layering system. Here’s a simple example with comments.
# create an input layer that takes 10x10 data
input = keras.Input(shape=(10,10), dtype="int32")# pass the input to the next layer which we call x.
# call the inputs on layer x with python functional syntax (in pink)
x = Dense(10, activation = 'relu)(input)# repeat this will as many times as needed,
# this time pass the previous layer to keep all the info
x = Dense(10, activation = 'relu)(x)# do the same thing with your output layer
output = Dense(2, activation='sigmoid')(x)# create the model object and pass your input and output layers
model = keras.models.Model(input, output, name = 'new_model')
Wala! the functional API at work. There are a couple of things I want to highlight. First, the variable names you see are just conventions introduced in the Keras documentation. You are allowed to call your layers whatever your want. Second, the model we create at the end is the base keras.models.Model class. This is cool because there’s the option to extend the base model class into something tailored to your specific modeling purposes and the extended class will work seamlessly with the functional API. Don’t worry about this more advanced case if you’re new, just something to keep in mind going forward.
“But wait…didn’t you create a sequential model anyways?”
Yes, this is a sequential model, but it didn’t have to be. Let’s take another look at the example. At each new layer, we redefine our x variable to be the current layer evaluated on all the previous ones. Let’s change that.
# create an input layer that takes 10x10 data
input = keras.Input(shape=(10,10), dtype="int32")# pass the input to the next layer which we call x.
# call the inputs on layer x with python functional syntax (in pink)
x1 = Dense(10, activation = 'relu)(input)# then we do something weird and repeat the same step
x2 = Dense(10, activation = 'relu)(input)# then we feed everything into a new layer called a concatenation layer
x = layers.concatenate([x1, x2])# then back to the output like last time
output = Dense(2, activation='sigmoid')(x)# create the model object and pass your input and output layers
model = keras.models.Model(input, output, name = 'newer_model')
So what happened? We created two pipelines: x1 and x2. These pipelines both take the same info from the input layer and then run them through a dense layer with 10 nodes. Keras has a specific layer in the layers API called concatenate that is used to combine pipelines into a single long feature. We do this to the two pipelines so all the info can be fed into the final output layer (which is the same as the first example.)
In math terms, this is called a non-linear graph. In general, the functional API can create any directed acyclic graph
What does this mean in practice?
It means the Keras world is a lot bigger than you thought. So you have different data types that need different layers for preprocessing? The functional API can handle that. What about nesting and concatenating recurrent networks with convolutional and normal connected layers? Yes, yes, and yes.
Another extremely, extremely important aspect of the Keras that often goes un-highlighted is the underlying structure for their model classes. In particular, every model can also be implemented as a layer. Sub-models is the word that comes to my mind. This opens up an entirely new world of intricate possibilities including multiple output predictions — each of which can have its own loss and optimizers.
Rather than ranting on about the functional API, I will close with some of the examples from the docs to see what types of architectures are possible. The following examples are pulled directly from the Keras functional API docs. They use another cool feature from Keras which lets us create visual representations of our model’s graph layouts. For information on these images or the underlying models please click the functional API image.
This model architecture contains three inputs with different preprocessing steps that then are used to predict two different pieces of information.
This is a recreation of a small ResNet architecture.
Please, please go to the Keras docs! All the source code for these more complex examples are hosted in the docs. They also have many more examples and other uses for the functional API not covered in this basic tour.
References
- The Functional API: Keras: https://keras.io/guides/functional_api/
