Keras model
Keras model
In this article, we are going to discuss the Keras model, its types, and working too. Before beginning, let’s have a look at what Keras exactly is.
Index
What is Keras
Types of Keras model
Sequential model
· What is a sequential model
· Deep learning model
Functional model
· What is a functional model in Keras
· Functional API in Keras
What is Keras?
Keras is a high-level neural network API, which is written in python. It is compatible with CNTK, TensorFlow, or Theano. It was developed to allow rapid experimentation.
Types of Keras model
Keras model consists mainly of two types. One is a sequential model, and the other is the functional model. Let’s discuss it one by one.
Sequential model
What is a sequential model?
The Sequential Model API is a way to build deep learning models where sequential class instances and model layers are generated and added.
Let’s take an example of using Keras, building a deep learning model.
Nowadays, deep learning has become the most common subset of machine learning. Using neural networks, deep learning is constructed. The neural network takes inputs, which are then processed in hidden layers using weights that are changed during training. Then the algorithm spits out the prediction. The weights are modified to identify trends to make accurate predictions. The consumer does not need to decide what patterns to search for — the neural network learns on its own.
Creating a Sequential model by passing a list of layer instances to the constructor is as follows:
Syntax
from Keras.models import Sequential
from Keras.layers import Dense, Activation
model = Sequential([
Dense(32, input_shape=(784,)),
Activation(‘relu’),
Dense(10),
Activation(‘softmax’),
])
Adding the layer by using the add() method as follow:
Syntax
model = Sequential()
model.add(Dense(32, input_dim=784))
model.add(Activation(‘relu’))
Specifying the shape of the input: The model needs to know what kind of input form it needs to assume.
Performing this task, there are certain possibilities:
· At the first layer, pass an input_shape argument. This is a tuple form in which tuple of integers or None entries, where none indicates that any positive integers, may be expected. The batch dimension is not included in the input shape.
· Like Dence, there are some 2d layers, which support the definition of their input form through the input dim argument, and some 3D temporary layers support input dim and input length arguments.
· If you ever need to define a fixed batch size for your inputs, you can pass a batch size argument to a layer. If you move both batch size=32 and input shape=(6, 8) to a layer, each batch of inputs will be required to have a batch shape (32, 6, 8).
Example of input_shape() is as follow:
Syntax
model = Sequential()
model.add(Dense(32, input_shape=(784,)))
Example of input_dim() as follow:
Syntax
model = Sequential()
model.add(Dense(32, input_dim=784))
Compilation: You need to configure the learning process that is performed using the compile method before training a model. It contains three arguments as follow:
· The optimizer may be the string identifier of an existing optimizer or an instance of the Optimizer class.
· The loss function is the target that the model should seek to mitigate. It can be a string identifier for an established loss function (such as categorical cross-entropy or mse), or it can be an objective function.
· You may need to set this to metrics=[‘accuracy] ‘for any classification issue.
Syntax
model.compile(optimizer=’rmsprop’,
loss=’categorical_crossentropy’,
metrics=[‘accuracy’])
model.compile(optimizer=’rmsprop’,
loss=’binary_crossentropy’,
metrics=[‘accuracy’])
model.compile(optimizer=’rmsprop’,
loss=’mse’)
import Keras.backend as K
def mean_pred(y_true, y_pred):
return K.mean(y_pred)
model.compile(optimizer=’rmsprop’,
loss=’binary_crossentropy’,
metrics=[‘accuracy’, mean_pred])
Training: Keras models are educated on Numpy input data arrays and labels. Usually, you can use the fit function to train a model. The syntax is given below:
Example of single model with 2 classes:
Syntax
model = Sequential()
model.add(Dense(32, activation=’relu’, input_dim=100))
model.add(Dense(1, activation=’sigmoid’))
model.compile(optimizer=’rmsprop’,
loss=’binary_crossentropy’,
metrics=[‘accuracy’])
import numpy as np
data = np.random.random((1000, 100))
labels = np.random.randint(2, size=(1000, 1))
batches of 32 samples
model.fit(data, labels, epochs=10, batch_size=32)
Example of a single model with 10 classes:
Syntax
model = Sequential()
model.add(Dense(32, activation=’relu’, input_dim=100))
model.add(Dense(10, activation=’softmax’))
model.compile(optimizer=’rmsprop’,
loss=’categorical_crossentropy’,
metrics=[‘accuracy’])
import numpy as np
data = np.random.random((1000, 100))
labels = np.random.randint(10, size=(1000, 1))
one_hot_labels = Keras.utils.to_categorical(labels, num_classes=10)
batches of 32 samples
model.fit(data, one_hot_labels, epochs=10, batch_size=32)
Example of Multilayer Perceptron (MLP) for multi-class softmax classification
Syntax
import Keras
from Keras.models import Sequential
from Keras.layers import Dense, Dropout, Activation
from Keras.optimizers import SGD
import numpy as np
x_train = np.random.random((1000, 20))
y_train = Keras.utils.to_categorical(np.random.randint(10, size=(1000, 1)), num_classes=10)
x_test = np.random.random((100, 20))
y_test = Keras.utils.to_categorical(np.random.randint(10, size=(100, 1)), num_classes=10)
model = Sequential()
model.add(Dense(64, activation=’relu’, input_dim=20))
model.add(Dropout(0.5))
model.add(Dense(64, activation=’relu’))
model.add(Dropout(0.5))
model.add(Dense(10, activation=’softmax’))
sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss=’categorical_crossentropy’,
optimizer=sgd,
metrics=[‘accuracy’])
model.fit(x_train, y_train,
epochs=20,
batch_size=128)
score = model.evaluate(x_test, y_test, batch_size=128)
Example of MLP for binary classification
Syntax
import numpy as np
from Keras.models import Sequential
from Keras.layers import Dense, Dropout
x_train = np.random.random((1000, 20))
y_train = np.random.randint(2, size=(1000, 1))
x_test = np.random.random((100, 20))
y_test = np.random.randint(2, size=(100, 1))
model = Sequential()
model.add(Dense(64, input_dim=20, activation=’relu’))
model.add(Dropout(0.5))
model.add(Dense(64, activation=’relu’))
model.add(Dropout(0.5))
model.add(Dense(1, activation=’sigmoid’))
model.compile(loss=’binary_crossentropy’,
optimizer=’rmsprop’,
metrics=[‘accuracy’])
model.fit(x_train, y_train,
epochs=20,
batch_size=128)
score = model.evaluate(x_test, y_test, batch_size=128)
Example of VGG-like convnet
Syntax
import numpy as np
import Keras
from Keras.models import Sequential
from Keras.layers import Dense, Dropout, Flatten
from Keras.layers import Conv2D, MaxPooling2D
from Keras.optimizers import SGD
x_train = np.random.random((100, 100, 100, 3))
y_train = Keras.utils.to_categorical(np.random.randint(10, size=(100, 1)), num_classes=10)
x_test = np.random.random((20, 100, 100, 3))
y_test = Keras.utils.to_categorical(np.random.randint(10, size=(20, 1)), num_classes=10)
model = Sequential()
model.add(Conv2D(32, (3, 3), activation=’relu’, input_shape=(100, 100, 3)))
model.add(Conv2D(32, (3, 3), activation=’relu’))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(64, (3, 3), activation=’relu’))
model.add(Conv2D(64, (3, 3), activation=’relu’))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(256, activation=’relu’))
model.add(Dropout(0.5))
model.add(Dense(10, activation=’softmax’))
sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss=’categorical_crossentropy’, optimizer=sgd)
model.fit(x_train, y_train, batch_size=32, epochs=10)
score = model.evaluate(x_test, y_test, batch_size=32)
Example of Sequence classification with LSTM
Syntax
from Keras.models import Sequential
from Keras.layers import Dense, Dropout
from Keras.layers import Embedding
from Keras.layers import LSTM
max_features = 1024
model = Sequential()
model.add(Embedding(max_features, output_dim=256))
model.add(LSTM(128))
model.add(Dropout(0.5))
model.add(Dense(1, activation=’sigmoid’))
model.compile(loss=’binary_crossentropy’,
optimizer=’rmsprop’,
metrics=[‘accuracy’])
model.fit(x_train, y_train, batch_size=16, epochs=10)
score = model.evaluate(x_test, y_test, batch_size=16)
Example of Sequence classification with 1D convolutions
Syntax
from Keras.models import Sequential
from Keras.layers import Dense, Dropout
from Keras.layers import Embedding
from Keras.layers import Conv1D, GlobalAveragePooling1D, MaxPooling1D
seq_length = 64
model = Sequential()
model.add(Conv1D(64, 3, activation=’relu’, input_shape=(seq_length, 100)))
model.add(Conv1D(64, 3, activation=’relu’))
model.add(MaxPooling1D(3))
model.add(Conv1D(128, 3, activation=’relu’))
model.add(Conv1D(128, 3, activation=’relu’))
model.add(GlobalAveragePooling1D())
model.add(Dropout(0.5))
model.add(Dense(1, activation=’sigmoid’))
model.compile(loss=’binary_crossentropy’,
optimizer=’rmsprop’,
metrics=[‘accuracy’])
model.fit(x_train, y_train, batch_size=16, epochs=10)
score = model.evaluate(x_test, y_test, batch_size=16)
Functional model
What is a functional model in Keras?
The Keras functional API offers a more robust way to identify models. It enables you to construct ad hoc acyclic network graphs.
Let’s starts the functional API in Keras:
The functional API in Keras is the way to define complex models, like multi-output models, directed acyclic graphs, etc. Before starting with functional API, we have assumed that you are already familiar with the sequential model.
The instance of a layer is callable and returns a tensor. To define a model, input and output tensors can be used. The model is trained as a sequential model in Keras.
Let’s take a simple example.
Syntax
from Keras.layers import Input, Dense
from Keras.models import Model
inputs = Input(shape=(784,))
returns a tensor
output_1 = Dense(64, activation=’relu’)(inputs)
output_2 = Dense(64, activation=’relu’)(output_1)
predictions = Dense(10, activation=’softmax’)(output_2)
model = Model(inputs=inputs, outputs=predictions)
model.compile(optimizer=’rmsprop’,
loss=’categorical_crossentropy’,
metrics=[‘accuracy’])
model.fit(data, labels)
Using functional API, it is easy to reuse trained models. By calling it a tensioner, you can treat any model as if it were a layer (Note that by calling a model, you’re not just reusing the model architecture, you’re also reusing its weights). Consider the syntax that given below:
Syntax
x = Input(shape=(784,))
y = model(x)
It can be possible, for example, to quickly create models that can process input sequences. You can transform an image classification model into a video classification model in just one section.
Syntax
from Keras.layers import TimeDistributed
input_sequences = Input(shape=(20, 784))
processed_sequences = TimeDistributed(model)(input_sequences)
Let’s discuss the multi-input and multi-output model in Keras functional API.
The key input will receive the headline as an integer sequence. The numbers range between 1 and 10,000, and the sequences range 100 words long. Consider the syntax given below:
Syntax
from Keras.layers import Input, Embedding, LSTM, Dense
from Keras.models import Model
import numpy as np
np.random.seed(0)
main_input = Input(shape=(100,), dtype=’int32', name=’main_input’)
x = Embedding(output_dim=512, input_dim=10000, input_length=100)(main_input)
lstm_out = LSTM(32)(x)
Now we will be inserting the auxiliary loss, allowing the LSTM and Embedding layer to be trained smoothly even though the key loss is much higher in the model.
Syntax
auxiliary_output = Dense(1, activation=’sigmoid’, name=’aux_output’)(lstm_out)
We then feed our auxiliary input data into the model by connecting it to the output of the LSTM:
Syntax
auxiliary_input = Input(shape=(5,), name=’aux_input’)
x = Keras.layers.concatenate([lstm_out, auxiliary_input])
x = Dense(64, activation=’relu’)(x)
x = Dense(64, activation=’relu’)(x)
x = Dense(64, activation=’relu’)(x)
main_output = Dense(1, activation=’sigmoid’, name=’main_output’)(x)
It defines a configuration of two inputs and two outputs:
Syntax
model = Model(inputs=[main_input, auxiliary_input], outputs=[main_output, auxiliary_output])
The model is assembled, and we give a weight of 0.2 to the auxiliary loss. You may use a list or dictionary to determine different loss weights or loss for each particular output.
Syntax
model.compile(optimizer=’rmsprop’, loss=’binary_crossentropy’,
loss_weights=[1., 0.2])
Using the list of input arrays and target arrays, we can train the model as follow:
Syntax
headline_data = np.round(np.abs(np.random.rand(12, 100) * 100))
additional_data = np.random.randn(12, 5)
headline_labels = np.random.randn(12, 1)
additional_labels = np.random.randn(12, 1)
model.fit([headline_data, additional_data], [headline_labels, additional_labels],
epochs=50, batch_size=32)
Since our inputs and outputs are named (we passed the “name” argument to them), we could also compile the model using:
Syntax
model.compile(optimizer=’rmsprop’,
loss={‘main_output’: ‘binary_crossentropy’, ‘aux_output’: ‘binary_crossentropy’},
loss_weights={‘main_output’: 1., ‘aux_output’: 0.2})
model.fit({‘main_input’: headline_data, ‘aux_input’: additional_data},
{‘main_output’: headline_labels, ‘aux_output’: additional_labels},
epochs=50, batch_size=32)
The model for inferencing, we use the following syntax:
Syntax
pred = model.predict([headline_data, additional_data])
Additionally,
Syntax
pred = model.predict([headline_data, additional_data])
Now, we are going to discuss the shared layers in Keras functional API.
Models that use shared layers are another good use for the functional API. We’re going to develop this using functional API.
Syntax
import Keras
from Keras.layers import Input, LSTM, Dense
from Keras.models import Model
tweet_a = Input(shape=(280, 256))
tweet_b = Input(shape=(280, 256))
The layer is shared across different inputs. We simply instantiate the layer once and then call it as many inputs as you want.
Syntax
shared_lstm = LSTM(64)
encoded_a = shared_lstm(tweet_a)
encoded_b = shared_lstm(tweet_b)
merged_vector = Keras.layers.concatenate([encoded_a, encoded_b], axis=-1)
predictions = Dense(1, activation=’sigmoid’)(merged_vector)
model = Model(inputs=[tweet_a, tweet_b], outputs=predictions)
model.compile(optimizer=’rmsprop’,
loss=’binary_crossentropy’,
metrics=[‘accuracy’])
model.fit([data_a, data_b], labels, epochs=10)
Now, let’s discuss the concept of layer “NODE”.
Whenever a layer is called on an input, a new tensor (output of layer) is generated, and a “node” is added to the layer, connecting the input tensor to the output tensor. As long as the layer is only connected to one input, there is no ambiguity, and the .output returns one output of the layer:
Syntax
a = Input(shape=(280, 256))
lstm = LSTM(32)
encoded_a = lstm(a)
assert lstm.output == encoded_a
Except if the layer has several inputs:
Syntax
a = Input(shape=(280, 256))
b = Input(shape=(280, 256))
lstm = LSTM(32)
encoded_a = lstm(a)
encoded_b = lstm(b)
lstm.output
After passing this code, it can give the following kind of error:
AttributeError: Layer lstm_1 has multiple inbound nodes,
hence the notion of “layer output” is ill-defined.
Use `get_output_at(node_index)` instead.
In the above case, the following code will work:
Syntax
assert lstm.get_output_at(0) == encoded_a
assert lstm.get_output_at(1) == encoded_b
Example of inception module
Syntax
from Keras.layers import Conv2D, MaxPooling2D, Input
input_img = Input(shape=(256, 256, 3))
tower_1 = Conv2D(64, (1, 1), padding=’same’, activation=’relu’)(input_img)
tower_1 = Conv2D(64, (3, 3), padding=’same’, activation=’relu’)(tower_1)
tower_2 = Conv2D(64, (1, 1), padding=’same’, activation=’relu’)(input_img)
tower_2 = Conv2D(64, (5, 5), padding=’same’, activation=’relu’)(tower_2)
tower_3 = MaxPooling2D((3, 3), strides=(1, 1), padding=’same’)(input_img)
tower_3 = Conv2D(64, (1, 1), padding=’same’, activation=’relu’)(tower_3)
output = Keras.layers.concatenate([tower_1, tower_2, tower_3], axis=1)
