Welcome to the complete Neural Network Regression in TensorFlow guide. This is an end-to-end guide you don’t need any experience in this field. We will explain everything. After reading this article you are able to understand the deep learning field and most of all solve any regression problem.
Keep reading
What Is A Regression Problem In Machine Learning
For example, if you build a model and that model predicts your friend’s age. Or if you are building a model predicting ( Apple stock price ) this type of problem is called a regression problem. A regression problem is a problem where you can predict continuous values. Some of the examples are a person’s age, weight, earnings, and price.
What Is Meant By Deep learning
In this topic, I wrote before in depth in this article. But for now, before learning deep learning, first understand where deep learning comes from.
Deep learning is a subfield of machine learning, and machine learning itself is a subfield of Artificial intelligence ( AI ).
Simple Definition What Is Artificial Intelligence
In general speaking, Artificial Intelligence ( AI ) is the process we make intelligent machines. Every AI developer’s goal is to use data and build intelligent machines.
Application Of Artificial Intelligence
Artificial intelligence is like a Sci-Fi movie type. Today’s news, research paper, website articles, and podcasts everyone talking about this field, it’s like a booming topic at this time. You know AI and you think where it’s used. I don’t see AI everywhere — Is this your question a good question? Actually, it’s widely used today, in every product you use, and every company in other terms AI is everywhere at this time. You watch a Youtube video and one thing notice Youtube recommended what video to watch next that’s AI. According to new research, AI is used today in 80% of companies that grow their company.
Machine learning Introduction
Now you have a basic understanding of the field of AI — and it’s time to know the subfield of Machine learning (ML). Machine learning is the process of building better AI. Machine learning is the field we are using algorithms to analyze the data, learn from that data, improve, and then the last step is to make a prediction about new data.
Understand this example — 🔻
We have lots of data, on heart disease patient records. So you built a machine learning model to predict heart disease patients. So our job is to train using ML algorithms and then check the new patient record and see how our model predicts.
There are many types of machine-learning algorithms that have different types of problems. Here is a list of common machine learning algorithms.
Regression
Decision Trees
Support Vector Machines (SVM)
Naive Bayes Classifiers
K-Nearest Neighbor
Artificial Neural Networks
Deep Learning Introduction
Now you understand the basics of Machine Learning (ML) and next is to learn the subfield of Deep Learning ( DL ). Yes, deep learning is a subfield of machine learning. Deep learning is an approach to how to improve machine learning so you can build better AI models. The goal is simply to improve AI intelligence.
Machine learning uses algorithms, and the same deep learning use algorithms are called Artificial Neural Networks (ANNs ). These algorithms are inspired by our human brain neurons. In the upcoming article, we discuss in-depth artificial neural networks . So sign up for our newsletter, so never miss an article the next time we publish.
Now it’s time to learn the regression problem. First, we created a small example dataset in ourselves and later we used the existing dataset.
import tensorflow as tf
print (tf.__version__) # Check the version tensorflow
# Output 2.9.2
Now it’s time to create some regression data. 🔻
# Import the important libraries we need.
import numpy as np
import matplotlib.pyplot as plt
# X is a our features -
X = np.array([-7.0, -4.0, -1.0, 2.0, 5.0, 8.0, 11.0, 14.0])
# y is a our labels
y = np.array([3.0, 6.0, 9.0, 12.0, 15.0, 18.0, 21.0, 24.0])
# let's visulaize our data
plt.scatter(X, y);
plt.title ("This is our demo data X and y ");
plt.xlabel("This is our X axis ");
plt.ylabel("This is our y axis ");
Understand the above code.⬆
We first import NumPy, in short form np. This library helps me calculate mathematics computation fast. If you learn more about NumPy step by step guide, read this article
Then we import the Matplotlib library, this library helps me visualize our data and model. If you want to learn more, read this article.
After we create our data, to work on regression problems. In this case, our data. ⬇
# X is a features and y is a label X = np.array([-7.0, -4.0, -1.0, 2.0, 5.0, 8.0, 11.0, 14.0]) y = np.array([3.0, 6.0, 9.0, 12.0, 15.0, 18.0, 21.0, 24.0])
What problems are we solving using this data? Our goal is to input X value [10 or any number] so what is the value y [?].
Meaning your input value X [10] so y is [20]. Our formula is (y = X + 10) . Now it’s time to solve this problem X[ 27 ] so what is y [? ].
Note 🔥: This is the most important thing in working neural network input and output shapes. Don’t worry too much this time, I will write in full depth in my next article. But for now, a little bit about know this.
Input shape — This layer takes input based on existing data.
Output shape — Output for predictions based on what data input shape or hidden layer.
In general, neural networks accept numbers and output numbers. These numbers typically act as tensors ( arrays ).
# Now it's time import important libray we need
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf # tf is short form tensorflow
# X is a our features
X = tf.constant([-7.0, -4.0, -1.0, 2.0, 5.0, 8.0, 11.0, 14.0])
# y is a our labels (y = X + 10)
y = tf.constant([3.0, 6.0, 9.0, 12.0, 15.0, 18.0, 21.0, 24.0])
# let's visulaize our data
plt.scatter(X, y);
plt.title ("This is our data X and y ");
plt.xlabel(" X axis ");
plt.ylabel(" Y axis ");
Neural Network Architecture Regression Model
This is the most common architecture in a neural network, but this is not the only way to write and solve regression problems. In deep learning there are many ways to solve problems. Now is the time to build our first deep learning model.
But before building a model, let’s understand what the steps are.
How To Build A Deep Learning Model In TensorFlow
Here are the steps you need to understand to build any deep learning model in Tensorflow. In TensorFlow three fundamental steps in creating and training the model. Creating a model, Compiling a model, and Fitting a model.
See the code below so you can understand better.⬇
# Set random seed using Tensorflow
tf.random.set_seed(42)
# Create a model using the Keras Sequential API
model = tf.keras.Sequential([
tf.keras.layers.Dense(1)
])
# Compile the model
model.compile(loss=tf.keras.losses.mae, # mae is short for mean absolute error
optimizer=tf.keras.optimizers.SGD(), # SGD is short for stochastic gradient descent
metrics=["mae"])
# Fit the model
model.fit(tf.expand_dims(X, axis=-1), y, epochs=5)
Now it’s boom 💥we create our first deep learning model and the next step is to make predictions.
# First see our X and y data..
print ("X is -- ", X)
print ("y is -- ", y)
model.predict([20.0])
OOO no 😞 our model prediction is not good, you accepted. Our model predicted ( y = 14) but the correct result is ( y = 30 ). Meaning we are building a not-a-good deep learning model. So it’s time to improve our model.
Here Is The Step To Improve Deep Learning Model
Adding layers
Increase the number of hidden layers
Change the activation function
Change the learning rate
Fitting more data
Fitting for longer ( Epochs 1 to 100 or so on )
Now it’s time to improve our deep learning model step by step. We can change things one at a time. See code below 🔻
# Set random seed using Tensorflow
tf.random.set_seed(42)
# Create a model using the Keras Sequential API
model = tf.keras.Sequential([
tf.keras.layers.Dense(1)
])
# Compile the model
model.compile(loss=tf.keras.losses.mae, # mae is short for mean absolute error
optimizer=tf.keras.optimizers.SGD(), # SGD is short for stochastic gradient descent
metrics=["mae"])
# Fit the model (This time we train for longer 100 epochs)
model.fit(tf.expand_dims(X, axis=-1), y, epochs=100)
# See our data
print ("X is -- ", X)
print ("y is -- ", y)
# Try to predict what is y The right answer is 30 (y = X + 10)
model.predict([20.0])
It’s better than before. This time our model predicts 35 — we are very close this time to the right answer. But for now, we can improve better so our mode can predict the right answer.
Now it’s time to create a bigger dataset same as above, but this time is big. Meaning fitting more data.
# X is a our Feature (but this time large dataset)
X = np.arange(-100, 100, 4)
X
# y is our labels
y = np.arange(-90, 110, 4)
y
$$This\ is\ our\ formula\ —\ y = X +10$$
# Check how many samples we have in both datasets
print("X dataset total sample have -- ", len(X))
print("y dataset total sample have -- ", len(y))
# Output
# X dataset total sample has -- 50
# y dataset total sample has -- 50
# Split data into train and test sets
X_train = X[:40] # first 40 examples (80% of data)
y_train = y[:40]
X_test = X[40:] # last 10 examples (20% of data)
y_test = y[40:]
# Check the length
print("X_train",len(X_train))
print("y_train is",len(y_train))
print("X_test is",len(X_test))
print("y_test is",len(y_test))
# Output
# X_train 40
# y_train is 40
# X_test is 10
# y_test is 10
Now is the time to visualize this data 🔻
plt.figure(figsize=(10, 7))
# Training data is red
plt.scatter(X_train, y_train, c='r', label='Training data')
# Testing data is black
plt.scatter(X_test, y_test, c='k', label='Testing data')
# Show the legend
plt.legend();
Build a model.
$$🤖$$
# Set random seed
tf.random.set_seed(42)
# Create a model
model = tf.keras.Sequential([
tf.keras.layers.Dense(1, input_shape=[1]) # define the input_shape to our model
])
# Compile model
model.compile(loss=tf.keras.losses.mae,
optimizer=tf.keras.optimizers.SGD(),
metrics=["mae"])
# Fiting model
model.fit(tf.expand_dims(X_train, axis=-1),y_train, epochs=100, verbose=0 )
Make predictions.
$$🤖 \ = \ 💯$$
# Make predictions (X_test) data model never seen before.
y_preds = model.predict(X_test)
# Output
# 1/1 [==============================] - 0s 174ms/step
y_preds
I know you don’t understand, but unfortunately, I don’t understand what’s going on with this number. So I always try to visualize predictions. Now it’s time to visualize our predictions.
Note 💡: Always remember when you need something again and again to create a function.
def model_predictions(training_data=X_train,
training_labels=y_train,
testing_data=X_test,
testing_labels=y_test,
predictions=y_preds):
plt.figure(figsize=(10, 7))
# Plot training data in blue
plt.scatter(training_data, training_labels, c="r", label="Training data")
# Plot test data in green
plt.scatter(testing_data, testing_labels, c="k", label="Testing data")
# Plot the predictions in red (predictions were made on the test data)
plt.scatter(testing_data, predictions, c="b", label="Predictions")
# Show the legend
plt.legend();
model_predictions(training_data=X_train,
training_labels=y_train,
testing_data=X_test,
testing_labels=y_test,
predictions=y_preds)
The prediction we see in our model does not predict well. Our goal is to build a model, learn from the red dots (X_train), and predicted black dots (X_test). But in this case, our prediction is outside of testing data, meaning there is so much gap in testing data and prediction. It’s really clear that we are not building a good deep learning model. So it is time to improve the model.
One thing to remember is always to visualize your data and model, this is good for working on any deep learning problem. To learn more about Matplotlib, read our this end-to-end guide.
Now is the time to improve our deep learning model 🤖
# Set random seed
tf.random.set_seed(42)
#
model_1 = tf.keras.Sequential([
tf.keras.layers.Dense(1)
])
# Compile the model
model_1.compile(loss=tf.keras.losses.mae,
optimizer=tf.keras.optimizers.SGD(),
metrics=['mae'])
# Fiting the model
model_1.fit(tf.expand_dims(X_train, axis=-1), y_train, epochs=100, verbose=0)
# Make a predictions for model_1
y_test_1 = model_1.predict(X_test)
model_predictions(predictions=y_test_1);
Not’a good model, let’s try another one. 🔻
# Set random seed
tf.random.set_seed(42)
# Create a model and this time we are add extra secound layer
model_2 = tf.keras.Sequential([
tf.keras.layers.Dense(1),
tf.keras.layers.Dense(1) # add a second layer
])
# Compile the model
model_2.compile(loss=tf.keras.losses.mae,
optimizer=tf.keras.optimizers.SGD(),
metrics=['mae'])
# Fiting the model
model_2.fit(tf.expand_dims(X_train, axis=-1), y_train, epochs=100, verbose=0)
# Make a predictions for model_2
y_test_2 = model_2.predict(X_test)
model_predictions(predictions=y_test_2);
It’s a good model, now try another one. 🔻
# Set random seed
tf.random.set_seed(42)
# Same model_2
model_3 = tf.keras.Sequential([
tf.keras.layers.Dense(1),
tf.keras.layers.Dense(1)
])
# Compile the model
model_3.compile(loss=tf.keras.losses.mae,
optimizer=tf.keras.optimizers.SGD(),
metrics=['mae'])
# Fit the model
model_3.fit(tf.expand_dims(X_train, axis=-1), y_train, epochs=500, verbose=0) # set verbose to 0 for you don't get any output
# Make a predictions for model_3
y_preds_3 = model_3.predict(X_test)
model_predictions(predictions=y_preds_3);
Now tell me which deep learning model is a good performance at this time. See above.
.
I think model_2 is good for this time. let’s see this visualization below.
When you build your deep learning model, the next step is to save the model. Here is a code example on how you can save your model.
# Save a model
model_2.save('my_best_model')
To learn more about Tensorflow’s savings model, read this article.
I will discuss this library in the article below if you are interested to learn more in-depth read this article below. 🔻
All the code in this article is here — download link.