ML - Regression problem

A simple regression example - 2D space y = ax + b.

This is a simple 2D regression as it help visualize the problem and the solution. This will be suitable for multiple inputs and outputs which is difficult to visualize in our limited 3D space.

The equation will be y = -5 * x +0.1 * random numbers between -5 and 5.

import torch
import matplotlib.pyplot as plt


#Data input
x = torch.arange(-5, 5, 0.1).view(-1, 1)
y = -5 * x + 0.1 * torch.randn(x.size())


#Define the model backbone
model = torch.nn.Linear(1, 1)                               #Model type - Linear
criterion = torch.nn.MSELoss()                              #measure the losses - Mean Squared Error between iput and target
optimizer = torch.optim.SGD(model.parameters(), lr = 0.1)   #Stochastic gradient desent - derivative to find the minimum

The function to train the model:

def train_model(iter):
    for epoch in range(iter):
        y1 = model(x)                                        #Put x into the model
        loss = criterion(y1, y)                              #Calculates the loss
        optimizer.zero_grad()                                #Finds the minimum 
        loss.backward()                                      #Computes the dloss/dx
        optimizer.step()                                     #Updates the value of x using gradient optimizer.zero_grad

net = train_model(10)                                        # 10 iterations are enough to get the regression

Let’s test the model for -4, -3, -2 and 2 plot against the linear equation.

#Test the model with one value -4
X = torch.tensor([[-4.0], [-3.0], [-2], [2]])

yhat = model(X)                 #Calculates y for x=-4 # yhat is the local minimum of a diferentiable funtion

plt.plot(x, y, 'r-')
plt.plot(X, yhat.detach(), 'bo')
plt.show()

This is the plot after 10 iterations:

And after only 5 iterations:

How about two iterations: