2D space with three groups of samples.

In a two-dimension space we have three groups of samples shown in the following image. 

 

 I will create a ML model that can predict to which group will belong to, a set of coordinates. In other words, can the model predict that 

  (-2,3) coordinate will correspond to group green. The data is called qwerties for lack of imagination. 

 Let’s import the modules required in python. I will use PyTorch for ML computation. 

 import numpy as np

import matplotlib.pyplot as plt

from sklearn.datasets import make_blobs

import matplotlib_inline 

matplotlib_inline.backend_inline.set_matplotlib_formats('svg')

import torch

import torch.nn as nn 

 I will give myself 300 qwerties, 100 for each group red, green and blue. I create some tensors and label the colours as 0, 1 and 2. The plot is shown at the beginning. 

 nrPoints = 300

X, y = make_blobs(nrPoints, centers=3, n_features=2, random_state=0)

x_red = X[np.where(y==0)[0],[0]]

y_red = X[np.where(y==0)[0],[1]]

x_blue = X[np.where(y==1)[0],[1]]

y_blue = X[np.where(y==1)[0],[0]]

x_green = X[np.where(y==2)[0],[0]]

y_green = X[np.where(y==2)[0],[1]]

data = torch.tensor(X).float()

labels = torch.tensor(y).long()

fig = plt.figure(figsize=(5,5))

plt.plot(x_red, y_red,'ro')

plt.plot(x_blue, y_blue, 'bs')

plt.plot(x_green, y_green,'g^')

plt.title('The qwerties!')

plt.xlabel('qwerty dimension 1')

plt.ylabel('qwerty dimension 2')

plt.show() 

 Here comes the model (artificial neural network). 

 It has 2 inputs, 3 outputs and 4 hidden layers in between. 

 #model architecture

ANNq = nn.Sequential(

 nn.Linear(2,4),

 nn.ReLU(),

 nn.Linear(4,3),

 nn.Softmax(dim=1)

)

#loss function

lossfunc = nn.CrossEntropyLoss()

#optimizer

optimizer = torch.optim.Rprop(ANNq.parameters(), lr=0.1) 

 A graphical representation of ANN with 3 inputs, one output and 2 hidden layers. 

 

 The fun part begins when we need to train the model using the following code. Here I programmed the model to educate himself for 500 times, record the losses and calculate the accuracy. 

 #Train model

numepochs = 500

#Initial losses

losses = torch.zeros(numepochs)

ongoingAcc = []

for epochi in range(numepochs):

 #forward pass

 yHat = ANNq(data)

 #compute loss

 loss = lossfunc(yHat,labels)

 losses[epochi] = loss

 #backprop

 optimizer.zero_grad()

 loss.backward()

 optimizer.step()

 #compute accuracy

 matches = torch.argmax(yHat,axis=1) == labels #boolean false or true

 matchesNumeric = matches.float() #convert to 0/1

 accuracyPct = 100 * torch.mean(matchesNumeric) #acerage and *100

 ongoingAcc.append( accuracyPct ) # add to the list of accuracy

#final forward pass

predictions = ANNq(data)

predlabels = torch.argmax(predictions,axis=1)

totalacc = 100 * torch.mean((predlabels == labels).float()) 

 Best way is to visualize the losses and accuracy. It seems to be a fast learner. He could’ve finish his self education after only 100 trials. 

 

 Another smart way to show some outcome of the model is to plot the probability of each group. One colour will have 100 probability and the other two will have 0. 

 

 We can save the trained model and later utilize the already trained model which is much faster. Complicated models can take hours or days to train. 

 #Save model

torch.save(ANNq.state_dict(),'traniedModel.pt')

#Another user can load the model

#Load the pre-trained model

loaded_model = ANNq

loaded_model.load_state_dict(torch.load('traniedModel.pt'))

loaded_model.eval() 

 Let’s test the trained model: 

 #Testing the model

X_test = [[np.random.randint(-4,4), np.random.randint(-1,6)]]

test_data = torch.tensor(X_test).float()

results = loaded_model(test_data)

print('test numers: ', X_test)

print(results.detach())

if results[0,0].detach() == 1 :

 print('red')

elif results[0,1].detach() == 1 :

 print('blue')

else:

 print('green')

 

 Here is the output: 

 test numbers: [[-4, 3]] 

 tensor([[2.7208e-40, 0.0000e+00, 1.0000e+00]]) 

 The coordinate correspond to group: green 

 And the final result: