This paper implements two classifiers: a softmax classifier and a perceptual machine classifier.
Softmax classifier
Softmax classification is a commonly used multi-category classification algorithm that maps the input data to a probability distribution.Softmax classification first obtains a vector from the input data by linear transformation, then performs an exponential function on each element in the vector, and finally normalizes the result of the exponential operation to obtain a probability distribution. This probability distribution can be interpreted as a probability estimate for each category.
define
Define a softmax classifier class.
class SoftmaxClassifier():
def __init__(self,input_size,output_size):
# Call the __init__() method of the parent class for initialization
super(SoftmaxClassifier,self).__init__()
# Define an object to map input features to output categories
= (input_size,output_size)
def forward(self,x):
x = (x) # Passed to the linear layer
return (x,dim=1) # To get a probability distribution
def compute_accuracy(self,output,labels):
preds = (output,dim=1) # Get predictive labels for each sample
correct = (preds == labels).item() # of correct forecasts calculated
accuracy = correct / len(labels) # Divide by total sample size to get accuracy
return accuracy
Define the three methods as above:
- __init__(self): constructor, run when the class is initialized, calls the __init__() method of the parent class for initialization
- forward(self): model forward computation process
- compute_accuracy(self): compute the model's prediction accuracy
train
Generate training data.
import numpy as np
# Generate random samples (with training and test data)
def generate_rand_samples(dot_num=100):
x_p = (3., 1, dot_num)
y_p = (3., 1, dot_num)
y = (dot_num)
C1 = ([x_p, y_p, y]).T
x_n = (7., 1, dot_num)
y_n = (7., 1, dot_num)
y = (dot_num)
C2 = ([x_n, y_n, y]).T
x_n = (3., 1, dot_num)
y_n = (7., 1, dot_num)
y = (dot_num)*2
C3 = ([x_n, y_n, y]).T
x_n = (7, 1, dot_num)
y_n = (3, 1, dot_num)
y = (dot_num)*3
C4 = ([x_n, y_n, y]).T
data_set = ((C1, C2, C3, C4), axis=0)
(data_set)
return data_set[:,:2].astype(np.float32),data_set[:,2].astype(np.int32)
X_train,y_train = generate_rand_samples()
y_train[y_train == -1] = 0
Setting up pre-training preliminaries and initializing the classifier
num_inputs = 2 # Input dimension size num_outputs = 4 # Output dimension size learning_rate = 0.01 # Learning rate num_epochs = 2000 # of training cycles # Normalize the data Subtract the mean from the data features and divide by the standard deviation X_train = (X_train - X_train.mean(axis=0)) / X_train.std(axis=0) y_train = y_train.astype() # Create the model and initialize it model = SoftmaxClassifier(num_inputs, num_outputs) criterion = () # Cross entropy loss optimizer = ((), lr=learning_rate) # SGD optimizer
Training.
# of training cycles traversed
for epoch in range(num_epochs):
outputs = model((X_train)) # Forward pass calculations
loss = criterion(outputs,(y_train)) # Calculate the loss between predicted output and true labeling
train_accuracy = model.compute_accuracy(outputs,(y_train)) # Calculate the accuracy of the model in the current training cycle
optimizer.zero_grad() # Clarify the gradient in the optimizer
() # Calculate the gradient of the loss to the model parameters
()
# Print information
if (epoch + 1) % 10 == 0:
print(f"Epoch [{epoch+1}/{num_epochs}], Loss: {():.4f}, Accuracy: {train_accuracy:.4f}")
Running:
Epoch [1820/2000], Loss: 0.9947, Accuracy: 0.9575
Epoch [1830/2000], Loss: 0.9940, Accuracy: 0.9600
Epoch [1840/2000], Loss: 0.9932, Accuracy: 0.9600
Epoch [1850/2000], Loss: 0.9925, Accuracy: 0.9600
Epoch [1860/2000], Loss: 0.9917, Accuracy: 0.9600
....
beta (software)
Generate Tests and Test:.
X_test, y_test = generate_rand_samples() # Generate test data
X_test = (X_test- (X_test)) / (X_test) # Normalization
y_test = y_test.astype()
predicts = model((X_test)) # Getting model output
accuracy = model.compute_accuracy(predicts,(y_test)) # Calculation accuracy
print(f'Test Accuracy: {accuracy:.4f}')
Output:
Test Accuracy: 0.9725
Drawing images.
# Drawing images x_min, x_max = X_test[:, 0].min() - 1, X_test[:, 0].max() + 1 y_min, y_max = X_test[:, 1].min() - 1, X_test[:, 1].max() + 1 xx, yy = ((x_min, x_max, 0.1), (y_min, y_max, 0.1)) Z = model((np.c_[(), ()], dtype=torch.float32)).argmax(dim=1).numpy() Z = () (xx, yy, Z, alpha=0.4) (X_test[:, 0], X_test[:, 1], c=y_test, s=20, edgecolor='k') ()
perceptron classifier (computing)
The implementation is similar to the softmax classifier described above, here the sigmod perceptron is implemented, using sigmod as the classification function, which maps the result of a linear transformation to a real value between 0 and 1, and is often used as the activation function in neural networks
The learning algorithm of the sigmoid perceptron is similar to that of a regular perceptron, which is also updated using stochastic gradient descent (SGD). The difference is that the output of the sigmoid perceptron is a probability value that needs to be transformed into category labels.
A threshold is usually used to determine the category to which the output value belongs, e.g., samples with an output value greater than 0.5 are categorized as positive and samples less than or equal to 0.5 are categorized as negative.
define
# Perceptron classifiers
class PerceptronClassifier():
def __init__(self, input_size,output_size):
super(PerceptronClassifier, self).__init__()
= (input_size,output_size)
def forward(self, x):
logits = (x)
return (logits)
def compute_accuracy(self, pred, target):
pred = (pred >= 0.5, 1, -1)
accuracy = (pred == target).sum().item() / (0)
return accuracy
Given an input vector (x1,x2,x3.... .xn), the output is y=σ(w⋅x+b)=1/(e^-(w⋅x+b))
train
Generate training set.
def generate_rand_samples(dot_num=100):
x_p = (3., 1, dot_num)
y_p = (3., 1, dot_num)
y = (dot_num)
C1 = ([x_p, y_p, y]).T
x_n = (6., 1, dot_num)
y_n = (0., 1, dot_num)
y = (dot_num)*-1
C2 = ([x_n, y_n, y]).T
data_set = ((C1, C2), axis=0)
(data_set)
return data_set[:,:2].astype(np.float32),data_set[:,2].astype(np.int32)
X_train,y_train = generate_rand_samples()
X_test,y_test = generate_rand_samples()
The process is similar to the softmax classifier described above.
num_inputs = 2 num_outputs = 1 learning_rate = 0.01 num_epochs = 200 # Normalize the data Subtract the mean from the data features and divide by the standard deviation X_train = (X_train - X_train.mean(axis=0)) / X_train.std(axis=0) # Create the model and initialize it model = PerceptronClassifier(num_inputs, num_outputs) optimizer = ((), lr=learning_rate) # SGD optimizer criterion = .binary_cross_entropy
Training:
# of training cycles traversed
for epoch in range(num_epochs):
outputs = model((X_train))
labels = (y_train).unsqueeze(1)
loss = criterion(outputs,())
train_accuracy = model.compute_accuracy(outputs, labels)
optimizer.zero_grad()
()
()
if (epoch + 1) % 10 == 0:
print(f"Epoch [{epoch+1}/{num_epochs}], Loss: {():.4f}, Accuracy: {train_accuracy:.4f}")
Output.
Epoch [80/200], Loss: -0.5429, Accuracy: 0.9550
Epoch [90/200], Loss: -0.6235, Accuracy: 0.9550
Epoch [100/200], Loss: -0.7015, Accuracy: 0.9500
Epoch [110/200], Loss: -0.7773, Accuracy: 0.9400
....
beta (software)
X_test, y_test = generate_rand_samples() # Generate test sets
X_test = (X_test - X_test.mean(axis=0)) / X_test.std(axis=0)
test_inputs = (X_test)
test_labels = (y_test).unsqueeze(1)
with torch.no_grad():
outputs = model(test_inputs)
accuracy = model.compute_accuracy(outputs, test_labels)
print(f"Test Accuracy: {accuracy:.4f}")
Drawing.
x_min, x_max = X_test[:, 0].min() - 1, X_test[:, 0].max() + 1 y_min, y_max = X_test[:, 1].min() - 1, X_test[:, 1].max() + 1 xx, yy = ((x_min, x_max, 100), (y_min, y_max, 100)) # Predict the category of each point Z = (model((((-1,1), (-1,1)), 1)), 1) Z = () # :: Mapping of classifications (xx, yy, Z, cmap=,alpha=0.0) # Drawing the line of demarcation w = ().numpy() # Weights b = ().numpy() # Bias x1 = (x_min, x_max, 100) x2 = (-b - w[0][0]*x1) / w[0][1] (x1, x2, 'k-') # Plotting sample points (X_train[:, 0], X_train[:, 1], c=y_train, cmap=) ()
The above is based on Pytorch to realize the detailed content of the example of the classifier, more information about Pytorch classifier please pay attention to my other related articles!