Logistic regression modeling based on MNIST dataset for decile tasks
Softmax Regression without hidden layers can only infer which number it is directly from the pixel points of the image, without the process of feature abstraction. Multi-layer neural networks relying on hidden layers, on the other hand, can combine higher-order features, such as horizontal lines, vertical lines, circles, etc., after which these higher-order features, or components, can then be combined to form numbers, and accurate matching and classification can be achieved.
import tensorflow as tf
import numpy as np
import input_data
print('Download and Extract MNIST dataset')
mnist = input_data.read_data_sets('data/', one_hot=True) # one_hot=True means the encoding format is 01 encoding
print("tpye of 'mnist' is %s" % (type(mnist)))
print("number of train data is %d" % (.num_examples))
print("number of test data is %d" % (.num_examples))
trainimg =
trainlabel =
testimg =
testlabel =
print("MNIST loaded")
"""
print("type of 'trainimg' is %s" % (type(trainimg)))
print("type of 'trainlabel' is %s" % (type(trainlabel)))
print("type of 'testimg' is %s" % (type(testimg)))
print("type of 'testlabel' is %s" % (type(testlabel)))
print("------------------------------------------------")
print("shape of 'trainimg' is %s" % (,))
print("shape of 'trainlabel' is %s" % (,))
print("shape of 'testimg' is %s" % (,))
print("shape of 'testlabel' is %s" % (,))
"""
x = (tf.float32, [None, 784])
y = (tf.float32, [None, 10]) # None is for infinite
w = (([784, 10])) # To facilitate direct initialization with zeros, Gaussian initialization is possible
b = (([10])) # 10 categorized tasks, 10 labels, so only 10 b's need to be initialized
pred = ((x, w) + b) # Predicted value of forward propagation
cost = tf.reduce_mean(-tf.reduce_sum(y*(pred), reduction_indices=[1])) # Cross entropy loss function
optm = (0.01).minimize(cost)
corr = ((pred, 1), (y, 1)) # () compares the index of the predicted value with the index of the real label, returns True if it is the same, or False if it is not.
accr = tf.reduce_mean((corr, tf.float32))
init = tf.global_variables_initializer() # Global parameter initializer
training_epochs = 100 # 100 iterations for all samples
batch_size = 100 # 100 samples selected for each iteration performed
display_step = 5
# SESSION
sess = () # Define a Session
(init) # Run the initialization operation in sess. #
# MINI-BATCH LEARNING
for epoch in range(training_epochs): # Loop every epoch
avg_cost = 0. # At first the loss value is defined as 0
num_batch = int(.num_examples/batch_size)
for i in range(num_batch): # Every batch makes a selection
batch_xs, batch_ys = .next_batch(batch_size) # With next_batch() it's possible to take the data one batch at a time.
(optm, feed_dict={x: batch_xs, y: batch_ys}) # run a bit to solve with gradient descent, pass x, y in via placeholder
avg_cost += (cost, feed_dict={x: batch_xs, y:batch_ys})/num_batch
# DISPLAY
if epoch % display_step == 0: # display_step was previously defined as 5, here it is printed every 5 epochs
train_acc = (accr, feed_dict={x: batch_xs, y:batch_ys})
test_acc = (accr, feed_dict={x: , y: })
print("Epoch: %03d/%03d cost: %.9f TRAIN ACCURACY: %.3f TEST ACCURACY: %.3f"
% (epoch, training_epochs, avg_cost, train_acc, test_acc))
print("DONE")
Iterating 100 times to run the model, eventually, it can achieve 92.2% accuracy on the test set, which is not bad, but it is not yet up to the practical level. The main application scenario for the recognition of handwritten numbers is the recognition of bank checks, which may cause serious consequences if the accuracy is not high enough.
Epoch: 095/100 loss: 0.283259882 train_acc: 0.940 test_acc: 0.922
Insert some knowledge points about the usage of some functions in tensorflow
sess = ()
arr = ([[31, 23, 4, 24, 27, 34],
[18, 3, 25, 0, 6, 35],
[28, 14, 33, 22, 30, 8],
[13, 30, 21, 19, 7, 9],
[16, 1, 26, 32, 2, 29],
[17, 12, 5, 11, 10, 15]])
existtensorflowPrinting in the center should be done using the.eval() (arr).eval() # Print the dimension of the matrix arr (arr).eval() # Print the size of the matrix arr (arr, 0).eval() # Print the index of the maximum value，parameters0For indexing by column，1For indexing by row
Above is the details of TensorFlow tutorial Softmax logistic regression to recognize handwritten digits MNIST dataset, for more information about Softmax logistic regression MNIST dataset handwritten recognition, please pay attention to my other related articles!