Recognition of

Handwriting by humans may seem as a very easy task but when done by a machine

is a very complex one. It is impossible for humans to expend so much of their

time to interpret characters apart from them analyzing the collected data. Our

main focus should be on analyzing the data rather than trying to interpret the

data in the first place. Apart from this, the manual interpretation of data may

not yield the right results since it may wary from person to person and hence it

is not accurate to a great extent and may take a lot of time and energy.

Algorithms using neural networks have made this task a lot easier and more

accurate. Therefore neural networks have been utilized with an aim to determine

the characters by recognizing those using algorithms, which use various rules

for prediction of handwritten characters. In this paper we discuss the

recognition of handwritten digits taken from the MNIST data set and calculate

the efficiency of our algorithm. This is done by training a neural network

using stochastic gradient descent and backpropagation.

Keywords

Digit recognition, Backpropagation, Mini batch

Stochastic Gradient

I.

Introduction

Handwriting is a form

of writing peculiar to a person with it varying in size, shape of letters,

spacing between letters. There are different styles of handwriting including

cursive, block letters, calligraphy, signature etc. It makes the task of

recognizing handwritten characters a bit complex for neural networks since it

has to predict the characters based on its ability to learn rather than being

explicitly programmed. According to Tom Mitchell “A computer program is

said to learn from experience E with respect to some class of tasks T and

performance measure P, if its performance at tasks in T, as measured by P,

improves with experience E.”link

A neural network

consists of neurons which are simple processing units and there are directed,

weighted connections between these neurons.

For a neuron j,

propagation function receives the outputs of other neurons and transforms them

in consideration of the weights into the network input that can be further

processed by the activation function.link

Mini batch gradient

descent used in the paper is a combination of batch gradient descent and

stochastic gradient descent algorithms. It calculates model error rate by

splitting data set into small batches.

The backpropagation

algorithm used in this paper is used for adjusting the weights in the neural

network. The algorithm works by comparing the actual output and the desired output

for a given input and calculates error value. The weights are adjusted based on

the error value. The error is first calculated at the output layer and then

distributed for the other layers.

II.

Proposed Methodology

Digit recognition is done by training a multi-layer

feedforward neural network by using mini batch stochastic gradient descent and

backpropagation algorithm.

The MNIST data set obtained from link1 contains a

modified version of the original training set of 60,000 images. The original

training set is split into a training set with 50,000 examples and a validation

set with 10,000 examples. This set is then used to train the neural network. Each image is represented

as numpy 1-dimensional array of 784 float values between 0 and 1. The labels

are numbers between 0 and 9 indicating which digit the image represents. link1

An

artificial neural network with sigmoid neurons is implemented. Therefore, the output

of each neuron is calculated using the sigmoid function.

The output of each neuron is given as. Where, w is the

weight, b is the bias and x is the input.

Initially, the weights and biases of the neural

network are initialized randomly using Gaussian distribution. They are later

adjusted by applying mini batch stochastic gradient descent and backpropagation.

The training data is split into a number of mini

batches. In each epoch, the training data is shuffled and split into mini

batches of a fixed size and gradient descent is applied. The neural network is

trained for a number of epochs. The gradient of the cost function is calculated

by using the backpropagation algorithm. This calculated gradient is then used

to update the weights and biases of the neural network. Starting from the

output layer and moving backwards, the biases and weights between connections

are adjusted.

After training the network during each epoch, the

trained network is tested using the 10,000 test images.