Abstract—Neural network, an adaptive

learning system designed based on human nervous system to solve complex

problems with ease and efficiently. Neural network can make decisions on a very

complex problem in no time. It uses several techniques such as adaptive

learning, pattern detection, associative mapping. Whereas conventional computer

does not have ability to perform adaptive learning. They can only solve fixed

set of problems for which they are designed.

Keywords—Neural

network; synaptic weight; neuron ; pattern recogintion; perceptron; Adaptive

Learning.

I. Introduction

Neural network has opened a whole new field of artificial

intelligence. Neural network uses different ways to analyze and solve complex

data. It consists of sets of programs and data structure which works like human

brain. Initially a neural network needs to be trained in-order to solve complex

problems or perform some peculiar tasks. As it performs adaptive learning, in

initial stages it tries to learn and store the results. The next time it

performs similar task it manipulates the information from its previous results

and try to perform same task in different ways. At the end it will compare the

results of the current test with its previous test results and use the most

efficient way perform similar tasks next time. In this case the neural network

works similar to human brain which also follows the same pattern for learning

new data. It is capable of performing several tasks together, and this is made

possible due to a feature called parallel tasking. It this feature helps to

improve its processing speed. Neural network is a rapid developing in field of

artificial intelligence and is significantly used in field such as robotics,

cardiovascular system diagnosing, marketing, credit evaluation system and many

more.

Where as in case of conventional computers also known as

Von-Neuman computers is not having the similar ability as the neural network to

perform adaptive learning. It is designed to perform a very specific and same

task i.e. used to work with similar data. If you assign data or task other than

what it is designed for, it will end up generating errors and in some circumstances,

it may not able to process the input data. It cannot perform tasks in parallel,

the way neural network can do. So, when comparing the processing speed for both

the computers neural network have upper edge. Both the computers have their own

pros and cons. For neural network, in some cases results can be unexpected and

in some circumstances incorrect too, whereas probability of von-neuman computer

generating error is less.

II. Mc

cullogh-pitts model

Fig. 1 Mc Cullogh-Pitts Model

Neurons work by processing

information, where they receive and provide information in form of spikes. Here

above-mentioned Fig. 1 represents Mc Cullogh-Pitts model for neuron, in which

we provide multiple inputs to the neuron and then each input is multiplied with

a variable weight W. Then these products are then added up to form a transfer

function and at the end you get output signal y on the other side. We will see

a detailed algorithm on how this thing actually works later in this paper.

A neuron network can be defined as a model of reasoning

based on the human brain. Our brain consists of a densely interconnected

network of nerve cells, the basic information-processing units called neurons.

Human brain consists of 10 billion neurons with 60 trillion connections between

them. Brain uses multiple neuron

simultaneously in order to perform its functions faster than fastest

conventional computers. Each neuron has a very simple structure but together

these neurons constitute a tremendous processing power. An artificial neural

network consists of many simple processors, which are known as neurons and are

analogous to the biological neurons of the brain. Mc Culloch & Pitts showed

how to encode any logical problem by an appropriate network of neurons. In

order to understand Mc Culloch & Pitts neuron we took example for our

study. Considering all the assumptions that they made during their study.

Suppose there is a neuron in a bird’s brain that has two receivers which are

directly or indirectly connected to bird’s eye. If the bird sees a round object

a signal is sent to first receiver informing the birds brain about the shape of

the object. But if any other shape is seen, no signal is sent. So, here first

receiver will detect whether shape of the object is round or not. Similarly,

the second receiver will generate signal when it sees blue color, so the second

receiver will inform the brain when it detects blue color. Notice that for both

receiver there is a question whose answer can be “yes” or “no” and a signal to

receiver is sent only when the answer is “yes”. So, in this case if both the

conditions are satisfied the brain will give signal to eat. So, the bird will

decide which object is of interest and act accordingly. In this case then it

knows the object is blue and round it will take it as blueberry and will

receive eat command from brain.

Table.1 Demonstrates operation of neurons.

Here all the signals sent to the

neuron and the signal that it sends out are all yes or no signals. These two-state

model “1” or “0” is one of the assumptions that was made in this theory.

III. Perceptrons

and their learning algorithm

A. Preceptron

Frank Rosenblatt in 1958 introduced a training algorithm

for artificial neural network, which is also called as perceptron. Perceptron

consists of single neuron with adjustable synaptic weights and a threshold i.e.

hard limit.

Fig.2 Single-layer two

input perceptron

Rosenblatt’s perceptron is based on the Mc Culloch and

Pitts neuron model. The weighted sum of these inputs is applied to a threshold

value=e by a hard limiter, which produces an output equal to +1 if its input is

positive & -1 if it is negative. The aim of the perceptron is to classify

inputs, x1, x2…. xn in two different sections or classes say A1 & A2. In an

elementary perceptron the n-dimensional space is divided by a hyper plane into

two decision regions, in our case these regions are called A1 & A2. Here

the linearly separated function defines the hyperplane.

Here is the threshold

value. There are different transfer functions for neural network and threshold

is one of them, the types of transfer functions are covered in further topics.

Linear separability in the perceptron for two-input perceptron and three-input

perceptron are given as follows.

Fig. 3.(a). Two-input perceptron

Fig. 3.(b). Three-input perceptron

Here in the above two figures we see the whole plane is

divided into two sections i.e. is A1 and A2. Similarly, perceptron divides the

input data in two sections and take decision accordingly.

B. Learning

process of a perceptron

Now we will be seeing how

perceptron learns to classify the given data or tasks. Perceptron perform this

using small adjustments in the weighs to reduce the difference between the

actual and desired output of the system. Initially these weights are randomly

assigned to each input as shown in Fig.1 which are usually in the range of -0.5

to 0.5 and then updated to obtain the output consistent with the training examples.

In learning process of neural network neurons need to perform several iterations

to solve similar kind of problem effectively, efficiently and with more

accuracy. Consider if at iteration p, the actual output is Y(p) and the Yd(p) is desired

output, then the error is given by

e(p)= Yd(p)-Y(p)

where p = 1, 2, 3…

Iteration p here refers to the pth

training example presented to the perceptron. So, if the error, e(p) is

positive, we need to increase perceptron output Y(p), but if it is negative, we

decrease Y(p).

Wi(p+1) = Wi(p)+ .xi(p).e(p)

Here is the rate at which the neuron learns, whose value is positive

constant less than unity. The perceptron learning rule was first proposed by

Rosenblatt in 1960. Using this rule, we can derive the perceptron training algorithm

for classification of data and task. This perceptron’s training algorithm is

divided into four steps. Going through each step one by one. Step1 is initialization.

In this step we set initial weights W1,W2,…,Wn

and threshold to random number in

the range of -0.5 to 0.5. If the error, e(p) is positive, we need to increase

perceptron output Y(p), but if it is negative, we decrease Y(p). Step 2 is

activation of perceptron by applying inputs x1(p), x2(p),…,

xn(p) and desired output Yd(p). We need to calculate the

actual output at the iteration p=1.

Here n is the number of perceptron

inputs and step is a step activation function. We can use function other than

step such as sigmoid, or any linear function in order to set the threshold

value. Step 3 is weight training, here we update the weighs of the perceptron.

Wi(p+1) = Wi(p) + Wi(p)

Here Wi(p) is the weight correction at iteration p. Here the

weight correction Wi(p) is computed by the delta rule.

Wi(p) = .xi(p).e(p)

Step4 is iteration. You need to

increase the value of p & repeat all the steps. You need to keep on performing

iteration until you get results with acceptable amount of error between desired

and actual output. The number of iteration needed to reach this level depends

on i.e. the rate at which the perceptron is learning.

Table.2 Perceptron learning Logical AND operation.

This way perceptron can learn the

logical operation and it has a limitation that a single layer perceptron cannot

learn Ex-or logic on its own.

C. Activation

of Neuron

The neuron computes the weighted sum of the input signals

and compares the result with a present threshold value . If the net input is less than the threshold, the neuron

output is -1. But if the input is greater than or equal to threshold, the

neuron becomes activated and its output attains a value +1. The neuron uses the

following transfer or activation function.

Y = {+1, if X >= 0

-1, if X< 0} This type of activation function is called a sign function. Fig.4 Different activation functions IV. multilayer neural network A. Architecture Architecture of neural network is divided into two types which are known as feed-forward network and feedback network. Feed-forward network allows signal travel in one direction only, there are no feedbacks or loops in the network. These kinds of network are extensively used in pattern recognition. Whereas Feedback networks can have signals travelling in both directions by using feedback loops in the network feedback networks are dynamic and their state is changing continuously until they reach an equilibrium point. They remain at equilibrium point until the input to the system changes. Once the input is changed the states starts to change again and keeps on changing until the network reaches new equilibrium. Multilayer perceptron is a feedforward neural network with one or more hidden layers. The network consists of an input layer of source neurons at least one middle or hidden layer of computational neurons and an output layer of neurons. The input signals are propagated in a forward direction on a layer-by-layer basis. Fig.5 Multilayer perceptron with two hidden layers The middle layers are called hidden layer because the computation neurons in the layer cannot be observed through the input/output behavior of the network. There is no standard way to know what the desired output of the hidden layer should be. On increasing the number of hidden layers, we can get more accurate results and we can finely tune the actual output to the desired output of the system. We tried to simulate a system with following mentioned transfer function in order to check this statement. This simulation was performed in MATLAB Simulink. Above mentioned is transfer function used for simulation Fig.6 (a) System with neural network We are providing step input to the system and using two different neural network i.e. the number of hidden alyers in each network are different. So in order to study this we use 10 & 20 hidden layer neural networks and then compare their resuults. We can clearly see the output from 20 hidden layer network is way more accurate than a 10 layer network, but the time required to process a 20 layer neural network is much higher than 10 hidden layer system. So you need to trade-off between accuracy and proccessing time according to the requirment of your application.