Site Loader
Rock Street, San Francisco


Abstract—Neural network, an adaptive
learning system designed based on human nervous system to solve complex
problems with ease and efficiently. Neural network can make decisions on a very
complex problem in no time. It uses several techniques such as adaptive
learning, pattern detection, associative mapping. Whereas conventional computer
does not have ability to perform adaptive learning. They can only solve fixed
set of problems for which they are designed.

network; synaptic weight; neuron ;  pattern recogintion; perceptron; Adaptive

We Will Write a Custom Essay Specifically
For You For Only $13.90/page!

order now

I.     Introduction

Neural network has opened a whole new field of artificial
intelligence. Neural network uses different ways to analyze and solve complex
data. It consists of sets of programs and data structure which works like human
brain. Initially a neural network needs to be trained in-order to solve complex
problems or perform some peculiar tasks. As it performs adaptive learning, in
initial stages it tries to learn and store the results. The next time it
performs similar task it manipulates the information from its previous results
and try to perform same task in different ways. At the end it will compare the
results of the current test with its previous test results and use the most
efficient way perform similar tasks next time. In this case the neural network
works similar to human brain which also follows the same pattern for learning
new data. It is capable of performing several tasks together, and this is made
possible due to a feature called parallel tasking. It this feature helps to
improve its processing speed. Neural network is a rapid developing in field of
artificial intelligence and is significantly used in field such as robotics,
cardiovascular system diagnosing, marketing, credit evaluation system and many

Where as in case of conventional computers also known as
Von-Neuman computers is not having the similar ability as the neural network to
perform adaptive learning. It is designed to perform a very specific and same
task i.e. used to work with similar data. If you assign data or task other than
what it is designed for, it will end up generating errors and in some circumstances,
it may not able to process the input data. It cannot perform tasks in parallel,
the way neural network can do. So, when comparing the processing speed for both
the computers neural network have upper edge. Both the computers have their own
pros and cons. For neural network, in some cases results can be unexpected and
in some circumstances incorrect too, whereas probability of von-neuman computer
generating error is less.

II.    Mc
cullogh-pitts model


Fig. 1 Mc Cullogh-Pitts Model


Neurons work by processing
information, where they receive and provide information in form of spikes. Here
above-mentioned Fig. 1 represents Mc Cullogh-Pitts model for neuron, in which
we provide multiple inputs to the neuron and then each input is multiplied with
a variable weight W. Then these products are then added up to form a transfer
function and at the end you get output signal y on the other side. We will see
a detailed algorithm on how this thing actually works later in this paper.

A neuron network can be defined as a model of reasoning
based on the human brain. Our brain consists of a densely interconnected
network of nerve cells, the basic information-processing units called neurons.
Human brain consists of 10 billion neurons with 60 trillion connections between
them.  Brain uses multiple neuron
simultaneously in order to perform its functions faster than fastest
conventional computers. Each neuron has a very simple structure but together
these neurons constitute a tremendous processing power. An artificial neural
network consists of many simple processors, which are known as neurons and are
analogous to the biological neurons of the brain. Mc Culloch & Pitts showed
how to encode any logical problem by an appropriate network of neurons. In
order to understand Mc Culloch & Pitts neuron we took example for our
study. Considering all the assumptions that they made during their study.
Suppose there is a neuron in a bird’s brain that has two receivers which are
directly or indirectly connected to bird’s eye. If the bird sees a round object
a signal is sent to first receiver informing the birds brain about the shape of
the object. But if any other shape is seen, no signal is sent. So, here first
receiver will detect whether shape of the object is round or not. Similarly,
the second receiver will generate signal when it sees blue color, so the second
receiver will inform the brain when it detects blue color. Notice that for both
receiver there is a question whose answer can be “yes” or “no” and a signal to
receiver is sent only when the answer is “yes”. So, in this case if both the
conditions are satisfied the brain will give signal to eat. So, the bird will
decide which object is of interest and act accordingly. In this case then it
knows the object is blue and round it will take it as blueberry and will
receive eat command from brain.


Table.1 Demonstrates operation of neurons.


Here all the signals sent to the
neuron and the signal that it sends out are all yes or no signals. These two-state
model “1” or “0” is one of the assumptions that was made in this theory. 

III.   Perceptrons
and their learning algorithm

A.   Preceptron

Frank Rosenblatt in 1958 introduced a training algorithm
for artificial neural network, which is also called as perceptron. Perceptron
consists of single neuron with adjustable synaptic weights and a threshold i.e.
hard limit.

            Fig.2 Single-layer two
input perceptron


Rosenblatt’s perceptron is based on the Mc Culloch and
Pitts neuron model. The weighted sum of these inputs is applied to a threshold
value=e by a hard limiter, which produces an output equal to +1 if its input is
positive & -1 if it is negative. The aim of the perceptron is to classify
inputs, x1, x2…. xn in two different sections or classes say A1 & A2. In an
elementary perceptron the n-dimensional space is divided by a hyper plane into
two decision regions, in our case these regions are called A1 & A2. Here
the linearly separated function defines the hyperplane.


Here  is the threshold
value. There are different transfer functions for neural network and threshold
is one of them, the types of transfer functions are covered in further topics.
Linear separability in the perceptron for two-input perceptron and three-input
perceptron are given as follows.


Fig. 3.(a). Two-input perceptron

Fig. 3.(b). Three-input perceptron

Here in the above two figures we see the whole plane is
divided into two sections i.e. is A1 and A2. Similarly, perceptron divides the
input data in two sections and take decision accordingly.

B.   Learning
process of a perceptron 

Now we will be seeing how
perceptron learns to classify the given data or tasks. Perceptron perform this
using small adjustments in the weighs to reduce the difference between the
actual and desired output of the system. Initially these weights are randomly
assigned to each input as shown in Fig.1 which are usually in the range of -0.5
to 0.5 and then updated to obtain the output consistent with the training examples.
In learning process of neural network neurons need to perform several iterations
to solve similar kind of problem effectively, efficiently and with more
accuracy. Consider if at iteration p, the actual output is Y(p) and the Yd(p) is desired
output, then the error is given by


e(p)= Yd(p)-Y(p)

where p = 1, 2, 3…


Iteration p here refers to the pth
training example presented to the perceptron. So, if the error, e(p) is
positive, we need to increase perceptron output Y(p), but if it is negative, we
decrease Y(p).


Wi(p+1) = Wi(p)+ .xi(p).e(p)


Here is the rate at which the neuron learns, whose value is positive
constant less than unity. The perceptron learning rule was first proposed by
Rosenblatt in 1960. Using this rule, we can derive the perceptron training algorithm
for classification of data and task. This perceptron’s training algorithm is
divided into four steps. Going through each step one by one. Step1 is initialization.
In this step we set initial weights W1,W2,…,Wn
and threshold   to random number in
the range of -0.5 to 0.5. If the error, e(p) is positive, we need to increase
perceptron output Y(p), but if it is negative, we decrease Y(p). Step 2 is
activation of perceptron by applying inputs x1(p), x2(p),…,
xn(p) and desired output Yd(p). We need to calculate the
actual output at the iteration p=1.


Here n is the number of perceptron
inputs and step is a step activation function. We can use function other than
step such as sigmoid, or any linear function in order to set the threshold
value. Step 3 is weight training, here we update the weighs of the perceptron.


Wi(p+1) = Wi(p) + Wi(p)


Here Wi(p) is the weight correction at iteration p. Here the
weight correction Wi(p) is computed by the delta rule.


Wi(p) = .xi(p).e(p)


Step4 is iteration. You need to
increase the value of p & repeat all the steps. You need to keep on performing
iteration until you get results with acceptable amount of error between desired
and actual output. The number of iteration needed to reach this level depends
on i.e. the rate at which the perceptron is learning.


Table.2 Perceptron learning Logical AND operation.


This way perceptron can learn the
logical operation and it has a limitation that a single layer perceptron cannot
learn Ex-or logic on its own.

C.   Activation
of Neuron

The neuron computes the weighted sum of the input signals
and compares the result with a present threshold value . If the net input is less than the threshold, the neuron
output is -1. But if the input is greater than or equal to threshold, the
neuron becomes activated and its output attains a value +1. The neuron uses the
following transfer or activation function.

Y = {+1, if X >= 0

       -1, if X< 0}   This type of activation function is called a sign function.     Fig.4 Different activation functions                                                   IV.   multilayer neural network A.   Architecture Architecture of neural network is divided into two types which are known as feed-forward network and feedback network. Feed-forward network allows signal travel in one direction only, there are no feedbacks or loops in the network. These kinds of network are extensively used in pattern recognition. Whereas Feedback networks can have signals travelling in both directions by using feedback loops in the network feedback networks are dynamic and their state is changing continuously until they reach an equilibrium point. They remain at equilibrium point until the input to the system changes. Once the input is changed the states starts to change again and keeps on changing until the network reaches new equilibrium. Multilayer perceptron is a feedforward neural network with one or more hidden layers. The network consists of an input layer of source neurons at least one middle or hidden layer of computational neurons and an output layer of neurons. The input signals are propagated in a forward direction on a layer-by-layer basis.   Fig.5 Multilayer perceptron with two hidden layers   The middle layers are called hidden layer because the computation neurons in the layer cannot be observed through the input/output behavior of the network. There is no standard way to know what the desired output of the hidden layer should be. On increasing the number of hidden layers, we can get more accurate results and we can finely tune the actual output to the desired output of the system. We tried to simulate a system with following mentioned transfer function in order to check this statement. This simulation was performed in MATLAB Simulink.      Above mentioned is transfer function used for simulation   Fig.6 (a) System with neural network   We are providing step input to the system and using two different neural network i.e. the number of hidden alyers in each network are different. So in order to study this we use 10 & 20 hidden layer neural networks and then compare their resuults. We can clearly see the output from 20 hidden layer network is way more accurate than a 10 layer network, but the time required to process a 20 layer neural network is much higher than 10 hidden layer system. So you need to trade-off between accuracy and proccessing time according to the requirment of your application. 

Post Author: admin


I'm Eunice!

Would you like to get a custom essay? How about receiving a customized one?

Check it out