Introduction to Neural Networks

Neural networks are simulation of biological neurons. Similar to dendrites in biological neurons, channels of data are responsible for carrying data between neurons. The strength of these channels is referred to as the weights associated with them.

A Simple Neuron

A simple biological neuron has 3 major parts:

• Dendrites : Act as the channel that gets signals to cell body with their strength as weights
• Cell Body : Collects input through dendrites and processes to produce output
• Axon : Responsible for transmitting signals to other neurons

Inspired from the structure of a biological neuron a single neuron can be explained though the following diagram:

The input brought by the input channels is summed or accumulated (Σ), further processing an output through the [f(Σ)] .

Neural Modeling

An American psychologist, William James came up with two important aspects of neural models which later on became the basics of neural networks :

• If two neurons are active in immediate succession or together, then they will excite each other on re-occurrence which will in turn grow the intensity of the connection between them
• The sum of signals a neuron receives accounts for it’s activity, where signals are proportional to the strength of the connection the signal passes through.

Simplest Neural Network

An Artificial Neuron is a simple model developed with approximation of a biological neuron by McCulloch-Pitts in 1940. It uses a threshold function to produce an output of either 0 or 1 and act as a classifier. This model has fixed weights and does not learn. This is the most simplest neural network from a biological perspective.

Consider a threshold neuron with ‘x¹’ and ‘x²’ as input and ‘t’ being the expected target output.

The neuron has weights ‘w¹’ and ‘w²’ respectively for the channels of input data. For calculating the output the net input ‘u’ is calculated as :

The net input for all such sets of inputs is calculated and then an arbitrary value of the threshold is chosen which lies between the lowest and the highest net input calculated. Let’s assume that a threshold of ‘u⁰’ is chosen.

The final output ‘y’ will be done in the following manner :

The classification is done on the basis of the value of threshold into two different categories. The above threshold function that classifies inputs using a simple neuron.

Conclusion

The neural networks we see today for multiple implementations draws inspiration from biological neurons. A threshold neuron is such simplest structure, which doesn’t adjust its weight and hence cannot learn. It can be used as a classifier on the basis of it’s threshold.

Don’t forget to give us your 👏 !