Here is something that would make you surprised. Do you think Neural networks are too complex jargon? No it’s not, it is simpler than people think. In this blog, the main objective of mine is to make you familiar with Deep Learning and Neural Networks. I would be discussing how neural networks work. What are the different segments in the Neural Network? How is input given to the neural network and how output is computed.
Let's start with a basic building block of neural networks that is a “Neuron” which is also called a perceptron. It takes inputs, does calculation and mathematics inside and gives out an output. The below image is an example of a 3 input neuron that are (x1,x2,x3) and corresponding are the weights (w1,w2,w3). After the input is fed into the neuron there is an “Aggregation Function” & “Activation Function” that comes into play.
The structure of Neuron
Here first inputs are multiplied by weights.
Addition of all weighted inputs with a bias b.
Sum is passed through an activation function.
Inside a neuron, the weights and inputs to the neurons make interaction and then are aggregated into a single value.
The way by which we calculate input from other earlier neurons is called “Aggregation function”.
There are many different aggregation functions that are used. Few are listed below:
Sum of Product
Product of Sum
Division of Sum
Division of Product
1. It’s main function is to activate the neuron for the required outputs.
2. It transforms linear input to non-linear so as to get good results.
3. It is also used to normalize the output between the range of (1 to -1).
4. Like aggregation functions you can define any function as activation functions.
5. Some of the mostly used Activation function are stated below:
Basic structure of a Neural Network
The above image shows the basic structure of a neural network that has inputs that are x1,x2 and so on. These inputs are connected to two different hidden layers and so on. And at last there is an output layer that is y1,y2 and so on.
It is a connection from form inputs towards output.
There is no connection that is connected to backwards.
The current layers input is the previous layer output.
There are no intra-layer connections that are present.
Input Layer: It’s function is to define the input vector. These usually form an input layer and there is only one layer that is present.
Hidden Layer: These layers constitute the intermediary node that divides the layer into boundaries. These form the hidden layers. We can model an arbitrary input-output relation if there are many hidden nodes.
Output Layer: This layer is responsible for output of the neural network. If there are two different classes there is only one output node.
First hidden layer extracts features.
Second hidden layer extracts features of features.
Output layers give the desired output.
Here,
Weight gives the matrix.
Vectors are formed from the output of the previous layer.
Activation function is applied point wise to the weighted times the input.
You need to check about the layers you want to use. (Dependent upon the data)
You need to take care of the neuron required in each layer.
Output and input always depends upon the problem statement but you can always choose or make a choice between neurons and hidden layers.
For given xi and yi
Calculate fw(xi) as an estimate of yi for all the samples.
Calculate the loss.
Make changes in w as to lower down the loss as little as possible.
Repeat the steps.
Loss function also known as cost function tells you about the performance of your model for making the desired predictions. It calculates the error for each training whereas cost functions computes the average loss function of all training samples. Assume there are total N data points in the data. We want to compute loss for all N data points that are present in the data. Then the loss can be computed using the below formula.
Loss function
If the loss will be less, the model would be able to generalize. But why is it so? This is because if you will decrease the error between the predicted value and the actual value, that means the model is performing well. How to lower down the loss lets us understand an algorithm known as Gradient Descent. It is called the optimization algorithm that is used widely as a loss function. It is a method to optimize neural networks. It is used to compute the minimum values for a respective function. It is also termed as Back Propagation.
For more information about the Gradient Descent, you can refer here. And for choosing the loss function you can refer here.
It is defined as the weights that are changed in different epochs (training of each layer is called as one epoch) are called Learning rate or step size. The learning rate is represented by a symbol called a greek letter (n). While the training is taking place the backpropagation computes the errors that are directly responsible for the weights of the node. The weights are ascended by step size instead of changing the whole weight. That meant a step size of 0.1 that is the default value would signify that weights in the networks are updated 0.1* (computed weight error) or 10% of the computed weight error for each period of time the weights are changed.
I would conclude this blog by stating that if the amount of the data increases the computation power also increases and that is where neural networks give you a good performance. In this blog, I have taken you through the concept of the neuron, neural network, aggregation functions, activation functions, different layers in the neural network, how to train a neural network, the role of optimization, and learning rate. I hope you might have got a basic idea behind the neural networks.
Reliance Jio and JioMart: Marketing Strategy, SWOT Analysis, and Working Ecosystem
READ MORE6 Major Branches of Artificial Intelligence (AI)
READ MORETop 10 Big Data Technologies
READ MOREIntroduction to Time Series Analysis: Time-Series Forecasting Machine learning Methods & Models
READ MOREWhat is the OpenAI GPT-3?
READ MORE7 types of regression techniques you should know in Machine Learning
READ MORE8 Most Popular Business Analysis Techniques used by Business Analyst
READ MOREHow Does Linear And Logistic Regression Work In Machine Learning?
READ MORE7 Types of Activation Functions in Neural Network
READ MOREDeep Learning - Overview, Practical Examples, Popular Algorithms
READ MORE
Comments
rajatktiwari1997
May 16, 2020Well Explained .
360digitmgsk
Oct 31, 2020Such a very useful article. Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article. <a rel="nofollow" href="https://360digitmg.com/india/cyber-security-course-training-in-hyderabad">360DigiTMG</a>