Difference Between the Batch size and Epoch in Neural Network
Have you ever been confused about the conceptualization of batch size and epoch in neural network? Believe me, most enthusiasts often suffer a lot to know the difference between these ideas, and I didn’t even realize it.
This article will discuss the concept of batch size and epoch and highlight the key points through visual representation.
What will you know after reading this article?
- What is a batch?
- What is an epoch?
- What is the difference between batch and epoch?
What is a batch size?
Batch size is the number of samples that usually pass through the neural network at one time. The batch size commonly referred to as mini-batch.
Having trouble understanding? don’t worry! let’s have a look at the image below
What are you observing in the image above? Some images are being fed to a neural network, and in the end, it is giving you a prediction, isn’t it?
Suppose we have 10 million of the dataset (images), In this case, if you train the model without defining the batch size, it will take a lot of computational time, and it will not be an efficient approach. That is why the concept of batch size has come up that you will not have to train each image separately, but you can train it through the batch size so that the model will be trained as a group. For example, if you define a batch size of 100, in that case, 100 sample images from your entire training dataset will be trained together as a group.
“The larger the batch size, the faster the model per epoch during training.”
One or more batches may be generated from a training dataset. Batch gradient descent is a learning algorithm that uses all training samples to generate a single batch. The learning algorithm is called stochastic gradient descent when the batch size is one sample. The learning algorithm is called mini-batch gradient descent when the batch size is more than one sample and less than the training dataset's size.
- Batch Gradient Descent. Batch Size = Size of Training Set
- Stochastic Gradient Descent. Batch Size = 1
- Mini-Batch Gradient Descent. 1 < Batch Size < Size of Training Set
The most popular batch sizes for mini-batch gradient descent are 32, 64, and 128 samples.
What is an epoch?
The batch size and epoch are not similar. Both have different working methods. An epoch is a term used in machine learning that refers to the number of passes the machine learning algorithm has made over the entire training dataset. The number of epochs equals the number of iterations if the batch size is the entire training dataset.
Line plots with epochs along the x-axis as time and the model’s error or ability on the y-axis are typical. These plots are also known as learning curves and help determine whether the model has overlearned, under-learned, or adequately fitted to the training dataset.
What is the difference between batch and epoch?
- Batch size: The batch size is the number of samples processed before updating the model. The number of epochs represents the total number of passes through the training dataset.
- Epoch: It indicates the number of passes of the entire training dataset the machine learning algorithm has completed
Conclusion
To conclude, this article briefly discusses batch size and epoch. These two concepts are not well understood by many; however, hopefully, this article will be useful for those who have started working on deep learning.
