Reference: stackoverflow

Without a bias neuron, each neuron takes the input and multiplies it by a weight, with nothing else added to the equation. So, for example, it is not possible to input a value of 0 and output 2. In many cases, it is necessary to move the entire activation function to the left or right to generate the required output values—this is made possible by the bias.

Although neural networks can work without bias neurons, in reality, they are almost always added, and their weights are estimated as part of the overall model.

sigmoid without bias

https://s3-us-west-2.amazonaws.com/secure.notion-static.com/f278597c-4e3a-49ec-b4ac-778d29cef022/Untitled.png

sigmoid with bias

https://s3-us-west-2.amazonaws.com/secure.notion-static.com/09f03bc1-3008-4bc2-b39f-712ab3a47b8b/Untitled.png