Imagine your self-driving car eyeballing a shadow on road: Is it puddle or rock? It doesn’t know for sure - but it must act anyway.
There’s a point where logic alone can’t handle things anymore. Because the world isn’t always true or false. At that moment, when things get uncertain - probability steps in to take control by extending the logic to deal with uncertainty.
In old school, we learned probability in a “frequentist” way - by repeating random experiments and counting how often things happen. Flip a coin million times and you’ll see roughly half heads, half tails or plucking petals off a flowers whispering “she loves me, she loves me not….” till you run out of flowers (while she’s probably busy reading this blog). That long-run ratio - that’s what frequentists call “probability”.
But here’s the thing: life isn’t a redo button. You can’t replay the same moment infinite times just to estimate how likely something is. So what happens when we are dealing with one unique event (events that aren’t repeatable)?
Like doctor saying, “There’s a 40% chance this patient has flu” or “probability of sun rising tomorrow”. In this case, we can’t create copies of patient just to repeat experiments to come up with an answer or we can’t consider having infinitely-many universes to define the probability.
That’s where the Bayesian view comes in: it treats probability as degree of belief given what we know.
It is surprising to know that: Bayesian and Frequentist views actually share the same mathematical umbrella but disagree on what probability represents. If your brain is wondering why then read the quote below :)
Back in 1926, this British philosopher named Ramsey figured out that if you just want to reason about uncertainty in a way that makes sense - you end up having to use the same math whether you think “how often things happen” or “how sure you are about something”. The math doesn’t care about your philosophy!
Bayesian thinking comes in when frequentist is not applicable - like remember self-driving car wondering about shadows can’t just decide if it’s rock or puddle by creating infinite scenarios in head!
Think of it as: “an update equation”.
Suppose, if your friend always comes late then you have a clear answer that she might be late today (cause you already know she’s late always). BUT she texted you saying “On my way early today!” Now you updated your belief as you got the new evidence.
Bayes Theorem is basically your brain’s update rule.