Last year I wrote a series of posts comparing frequentist and Bayesian approaches to various problems:

• In Frequentism and Bayesianism I: a Practical Introduction [perma] I gave an introduction to the main philosophical differences between frequentism and Bayesianism, and showed that for many common problems the two methods give basically the same point estimates.
• In Frequentism and Bayesianism II: When Results Differ [perma] I went into a bit more depth on when frequentism and Bayesianism start to diverge, particularly when it comes to the handling of nuisance parameters.
• In Frequentism and Bayesianism III: Confidence, Credibility, and why Frequentism and Science Don't Mix [perma] I talked about the subtle difference between frequentist confidence intervals and Bayesian credible intervals, and argued that in most scientific settings frequentism answers the wrong question.

• In Frequentism and Bayesianism IV: How to be a Bayesian in Python [perma] I compared three Python packages for doing Bayesian analysis via MCMC: emcee, pymc, and pystan.

Here [perma] I am going to dive into an important topic that I've not yet covered: model selection. We will take a look at this from both a frequentist and Bayesian standpoint, and along the way gain some more insight into the fundamental philosophical divide between frequentist and Bayesian methods, and the practical consequences of this divide.

My quick, TL;DR summary is this: for model selection, frequentist methods tend to be conceptually difficult but computationally straightforward, while Bayesian methods tend to be conceptually straightforward but computationally difficult.

• hide

  Part V

  Intro