While employing statistically significant data is the gold standard for validation, relying solely on metrics has its challenges.

  1. Metrics can't tell you the "why" of things
  2. Isolating and fixing issues can take a long time, especially in the early stages when you may not have a lot of traffic to test with.

Even though we attempt to apply the Scientific Method to innovation, it’s important to point out the goals of scientists and entrepreneurs are different:

For this reason, entrepreneurs cannot afford to exert the same rigor to experiments as scientists and instead seek to find signals in the noise.

A key mindset for doing this is employing a two-phase validation process: Validate Qualitative, Verify Quantitatively.