This article by John Palmer uses the simply brilliant analogy of A/B testing to demonstrate the limits of making improvements in governing structure and in the policies those governments implement. Capitalism sets up a continuous A/B test where people’s ability to spend money and judge the results of their spending result in incremental improvements in the businesses available in the market.
However in successfully applying the tactic to governments, there is a critical point that must be investigated: A/B testing only works as a strategy if the metrics for determining whether A or B should win out are universally established.
Prior to A/B testing new features, product managers agree upon the metrics they will measure: conversion rate, click-through-rate, page impressions, web page bounce, exit rates, revenue lift, etc.
In the capitalism example John uses, we hypothetically measure which business “wins” the A/B test by the amount of users the platforms has and the amount those users are willing to pay for that solution. Solution B should hypothetically win in the marketplace if it can keep making solutions because it has enough money via paying customers. However this is complicated by the fact that so many businesses are propped up initially by VC funding. In many ways, VCs get to determine the outcome of an A/B test more than individual consumers.
So what metrics should we use for determining which version of government is better in order for the decision to be informed by the actual citizens who live there?
If we can assume that a good government = functioning society and good quality of life, we should measure indications of this societal success:
In addition, we should measure the effectiveness of the actual governmental infrastructure: