I’ve spent much of my life learning about values-misalignment, which covers everything above.

This is a quick overview of the problem and why common design approaches (including mechanism designXDdesign thinking, and red teaming) don’t address it well. Based on why those fail, I’ll outline of an alternative.

Values-misalignment

Values-misalignment is ubiquitous in human life. It’s there in a shallow dinner conversation among people who’d rather go deep; there when an organization’s promotions process makes honest people withhold information; there when an “egalitarian” team structure gives rise to hidden power dynamics; there when attempts to make the world “more open and connected” give rise to political polarization on an unprecedented scale.

**You can tell people want to make fewer values-misaligned things.**There are popular essays, like Meditations on Moloch. There are talking heads like Eric Weinstein and Daniel Schmachtenberger. And academic fields like social choicegame theory, and mechanism design purport to deal with values-alignment, with a vast literature on market and coordination failures.

To make the problem clearer, let’s divide it in two:

Even with the Easy Problem, we’ve made very little progress. Usually, good people fail to anticipate the social consequences of what they build, even when they take time to think.

Existing Approaches

Imagine you’re working on something. Your team has rough agreement about what’d be good and bad, and can afford to design thoughtfully.

What kind of thinking should you do together?

I can’t cover all the existing approaches, but the most popular are plainly inadequate:

Next up are design subfields like mechanism design, experience design, and speculative futurism. Each is inadequate in its own way: