Against Optimization

We are bombarded by messages telling us to worship the gods of efficiency and optimization, life hacking our way to prosperity. It's a trap. Resilience is a smarter, sturdier goal.

Thank you for reading The Garden of Forking Paths. This edition is for everyone, but do consider upgrading to a paid subscription to support my work and to fully unlock all 170+ essays. You can also support my work by buying my book FLUKE: Chance, Chaos, and Why Everything We Do Matters—coming soon in paperback in the US and the UK.

---

I: The Optimization Ethic and the Spirit of Capitalism

YouTube influencers, “smart thinking” guru authors, overnight TikTok celebrities are all cashing in on a seemingly unquenchable thirst for slaying our perceived mortal enemy: “unproductive” time.

In 1905, father of sociology Max Weber wrote The Protestant Ethic and the Spirit of Capitalism. While his thesis is contested by historians, he highlighted an important social dynamic: the rise of widespread beliefs that hard work, productivity, and economic output are direct proxies for individual virtue and self-worth. For Protestants, Weber argued, the truest path to revealing godliness was through endless toil, a daily bodily prostration to the divine.

Today, religiosity has plummeted across a variety of Western countries, but that “spirit of capitalism” dominates most modern lives. However, these days, it’s not enough to work hard; to get ahead, you have to work smart. Maximize your output. Minimize your inefficiency. Optimize your life.

The most recent episode of one of America’s top podcasts—the Huberman Lab—is titled “Optimize your learning and creativity with science-based tools,” as though creativity is a machine. Oil your brain with the right mental lubricant, feed it a diet of a few peer-reviewed papers, and presto: become the next Picasso, Shakespeare, or Einstein. (Or, more realistically, someone who tells disembodied heads in rectangular boxes on a Microsoft Teams meeting about an innovative brainwave for better “synergy” to boost profits).

These dystopian social tendencies have worsened through the quantification of everything, in which we most value what can be measured and translated into data. From Q3 benchmarks to sleep scores and step counts, data can help guide us, but too often we live in a world where abstract passions are discounted in favor of metrics. Whether it’s courting risk by foolishly “moneyballing” everything, using data tracking to monitor internet-connected egg cartons and salt shakers, or slathering the internet with a thick coating of “life hack” articles, it is clear that many people are devoted to a new religion.

The new god is called Optimization—and the disciples are legion.

The true believers, however, aren’t just lifehack bros who lurk in YouTube comments sections; instead, they run multinational companies, orchestrate international trade, reshape economies. They are the Prophets of Profits, the one percenters who get richer by squeezing out the last one percent of inefficiency from a social system that is endlessly deemed to be “under-optimized.”

The problem, however, is that there’s a trade-off being ignored. What one person mistakes as inefficiency may actually be resilience. Rather than a demon to be slayed by a McKinsey exorcism, social slack is required for robustness. From modern social systems to our individual lives, we are over-optimized, courting disaster because we are deliberately slicing away the sinews that make ourselves and our world sturdier.

New research, fresh insights into complexity science, and lessons from the natural world offer cautionary tales, warning us of our blind devotion to the optimization gospel. We would be wise to pay attention, as each source of older wisdom unmasks the perils of following these newer, false apostles. And these lessons should give us pause, from how we think about global systems that we rely on to run our infinitely complex world to charting a new path in how we structure our daily existence within the pervasive force of “Hustle Culture.”

Let’s start with the social systems.

II: Criticality and Slack, from the Suez to Estonia

In 2021, a gust of wind hit a container ship nearly twice as long as the Titanic. The ship twisted sideways, both its ends wedged awkwardly into the banks of the Suez Canal. The Ever Given was stuck—and, in an instant, the shipping pipeline for 12 percent of global trade was blocked.

For six days, supply chains broke down, products were spoiled, and delays piled up, sending shockwaves across the global economy. One analysis suggested that the total economic damage from that one boat twisting sideways was $73 billion. Never before in human history could one boat cause a global economic calamity, clear evidence of a system optimized to the point of fragility.

The Ever Given, in a photograph taken from the International Space Station.

More recently, in late December 2024, the Estlink 2 power cable connecting Estonia and Finland was severed, most likely by a vessel operated under Russia’s “Shadow Fleet.” It was a major setback, damage that will take months to fully repair. But there were no blackouts. Unlike the Suez Canal incident, the Estonian power grid was resilient, able to withstand an unexpected blow.

When you consider these two systems, it’s obvious there’s a direct trade-off between optimization and resilience. For any “just in time” manufacturing processes that require deliveries within hours of their scheduled arrival time, a six day delay would be catastrophic. By contrast, any manufacturing process that needed goods to arrive within a week of the expected arrival time would have been fine — the six day delay would have been inconvenient, but there would have been a sufficient buffer to absorb the shock.

Similarly, smart, well-designed power grids are often engineered for robustness and curated for flexibility, able to respond to unexpected surges, unusually low demand, or a severed undersea link—all in an instant. When one node fails, the network adapts, creating resilience through redundancy. Inevitably, that will introduce some inefficiency into the system.

We often call such inefficiency slack. But the obvious lesson we too easily ignore in the modern world is this: slack is often both necessary and desirable. Our ability to predict the future is limited, and in an era of hyper-uncertainty, relying on ever-more precise past data only yields ever-more misguided certainty that the patterns of the past will be a reliable guide to our future.

New research from the University of Oxford’s Institute for New Economic Thinking confirms this point. Using sophisticated agent-based models, the researchers demonstrate a new form of over-optimization that they call “timeliness criticality.”

In complex systems—which includes pretty much everything humans arrange themselves to do in large groups—we can inadvertently reach a state of criticality. (In more conversational English, this is often referred to as a “tipping point” or a “phase transition” or an “avalanche”). These terms highlight a simple but important dynamic: a system that’s put under too much strain can rapidly change, sometimes collapsing—with catastrophic consequences.¹

The new Oxford complexity research provides compelling evidence of how this breakdown occurs from delays in fragile systems—mathematically modelling a Suez Canal-like event. They demonstrate that when buffers are razor-thin, even minor delays in a system can cascade across nodes in a network, compounding and creating cataclysmic “shocks”.

But “shock” is a misnomer; it’s not some unexpected bolt from the blue, but too often the inevitable result of a system designed to be brittle, precisely because it’s over-optimized, made that way by people who obsess over inefficiency but neglect resilience.

III: A Typology of Optimization

Confusion arises because the same word—optimization—is used to mean different things in different contexts. It’s useful, then, to categorize three particularly relevant kinds of optimization, as only two of them tend to come with major risks attached.

• Optimization for absolute efficiency

• Optimization for goals

• Optimization for accuracy

The Suez Canal debacle was created by an over-optimization on efficiency. The attempt to eliminate any buffer in the system in pursuit of slightly higher profit margins created a cascading disaster from one minor mistake. Optimizing for efficiency often comes with a tradeoff: less resilience and reduced ability to adapt to uncertain, fluctuating conditions. At a moment when the world is changing faster than ever before, banking on a system that is tailor-made to break down in the face of the uncertain and the unexpected is a terrible bet for humanity.

Optimizing for goals rather than pure efficiency can be wise, but also comes with risks—particularly when the wrong goal is fetishized because it can be easily measured, or when goal setting eclipses the intrinsic purpose of the activity itself.

For example, in 2014, the United States government spent $416 million to help empower women in Afghanistan. The program targeted 75,000 women with a series of training workshops. All the metrics looked good, “outputs” carefully ticked off the checklists.

What did it achieve? The goals were measurable: workshops delivered, training hours tallied, money out the door. But those goals turned out to be meaningless.

A later evaluation found that just 55 women got jobs out of the program. Worse, it was unclear whether the spending had played any role in that achievement. Even if it had, that’s a cost of $7.5 million per job—in a country where the average yearly income was a few hundred dollars. (If the US government had just handed over $416 million in bags of cash split to the 75,000 women, it would have worked out to $5,500 apiece, about a decade of wages—a far better use of the money).

Finally, there’s optimizing a process for accuracy, which comes with fewer potential risks. Take cancer diagnoses, for example. We want them to be as close to perfect as possible. The endless drumbeat for progress in these contexts is wise. (There are typically no tradeoffs in a closed, straightforward, stable system being optimized for accuracy, but the dynamics of a cancer diagnostic system are completely different from the far more complex, unruly worlds of modern economics or geopolitics).

But what are the lessons for us?

IV: The Barren Desolation of Optimization Culture

Beyond social systems, over-optimizing for efficiency or goals on an individual level can suck the joy out of existence, reducing life’s dazzling unquantifiable flourishes into mere “inefficiencies” to be excised. As I previously wrote, on the relentless goal setting of “hustle culture”:

Hustle culture is the pinnacle of what I call a checklist existence, the ultimate form of a world in which every box we tick gets replaced by yet another one, the same way that e-mail inboxes are the ever-regenerating many-headed hydras that plague our daily lives. We slay them endlessly, hoping that each slash of the delete key or reply button will get us closer to that mythic allure of “inbox zero,” a profoundly dystopian goal. In that never-ending battle, which we always lose, the checklist itself becomes the achievement, an utterly bizarre, tragicomic approach to living—and yet one that, like most of us, I struggle to resist. We are, too often, chained to our checklists, inmates held inside our own inboxes.

There is a problem with this lifestyle, which seeps through every aspect of modern society. I call it the checklist paradox. It works like this:

Doing more often means savoring life less.

When I was conducting research for this essay, I came across a poignant Reddit thread. One post, by a now-deleted user, sums up the emptiness of lived optimization, where goals and efficiency are deemed the Holy Grail of existence:

I've reached a point where I feel guilty for having a slight bit of a beer belly, where I can't wake up at 9 am without feeling like a lazy sack of shit who has wasted most of the day, where I can't have an evening bottle of cold beer without thinking of its potential impact on my sleep, where I can't have a hot shower without berating myself for lacking the discipline for a cold shower.

This is the kind of crushing social pressure that Jia Tolentino critiques in her essay “Always Be Optimizing,” in which women are taught they must “look like an Instagram.” But whether it’s slight beer bellies or other illusory imperfections, “in these pursuits,” Tolentino writes, “most pleasures end up being traps, and every public-facing demand escalates in perpetuity. Satisfaction remains, under the terms of the system, necessarily out of reach.” And when you feel unsatisfied, the implicit answer from our culture is always the same: you didn’t optimize enough. It’s a system that can never be beaten.

In reply to the Reddit user above, another commenter offered astute advice, the kind that provides an antidote to Tolentino’s description of modern social despair and feelings of never being good enough.

“I stopped trying to create who I was... then I stopped trying to discover who I was... and now I allow who I am.”

This isn’t an invitation to resigned complacency, but rather a corrective compass: a reminder that personal striving should be guided by internal motivation, not to satisfy some unicorn-like social fantasy about the perfectly optimized life—astonishingly efficient, ruthlessly goal-oriented, and utterly nightmarish.

We love to achieve greatness through hard work and passion, but many of the pinnacles of human emotion emerge unexpectedly, often within the slack that we allow for ourselves to truly live. As with social systems, we all need a buffer.

V: Nature’s Robustness and Survival of the Resilient

For most of the big decisions we make—about how to govern our societies or how to structure our individual lives—there is a better, wiser strategy for us to follow. Topple the churches to the god of Optimization. Replace them with shrines to a wiser, more caring deity: Resilience.

To see why, we need to draw on lessons from unexpected places: the shells of molluscs, the carefully engineered robustness of ant colonies, and by debunking the mistaken interpretations of evolutionary biology that have infected the dominant—but incorrect—view as to how our world works.

The popular reduction of evolutionary principles to “survival of the fittest”—with overtones of relentless, flawless optimization—is a tragic mistake. (Many incorrectly attribute the phrase to Charles Darwin, but it was first coined by Herbert Spencer). While it is true that evolution does often fine-tune species to greater fitness over time through natural selection, the ultimate engine of evolution is survival and reproduction—which often requires robustness and the ability to adapt to uncertainty.²

A hyper-optimized species that can only survive in one environment will get wiped out if that environment changes. That’s one reason why evolution routinely works in unexpected ways, through what the brilliant evolutionary biologist Zachary Blount calls “the genomic junk drawer.” The specific evolutionary path that a species took—along with plenty of accidental, contingent events along the way—leaves extra stuff in the genome that might at first appear to be junk.

The awe-inspiring genius of our natural world is that evolution provides a mechanism to repurpose that genomic “slack” into something more useful when the environment changes. It’s the evolutionary wizardry of resilient adaptation. That’s why, as Daniel Milo argues, a huge range of lasting species are defined not by optimal solutions, but by “good enough” ones. It’s not survival of the perfectly optimized, but survival of the resilient, as only the most robust inherit the Earth.

For example, nacre, or “mother of pearl,” is one of the oldest and most unchanged biomechanical structures on Earth. With a stunningly beautiful lustre, it gives pearls their sheen and adorns the inner shell of some molluscs. It is largely the same structure from when it first emerged roughly 530 million years ago. (Modern humans have been around for only about 250,000 years, so we might have something to learn from this longstanding byproduct of evolutionary pressure).

Nacre persists because nature is an engineering marvel, producing an ingenious structure that offers a parable for us. The short version is this: at the nano-level, the nacre on mollusc shells has a series of flawed, interconnecting parts that are decidedly un-optimized. The flaws lock together in an irregular brick and mortar pattern, where the “mortar” is organic material that, if needed, can be squeezed out when the material is put under strain.

Moreover, the unique structure creates discontinuities, so if one part cracks, the damage is contained, isolated, decoupled from the rest of the material. To an untrained eye, the structure looks woefully inefficient, wasteful, badly designed. Instead, it’s one of the strongest substances in the world.

Nacre under electron microscopy.

This structure provides two initial key lessons for humans—both in our social systems and in our lives. Resilience can often be produced by systems that feature:

• Diversity (lots of different kinds of components that work together are more robust than a uniform single structure, just as the Estonian power supply was augmented by a wide array of other electricity sources when one cable was severed);

• Redundancy (systems that are designed to work even after an unexpected failure or setback are more robust, illustrated by the Suez Canal, which had no backup option when the route became blocked).

The third lesson comes not from molluscs but from ants. It’s resilience from what I call decoupled connectivity, the idea that robustness comes from interconnected support networks—but also that one needs to be able to sever a destructive node when it becomes toxic. Connectivity allows a system to repair itself when under strain, while decoupling allows isolation to contain a devastating cascade.³

When ant colonies face a disease outbreak, for example, they exhibit ingenious behaviors. If the outbreak is merely of a mild fungal infection, then connectivity saves the colony, as “nurse” ants are swiftly deployed to administer “a formic acid antimicrobial poison to their patients whilst grooming them.” Without the connected network, a mild outbreak could become a devastating epidemic.

However, if the outbreak does become more severe, maintaining connectivity could prove fatal. Then, the colony will pursue more extreme strategies, either of isolation—keeping infected ants away from the healthy ones—or of killing diseased individuals, pruning that node off from the colony altogether.⁴ Since outbreaks don’t happen all the time, keeping these mechanisms in place could be thought of as a form of inefficient slack. But when disease strikes, it’s the slack that saves the colony from existential risk.

One of nature’s overarching lessons is this: what may look to a naive human eye as waste, or inefficiency, or under-optimized slack is often evolution’s secret weapon, providing the adaptive resilience to survive in an ever-changing world.

VI: The Cost of Optimization and the Loss of Resonance

The attempted assertion of perfect control over an uncontrollable world isn’t just a fool’s errand that will always end in disappointment; it’s also a blueprint for a miserable life. Hartmut Rosa, the German sociologist who focuses on malaise created by the “social acceleration” demanded by modern life, implores us to focus less on asserting control and more on resonance, those moments when checklists are obliterated from our minds and we feel, in Rosa’s language, like a “vibrating wire,” resonating with our world.

“It is only in encountering the uncontrollable that we really experience the world. Only then do we feel touched, moved, alive,” Rosa says. That doesn’t mean abandoning striving, or giving up, or passively floating through life. But when everything becomes an instrumental goal to be optimized, intrinsic passion falls by the wayside, an infinite regress where each task and goal gives way to yet another—until you die. Worse, along the way, personal resilience is jettisoned, a casualty of a false belief that life hacks are gospel and life slack is waste.

Social systems, like individual lives, are made fragile by the optimization creed, sacrificed to the God of Efficiency. Goal setting unlocks achievement, efficiency is mostly good, and striving for a better self and a better world forges progress. But when we take these ambitions to their limit, we stretch to the breaking point, engineering fragility. And that leaves us with a potent reminder that you’ll rarely see mentioned in the productivity industry, which seeks to cash in on transforming the unruly beauty of humanity into optimized metric-driven drones:

Few humans on their deathbed have celebrated their achievement of “Inbox Zero.”

Thank you for reading The Garden of Forking Paths. As you might imagine, these essays take quite a lot of work, so if you’d like to keep my writing sustainable, please consider upgrading to a paid subscription. I rely exclusively on reader support—and you’ll get full access to over 170+ essays from the archive, all the low price of just $4 /month.

1. Another realm of related research refers to “critical slowing down,” in which systems fail to return to a comfortable equilibrium after being perturbed—a bit like nature’s early warning system, which I describe in Fluke.

2. I am glossing over a lot of nuance and many caveats here in evolutionary biology.

3. The ability of nacre to isolate cracks is an example of decoupling.

4. Interestingly, ant colonies exhibit robustness in other ways. Worker ants are able to flexibly carry out tasks so that if a forager dies, a different ant can adapt and take on that “job.” It’s only when ant colonies reach a large enough population that they begin to specialize with specific body types for different tasks, precisely because there are so many individuals that the death of some of the colony can easily be replaced by others. This form of redundancy and resilience is captured by the size-complexity hypothesis.