The Efficiency Myth

I hate efficiency. I hate it with a passion. It always seems to drive people into making absurd and dangerous decisions. In a world where the future is unknowable, that is where uncertainty reins supreme, it is a very stupid strategy to attempt to be efficient. Dinosaurs were very efficient. Supremely so. They thus ruled the earth for a length of time that makes us look like tiny and insignificant amateurs. Their problem was that they became too efficient. They stopped thinking. They had no back up plan. They had no redundancy. So they could not withstand a shock in their environment. The unknown eventually popped up and rendered all that efficiency as monumentally inefficient. I realize that this is a gross simplification, but bear with me, it’s an analogy.

Or, for the more modern amongst us, think of the Maginot line. A perfect defense system designed to withstand all that could be thrown against it. But not too good if the enemy simply drives around it.

Efficiency, it seems, is entirely contextual. What works well today and thus appears to be the height of elegant engineering, with efficiency fairly oozing from every corner, will collapse in an undignified heap tomorrow when the earth shifts, the environment or tastes change, or when new technologies simply make it all seem so quaint.

So I hate efficiency because it feels and looks like a fool’s game.

I say keep something in reserve. Because you never know.

The problem is that other people adore efficiency.

Like the bankers who thought nothing of gambling with their balance sheets by deploying a 40:1 leverage ratio. Why worry? They had thought through every possibility. Their thinking allowed them to stretch to the very edge. Efficiency demanded they squeeze every ounce of profit from their equity and not allow it to sit around under deployed. That word “under” implies inefficient deployment.

Efficient people always feel fine going to extremes. They know the future sufficiently to do so. They see no reason to build in some spare capacity. They just know the answer.

Besides it is cheaper being efficient. Cheaper because every ounce of resource is at work. Nothing is left idle. There is no need for a reserve. No need for a shock absorber.

At the heart of the efficiency error is a dichotomy to do with knowledge and the way we store and use it.

When I discuss knowledge in the context of business I like to refer to “primary” and “secondary” kinds of knowledge. Dinosaurs are a good example of relying exclusively on the primary sort. Primary knowledge is compressed into simple routines. It is the kind of knowledge that says “when this happens, respond by doing x”. Easy. Cheap to store. Easily encoded. Easily replicated. Very easy to manage. And produces the same result every time. Businesses love this kind of knowledge. It lies at the heart of the dumbing down in every large business. It makes the cost of management lower because you don’t need much management overhead to get consistent results.

Until, of course something changes. As in the environment shifting.

Then all that supremely efficient knowledge is rendered not just useless, but dangerous. Organizations who pride themselves on their efficiency are betting that their environment will justify their knowledge. They have, either explicitly or implicitly, planned that they know the future.

DNA works this way. Living things adapt to their environments through trial and error and then pass into future generations all those survival tricks nicely encoded into DNA. Most living things don’t add to this knowledge. They simply re-enact the actions of their ancestors whenever a similar set of problems pops up. This is why Popper called all life problem solving. DNA represents an accumulated set of responses to an historical set of problems. The bet being that this set of problems will be a good example of what will happen in the future. that’s a big bet. And it works only over limited periods. Change inevitably invalidates it.

But if that bet pays off repeatedly the living organism gradually concentrates itself on its niche and works its way up a fitness peak. Such a peak being a representation of repetitive success driving ever more specialization at the tasks that appear to work. The higher the fitness peak, the greater the past success of whatever knowledge the organism deploys in order to survive. In other words the more efficient the organism is with respect to its surroundings.

We all know where this goes.

The more specialized the knowledge, or the more leveraged against its knowledge something is, the more vulnerable it is to change.

Efficiency then becomes a very bad thing.

Imagine that you have just struggled to squeeze every ounce of cost out of your system. It costs a great deal to carry around excess knowledge. Brains use a great deal of energy. The less you need them the lower cost you can be. So you arrive at your low cost/high leverage peak and look out across the landscape. And. Darn. There’s a higher peak. Someone is more efficient. Worse. You are in competition with that other person. You are doomed.

Efficiency is not just relative to the environment. It is also relative to other solutions to the same problem. That means to other knowledge. You can never rest assured you have the right answer. Someone else may have a better idea.

Having climbed laboriously to the top of your hill you are faced with deconstructing your efficiency and starting over in order to compete. Most likely you will succumb before you succeed in re-engineering yourself. Too bad. You should have left space for the very wasteful, but entirely necessary secondary knowledge.

Such secondary knowledge is embodied in what I refer to as roles – as opposed to the routines of primary knowledge. The crucial difference being that a role allows ad hoc response based upon an evaluation of changes in context. It is still rule based, but now it incorporates feedback. It thus involves learning. Secondary knowledge, by its nature, is high cost to deploy. It involves lugging around all sorts of unused rules that may or may not ever be deployed in action. It involves time to implement the feedback loop. It involves approximation rather than exact response. It allows for making do, rather than locating and then replicating a perfect idea. It also is very difficult to control since it allows each instance of a problem to produce a unique response. “Whatever works” becomes the operating plan. Rather than “stick to this since we know it works”.

There is always a tension between primary and secondary knowledge. Business prefers primary at all times since it is cheaper. Adaptation requires secondary since it allows change. Evolution has used both, but the emphasis is on primary knowledge with the result that failed knowledge implies extinction. Dinosaurs being a good example. Perfect for a very long time. Constant evolution along a path that then became, suddenly, a poor one. Highly efficient. And then not at all efficient.

The preferable strategy seems to be a bit of both.

Lots of embedded primary knowledge, coupled with a goodly amount of adaptive learning ability. Just enough redundancy to pivot when necessary. And enough efficiency to motor through the smooth periods at relatively low cost. This balancing act puts a great emphasis on sufficiency. “Satisficing” as Simon called it. Just enough of everything to manage. With some latitude along the way.

Humans are a great example of this. Poorly designed eyes. Rotten hearing. Weak sense of smell. Not too big. Not too small. We get by. We remain the same tubular body plan of our distant ancestors – think about the topology for a moment – but with a ton of adaptive equipment added. We manage our environment as best we can so that we avoid being entirely context dependent. We can occupy all sorts of niches. We rely heavily on secondary knowledge to solve problems on the fly. In fact we are supremely ad hoc. We get by.

Which makes me wonder why we constantly throw up dreamlike ideas of perfection. So many great thinkers have been beguiled into thinking that they can develop the perfect system. That epitome of efficiency we all long for. Or so they presume. Our history is full of utopias that look silly in the light of consequent events. We are afflicted by the lingering seduction or many more. They all promise efficiency. They feed our apparent need to lower the emotional and physical cost of surviving. We all dream of a life unencumbered by the vagaries of uncertainty. Death in particular seems to be a problem. So we circumvent uncertainty by pretending to know and control the future. We aspire to be efficient.

The result of this arrogant folly is always the same. Whenever the utopia is translated into action or policy here in the uncertain world, it ends up being a flop. Sometimes momentously so.

But uncertainty is relentless. And the past no map for the future.

So learning and adaptation are necessary for survival or growth.

Whereas efficiency allows for neither.

Beware of the utopias that incorporate efficiency. They are dangerous and misleading. Avoid them as the plague they are. Stick with the wasteful and vague comfort of muddling through. Piecemeal progress that allows for adaptation and does not presume to know the future may feel intellectually messy, but the mess won’t be you being trampled underfoot by something you didn’t plan for. Better yet: you won’t be taken for one of those hubris ridden fools who thinks they know everything.