Welcome to the bloggy home of Noah Brier. I'm the co-founder of Percolate and general internet tinkerer. This site is about media, culture, technology, and randomness. It's been around since 2004 (I'm pretty sure). Feel free to get in touch. Get in touch.

You can subscribe to this site via RSS (the humanity!) or .

Framework of the Day: Known Unknowns

As some of you may know I’ve been collecting mental models and working on a book for a little while now (it’s been going pretty slow since my daughter was born in January). This is more notes than chapter, but I still thought it was worth sharing. If you like this I’m happy to do more in the future (I wrote about the pace layers framework in my last post). Oh, and if you haven’t already, sign up to get my new blog posts by email, it’s the best way to keep up.

By all accounts Donald Rumsfeld was a man who didn’t suffer from a shortage of self-confidence. Whether it was Meet the Press, Errol Morris’s documentary Unknown Known (it’s also worth reading the four-part series Morris wrote on Rumsfeld/the documentary for the New York Times), or a grilling from Jon Stewart on the Daily Show, he always seemed supremely satisfied with his own certainty. Which must have made the public response to what’s become his most famous comment all the more vexing. At a Department of Defense briefing in February, 2002, then Secretary of Defense Rumsfeld was asked about evidence to support claims of Iraq helping to supply terrorist organizations with weapons of mass destruction. “Because,” the questioner explained, “there are reports that there is no evidence of a direct link between Baghdad and some of these terrorist organizations.”

Rumsfeld famously replied:

Reports that say that something hasn’t happened are always interesting to me, because as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns — the ones we don’t know we don’t know. And if one looks throughout the history of our country and other free countries, it is the latter category that tend to be the difficult ones.

While it’s a mouthful and the context shouldn’t be lost, there’s a useful framework buried in Rumsfeld’s dodge. It looks something like this:

(Give the whole article from the Project Management Institute on how to apply known unknowns to project management a read.)

Rumsfeld went on to title his memoir Known and Unknown, and explained his perspective on its meaning early in the book:

At first glance, the logic may seem obscure. But behind the enigmatic language is a simple truth about knowledge: There are many things of which we are completely unaware—in fact, there are things of which we are so unaware, we don’t even know we are unaware of them. Known knowns are facts, rules, and laws that we know with certainty. We know, for example, that gravity is what makes an object fall to the ground. Known unknowns are gaps in our knowledge, but they are gaps that we know exist. We know, for example, that we don’t know the exact extent of Iran’s nuclear weapons program. If we ask the right questions we can potentially fill this gap in our knowledge, eventually making it a known known. The category of unknown unknowns is the most difficult to grasp. They are gaps in our knowledge, but gaps that we don’t know exist. Genuine surprises tend to arise out of this category. Nineteen hijackers using commercial airliners as guided missiles to incinerate three thousand men, women, and children was perhaps the most horrific single unknown unknown America has experienced.

Rumsfeld was obsessed with Pearl Harbor. In his memoir he quotes a foreword written by game theorist/nuclear strategist Thomas Schelling that introduced a book about the attack by Roberta Wohlstetter. Schelling wrote (emphasis mine):

If we think of the entire U.S. government and its far-flung military and diplomatic establishment, it is not true that we were caught napping at the time of Pearl Harbor. Rarely has a government been more expectant. We just expected wrong. And it was not our warning that was most at fault, but our strategic analysis. We were so busy thinking through some “obvious” Japanese moves that we neglected to hedge against the choice that they actually made.

And it was an “improbable” choice; had we escaped surprise, we might still have been mildly astonished. (Had we not provided the target, though, the attack would have been called off.) But it was not all that improbable. If Pearl Harbor was a long shot for the Japanese, so was war with the United States; assuming the decision on war, the attack hardly appears reckless. There is a tendency in our planning to confuse the unfamiliar with the improbable. The contingency we have not considered seriously looks strange; what looks strange is thought improbable; what is improbable need not be considered seriously.

In other words, unknown unknowns.

Outside of politics, the framework is a useful way to categorize risk/uncertainty in life or business. I got interested and dug around a bit to find the historical context for the idea, which led me in a few different directions.

Rumsfeld credits William R. Graham at NASA with first introducing him to the concept in the late-90s, though it turns out to go back a lot further than that. The oldest reference I could find comes from a 1968 issue of the Armed Forces Journal International. In the article “The ‘Known Unknowns’ And The ‘Unknown Unknowns'” about the procurement of new weapons. The article opens like this:

Cheyenne was the first major Army weapon to be developed under DoD’s sometimes controversial contract definition procedures. General Bunker put the process in perspective by pointing out that no procedural system can entirely eliminate “surprises” from happening during development of a complex weapons system, and that contract definition wasn’t expected to. “But,” he pointed out, “there are two kinds of technical problems: there are the known unknowns, and the unknown unknowns. Contract definition has helped eliminate the known unknowns. It cannot eliminate completely potential cost overruns, because these are due largely to the unknown unknowns.”

The term pops up throughout the 70s in relation to military procurement. Sometime in there some folks also start using the term “unk-unks” to refer to the most dangerous of the four boxes. Here it is in context from a 1982 New Yorker piece on the airplane industry:

The excitement of this business lies in the sweep of the uncertainties. Matters as basic as the cost of the product — the airplane — and its break-even point are obscure because so much else is uncertain or unclear. The fragility of the airline industry does, of course, create uncertainties about the size and the reliability of the market for a new airplane or a new variant of an existing airplane. Then, there is a wide range of unknowns, for which an arbitrarily fixed amount of money must be set aside in the development budget. Some of these are so-called known unknowns; others are thought of as unknown unknowns and are called “unk-unks.” The assumption is that normal improvements in an airplane program or an engine program will create problems of a familiar kind that add to the costs; these are the known unknowns. The term “unk-unks” is used to cover less predictable contingencies; the assumption is that any new airplane or engine intended to advance the state of the art will harbor surprises in the form of problems that are wholly unforeseen, and perhaps even novel, and these must be taken account of in the budget.

Some are even trying to use it as a kind of code word for breakthrough innovations.

Finally, although it’s not clear they’re connected, there’s a very similar framework from psychologists Joseph Luft and Harrington Ingham from 1955 called the Johari Window. The model attempts to visualize the effects of our knowledge of self and how that works in relation to the knowledge of others:

Quadrant I, the area of free activity, refers to behavior and motivation known to self and known to others.
Quadrant II, the blind area, where others can see things in ourselves of which we are unaware.
Quadrant III, the avoided or hidden area, represents things we know but do not reveal to others (e.g, a hidden agenda or matters about which we have sensitive feelings)
Quadrant IV, area of unknown activity. Neither the individual nor others are aware of certain behaviors or motives: Yet we can assume their existence because eventually some of these things become known, and it motives were influencing relationships all along.

Despite the context for the original quote, the idea is a useful way to think about strategy and understand the various risks you might face.

Framework of the Day posts:

August 23, 2018 // This post is about: , , , , , , , ,

Framework of the Day: Pace Layers

As some of you may know I’ve been collecting mental models and working on a book for a little while now (it’s been going pretty slow since my daughter was born in January). This is more notes than chapter, but I still thought it was worth sharing. If you like this I’m happy to do more in the future. Oh, and if you haven’t already, sign up to get my new blog posts by email, it’s the best way to keep up.

Pace Layers

This one comes from Stewart Brand and is a way to explain the different speed various layers of society moves. The outer layer, fashion, is the quickest, while the innermost layer, nature, moves most slowly. Each layer interacts with one another as inventions and ideas get digested. As Brand explains:

The job of fashion and art is to be froth—quick, irrelevant, engaging, self-preoccupied, and cruel.  Try this!  No, no, try this!  It is culture cut free to experiment as creatively and irresponsibly as the society can bear.  From all that variety comes driving energy for commerce (the annual model change in automobiles) and the occasional good idea or practice that sifts down to improve deeper levels, such as governance becoming responsive to opinion polls, or culture gradually accepting “multiculturalism” as structure instead of just entertainment.

Brand’s inspiration for the framework came from an architect named Frank Duffy who encouraged builders not to think of a building as a single entity, but as a set of layers operating at different timescales. Duffy included four timescales: Shell, services, scenery, and sets (represented below).

Brand picked up on Duffy’s work and adapted it to a kind of proto-pace layer framework in his 1994 book How Buildings Learn: What Happens After They’re Built, expanding it to six S’s and including this handy diagram:

Brand eventually adapted that into the pace layer framework at the top in his 2008 book The Clock of the Long Now: Time and Responsibility (the chapter on pace layers was edited and republished last year in MIT’s Journal of Design and Science). If you want more, here’s a great writeup from Eric Nehrlich on a conversation about pace layers between Brand and Paul Saffo. Nehrlich calls out this slide from the presentation, which is quite helpful for understanding how the layers work:

(The whole talk is posted at the Long Now Blog if you’re so inclined.)

The framework has been picked up and adapted by many, but one of the more notable versions for me comes from Gartner as a way to think about your enterprise software strategy. They break enterprise software into three “layers”:

– Systems of Record — Established packaged applications or legacy homegrown systems that support core transaction processing and manage the organization’s critical master data. The rate of change is low, because the processes are well-established and common to most organizations, and often are subject to regulatory requirements.
– Systems of Differentiation — Applications that enable unique company processes or industry-specific capabilities. They have a medium life cycle (one to three years), but need to be reconfigured frequently to accommodate changing business practices or customer requirements.
– Systems of Innovation — New applications that are built on an ad hoc basis to address new business requirements or opportunities. These are typically short life cycle projects (zero to 12 months) using departmental or outside resources and consumer-grade technologies.

Each layer has it’s own pace of change, lifetime, planning horizon, governance model, and many other unique differentiators:

All in all, the overarching shearing/pace layers framework (many layers which interact with each other and operate at different speeds) is something I’ve found useful in various spheres in addition to the society, architecture, and enterprise software examples above. Inside a company, for instance, you conduct various activities that exist in a similar set of layers ranging from long-term planning and brand building to quarterly goals or roadmaps to two week sprints to weekly exec meetings and then the daily work. It’s a useful way to spot where you’re overloaded with meetings (too many weekly check-ins, not enough monthly lookbacks) or understand where you’re falling down (not doing a good enough job translating the medium term to the long term).

Framework of the Day posts:

August 21, 2018 // This post is about: , , , , , , , , ,