If you’re not a regular reader of my site, please do me a favor and subscribe to the email (it comes infrequently — whenever I add a post here). I write about business, technology, history, mental models (a lot of those), and all the random interesting stuff I’m reading about. It’s a hodge-podge and I hope you’ll enjoy.
At the beginning of last year I decided I was going to spend more time reading books in 2018. I set myself a goal of 30 on Goodreads and blew past that by year end. While it felt good and something I’m looking to reproduce in 2019, it left me wondering whether I’d actually read enough articles to put together my favorite longform list.
As I was wondering this, a series of plagues befell my house and knocked me off my feet (and computer) for just about two full weeks (it was not fun). When I was finally feeling better this week I thought I’d at least see how many articles I favorited in Instapaper and try to figure out whether a list was feasible. Sixty-something links later I realized I read enough to put it together, so here it is. Some usual caveats apply:
- This is a list of my favorites. It’s not meant to be conclusive and I know I missed lots of great stuff (especially this year with my focus on books).
- You’ll notice a concentration of articles from The New Yorker and New York Times Magazine. That’s because I get both of those delivered. Again, this isn’t meant to be the definitive list of best articles of the year (if you want that I’d head over to longform.org).
- The way I put this together is first I just list out all the articles I favorited in Instapaper. You can find those at the bottom of this post and also follow them (and favorites from YouTube) at my @heyitsinstafavs automated Twitter account. After I list them all out I just look over the list again and pull out all the ones I specifically remember with the thought that those were the ones that had the biggest impact on me.
- Everything is categorized and my picks are in bold throughout. The full list of favorites is at the bottom.
Finally, if you get through this whole list and want more, here are my past versions:
Okay, onto the list:
If you’ve ever read one of these lists before you’ll know that I love David Grann. Basically anything he writes automatically makes the cut. In 2012 that was his story about an American who fought in the Cuban revolution titled “The Yankee Comandante” and in 2011 it was “A Murder Foretold”. After a few years off to publish his excellent book Killers of the Flower Moon, he was back in the New Yorker in February with “White Darkness”, an amazing story of a solitary journey across Antarctica. (It’s also out as a book now, though I think it’s just the article with some more pictures.) What’s amazing about Grann’s writing obviously starts with the stories he finds, but as you read you realize it’s more about the characters that make up those stories. Somehow he always seems to discover people who both live amazing adventures and are also poets, or something close to it. (As an aside, if you haven’t read Grann’s book of essays The Devil and Sherlock Holmes do yourself a favor and get on that or at least pick a few from his New Yorker profile page.)
[“The White Darkness” – David Grann – New Yorker – February 12, 2018]
This is another category where there was really only one entry for me. It comes from the New York Times’ Rukmini Callimachi, who will show up again when we get to favorite podcasts. Callimachi’s work is pure effort and for her huge feature on the Isis she found and sifted through thousands of internal documents and receipts to piece together the story of how the terror organization actually governed in Iraq. The work in and of itself makes the piece worthy for this list, but what makes the article so memorable for me is how it exposed the banality of Isis’s approach in Iraq. “ISIS built a state of administrative efficiency that collected taxes and picked up the garbage,” she writes. “[T]he group at times offered better services and proved itself more capable than the government it had replaced.” If you had to sum the piece up in one sentence it would probably be this one: “The world knows the Islamic State for its brutality, but the militants did not rule by the sword alone. They wielded power through two complementary tools: brutality and bureaucracy.” (If you’re interested in learning more about how the documents were collected or what’s happening with them, there’s a Q&A on the Times’ site. You should also check out Callimachi on the longform podcast for an inside look on how she approaches her work.)
[“The Isis Files: When Terrorists Run City Hall” – Rukmini Callimachi – New York Times – April 4, 2018]
Health & Parenting
I’ve got three entrants for this one and they’re all pretty different.
The first is all about parenting. I would have sworn “The diabolical genius of the baby advice industry” came out last year, but apparently it was from the beginning of this one. It makes sense I wouldn’t have much sense of time, though, since it came out right around when my second daughter was born. The article plainly spells out how big of an industry parenting advice is and how much its foundation is built on bullshit. The bit I remember best is this aside about how statistically insignificant most baby advice really is:
“(Parenting experts who are childless, such as the “queen of routine” Gina Ford, author of the unavoidable Contented Little Baby series, attract a lot of sharp words for it, but this seems unfair. Where Ford has direct experience of parenting none of the 130 million babies born on Earth each year, most gurus only have direct experience of parenting two or three babies, which isn’t much better as a sample size. The assumption that whatever worked for you will probably work for everyone, which is endemic in the self-help world, reaches an extreme in the pages of baby books.)”
The other two are a lot more serious. First is an amazing piece from Guardian writer Hannah Jane Parkinson about her own struggle with mental illness (and specifically bipolar disorder). Parkinson is a very good writer and there’s something about reading an accomplished journalist using her work to explain why her work is so hard that’s particularly impactful. The article’s title “It’s nothing like a broken leg” comes from this passage:
In the last few years I have lost count of the times mental illness has been compared to a broken leg. Mental illness is nothing like a broken leg.
In fairness, I have never broken my leg. Maybe having a broken leg does cause you to lash out at friends, undergo a sudden, terrifying shift in politics and personality, or lead to time slipping away like a Dali clock. Maybe a broken leg makes you doubt what you see in the mirror, or makes you high enough to mistake car bonnets for stepping stones (difficult, with a broken leg) and a thousand other things.
I would also highly recommend this short piece from Parkinson published at the end of December about her riding buses around London in the middle of the night when things get particularly bad.
Finally, my pick for this category comes from a New York Times Magazine story titled “Why America’s Black Mothers and Babies Are in a Life-or-Death Crisis.” The article shocked and saddened me as it spelled out just how inadequate the care pregnant black mothers receive. Education and income, as the article explains, don’t explain it. “In fact, a black woman with an advanced degree is more likely to lose her baby than a white woman with less than an eighth-grade education.” Most shocking to me was this story about how the institutional racism embedded in the system manifests itself in obviously bad science amongst doctors:
In 2016, a study by researchers at the University of Virginia examined why African-American patients receive inadequate treatment for pain not only compared with white patients but also relative to World Health Organization guidelines. The study found that white medical students and residents often believed incorrect and sometimes “fantastical” biological fallacies about racial differences in patients. For example, many thought, falsely, that blacks have less-sensitive nerve endings than whites, that black people’s blood coagulates more quickly and that black skin is thicker than white. For these assumptions, researchers blamed not individual prejudice but deeply ingrained unconscious stereotypes about people of color, as well as physicians’ difficulty in empathizing with patients whose experiences differ from their own. In specific research regarding childbirth, the Listening to Mothers Survey III found that one in five black and Hispanic women reported poor treatment from hospital staff because of race, ethnicity, cultural background or language, compared with 8 percent of white mothers.
Go read the whole thing and when you’re done please donate to the Birthmark Doula Collective who are trying to help change the care black mothers receive.
[“Why America’s Black Mothers and Babies Are in a Life-or-Death Crisis” – Linda Villarosa – New York Times Magazine – April 11, 2018]
As you may or may not know I’m a big NBA fan. The league is as good as its ever been and has been fundamentally transforming the way the game is played for the last ten years or so. I’ve spent a fair amount of time trying to explain this to friends and now, thanks to Kevin Arnovitz and Kevin Pelton, I can just send them the article “How the NBA got its groove back.” In the piece, the Kevins spell out how much faster the league got over the last decade, starting with the D’Antoni/Nash. In fact, as the article explains, those 2004-05 Suns wouldn’t even be considered a fast team by today’s standards. “Back then, Phoenix’s 98.6 pace was more than a possession per game faster than that of any other NBA team. In 2017-18, the average team had 99.6 possessions per 48 minutes, and the Suns’ 2004-05 pace would have ranked 19th in the league.” It’s a fun time to be an NBA fan. (Honorable mention sports story has to go to The Ringer’s insane “The Curious Case of Bryan Colangelo and the Secret Twitter Account”.)
[“How the NBA got its groove back” – Kevin Arnovitz and Kevin Pelton – ESPN – May 24, 2018]
Society isn’t a perfect title for this category, but it gets at it. I’ve got two pieces here. The first comes from psychology professor Alison Gopnik (who shows up in the podcast list as well). Although not terribly wrong, I thought her review of Steven Pinker’s Enlightenment Now offered a really interesting rebuttal to the macro “everything is getting better” story. Gopnik opens the piece by stating her credentials: She’s a scientist, professor, and “card-carrying true believer in liberal Enlightenment values.” But she doesn’t think we can, or should, push aside local needs and values for the global:
The weakness of the book is that it doesn’t seriously consider the second part of the conversation—the human values that the young woman from the small town talks about. Our local, particular connections to just one specific family, community, place, or tradition can seem irrational. Why stay in one town instead of chasing better opportunities? Why feel compelled to sacrifice your own well-being to care for your profoundly disabled child or fragile, dying grandparent, when you would never do the same for a stranger? And yet, psychologically and philosophically, those attachments are as central to human life as the individualist, rationalist, universalist values of classic Enlightenment utilitarianism. If the case for reason, science, humanism, and progress is really going to be convincing—if it’s going to amount to more than preaching to the choir—it will have to speak to a wider spectrum of listeners, a more inclusive conception of flourishing, a broader palette of values.
In some ways there’s a similar theme in my pick for this category. “Pay the Homeless” is all about the local realities. It’s an argument against the idea that giving money to someone asking for it is somehow not good for the system as a whole or that individual specifically:
Yet on the whole, all the evidence, from the statistical to the spiritual, points in one direction: if you can give, you should give. It won’t solve the problems of mass homelessness or impoverishment. But it will improve someone’s life ever so slightly and briefly. “People are in dire straits and raising money for bare necessities,” Jerry Jones, policy director at the Inner City Law Center, told me. They might be trying to collect enough to pay for a room for the night. They might need bus fare or gas to get to an appointment.
[“Pay the Homeless” – Bryce Covert – Longreads – June, 2018]
I try to read just about anything Tim Harford writes. His ability to take complicated topics and simplify them are impressive and inspiring. Last year’s piece on Arrow’s impossibility theorem is something I still think about and his BBC Pop-Up Ideas series was absolutely amazing, particularly this episode on the idea of feral cities. This year his FT feature “Why big companies squander brilliant ideas” introduced me to the idea of architectural innovation, which I wrote about in some length in my Framework of the Day piece on Conway’s Law. If you enjoy the piece I would also suggest checking out the reading list, which includes links to all the source material.
[“Why big companies squander brilliant ideas” – Tim Harford – Financial Times – September 8, 2018]
There aren’t that many pieces that stood out in the world of politics for me this year. I suspect that’s because I actively avoided them (as opposed to last year). With that said, one writer and two pieces stood out for me this year.
First, the writer. I don’t know of anyone I turned to more this year to get the pulse of the country than the Washington Post’s media columnist Margaret Sullivan. She doesn’t have a piece on the list because her writing is more ephemeral. In a world where the President is a TV star, it seems appropriate that the most important columnist is someone who has a deep understanding of how the media functions. A few of her pieces that stood out: “Publishing that anonymous New York Times article wasn’t ‘gutless.’ But writing it probably was.”, “It’s high time for media to enter the No Kellyanne Zone — and stay there”, “Don’t forget how the movement that changed Hollywood started: With great reporting”, and “Enough, already, with anything Steve Bannon has to say. We got it the first time.”. She’s who I turned to when I wanted to understand the importance of a moment.
Okay, back to the longform. First up is Jane Mayer’s masterful profile of Christopher Steele, the author of the infamous Trump Dossier. The piece didn’t necessarily break new ground (after all, Buzzfeed had already published the whole thing), but it did paint an interesting picture for the man who wrote it, his credibility, and the accuracy, or lack thereof, that his methods may have led to.
My pick for politics goes to Robert Draper’s New York Times Magazine profile of House leader Nancy Pelosi. We’re about to hear A LOT about Pelosi as she battles Trump and the Republicans over the next two years and, for me at least, my knowledge and understanding of her was surface at best. The profile is pretty unvarnished and paints Pelosi as a pure politician who knows how to operate as well or better than anyone out there. I suspect we’re going to see a lot of stereotypical framing of Pelosi because she’s a woman and this felt like a good foundation to build understanding. I thought this bit about how much of her perception, even amongst Democrats I’d argue, has been shaped by the Republicans was particularly interesting:
Still, Pelosi’s foremost liability is the effectiveness of the attacks against her. In 2010, Republicans spent $65 million attacking Pelosi in ads; the Republican National Committee hung a banner from its headquarters that read FIRE PELOSI. The attacks have often borne more than a tinge of sexism; in 2012, when Pelosi, as minority leader, wielded less power than the Senate’s Democratic majority leader, Harry Reid, Republicans’ negative television ads were seven times as likely to mention Pelosi as Reid, according to the Wesleyan Media Project, which tracks political advertising. The 2010 onslaught took its toll on Pelosi’s public standing — her favorable rating dropped into the 20s — but otherwise did not faze her. She made clear to her caucus members that they should do whatever it took to win, even if it meant publicly distancing themselves from her. “I don’t know anyone in the world with thicker skin, or anyone about whom more callous things have been said, and she just truly doesn’t care,” a former Pelosi staff member told me. “There’s a small constituency she cares about: her members.”
[“Nancy Pelosi’s Last Battle” – Robert Draper – New York Times Magazine – November 19, 2018]
I listened to fewer podcasts this year thanks to the introduction of audiobooks into my media diet. With that said, there were a few that stood out. Rather than specific episodes, though, this year my favorites felt more like shows in their entirety. This might be because I explored fewer new podcasts this year or just because there were a few exceptional short series that came out in 2018.
First off is Reply All. As far as week-after-week quality goes, it’s hard to beat these guys. Two (really three) episodes in particular stood our for me:
- “Invcel”: “How a shy, queer Canadian woman accidentally invented one of the internet’s most toxic male communities.”
- “The Crime Machine, Part 1” & “The Crime Machine, Part 2”: “New York City cops are in a fight against their own police department. They say it’s under the control of a broken computer system that punishes cops who refuse to engage in racist, corrupt policing. The story of their fight, and the story of the grouchy idealist who originally built the machine they’re fighting.”
Next up is American Fiasco, Roger Bennet’s ten-part series on the disaster that was America’s 1998 World Cup.
Finally, and my real pick, is Rukmini Callimachi’s ten-part series Caliphate. The podcast follows Callimachi as she reports on the Islamic State and the fall of Mosul. It’s an extraordinary piece of reporting with the kinds of twists and turns that we’ve come to expect in great podcasts these days. Again, I can’t say enough about Callimachi’s work this year between Caliphate and The Isis Files.
[Calpihate – April, 2018]
Not This Year
Finally, because I can’t resist, I read a bunch of longform that was amazing and didn’t come out this year. Although it doesn’t officially fit my rules, I’m going to include a few picks as a way to wrap things up.
I already talked about David Grann at length, so I won’t spend too much time introducing the amazing 2008 New Yorker story I reread titled “True Crime: A Postmodern Murder Mystery.” It’s insane. Go read it.
I think I read “Promthea Unbound” just after I put together last year’s list otherwise I have to assume it would have made the cut. It’s the extraordinary (sorry, I’m running out of superlatives) story of a child genius and her mom and how they got through life together.
Finally, my pick for “Not This Year” maybe shouldn’t officially even count as longform, but I’m making the rules and I say short stories are allowed. “The Ones Who Walk Away from Omelas” is a short story by Ursula Le Guin that feels as appropriate today (if not more) than it must have when it was published in 1973. It’s about the costs were willing to take on to live happily. I’ll leave it at that so you can enjoy.
[“The Ones Who Walk Away from Omelas” – Ursula Le Guin – October, 1973]
Alright, that’s it, thanks for reading. Here are the picks all wrapped up:
The Full Lists
Articles (in chronological order):
- “Parking for Gold” – Geoff Manaugh – The Atlantic – January 2, 2018
- “The Case for the Subway” – Jonathan Mahler – New York Times Magazine – January 3, 2018
- “Making China Great Again” – Evan Osnos – New Yorker – January 8, 2018
- “My Father’s Body, at Rest and in Motion” – Siddhartha Mukherjee – New Yorker – January 8, 2018
- “The Strange Brands in Your Instagram Feed” – Alexis C. Madrigal – The Atlantic – January 10, 2018
- “The diabolical genius of the baby advice industry” – Oliver Burkeman – The Guardian – January 16, 2018
- “The Rising Pressure of the #MeToo Backlash” – Jia Tolentino – New Yorker – January 24, 2018
- “To Be, or Not to Be” – Masha Gessen – New York Review of Books – Feburary 8, 2018
- “Has Anyone Seen the President?” – Michael Lewis – Bloomberg – February 9, 2018
- “Inside The Two Years That Shook Facebook—And The World” – Nicholas Thompson and Fred Vogelstein – Wired – February 12, 2018
- “The White Darkness” – David Grann – New Yorker – February 12, 2018
- “What has really been going on with Markelle Fultz?” – Kyle Neubeck – Philly Voice – February 12, 2018
- “Jerry and Marge Go Large” – Jason Fagone – Highline – February 28, 2018
- “I Got a Story to Tell” – Steve Francis – Player’s Tribune – March 8, 2018
- “Christopher Steele, the Man Behind the Trump Dossier” – Jane Mayer – New Yorker – March 12, 2018
- “A $1.6 Billion Spotify Lawsuit Is Based On A Law Made For Player Pianos” – Sarah Jeong – The Verge – March 14, 2018
- “Why Earth’s History Appears So Miraculous” – Peter Brannan – The Atlantic – March 15, 2018
- “Tragically Lost in Joshua Tree’s Wild Interior” – Geoff Manaugh – New York Times Magazine – March 22, 2018
- “When Truth and Reason Are No Longer Enough” – Allison Gopnik – The Atlantic – April, 2018
- “The Isis Files: When Terrorists Run City Hall” – Rukmini Callimachi – New York Times – April 4, 2018
- “Why America’s Black Mothers and Babies Are in a Life-or-Death Crisis” – Linda Villarosa – New York Times Magazine – April 11, 2018
- “The Young and the Reckless” – Brendan Koerner – Wired – April 17, 2018
- “Whose Story (and Country) Is This?” – Rebecca Solnit – Literary Hub – April 18, 2018
- “How American Racism Influenced Hitler” – Alex Ross – New Yorker – April 30, 2018
- “Bad TV” – Andrea Long Chu – n+1 – Spring, 2018
- “The Gambler Who Cracked the Horse-Racing Code” – Kit Chellel – Bloomberg Businessweek – May 3, 2018
- “The Man Who Cracked the Lottery” – Reid Forgrave – New York Times Magazine – May 3, 2018
- “I’m Not Black, I’m Kanye” – Ta-Nehesi Coates – May 7, 2018
- “The Promise of Vaping and the Rise of Juul” – Jia Tolentino – New Yorker – May 14, 2018
- “How Fortnite Captured Teens’ Hearts and Minds” – Nick Paumgarten – New Yorker – May 21, 2018
- “Trump vs. the ‘Deep State'” – Evan Osnos – New Yorker – May 21, 2018
- “How the NBA got its groove back” – Kevin Arnovitz and Kevin Pelton – ESPN – May 24, 2018
- “Maybe She Had So Much Money She Just Lost Track of It Somebody had to foot the bill for Anna Delvey’s fabulous new life. The city was full of marks.” – Jessica Pressler – New York Magazine – May 28, 2018
- “The Curious Case of Bryan Colangelo and the Secret Twitter Account” – Ben Detrick – The Ringer – May 29, 2018
- “The Ultimate Humiliation” – Sarah Nicole Prickett – n+1 – May 30, 2018
- “Pay the Homeless” – Bryce Covert – Longreads – June, 2018
- “Stephen A. Smith Won’t Stop Talking” – Vinson Cunningham – New Yorker – June 25, 2018
- “‘It’s nothing like a broken leg’: why I’m done with the mental health conversation” – Hannah Jane Parkinson – The Guardian – June 30, 2018
- “How a Notorious Gangster Was Exposed by His Own Sister” – Patrick Radden Keefe – New Yorker – August 6, 2018
- “Why big companies squander brilliant ideas” – Tim Harford – Financial Times – September 8, 2018
- “What Termites Can Teach Us” – Amia Srinivasan – New Yorker – September 17, 2018
- “The Comforting Fictions of Dementia Care” – Larissa MacFarquhar – New Yorker – October 8, 2018
- “This Melissa McCarthy Story Just Might (Maybe? Possibly?) Cheer You Up” – Taffy Brodesser-Akner – New York Times Magazine – October 17, 2018
- “What if the Placebo Effect Isn’t a Trick?” – Gary Greenberg – New York Times Magazine – November 7, 2018
- “Nancy Pelosi’s Last Battle” – Robert Draper – New York Times Magazine – November 19, 2018
- “The Mystery of the Havana Syndrome” – Adam Entous and Jon Lee Anderson – New Yorker – November 19, 2018
- “Carmelo Anthony is the last great American ball hog” – Kirk Goldsberry – ESPN – December 6, 2018
Podcasts (in chronological order):
- “The Hollywood Edition” – Slate Money – March 31, 2018
- Calpihate – April, 2018
- “Jason Alexander” – WTF with Marc Maron – April 5, 2018
- “Alison Gopnik on ‘The wrong way to think about parenting, plus the downsides of modernity'” – Rationally Speaking – April 29, 2018
- “Invcel” – Reply All – May 10, 2018
- “Data, Decisions, and Basketball with Sam Hinkie” – Invest Like the Best – May 22, 2018
- American Fiasco – June, 2018
- “Charles Oakley Talks MJ Stories, LeBron, and His Favorite Fights” – The Bill Simmons Podcast – June 8, 2018
- “General Chapman’s Last Stand” – Revisionist History – June 13, 2018
- “The Crime Machine, Part 1” & “The Crime Machine, Part 2” – Reply All – October 11, 2018
- “Negative Mount Pleasant” – Reply All – December 6, 2018
Articles [Not 2018] (in chronological order):
- “The Last Question” – Isaac Asimov – November, 1956
- “The Ones Who Walk Away from Omelas” – Ursula Le Guin – October, 1973
- “True Crime: A Postmodern Murder Mystery” – David Grann – New Yorker – February 11, 2008
- “The Last Days of Stealhead Joe” – Ian Frazier – Outside – August 21, 2013
- “After decades of defeat, Caltech finds formula for winning in conference” – Chris Ballard – Sports Illustrated – November 23, 2015
- “The Falling Man: An Unforgettable Story” – Tom Junod – September 9, 2016
- “Promethea Unbound” – Mike Mariani – Atavist Magaqzine – October 27, 2017
- “Downward Spiral” – David Roth – The Baffler – December 4, 2017
I started and stopped this post four times as I tried to find the right way to open. Eventually I got tired of searching and figured it was easiest to just jump off the note I wrote to myself in Google Keep after the idea popped into my head:
That might not make so much sense (yet), but like any good note it captured enough of the concept that I remembered what I was thinking when I wrote it. I jotted it down as I was prepping for a webinar I did last week offering up some predictions for marketing in 2019. I was getting worked up (as I’m wont to do) about how much it bugs me when everyone in marketing talks about AI as if they have any idea what it really means or the implications. Someone asked why it bothered me so much and my answer, which kind of just poured out, was that once everyone starts agreeing about something (and saying it endlessly) it becomes less and less meaningful. This is not just some soft definition of the word meaning, though, it literally has less information.
A few months ago I wrote about Claude Shannon and information theory. Shannon wrote a seminal paper in 1948 called “A Mathematical Theory of Communication“. In it he defined the measure of information as, effectively, its unexpectedness (he called it entropy). The more random, the more information. This is precisely what bits measure (you can think of it as the number of yes/no questions it would take to get to the answer). What happens when you compress a photo? You take away the randomness. That’s why otherwise complex surfaces like sky or skin might come to look a bit pixelated: The compression algorithm is constraining the number of hues available in order to bring down the entropy (and therefore the file size) of the whole photo.
What does that mean for marketing buzzwords?
Well, as everyone starts to say the same thing and continue to offer little behind it, it becomes more and more expected and, therefore, starts to carry less and less information. When people layer on top of those buzzwords with real examples or alternative ideas, they return some randomness (and therefore information) to the concept. At their best, marketing contrarians are attempting to breathe some life into words and ideas that have otherwise lost their information content.
I don’t really like to think of myself as a contrarian because I think that often carries with it some notion of being different for the sake of being different (and trolling). Rather, I think if everyone is following one strategy or idea, the value of being the next person to jump on board is incrementally less (especially when that idea is poorly defined/understood). In a way it’s like an anti-network effect.
What can contrarians teach us?
That guy with the sad face is former Philadelphia 76ers General Manager Sam Hinkie. He is a contrarian. For the uninitiated, the brief history here is that Hinkie carried out one of the most radical transformation experiments in recent sports history. He put aside any notion that his team was trying to win and traded everything away for more ping pong balls in the NBA’s rookie lottery. In basketball, where only five guys from a team are on the floor at any one time, a superstar can have a massive impact on a team’s success. In a 2016 episode of his podcast Revisionist History, Malcolm Gladwell called basketball a “strong link” sport because a team’s success is best predicted by the quality of its best player (as opposed to soccer where it’s based on the worst player on the field he explains). Hinkie figured this out. He also figured out the best way to get one of those superstars was to draft them and to do that you had to have an early first round pick. In the end, he took a lot of heat for his strategy (even though he was following all the rules) and was eventually pushed out by the league office (and replaced by someone whose wife kept secret Twitter accounts where she posted lots of critical comments about players on the Sixers as well as some private medical information … whoops).
Back to Hinkie’s letter. It was leaked and provided an amazing view into the psyche of someone who was willing to be a pariah. In it he paints an interesting picture of the connection between contrarianism and traditionalism.
Here he is on contrarianism:
To develop truly contrarian views will require a never-ending thirst for better, more diverse inputs. What player do you think is most undervalued? Get him for your team. What basketball axiom is most likely to be untrue? Take it on and do the opposite. What is the biggest, least valuable time sink for the organization? Stop doing it. Otherwise, it’s a big game of pitty pat, and you’re stuck just hoping for good things to happen, rather than developing a strategy for how to make them happen.
And on traditionalism:
While contrarian views are absolutely necessary to truly deliver, conventional wisdom is still wise. It is generally accepted as the conventional view because it is considered the best we have. Get back on defense. Share the ball. Box out. Run the lanes. Contest a shot. These things are real and have been measured, precisely or not, by thousands of men over decades of trial and error. Hank Iba. Dean Smith. Red Auerbach. Gregg Popovich. The single best place to start is often wherever they left off.
Let’s bring it back to buzzwords.
So basically Hinkie’s argument is that the most appropriate way to be a contrarian is to also be a traditionalist: To be a respectful student of the underlying principles while also constantly probing and questioning whether they still make sense. One of the things that surprises me about the marketing industry is how often people miss this tradeoff. In an attempt to play the contrarian they shun traditional wisdom, but at the same time they repeat empty phrases and approaches at every conference that will let them on stage.
I actually think one of the reasons Byron Sharp’s book How Brands Grow has picked up as much steam as it has is because it strikes a good balance between these things. It’s a contrarian take (loyalty shouldn’t be a goal because it’s an outcome) but at the same time it’s deeply rooted in some traditional marketing ideas (marketshare, reach, and creativity to name three). This is a tough balance to strike, but when someone hits the spot is has the opportunity to really resonate.
Unfortunately, most of the time the industry misses the market by a lot. What we end up with a bunch of anti-historical/anti-intellectual slogans that get repeated ad-infinitum. It’s lots of words and little information.
This being Giving Tuesday, I thought it was appropriate to write up something up about a non-profit I think is worth supporting: Your local library. As always, if you enjoy these posts please sign up for my email to not miss any posts and always feel free to share with friends. Thanks.
At the beginning of the year I decided I was going to try to read more books this year. I don’t remember it was a resolution or what, but I set a goal of thirty and set on my way, tracking everything on Goodreads (which has legitimately become one of my favorite networks over the last eleven months). This post isn’t about my book list, though (I’ll wrap that up at the end of the year), but rather about the library. Most of the books I’ve read this year have been borrowed using Libby, the app offered by Overdrive which manages e-book borrowing for most libraries (including the New York Public and Brooklyn Public, where I belong).
When I’ve told people about my library habits I’ve gotten two reactions: There’s a group who is amazed that you can borrow Kindle books and promises to immediately go out and get themselves a card and there’s another who tells me they tried it, but the borrowing just didn’t work for them. They can never find books they want, they explain, and when they do finally find a good ebook to borrow, they are always on hold. I can’t say much about not finding books you want except that I’ve managed to find lots books this year that were both well worth reading and immediately available. But that’s not the point of this post. I want to talk about book holds and how they’re better thought of as a feature of the library, not a bug. Let me explain.
Back in 2006 I vividly remember reading about behavioral economics for the first time. I had somehow run across an article about it from Harvard Magazine and there was one bit in particular, about how pre-committing to something can help us work against our instinct to take the easy way out, that fascinated me at the time and still rattles around in my brain to this day. The basic idea, now commonly understood with the rising prominence of behavioral economics, is that humans do a very bad job of the value of things in the future. As a result, we are constantly doing things that give us pleasure in the short term and not the long term. In other words, we promise tomorrow’s self it will read that important book, watch that critically acclaimed film, or finally hit the gym while today’s self enjoys that trashy novel, watches another dumb sitcom episode, and drinks a few beers with friends instead of exercising.
But there’s a trick to dealing with our irrationality and it’s called pre-committing. The article offered up an analogy by way of Homer:
The goddess Circe informs Odysseus that his ship will pass the island of the Sirens, whose irresistible singing can lure sailors to steer toward them and onto rocks. The Sirens are a marvelous metaphor for human appetite, both in its seductions and its pitfalls. Circe advises Odysseus to prepare for temptations to come: he must order his crew to stopper their ears with wax, so they cannot hear the Sirens’ songs, but he may hear the Sirens’ beautiful voices without risk if he has his sailors lash him to a mast, and commands them to ignore his pleas for release until they have passed beyond danger. “Odysseus pre-commits himself by doing this,” Laibson explains.
Sometimes, as the analogy goes, we’ve got to bind ourselves to the mast of what’s good for us to actually make it happen. Back in 2006 my favorite example of pre-commitment was Netflix. If you can remember back to its days before life as a streaming service, you put a DVD at the bottom of your queue as you chose it and it slowly moved up the list as you watched and returned movies. The beauty of the system was that it disconnected what you wanted to watch from what you actually watched by splitting them up as two different functions (largely the result of needing to mail out DVDs). What it meant for me was that I watched a bunch of great films I’d always wanted to see because they showed up in my mailbox and I didn’t have another good choice. Instead of just watching another mindless procedural crime drama (not that there’s anything wrong with that), I finally got around to watching films from Alfred Hitchcock, Woody Allen, Orson Welles, and a bunch of other filmmakers that had been permanently relegated to deep depths of my mental movie queue.
So back to the library. If you haven’t borrowed an e-book before it works just like borrowing a physical one: The library has a set number of digital copies and if they’re all out at the moment then you get put on a waitlist. The longest you can borrow a book is 21 days and there are no renewals. That means holds often come through at inopportune times. Sometimes that means skipping the book altogether, but more often I’ve found it was just the push I needed to read something I wanted to read in the past but wouldn’t have necessarily made time for in the present.
In other words, in case you needed a reason to appreciate the public library outside the amazing civic resource it is (one branch of the New York Public Library announced they were lending ties, briefcases, and handbags to people who needed them for job interviews earlier this year), the borrowing mechanism can actually help you fight some of your more irrational tendencies.
Finally, because I can’t resist, if you appreciate the library it’s worth giving a donation if you can afford it. If you think of all the money you spend on Netflix and the like, it’s hopefully not too much of a hardship to offer your local library a few dollars a month. They’d surely appreciate it.
It’s been awhile since I did a Remainders posts so I figured I’d throw one together. In theory it’s all the other stuff I didn’t get a chance to blog about. In reality, it’s pretty much everything I’ve been reading that isn’t about mental models/frameworks (and even some of that). You can find previous versions filed under Remainders and, as always, if you enjoy the writing, please subscribe by email and pass around.
Let’s start with some books. Here’s what I’ve read in the last three months (in order of when they were read):
- Judas: How a Sister’s Testimony Brought Down a Criminal Mastermind (Astrid Holleeder): Inspired by the New Yorker story by Patrick Radden Keefe about a Dutch woman who eventually testified about her mobster brother, I decided to dig into the English translation. It was a lot more difficult to read than I expected. The New Yorker story, because of length, isn’t able to go into the extensive psychological abuse Holleeder’s brother put his family through. I found it emotionally exhausting about two-thirds into the book.
- Countdown to Zero Day (Kim Zetter): As far as I know this is the definitive book on Stuxnet, the digital weapon that targeted the Iranian nuclear facility at Natanz.
- Complexity: A Guided Tour (Melanie Mitchell): Easily one of my favorite books of the year. I’ve read lots about complexity theory, but nothing that pulled all the various strings together so well. (This also helped send me down a deep physics rabbit hole that I’ve yet to emerge from.)
- My Holiday in North Korea: The Funniest/Worst Place on Earth (Wendy Simmons): I really loved the graphic novel Pyongyang and thought I’d give this travelogue a try when I saw it sitting on a shelf at the bookstore. It was a fine book to read alongside some of the heavier stuff I’ve been reading lately.
- Remote: Office Not Required (Jason Fried): This book sucked, but at least the Audible narration was slow enough that I could crank it up to 2x speed.
- Einstein 1905: The Standard of Genius (John S. Rigden): Like I said, I’ve been falling deeper into a physics rabbit hole, and as part of that I’ve been watching a bunch of physics and math lectures on YouTube. One of the ones I watched was Douglas Hofstadter essentially trying to recreate a talk he once saw the John Rigden, the author of this book, give in 2005. The book, and the talk, are about the ideas behind Einstein’s five papers of 1905 (four of which are considered foundational in physics).
- The Undoing Project: A Friendship That Changed Our Minds (Michael Lewis): I am almost embarrassed to admit I still haven’t read Daniel Kahneman’s Thinking, Fast and Slow (it’s on the list, I swear), so Michael Lewis on the relationship between Kahneman and Taversky is the next best thing. Related: Malcolm Gladwell interviewing Lewis about the book.
- Perfect Rigor: A Genius and the Mathematical Breakthrough of the Century (Masha Gessen): Masha Gessen’s biography (I guess you could call it that) of Grigori Perelman, the eccentric mathematician who solved the Poincare Conjecture (one of the seven Millenium Problems from the Clay Institute) and then disappeared.
- Jorge Luis Borges: The Last Interview: and Other Conversations (Jorge Luis Borges): A long and fascinating conversation with Borges.
- Hit Refresh: The Quest to Rediscover Microsoft’s Soul and Imagine a Better Future for Everyone (Satya Nadella): Like just about everyone, I’m super impressed with everything Microsoft has done since promoting Nadella to CEO. Although this book promises to be about how it’s all happening, it’s about 75% a commercial for Microsoft’s vision for the future (which although it could be right, is not particularly interesting or original).
- Measure What Matters: How Google, Bono, and the Gates Foundation Rock the World with OKRs (John Doerr): A mostly interesting read about the OKR (objectives and key results) goal setting system.
- A Brief History of Time (Stephen Hawking): If you find yourself in a physics rabbit hole, this seems like something worth reading …
- Dreamtigers (Jorge Luis Borges): I read about this in the Borges interview book. He basically explained that his publisher asked for a book and so he collected a bunch of poems and stories that were sitting around his house and hadn’t been published and stuck it together.
Okay, onto some other reading, etc. …
This Wired piece about the possibility of a coming “AI cold war” has two particularly interesting strings in it: One is a fundamental question about the nature of technology and its relationship with democracy (put simply: is the internet better structured to support or defeat democratic ideals) and the other is about how China (and the US) will use 5G as a power play (“If you are a poor country that lacks the capacity to build your own data network, you’re going to feel loyalty to whoever helps lay the pipes at low cost. It will all seem uncomfortably close to the arms and security pacts that defined the Cold War.”)
I’ve been fascinated by the mysterious attacks against Americans in Cuba since I read about them (probably over a year ago now). I was excited to see the New Yorker finally dig in.
We’ve been having lots of trouble convincing our three-year-old to wear a coat in the cold. Turns out its pretty normal.
The Chronicle of Higher Education asked a bunch of academics for their most influential academic book of the last twenty years. Lots of interesting things to read here.
This is from earlier in the year, but it’s worth re-reading Bruce Schneier’s piece on securing elections. More recently he had a good one on mobile phone security.
- Benoît Mandelbrot (of fractal fame) is apparently responsible (at least in part) for the introduction of passwords at IBM. From When Einstein Walked with Gödel (which I’m reading now), “When his son’s high school teacher sought help for a computer class, Mandelbrot obliged, only to find that soon students all over Westchester County were tapping into IBM’s computers by using his name. ‘At that point, the computing center staff had to assign passwords,’ he says. ‘So I can boast-if that’s the right term-of having been at the origin of the police intrusion that this change represented.'”
- Also from the same book, the low numerals are meant to be representative of the number of things they are. Since that makes no sense, here’s the quote from the book: “Even Arabic numerals follow this logic: 1 is a single vertical bar; 2 and 3 began as two and three horizontal bars tied together for ease of writing.”
- When you get helium super cold very strange stuff starts happening.
- A Rochester garbage plate “is your choice of cheeseburger, hamburger, Italian sausages, steak, chicken, white or red hots*, served on top of any combination of home fries, french fries, baked beans, and/or macaroni salad.”
- There’s a taxonomy of parking garage design (image below).
Barkley Marathons sound awful.
This hit close to home:
It took 200 years for them to start making brown point shoes for non-white ballet dancers …
There’s apparently a big conversation going on in the machine learning community about whether ML is alchemy:
Rahimi believes contemporary machine learning models’ successes — which are mostly based on empirical methods — are plagued with the same issues as alchemy. The inner mechanisms of machine learning models are so complex and opaque that researchers often don’t understand why a machine learning model can output a particular response from a set of data inputs, aka the black box problem. Rahimi believes the lack of theoretical understanding or technical interpretability of machine learning models is cause for concern, especially if AI takes responsibility for critical decision-making.
This is a park covered in spiderwebs:
Tangentially related, here’s how corporate America contributes to politics by industry:
The Article Group email list is worth subscribing to. Back issues here.
I loved this quote from philosopher Daniel Dennet’s talk on what he calls intelligent design (don’t worry, it’s not the same):
Stochastic terrorism is one of those ideas you read once and think about from then on …
I don’t know where I fall on this, but I found Douglas Rushkoff’s argument that universal basic income is a scam being put forward by technology companies fascinating:
Uber’s business plan, like that of so many other digital unicorns, is based on extracting all the value from the markets it enters. This ultimately means squeezing employees, customers, and suppliers alike in the name of continued growth. When people eventually become too poor to continue working as drivers or paying for rides, UBI supplies the required cash infusion for the business to keep operating.
Adam Davidson had a good Twitter thread about “both-sidism” in political reporting.
Wired on “it’s not a bug, it’s a feature”.
The changing landscape of business expenses:
It seems like one out of 100 Player’s Tribune articles are amazing. This one from former Clipper Darius Miles fits the bill.
I’ve been really enjoying John Horgan’s Scientific American blog “Cross-Check”.
David Grann, who is probably my favorite author, snuck a book out without me knowing. Called White Darkness, it appears to be an expanded version of his New Yorker article about Antarctic explorers from earlier this year (one of my favorites).
Alright, I’m going to cut this here … I’m only caught up to late October, so look out for a part two. Thanks for reading.
If you haven’t read any of these yet, the gist is that I’m writing a book about mental models and writing these notes up as I go. You can find links at the bottom to the other frameworks I’ve written. If you haven’t already, please subscribe to the email and share these posts with anyone you think might enjoy them. I really appreciate it.
The vast majority of the models I’ve written about were ones that I discovered at one time or another and have adopted for my own knowledge portfolio. The Variance Spectrum, on the other hand, I came up with. Its origin was in trying to answer a question about why there wasn’t a centralized “system of record” for marketing in the same way you would find one in finance (ERP) or sales (CRM). My best answer was that the output of marketing made it particularly difficult to design a system that could satisfy the needs of all its users. Specifically, I felt as though the variance of marketing’s output, the fact that each campaign and piece of content is meant to be different than the one that came before it, made for an environment that at first seemed opposed to the basics of systemization that the rest of a company had come to accept.
To illustrate the idea I plotted a spectrum. The left side represented zero variance, the realm of manufacturing and Six Sigma, and the right was 100 percent variance, where R&D and innovation reign supreme.
While the poles of the spectrum help explain it, it’s what you place in the middle that makes it powerful. For example, we could plot the rest of the departments in a company by the average variance of their output (finance is particularly low since so much of the department’s output is “governed” — quite literally the government sets GAAP accounting standards and mandates specific tax forms). Sales is somewhere in the middle: A pretty good mix of process and methodology plus the “art of the deal”. Marketing, meanwhile, sits off to the right, just behind R&D.
But that’s just the first layer. Like so many parts of an organization (and as described in my essays on both The Parable of Two Watchmakers and Conway’s Law), companies are hierarchical and at any point in the spectrum you can drill in and find a whole new spectrum of activities that range from low variance to high variance. That is, while finance may be “low variance” on average thanks to government standards, forecasting and modeling is most certainly a high variance function: Something that must be imagined in original ways depending on a number of variables include the company, and its products and markets (to name a few). Zooming in on marketing we find a whole new set of processes that can themselves be plotted based on the variance of their output, with governance far to the low variance side and creative development clearly on the other pole. Another way to articulate these differences is that the low variance side represents the routine processes and the right the creative.
While I haven’t seen anyone else plot things quite this way, this idea, that there are fundamentally different kinds of tasks within a company, is not new. Organizational theorists Richard Cyert, Herbert Simon, and Donald Trow, also noted this duality in paper from 1956 called “Observation of a Business Decision“:
At one extreme we have repetitive, well-defined problems (e.g., quality control or production lot-size problems) involving tangible considerations, to which the economic models that call for finding the best among a set of pre-established alternatives can be applied rather literally. In contrast to these highly programmed and usually rather detailed decisions are problems of a non-repetitive sort, often involving basic long-range questions about the whole strategy of the firm or some part of it, arising initially in a highly unstructured form and requiring a great deal of the kinds of search processes listed above. In this whole continuum, from great specificity and repetition to extreme vagueness and uniqueness, we will call decisions that lie toward the former extreme programmed, and those lying toward the latter end non-programmed. This simple dichotomy is just a shorthand for the range of possibilities we have indicated.
This also introduces an interesting additional way to think about the spectrum: The left side is representative of those ideas where you have the most clarity about the final goal (in manufacturing you know exactly what you want the output to look like when it’s done) and the right the most ambiguity (the goal of R&D is to make something new). For that reason, high variance tasks should also fail far more often than their low variance counterparts: Nine out of ten new product ideas might be a good batting average, but if you are throwing away 90 percent of your manufactured output you’ve massively failed.
Even though it may be tempting, that’s not a reason to focus purely on the well-structured, low-variance problems, as Richard Cyert laid out in a 1994 paper titled “Positioning the Organization“:
It is difficult to deal with the uncertainty of the future, as one must to relate an organization to others in the industry and to events in the economy that may affect it. One must look ahead to determine what forces are at work and to examine the ways in which they will affect the organization. These activities are less structured and more ambiguous than dealing with concrete problems and, therefore, the CEO may have trouble focusing on them. Many experiments show that structured activity drives out unstructured. For example, it is much easier to answer one’s mail than to develop a plan to change the culture of the organization. The implications of change are uncertain and the planning is unstructured. One tends to avoid uncertainty and to concentrate on structured problems for which one can correctly predict the solutions and implications.
Going a level deeper, another way to cut the left and right sides of the spectrum is based on the most appropriate way to solve the problem. For the routine tasks you want to have a single way of doing things in an attempt to push down the variance of the output while on the high variance side you have much more freedom to try different approaches. In software terms this can be expressed as automation and collaboration respectively.
While this is primarily a framework for thinking about process, there’s a more personal way to think about the variance spectrum as it relates to giving feedback to others. It’s a common occurrence that employees over-or-misinterpret the feedback of more senior members of the team. I experienced this many times myself in my role as CEO. Because words are often taken literally from the leader of a company, an aside about something like color choice in a design comp can be easily misconstrued as an order to change when it wasn’t meant that way. The variance spectrum in that context can be used to make explicit where the feedback falls: Is it a low variance order you expect to be acted on or a high variance comment that is simply your two cents? I found this could help avoid ambiguity and also make it more clear I respected their expertise.
- Cyert, R. M., Simon, H. A., & Trow, D. B. (1956). Observation of a business decision. The Journal of Business, 29(4), 237-248.
- Cyert, R. M. (1994). Positioning the organization. Interfaces, 24(2), 101-104.
- Dong, J., March, J. G., & Workiewicz, M. (2017). On organizing: an interview with James G. March. Journal of Organization Design, 6(1), 14.
- March, J. G. (2010). The ambiguities of experience. Cornell University Press.
- Simon, H. A. (2013). Administrative behavior. Simon and Schuster.
- Stene, E. O. (1940). An approach to a science of administration. American Political Science Review, 34(6), 1124-1137.
Framework of the Day posts:
Thanks again for reading and for all the positive feedback. Please keep it coming. If you haven’t read any of these yet, the gist is that I’m writing a book about mental models and writing these notes up as I go. You can find links at the bottom to the other frameworks I’ve written. If you haven’t already, please subscribe to the email and share these posts with anyone you think might enjoy them. I really appreciate it.
Credit: Organizational Charts by Manu CornetI first ran into Conway’s Law while helping a brand redesign their website. The client, a large consumer electronics company, was insistent that the navigation must offer three options: Shop, Learn, and Support. I valiantly tried to convince them that nobody shopping on the web, or anywhere else, thought about the distinction between shopping and learning, but they remained steadfast in their insistence. What I eventually came to understand is that their stance wasn’t born out of customer need or insight, but rather their own organizational chart, which shockingly included a sales department, a marketing department, and a support department.
“Organizations which design systems (in the broad sense used here) are constrained to produce designs which are copies of the communication structures of these organizations.” That’s the way computer scientist and software engineer Melvin Conway put it in a 1968 paper titled “How Do Committees Invent?” His point was that the choices we make before start designing any system most often fundamentally shapes the final output. Or, as he put it, “the very act of organizing a design team means that certain design decisions have already been made.”
Why does this happen, where does it happen, and what can we do about it? That’s the goal of this essay, but before I get there we’ve got to take a short sojourn into the history of the concept. As I mentioned, the idea in its current form came from Melvin Conway in May of 1968. In the article he cited a few key sources as inspiration including economist John Kenneth Galbraith and historian C. Northcote Parkinson, who’s 1957 book Parkinson’s Law and Other Studies in Administration was particularly influential in spelling out the ever-increasing complexity that any bureaucratic organization will create. Finally, judging by the focus on modularity in Conway’s writing, it seems clear he was also inspired by Herbert Simon’s work, in particular his “Architecture of Complexity” paper and the Parable of Two Watchmakers (which I wrote about earlier).
Parkinson aside (who did so mostly in jest), very few have the chutzpah to actually name a law after themselves and Conway wasn’t responsible for the law’s coining. That came a few months after the “Committees” article was published from a fan and fellow computer scientist George Mealy. In his paper for the July 1968 National Symposium on Modular Programming (which I seem to be one of the very few people to have actually tracked down), Mealy examined four bits of “conventional wisdom” that surrounded the development of software systems at the time. Number four came directly from Conway: “Systems resemble the organizations that produced them.” The naming comes 3 pages in:
Our third aphorism-“if one programmer can do it in one year, two programmers can do it in two years”-is merely a reflection of the great difficulty of communication in a large organization. The crux of the problem of giganticism [sic] and system fiasco really lies in the fourth dogma. This — “systems resemble the organizations that produced them” — has been noticed by some of us previously, but it appears not to have received public expression prior to the appearance of Dr. Melvin E. Conway’s penetrating article in the April 1968 issue of Datamation. The article was entitled “How Do Committees Invent?”. I propose to call my preceding paraphrase of the gist of Conway’s paper “Conway’s Law”.
While most, including Conway on his own website, credit Fred Brooks’ 1975 Mythical Man Month with naming the law, it seems that Mealy deserves the credit (though Brooks’ book is surely the reason so many know about Conway’s important concept).Back to the questions at hand: Why does this happen, where does it happen, and what can we do about it?
Let’s start with the why. This seems like it should be easy to answer, but it’s actually not. The answer starts with some basics of hierarchy and modularity that Herbert Simon offered up in his Parable of Two Watchmakers: Mainly, breaking a system down into sets of modular subsystems seems to be the most efficient design approach in both nature and organizations. For that reason we tend to see companies made up of teams which are then made up of more teams and so-on. But that still doesn’t answer the question of why they tend to design systems in their image. To answer that we turn to some of the more recent research around the “mirroring hypothesis,” which (in simplified terms) is an attempt to prove out Conway’s Law. Carliss Baldwin, a professor at Harvard Business School, seems to be spearheading much of this work and has been an author on two of the key papers on the subject. Most recently, “The mirroring hypothesis: theory, evidence, and exceptions” is a treasure trove of information and citations. Her theory as to why mirroring occurs is essentially that it makes life easier for everyone who works at the company:
The mirroring of technical dependencies and organizational ties can be explained as an approach to organizational problem-solving that conserves scarce cognitive resources. People charged with implementing complex projects or processes are inevitably faced with interdependencies that create technical problems and conflicts in real time. They must arrive at solutions that take account of the technical constraints; hence, they must communicate with one another and cooperate to solve their problems. Communication channels, collocation, and employment relations are organizational ties that support communication and cooperation between individuals, and thus, we should expect to see a very close relationship—technically a homomorphism—between a network graph of technical dependencies within a complex system and network graphs of organizational ties showing communication channels, collocation, and employment relations.
It’s all still a bit circular, but the argument that in most cases a mirrored product is both reasonably optimal from a design perspective (since organizations are structured with hierarchy and modularity) and also cuts down the cognitive load by making it easy for everyone to understand (because it works like an org they already understand) seems like a reasonable one. The paper then goes on to survey the research to understand what kind of industries mirroring is most likely to occur and the answer seems to be everywhere. They found evidence from across expected places like software and semiconductors, but also automotive, defense, sports, and even banking and construction. For what it’s worth, I’ve also seen it across industries in marketing projects throughout my own career.
That’s the why and the where, which only leaves us with the question of what an organization can do about it. Here there seem to be a few different approaches. The first one is to do nothing. After all, it may well be the best way to design a system for that organization/problem. The second is to find an appropriate balance. If you buy the idea that some part of mirroring/Conway’s Law is simply about making it easier to understand and maintain systems, than its probably good to keep some mirroring. But it doesn’t need to be all or nothing. In the aforementioned paper, Baldwin and her co-authors have a nice little framework for thinking about different approaches to mirroring depending on the kind of business:
As you see at the bottom of the framework you have option three: “Strategic mirror-breaking.” This is also sometimes called an “inverse Conway maneuver” in software engineering circles: An approach where you actually adjust your organizational model in order to change the way your systems are architected. Basically you attempt to outline the type of system design you want (most of the time it’s about more modularity) and you back into an org structure that looks like that.
In case it seems like all this might be academic, the architecture of organizations has been shown to have a fundamental on the company’s ability to innovate. Tim Harford recently wrote a piece for the Financial Times that heavily quotes a 1990 paper by an economist named Rebecca Henderson titled “Architectural Innovation: The Reconfiguration of Existing Product Technologies and the Failure of Established Firms.” The paper outlines how the organizational structure of companies can prevent them from innovating in specific ways. Most specifically the paper describes the kind of innovation that keeps the shape of the previous generation’s product, but completely rewires it: Think film cameras to digital or the Walkman to MP3 players. Here’s Harford describing the idea:
Dominant organisations are prone to stumble when the new technology requires a new organisational structure. An innovation might be radical but, if it fits the structure that already existed, an incumbent firm has a good chance of carrying its lead from the old world to the new.
A case study co-authored by Henderson describes the PC division as “smothered by support from the parent company”. Eventually, the IBM PC business was sold off to a Chinese company, Lenovo. What had flummoxed IBM was not the pace of technological change — it had long coped with that — but the fact that its old organisational structures had ceased to be an advantage. Rather than talk of radical or disruptive innovations, Henderson and Clark used the term “architectural innovation”.
Like I said before, it’s all quite circular. It’s a bit like the famous quote “We shape our tools and thereafter our tools shape us.” Companies organize themselves and in turn design systems that mirror those organizations which in turn further solidify the organizational structure that was first put in place. Conway’s Law is more guiding principle than physical property, but it’s a good model to keep in your head as you’re designing organizations or systems (or trying to disentangle them).
- Arrow, K. J. (1985). Informational structure of the firm. The American Economic Review, 75(2), 303-307.
- Brunton-spall, Michael (2 Nov. 2015.). The Inverse Conway Manoeuvre and Security – Michael Brunton-Spall – Medium. Medium. Retrieved from https://medium.com/@bruntonspall/the-inverse-conway-manoeuvre-and-security-55ee11e8c3a9
- Colfer, L. J., & Baldwin, C. Y. (2016). The mirroring hypothesis: theory, evidence, and exceptions. Industrial and Corporate Change, 25(5), 709-738.
- Conway, Melvin E. “How do committees invent.” Datamation 14.4 (1968): 28-31.
- Conway, Melvin E. “The Tower of Babel and the Fighter Plane.” Retrieved from http://melconway.com/keynote/Presentation.pdf
- Evans, Benedict (31 Aug. 2018.). Tesla, software and disruption. Benedict Evans. Retrieved from https://www.ben-evans.com/benedictevans/2018/8/29/tesla-software-and-disruption
- Galbraith, J. K. (2001). The essential galbraith. HMH.
- Harford, Tim. (6 Sept. 2018.). Why big companies squander good ideas. Financial Times. Retrieved from https://www.ft.com/content/3c1ab748-b09b-11e8-8d14-6f049d06439c
- Henderson, R. M., & Clark, K. B. (1990). Architectural innovation: The reconfiguration of existing product technologies and the failure of established firms. Administrative science quarterly, 9-30.
- Hvatum, L. B., & Kelly, A. (2005). What do I think about Conway’s Law now?. In EuroPLoP (pp. 735-750).
- Lee, J. A. (1995). International biographical dictionary of computer pioneers. Taylor & Francis.
- MacCormack, A., Baldwin, C., & Rusnak, J. (2012). Exploring the duality between product and organizational architectures: A test of the “mirroring” hypothesis. Research Policy, 41(8), 1309-1324.
- MacDuffie, J. P. (2013). Modularity‐as‐property, modularization‐as‐process, and ‘modularity’‐as‐frame: Lessons from product architecture initiatives in the global automotive industry. Global Strategy Journal, 3(1), 8-40.
- Mealy, George, “How to Design Modular (Software) Systems,” Proc. Nat’l. Symp. Modular Programming, Information & Systems Institute, July 1968.
- Newman, Sam. (30 Jun. 2014.). Demystifying Conway’s Law. ThoughtWorks. Retrieved from https://www.thoughtworks.com/insights/blog/demystifying-conways-law
- Parnas, D. L. (1972). On the criteria to be used in decomposing systems into modules. Communications of the ACM, 15(12), 1053-1058.
- Software Engineering Radio. Kevin Goldsmith on Architecture and Organizational Design : Software Engineering Radio. Se-radio.net. Retrieved from http://www.se-radio.net/2018/07/se-radio-episode-331-kevin-goldsmith-on-architecture-and-organizational-design/
- Van Dusen, Matthew (19 May 2016.). A principle called “Conway’s Law” reveals a glaring, biased flaw in our technology. Quartz. Retrieved from https://qz.com/687457/a-principle-called-conways-law-reveals-a-glaring-biased-flaw-in-our-technology/
Framework of the Day posts:
I’m still hard at work on writing up Conway’s Law, so sharing something I wrote a few months ago that I haven’t posted yet. If you are following along, I’m working on a book about the frameworks we all use to understand the world and these are some drafts of the work. I appreciate any feedback and hope you’ll subscribe by email if you haven’t. Thanks for reading.
Most people know the Pareto principle by it’s more common name, “the 80/20 rule.” It’s story starts in the late-1800s with the Italian economist Vilfredo Pareto. Responsible for a number of economic breakthroughs, Pareto became particularly interested in the distribution of income. After collecting wealth and tax data from a variety of countries, he noticed a consistent pattern in the distribution. Originally outlined in his first major work, Cours d’Économie Politique, Pareto had discovered that across countries 20 percent of the population seemed to control around 80 percent of the income.
Source: “The Curve of the Distribution of Wealth.” History of Economic Ideas 17.1 (Translation: 2009)
Although he had uncovered the phenomena, Pareto wasn’t sure why it existed:
It is not easy to understand a priori how and why this should happen. As I said in my Cours, it seems to me probable that the income curve is in some way dependent on the law of the distribution of the mental and physiological qualities of a certain number of individuals. If such is really the case, we can catch a glimpse of the reason why approximately the same law is to be found in the most varied manifestations of human activity. But, instead of seeing those phenomena only in dim outlines, we would like to perceive them clearly and precisely, and up till now I have not succeeded in doing so.
The specifics of 80 and 20 aren’t critical, the point is that a small portion of a specific population tends to account for a large portion of some other resource. As time has gone on we’ve found evidence for Pareto’s discovery in more and more systems: Just a few scientific papers grab most of the citations, a small portion of a company’s customers tend be responsible for large percent of its profits, a tiny number of users tends to make up the vast majority of the customer service requests, and a “vital few” factory defects account for the bulk of the production issues.
It’s that last one about factories that we have to thank for the popularity of the Pareto principle. Quality control pioneer (and catchy name-coiner) Joseph Juran explains:
It was during the late 1940s, when I was preparing the manuscript for Quality Control Handbook, First Edition, that I was faced squarely with the need for giving a short name to the universal. In the resulting write-up under the heading “Maldistribution of Quality Losses,” I listed numerous instances of such maldistribution as a basis for generalization. I also noted that Pareto had found wealth to be maldistributed. In addition, I showed examples of the now familiar cumulative curves, one for maldistribution of wealth and the other for maldistribution of quality losses. The caption under these curves reads “Pareto’s principle of unequal distribution applied to distribution of wealth and to distribution of quality losses.”
Juran went on to become an important management thinker and the Pareto principle spread through industry and the broader world. At this point the 80/20 rule has become a basic and helpful mental model that many managers understand.
But we still haven’t answered Pareto’s original question: What it is about human nature that causes this massive imbalance to continually emerge in such a variety of systems? To answer that we turn to Albert-László Barabási and his study of networks. As the web was emerging, Barabási and his colleagues were busy analyzing the new and rich datasets it generated. Every time they dug in, the same odd pattern emerged.
In one of their studies, the team set up a crawler to look at how different web pages linked to each other. Expecting to see a bell curve, they instead spotted something very different: “the network our robot brought back from its journey had many nodes with a few links only, and a few hubs with an extraordinarily large number of links.” Barabási continues, “The biggest surprise came when we tried to fit the histogram of the node connectivity on a so-called log-log plot. The fit told us that the distribution of links on various Webpages precisely follows a mathematical expression called a power law.”
What made this discovery so important was that power laws are a signal that you’re not working with random data. If you chart random (or more precisely disconnected) data points, like the heights of people in your town or the scores of students on a test, you see a bell curve distribution. However, if you chart non-random interdependent data points you get the power curve that Barabási kept seeing:
Power laws rarely emerge in systems completely dominated by a roll of the dice. Physicists have learned that most often they signal a transition from disorder to order. Thus the power laws we spotted on the Web indicated, for the first time in precise mathematical terms, that real networks are far from random. Complex networks finally started to speak to us in a language that scientists trained in self-organization and complexity could finally understand. They spoke of order and emerging behavior. We just needed to listen carefully.
So we come full-circle back to Pareto, who once explained that, “The molecules in the social system are interdependent in space and in time. Their interdependence in space becomes apparent in the mutual relations that subsist between social phenomena.” The 80/20 rule is present in systems where there are self-organizing interdependent parts and its subject to the same cumulative advantage mechanics we saw with popular music. That’s why the pattern emerges so often in companies and markets: It means a huge number of forces are pushing and, critically, reacting to each other at the same time.
As should be reasonably obvious, the 80/20 rule has a number of important effects and implications for everyday business and life (many of which will come up in other models). First, understanding when you’re working in a system susceptible to the Pareto principle is critical. Once understood, being able to accurately isolate the 20 percent and find ways to make it less interdependent can fundamentally alter the balance of the equation. One of the simplest conclusions to be drawn from the 80/20 rule is that sometimes you need to fire a customer or an employee who is responsible for eating up the majority of your resources, as painful as that choice may be.
- Alexander, James. “Vilfredo Pareto: Sociologist and Philosopher.” Ihr.org. n.d. Web. 17 Dec. 2017. <http://www.ihr.org/jhr/v14/v14n5p10_Alexander.html>
- Aspers, Patrik. “Crossing the boundary of economics and sociology: The case of Vilfredo Pareto.” American Journal of Economics and Sociology 60.2 (2001): 519-545.
- Bunkley, Nick. “Joseph Juran, 103, Pioneer in Quality Control, Dies.” Nytimes.com. 3 Mar. 2008. Web. 17 Dec. 2017. <https://www.nytimes.com/2008/03/03/business/03juran.html>
- Chipman, John S. “Pareto: manuel of political economy.” English translation, available at http://www.econ.umn.edu/~jchipman/DALLOZ5.pdf, of ‘Pareto: Manuel di d’Économie Politique’ in Dictionnaire des grandes oeuvres d’économise, X Greffe, j. Lallemant and M De Vroey (eds), Paris: Dalloz (2002): 424-433.
- Cirillo, Renato. “Was Vilfredo Pareto Really a ‘Precursor’ of Fascism.?.” American Journal of Economics and Sociology 42.2 (1983): 235-246.
- Crawford, Walt. “Exceptional institutions: libraries and the Pareto principle.” American Libraries 32.6 (2001): 72-74.
- Edgeworth, F. Y., and Vilfredo Pareto. “Controversy Between Pareto and Edgeworth.” Giornale degli Economisti e Annali di Economia 67.3 (2008): 425-440.
- Hazlitt, Henry. “Pareto’s Picture of Society: His Monumental Work Covers an Enormous Field of Knowledge.” New York Times (May 26, 1935).
- Juran, Joseph M. “Pareto, lorenz, cournot, bernoulli, juran and others.” (1950).
- Juran, Joseph, and A. Blanton Godfrey. “Quality handbook.” Republished McGraw-Hill (1999).
- Juran, Joseph M. “The non-Pareto principle; mea culpa.” Quality Progress 8.5 (1975): 8-9.
- Juran, Joseph M. “Universals in management planning and controlling.” Management Review 43.11 (1954): 748-761.
- Koch, Richard. The 80/20 principle: the secret to achieving more with less. Crown Business, 2011.
- Lopreato, Joseph. “Notes on the work of Vilfredo Pareto.” Social Science Quarterly (1973): 451-468.
- Mandelbrot, Benoit, and Richard L. Hudson. The Misbehavior of Markets: A fractal view of financial turbulence. Basic books, 2007.
- Moore, H. L. “Cours d’Économie Politique. By VILFREDO PARETO, Professeur à l’Université de Lausanne. Vol. I. Pp. 430. I896. Vol. II. Pp. 426. I897. Lausanne: F. Rouge.” The ANNALS of the American Academy of Political and Social Science 9.3 (1897): 128-131.
- Pareto, Vilfredo. “Supplement to the Study of the Income Curve.” Giornale degli Economisti e Annali di Economia 67.3 (2008): 441-451.
- Pareto, Vilfredo. “The Curve of the Distribution of Wealth.” History of Economic Ideas 17.1 (2009): 132-143.
- Pareto, Vilfredo. The mind and society: Trattato di sociologia generale. AMS Press, 1935.
- Tarascio, Vincent J. “The Pareto law of income distribution.” Social Science Quarterly (1973): 525-533.
Framework of the Day posts:
Another framework of the day. If you haven’t read the others, the links are all at the bottom. I’m working on a book of mental models and sharing some of the research and writing as I go. This post actually started in writing about Conway’s Law, which is coming soon. I felt like I had to get this out first, as I would need to rely on some of the research in giving the Law its due. Thanks for reading and please let me know what you think, pass this link on, and subscribe to the email if you haven’t done it already. Thanks for reading.
This framework is a little different than the ones before as it doesn’t come with a nice diagram or four box. Rather, the Parable of Two Watchmakers is just that: A story about two people putting together complicated mechanical objects. The parable comes from a paper called “The Architecture of Complexity” written by Nobel-prize winning economist Herbert Simon (you might remember Simon from the theory of satisficing). Beyond being a brilliant economist, Simon was also a major thinker in the worlds of political science, psychology, systems, complexity, and artificial intelligence (in doing this research he climbed up the ranks of my intellectual heroes).
In his 1962 he laid out an argument for how complexity emerges, which is largely focused on the central role of hierarchy in complex systems. To start, let’s define hierarchy so we’re all on the same page. Here’s Simon:
Etymologically, the word “hierarchy” has had a narrower meaning than I am giving it here. The term has generally been used to refer to a complex system in which each of the subsystems is subordinated by an authority relation to the system it belongs to. More exactly, in a hierarchic formal organization, each system consists of a “boss” and a set of subordinate subsystems. Each of the subsystems has a “boss” who is the immediate subordinate of the boss of the system. We shall want to consider systems in which the relations among subsystems are more complex than in the formal organizational hierarchy just described. We shall want to include systems in which there is no relation of subordination among subsystems. (In fact, even in human organizations, the formal hierarchy exists only on paper; the real flesh-and-blood organization has many inter-part relations other than the lines of formal authority.) For lack of a better term, I shall use hierarchy in the broader sense introduced in the previous paragraphs, to refer to all complex systems analyzable into successive sets of subsystems, and speak of “formal hierarchy” when I want to refer to the more specialized concept.
So it’s more or less the way we think of it, except he is drawing a distinction to the formal hierarchy we see in an org chart where each subordinate has just one boss and the informal hierarchy that actually exists inside organizations, where subordinates interact in a variety of ways. And he points out the many complex systems we find hierarchy, including biological systems, “The hierarchical structure of biological systems is a familiar fact. Taking the cell as the building block, we find cells organized into tissues, tissues into organs, organs into systems. Moving downward from the cell, well-defined subsystems — for example, nucleus, cell membrane, microsomes, mitochondria, and so on — have been identified in animal cells.”
The question is why did all these systems come to be arranged this way and what can we learn from them? Here Simon turns to story:
Let me introduce the topic of evolution with a parable. There once were two watchmakers, named Hora and Tempus, who manufactured very fine watches. Both of them were highly regarded, and the phones in their workshops rang frequently — new customers were constantly calling them. However, Hora prospered, while Tempus became poorer and poorer and finally lost his shop. What was the reason?
The watches the men made consisted of about 1,000 parts each. Tempus had so constructed his that if he had one partly assembled and had to put it down — to answer the phone say— it immediately fell to pieces and had to be reassembled from the elements. The better the customers liked his watches, the more they phoned him, the more difficult it became for him to find enough uninterrupted time to finish a watch.
The watches that Hora made were no less complex than those of Tempus. But he had designed them so that he could put together subassemblies of about ten elements each. Ten of these subassemblies, again, could be put together into a larger subassembly; and a system of ten of the latter subassemblies constituted the whole watch. Hence, when Hora had to put down a partly assembled watch in order to answer the phone, he lost only a small part of his work, and he assembled his watches in only a fraction of the man-hours it took Tempus.
Whether the complexity emerges from the hierarchy or the hierarchy from the complexity, he illustrates clearly why we see this pattern all around us and articulates the value of the approach. It’s not just hierarchy, he goes on to explain, but also modularity (which he refers to as near-decomposability) that appears to be a fundamental property of complex systems. That is, each of the subsystems operates both independently and as part of the whole. As Simon puts it, “Intra-component linkages are generally stronger than intercomponent linkages” or, even more simply, “In a formal organization there will generally be more interaction, on the average, between two employees who are members of the same department than between two employees from different departments.”
Why is that? Well, for one, it’s an efficiency thing. Just as we see inside organizations, we want to use specialized resources in a specialized way. But beyond that, as Simon outlines in the parable, it’s also about resiliency: By relying on subsystems you have a defense against catastrophic failure when one piece of the whole breaks down. Just as Hora was able to quickly start building again when he put something down, any system made up of subsystems should be much more capable of dealing with changes in environment. It works in organisms, companies, and even empires, as Simon pointed out in The Sciences of the Artificial:
We have not exhausted the categories of complex systems to which the watchmaker argument can reasonably be applied. Philip assembled his Macedonian empire and gave it to his son, to be later combined with the Persian subassembly and others into Alexander’s greater system. On Alexander’s death his empire did not crumble to dust but fragmented into some of the major subsystems that had composed it.
Hopefully the application of this framework is pretty clear (and also instructive) in every day business life. Interestingly, Simon’s theories were the ultimate inspiration for a management fad we saw burn bright (and flame out) just a few years ago: Holacracy, the fluid organizational structure made up of self-organizing teams. Invented by Brian Robertson and made famous by Tony Hsieh and Zappos, the method (it’s a registered trademark) is based on ideas about “holons” from Hungarian author and journalist Arthur Koestler. In his 1967 book The Ghost in the Machine, Koestler repeats Simon’s story of Tempus and Hora and then goes on to theorize that holons (a name he coined “from the Greek holos—whole, with the suffix on (cf. neutron, proton) suggesting a particle or part”) are “meant to supply the missing link between atomism and holism, and to supplant the dualistic way of thinking in terms of ‘parts’ and ‘wholes,’ which is so deeply engrained in our mental habits, by a multi-levelled, stratified approach. A hierarchically-organized whole cannot be “reduced” to its elementary parts; but it can be ‘dissected’ into its constituent branches of holons, represented by the nodes of the tree-diagram, while the lines connecting the holons stand for channels of communication, control or transportation, as the case may be.”
Holacracy aside, there’s a ton of goodness in the parable and the architecture of modularity that it posits as critical. It’s not an accident that every company is built this way and as we think about those companies designing systems, it’s also not surprising many of those should also follow suit (a good lead-in for Conway’s Law, which is up next). Although I’m pretty out of words at this point, Simon also applies the same hierarchy/modularity concept to problem solving and there’s a pretty good argument to be made that the “latticework of models” Charlier Munger described in his 1994 USC Business School Commencement Address would fit the framework.
- Egidi, Massimo, and Luigi Marengo. “Cognition, institutions, near decomposability: rethinking Herbert Simon’s contribution.” (2002).
- Egidi, Massimo. “Organizational learning, problem solving and the division of labour.” Economics, bounded rationality and the cognitive revolution. Aldershot: Edward Elgar (1992): 148-73.
- Koestler, Arthur, and John R. Smythies. Beyond Reductionism, New Perspectives in the Life Sciences [Proceedings of] the Alpbach Symposium . (1972).
- Koestler, Arthur. “The ghost in the machine.” (1967).
- Radner, Roy. “Hierarchy: The economics of managing.” Journal of economic literature 30.3 (1992): 1382-1415.
- Simon, Herbert A. “Near decomposability and the speed of evolution.” Industrial and corporate change 11.3 (2002): 587-599.
- Simon, Herbert A. “The Architecture of Complexity.” Proceedings of the American Philosophical Society 106.6 (1962): 467-482.
- Simon, Herbert A. “The science of design: Creating the artificial.” Design Issues (1988): 67-82.
- Simon, Herbert A. The sciences of the artificial. MIT press, 1996.
Framework of the Day posts:
As some of you may know I’ve been collecting mental models and working on a book for a little while now (it’s been going pretty slow since my daughter was born in January). This is more notes than chapter, but I still thought it was worth sharing. If you like this I’m happy to do more in the future (I wrote about the pace layers framework in my last post). Oh, and if you haven’t already, sign up to get my new blog posts by email, it’s the best way to keep up.
By all accounts Donald Rumsfeld was a man who didn’t suffer from a shortage of self-confidence. Whether it was Meet the Press, Errol Morris’s documentary Unknown Known (it’s also worth reading the four-part series Morris wrote on Rumsfeld/the documentary for the New York Times), or a grilling from Jon Stewart on the Daily Show, he always seemed supremely satisfied with his own certainty. Which must have made the public response to what’s become his most famous comment all the more vexing. At a Department of Defense briefing in February, 2002, then Secretary of Defense Rumsfeld was asked about evidence to support claims of Iraq helping to supply terrorist organizations with weapons of mass destruction. “Because,” the questioner explained, “there are reports that there is no evidence of a direct link between Baghdad and some of these terrorist organizations.”
Rumsfeld famously replied:
Reports that say that something hasn’t happened are always interesting to me, because as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns — the ones we don’t know we don’t know. And if one looks throughout the history of our country and other free countries, it is the latter category that tend to be the difficult ones.
While it’s a mouthful and the context shouldn’t be lost, there’s a useful framework buried in Rumsfeld’s dodge. It looks something like this:
(Give the whole article from the Project Management Institute on how to apply known unknowns to project management a read.)
Rumsfeld went on to title his memoir Known and Unknown, and explained his perspective on its meaning early in the book:
At first glance, the logic may seem obscure. But behind the enigmatic language is a simple truth about knowledge: There are many things of which we are completely unaware—in fact, there are things of which we are so unaware, we don’t even know we are unaware of them. Known knowns are facts, rules, and laws that we know with certainty. We know, for example, that gravity is what makes an object fall to the ground. Known unknowns are gaps in our knowledge, but they are gaps that we know exist. We know, for example, that we don’t know the exact extent of Iran’s nuclear weapons program. If we ask the right questions we can potentially fill this gap in our knowledge, eventually making it a known known. The category of unknown unknowns is the most difficult to grasp. They are gaps in our knowledge, but gaps that we don’t know exist. Genuine surprises tend to arise out of this category. Nineteen hijackers using commercial airliners as guided missiles to incinerate three thousand men, women, and children was perhaps the most horrific single unknown unknown America has experienced.
Rumsfeld was obsessed with Pearl Harbor. In his memoir he quotes a foreword written by game theorist/nuclear strategist Thomas Schelling that introduced a book about the attack by Roberta Wohlstetter. Schelling wrote (emphasis mine):
If we think of the entire U.S. government and its far-flung military and diplomatic establishment, it is not true that we were caught napping at the time of Pearl Harbor. Rarely has a government been more expectant. We just expected wrong. And it was not our warning that was most at fault, but our strategic analysis. We were so busy thinking through some “obvious” Japanese moves that we neglected to hedge against the choice that they actually made.
And it was an “improbable” choice; had we escaped surprise, we might still have been mildly astonished. (Had we not provided the target, though, the attack would have been called off.) But it was not all that improbable. If Pearl Harbor was a long shot for the Japanese, so was war with the United States; assuming the decision on war, the attack hardly appears reckless. There is a tendency in our planning to confuse the unfamiliar with the improbable. The contingency we have not considered seriously looks strange; what looks strange is thought improbable; what is improbable need not be considered seriously.
In other words, unknown unknowns.
Outside of politics, the framework is a useful way to categorize risk/uncertainty in life or business. I got interested and dug around a bit to find the historical context for the idea, which led me in a few different directions.
Rumsfeld credits William R. Graham at NASA with first introducing him to the concept in the late-90s, though it turns out to go back a lot further than that. The oldest reference I could find comes from a 1968 issue of the Armed Forces Journal International. In the article “The ‘Known Unknowns’ And The ‘Unknown Unknowns'” about the procurement of new weapons. The article opens like this:
Cheyenne was the first major Army weapon to be developed under DoD’s sometimes controversial contract definition procedures. General Bunker put the process in perspective by pointing out that no procedural system can entirely eliminate “surprises” from happening during development of a complex weapons system, and that contract definition wasn’t expected to. “But,” he pointed out, “there are two kinds of technical problems: there are the known unknowns, and the unknown unknowns. Contract definition has helped eliminate the known unknowns. It cannot eliminate completely potential cost overruns, because these are due largely to the unknown unknowns.”
The term pops up throughout the 70s in relation to military procurement. Sometime in there some folks also start using the term “unk-unks” to refer to the most dangerous of the four boxes. Here it is in context from a 1982 New Yorker piece on the airplane industry:
The excitement of this business lies in the sweep of the uncertainties. Matters as basic as the cost of the product — the airplane — and its break-even point are obscure because so much else is uncertain or unclear. The fragility of the airline industry does, of course, create uncertainties about the size and the reliability of the market for a new airplane or a new variant of an existing airplane. Then, there is a wide range of unknowns, for which an arbitrarily fixed amount of money must be set aside in the development budget. Some of these are so-called known unknowns; others are thought of as unknown unknowns and are called “unk-unks.” The assumption is that normal improvements in an airplane program or an engine program will create problems of a familiar kind that add to the costs; these are the known unknowns. The term “unk-unks” is used to cover less predictable contingencies; the assumption is that any new airplane or engine intended to advance the state of the art will harbor surprises in the form of problems that are wholly unforeseen, and perhaps even novel, and these must be taken account of in the budget.
Some are even trying to use it as a kind of code word for breakthrough innovations.
Finally, although it’s not clear they’re connected, there’s a very similar framework from psychologists Joseph Luft and Harrington Ingham from 1955 called the Johari Window. The model attempts to visualize the effects of our knowledge of self and how that works in relation to the knowledge of others:
Quadrant I, the area of free activity, refers to behavior and motivation known to self and known to others.
Quadrant II, the blind area, where others can see things in ourselves of which we are unaware.
Quadrant III, the avoided or hidden area, represents things we know but do not reveal to others (e.g, a hidden agenda or matters about which we have sensitive feelings)
Quadrant IV, area of unknown activity. Neither the individual nor others are aware of certain behaviors or motives: Yet we can assume their existence because eventually some of these things become known, and it motives were influencing relationships all along.
Despite the context for the original quote, the idea is a useful way to think about strategy and understand the various risks you might face.
- Andrews, Walter. (1968). The “Known Uknnowns” And The “Unknown Unknowns”. Armed Forces Journal, p. 14-15.
- BBC NEWS | Magazine | What we know about ‘unknown unknowns’. (2018). News.bbc.co.uk. Retrieved 28 September 2018, from http://news.bbc.co.uk/2/hi/uk_news/magazine/7121136.stm
- Defense.gov Transcript: DoD News Briefing – Secretary Rumsfeld and Gen. Myers . (2002). Archive.defense.gov. Retrieved 28 September 2018, from http://archive.defense.gov/Transcripts/Transcript.aspx?TranscriptID=2636
- Graham, D. (2014). Rumsfeld’s Knowns and Unknowns: The Intellectual History of a Quip. The Atlantic. Retrieved 28 September 2018, from https://www.theatlantic.com/politics/archive/2014/03/rumsfelds-knowns-and-unknowns-the-intellectual-history-of-a-quip/359719/
- Hevesi, D. (2015). Roberta Wohlstetter, 94, Military Policy Analyst, Dies. Nytimes.com. Retrieved 28 September 2018, from https://www.nytimes.com/2007/01/11/obituaries/11wohlstetter.html
- Kim, S. D. (2012). Characterizing unknown unknowns. Paper presented at PMI® Global Congress 2012—North America, Vancouver, British Columbia, Canada. Newtown Square, PA: Project Management Institute.
- Kirkpatrick, L.B. Book review of Pearl Harbor: Warning and Decision by Roberta Wohlstetter — Central Intelligence Agency. (1993). Cia.gov. Retrieved 28 September 2018, from https://www.cia.gov/library/center-for-the-study-of-intelligence/kent-csi/vol7no3/html/v07i3a13p_0001.htm
- Grimes, W. (2016). Thomas C. Schelling, Master Theorist of Nuclear Strategy, Dies at 95. Nytimes.com. Retrieved 28 September 2018, from https://www.nytimes.com/2016/12/13/business/economy/thomas-schelling-dead-nobel-laureate.html
- Luft, Joseph, and Harry Ingham. “The johari window.” Human Relations Training News 5.1 (1961): 6-7.
- Morris, E. (2014). The Certainty of Donald Rumsfeld (Part 1). Opinionator. Retrieved 28 September 2018, from https://opinionator.blogs.nytimes.com/2014/03/25/the-certainty-of-donald-rumsfeld-part-1/
- Morris, E. (2014). The Certainty of Donald Rumsfeld (Part 2). Opinionator. Retrieved 28 September 2018, from https://opinionator.blogs.nytimes.com/2014/03/26/the-certainty-of-donald-rumsfeld-part-2/
- Morris, E. (2014). The Certainty of Donald Rumsfeld (Part 3). Opinionator. Retrieved 28 September 2018, from https://opinionator.blogs.nytimes.com/2014/03/27/the-certainty-of-donald-rumsfeld-part-3/
- Morris, E. (2014). The Certainty of Donald Rumsfeld (Part 4). Opinionator. Retrieved 28 September 2018, from https://opinionator.blogs.nytimes.com/2014/03/28/the-certainty-of-donald-rumsfeld-part-4/
- Mullins, J. W. (2007). Discovering” Unk-Unks”. MIT Sloan Management Review, 48(4), 17.
- Newhouse, J. (1982). A sporty game: I. betting the Company. The New Yorker, 48-105.
- Ramasesh, R. V., & Browning, T. R. (2014). A conceptual framework for tackling knowable unknown unknowns in project management. Journal of Operations Management, 32(4), 190-204.
- Rumsfeld, D. (2011). Known and unknown: a memoir. Penguin.
- Schelling, Thomas C. “Meteors, Mischief, and War.” Bulletin of the Atomic Scientists 16.7 (1960): 292-300.
- Steyn, M. (2003). Rummy speaks the truth, not gobbledygook. Telegraph.co.uk. Retrieved 28 September 2018, from https://www.telegraph.co.uk/comment/personal-view/3599959/Rummy-speaks-the-truth-not-gobbledygook.html
- Wilson, George C. (1969). The Washington Post, p. A1.
- Wohlstetter, Roberta. Pearl Harbor: warning and decision. Stanford University Press, 1962.
- Wright, Robert A. (1970). Lockheed’s Illness Is Contagious. New York Times.
Framework of the Day posts:
As some of you may know I’ve been collecting mental models and working on a book for a little while now (it’s been going pretty slow since my daughter was born in January). This is more notes than chapter, but I still thought it was worth sharing. If you like this I’m happy to do more in the future. Oh, and if you haven’t already, sign up to get my new blog posts by email, it’s the best way to keep up.
This one comes from Stewart Brand and is a way to explain the different speed various layers of society moves. The outer layer, fashion, is the quickest, while the innermost layer, nature, moves most slowly. Each layer interacts with one another as inventions and ideas get digested. As Brand explains:
The job of fashion and art is to be froth—quick, irrelevant, engaging, self-preoccupied, and cruel. Try this! No, no, try this! It is culture cut free to experiment as creatively and irresponsibly as the society can bear. From all that variety comes driving energy for commerce (the annual model change in automobiles) and the occasional good idea or practice that sifts down to improve deeper levels, such as governance becoming responsive to opinion polls, or culture gradually accepting “multiculturalism” as structure instead of just entertainment.
Brand’s inspiration for the framework came from an architect named Frank Duffy who encouraged builders not to think of a building as a single entity, but as a set of layers operating at different timescales. Duffy included four timescales: Shell, services, scenery, and sets (represented below).
Brand picked up on Duffy’s work and adapted it to a kind of proto-pace layer framework in his 1994 book How Buildings Learn: What Happens After They’re Built, expanding it to six S’s and including this handy diagram:
Brand eventually adapted that into the pace layer framework at the top in his 2008 book The Clock of the Long Now: Time and Responsibility (the chapter on pace layers was edited and republished last year in MIT’s Journal of Design and Science). If you want more, here’s a great writeup from Eric Nehrlich on a conversation about pace layers between Brand and Paul Saffo. Nehrlich calls out this slide from the presentation, which is quite helpful for understanding how the layers work:
(The whole talk is posted at the Long Now Blog if you’re so inclined.)
The framework has been picked up and adapted by many, but one of the more notable versions for me comes from Gartner as a way to think about your enterprise software strategy. They break enterprise software into three “layers”:
– Systems of Record — Established packaged applications or legacy homegrown systems that support core transaction processing and manage the organization’s critical master data. The rate of change is low, because the processes are well-established and common to most organizations, and often are subject to regulatory requirements.
– Systems of Differentiation — Applications that enable unique company processes or industry-specific capabilities. They have a medium life cycle (one to three years), but need to be reconfigured frequently to accommodate changing business practices or customer requirements.
– Systems of Innovation — New applications that are built on an ad hoc basis to address new business requirements or opportunities. These are typically short life cycle projects (zero to 12 months) using departmental or outside resources and consumer-grade technologies.
Each layer has it’s own pace of change, lifetime, planning horizon, governance model, and many other unique differentiators:
All in all, the overarching shearing/pace layers framework (many layers which interact with each other and operate at different speeds) is something I’ve found useful in various spheres in addition to the society, architecture, and enterprise software examples above. Inside a company, for instance, you conduct various activities that exist in a similar set of layers ranging from long-term planning and brand building to quarterly goals or roadmaps to two week sprints to weekly exec meetings and then the daily work. It’s a useful way to spot where you’re overloaded with meetings (too many weekly check-ins, not enough monthly lookbacks) or understand where you’re falling down (not doing a good enough job translating the medium term to the long term).
- Brand, Stewart. The clock of the long now: Time and responsibility. Basic Books, 2008.
- Brand, Stewart. How buildings learn: What happens after they’re built. Penguin, 1995.
- Brand, S. (2018). Pace Layering: How Complex Systems Learn and Keep Learning. Journal of Design and Science. https://doi.org/10.21428/7f2e5f08
- Duffy, Francis. “Measuring building performance.” Facilities 8.5 (1990): 17-20.
- Gartner.com. (2012). Gartner Says Adopting a Pace-Layered Application Strategy Can Accelerate Innovation. [online] Available at: https://www.gartner.com/newsroom/id/1923014 [Accessed 28 Sep. 2018].
- Mesaglio, Mary & Matthew Hotle. “Pace-Layered Application Strategy and IT Organizational Design: How to Structure the Application Team for Success.” Gartner, 2016.
- Nehrlich, E. (2015). Stewart Brand and Paul Saffo at the Interval. [online] Nehrlich.com. Available at: http://www.nehrlich.com/blog/2015/02/11/stewart-brand-and-paul-saffo-at-the-interval/ [Accessed 28 Sep. 2018].
Framework of the Day posts:
« Older posts