Conway’s Law [Framework of the Day]

Thanks again for reading and for all the positive feedback. Please keep it coming. If you haven’t read any of these yet, the gist is that I’m writing a book about mental models and writing these notes up as I go. You can find links at the bottom to the other frameworks I’ve written. If you haven’t already, please subscribe to the email and share these posts with anyone you think might enjoy them. I really appreciate it.

Credit: Organizational Charts by Manu CornetI first ran into Conway’s Law while helping a brand redesign their website. The client, a large consumer electronics company, was insistent that the navigation must offer three options: Shop, Learn, and Support. I valiantly tried to convince them that nobody shopping on the web, or anywhere else, thought about the distinction between shopping and learning, but they remained steadfast in their insistence. What I eventually came to understand is that their stance wasn’t born out of customer need or insight, but rather their own organizational chart, which shockingly included a sales department, a marketing department, and a support department.

“Organizations which design systems (in the broad sense used here) are constrained to produce designs which are copies of the communication structures of these organizations.” That’s the way computer scientist and software engineer Melvin Conway put it in a 1968 paper titled “How Do Committees Invent?” His point was that the choices we make before start designing any system most often fundamentally shapes the final output.¹ Or, as he put it, “the very act of organizing a design team means that certain design decisions have already been made.”

Why does this happen, where does it happen, and what can we do about it? That’s the goal of this essay, but before I get there we’ve got to take a short sojourn into the history of the concept. As I mentioned, the idea in its current form came from Melvin Conway in May of 1968. In the article he cited a few key sources as inspiration including economist John Kenneth Galbraith and historian C. Northcote Parkinson, who’s 1957 book Parkinson’s Law and Other Studies in Administration was particularly influential in spelling out the ever-increasing complexity that any bureaucratic organization will create.² Finally, judging by the focus on modularity in Conway’s writing, it seems clear he was also inspired by Herbert Simon’s work, in particular his “Architecture of Complexity” paper and the Parable of Two Watchmakers (which I wrote about earlier).

Parkinson aside (who did so mostly in jest), very few have the chutzpah to actually name a law after themselves and Conway wasn’t responsible for the law’s coining. That came a few months after the “Committees” article was published from a fan and fellow computer scientist George Mealy. In his paper for the July 1968 National Symposium on Modular Programming (which I seem to be one of the very few people to have actually tracked down), Mealy examined four bits of “conventional wisdom” that surrounded the development of software systems at the time. Number four came directly from Conway: “Systems resemble the organizations that produced them.” The naming comes 3 pages in:

Our third aphorism-“if one programmer can do it in one year, two programmers can do it in two years”-is merely a reflection of the great difficulty of communication in a large organization. The crux of the problem of giganticism [sic] and system fiasco really lies in the fourth dogma. This — “systems resemble the organizations that produced them” — has been noticed by some of us previously, but it appears not to have received public expression prior to the appearance of Dr. Melvin E. Conway’s penetrating article in the April 1968 issue of Datamation. The article was entitled “How Do Committees Invent?”. I propose to call my preceding paraphrase of the gist of Conway’s paper “Conway’s Law”.

While most, including Conway on his own website, credit Fred Brooks’ 1975 Mythical Man Month with naming the law, it seems that Mealy deserves the credit (though Brooks’ book is surely the reason so many know about Conway’s important concept).³Back to the questions at hand: Why does this happen, where does it happen, and what can we do about it?

Let’s start with the why. This seems like it should be easy to answer, but it’s actually not. The answer starts with some basics of hierarchy and modularity that Herbert Simon offered up in his Parable of Two Watchmakers: Mainly, breaking a system down into sets of modular subsystems seems to be the most efficient design approach in both nature and organizations. For that reason we tend to see companies made up of teams which are then made up of more teams and so-on. But that still doesn’t answer the question of why they tend to design systems in their image. To answer that we turn to some of the more recent research around the “mirroring hypothesis,” which (in simplified terms) is an attempt to prove out Conway’s Law. Carliss Baldwin, a professor at Harvard Business School, seems to be spearheading much of this work and has been an author on two of the key papers on the subject. Most recently, “The mirroring hypothesis: theory, evidence, and exceptions” is a treasure trove of information and citations. Her theory as to why mirroring occurs is essentially that it makes life easier for everyone who works at the company:

The mirroring of technical dependencies and organizational ties can be explained as an approach to organizational problem-solving that conserves scarce cognitive resources. People charged with implementing complex projects or processes are inevitably faced with interdependencies that create technical problems and conflicts in real time. They must arrive at solutions that take account of the technical constraints; hence, they must communicate with one another and cooperate to solve their problems. Communication channels, collocation, and employment relations are organizational ties that support communication and cooperation between individuals, and thus, we should expect to see a very close relationship—technically a homomorphism—between a network graph of technical dependencies within a complex system and network graphs of organizational ties showing communication channels, collocation, and employment relations.

It’s all still a bit circular, but the argument that in most cases a mirrored product is both reasonably optimal from a design perspective (since organizations are structured with hierarchy and modularity) and also cuts down the cognitive load by making it easy for everyone to understand (because it works like an org they already understand) seems like a reasonable one.⁴ The paper then goes on to survey the research to understand what kind of industries mirroring is most likely to occur and the answer seems to be everywhere. They found evidence from across expected places like software and semiconductors, but also automotive, defense, sports, and even banking and construction. For what it’s worth, I’ve also seen it across industries in marketing projects throughout my own career.

That’s the why and the where, which only leaves us with the question of what an organization can do about it. Here there seem to be a few different approaches. The first one is to do nothing. After all, it may well be the best way to design a system for that organization/problem. The second is to find an appropriate balance. If you buy the idea that some part of mirroring/Conway’s Law is simply about making it easier to understand and maintain systems, than its probably good to keep some mirroring. But it doesn’t need to be all or nothing. In the aforementioned paper, Baldwin and her co-authors have a nice little framework for thinking about different approaches to mirroring depending on the kind of business:

As you see at the bottom of the framework you have option three: “Strategic mirror-breaking.” This is also sometimes called an “inverse Conway maneuver” in software engineering circles: An approach where you actually adjust your organizational model in order to change the way your systems are architected.⁵ Basically you attempt to outline the type of system design you want (most of the time it’s about more modularity) and you back into an org structure that looks like that.

In case it seems like all this might be academic, the architecture of organizations has been shown to have a fundamental on the company’s ability to innovate. Tim Harford recently wrote a piece for the Financial Times that heavily quotes a 1990 paper by an economist named Rebecca Henderson titled “Architectural Innovation: The Reconfiguration of Existing Product Technologies and the Failure of Established Firms.” The paper outlines how the organizational structure of companies can prevent them from innovating in specific ways. Most specifically the paper describes the kind of innovation that keeps the shape of the previous generation’s product, but completely rewires it: Think film cameras to digital or the Walkman to MP3 players. Here’s Harford describing the idea:

Dominant organisations are prone to stumble when the new technology requires a new organisational structure. An innovation might be radical but, if it fits the structure that already existed, an incumbent firm has a good chance of carrying its lead from the old world to the new.

A case study co-authored by Henderson describes the PC division as “smothered by support from the parent company”. Eventually, the IBM PC business was sold off to a Chinese company, Lenovo. What had flummoxed IBM was not the pace of technological change — it had long coped with that — but the fact that its old organisational structures had ceased to be an advantage. Rather than talk of radical or disruptive innovations, Henderson and Clark used the term “architectural innovation”.

Like I said before, it’s all quite circular. It’s a bit like the famous quote “We shape our tools and thereafter our tools shape us.” Companies organize themselves and in turn design systems that mirror those organizations which in turn further solidify the organizational structure that was first put in place. Conway’s Law is more guiding principle than physical property, but it’s a good model to keep in your head as you’re designing organizations or systems (or trying to disentangle them).

Footnotes:

1. He was writing mostly about software systems, but as you’ll see it’s much more broadly applicable.↑
2. Here’s how Conway explains Parkinson’s complexity concept: “As each new brand is created it justifies itself by challenging the established order. Thus, after a while, the organization is fully occupied in internal political warfare.”↑
3. As an aside, it’s hard not to think that Mealy’s third point about what one programmer can do versus two sounds a lot like Fred Brooks’ “mythical man month” concept. Mealy worked with Brooks on OS/360 and in the book Computer Pioneers by J.A.N. Lee it’s mentioned that Mealy’s Law was also named at the 1968 symposium: “There is an incremental programmer who, when added to a project, consumes more resources than are made available.” Sounds pretty similar to me.↑
4. There’s a very interesting point about the role of “information hiding” in pushing companies into Conway’s Law. Essentially the idea is that companies naturally hide information within teams or departments for the sake of simplicity across the rest of the company. It would only make things more complicated, for instance, if the finance team exposed the detailed rules of GAAP accounting instead of just distributing a monthly GAAP accounting report. “Information hiding as a means of controlling complexity is a fundamental principle underlying the mirroring hypothesis. With information hiding, each module in a technical system is informationally isolated from other modules within a framework of system design rules. This means that independent individuals, teams, or firms can work separately on different modules, yet the modules will work together as a whole (Baldwin and Clark, 2000).”↑
5. If you’re interested in the idea you should check out the episode of Software Engineering Radio with engineering leader Kevin Goldsmith.↑

Bibliography:

• Arrow, K. J. (1985). Informational structure of the firm. The American Economic Review, 75(2), 303-307.
• Brunton-spall, Michael (2 Nov. 2015.). The Inverse Conway Manoeuvre and Security – Michael Brunton-Spall – Medium. Medium. Retrieved from https://medium.com/@bruntonspall/the-inverse-conway-manoeuvre-and-security-55ee11e8c3a9
• Colfer, L. J., & Baldwin, C. Y. (2016). The mirroring hypothesis: theory, evidence, and exceptions. Industrial and Corporate Change, 25(5), 709-738.
• Conway, Melvin E. “How do committees invent.” Datamation 14.4 (1968): 28-31.
• Conway, Melvin E. “The Tower of Babel and the Fighter Plane.” Retrieved from http://melconway.com/keynote/Presentation.pdf
• Evans, Benedict (31 Aug. 2018.). Tesla, software and disruption. Benedict Evans. Retrieved from https://www.ben-evans.com/benedictevans/2018/8/29/tesla-software-and-disruption
• Galbraith, J. K. (2001). The essential galbraith. HMH.
• Harford, Tim. (6 Sept. 2018.). Why big companies squander good ideas. Financial Times. Retrieved from https://www.ft.com/content/3c1ab748-b09b-11e8-8d14-6f049d06439c
• Henderson, R. M., & Clark, K. B. (1990). Architectural innovation: The reconfiguration of existing product technologies and the failure of established firms. Administrative science quarterly, 9-30.
• Hvatum, L. B., & Kelly, A. (2005). What do I think about Conway’s Law now?. In EuroPLoP (pp. 735-750).
• Lee, J. A. (1995). International biographical dictionary of computer pioneers. Taylor & Francis.
• MacCormack, A., Baldwin, C., & Rusnak, J. (2012). Exploring the duality between product and organizational architectures: A test of the “mirroring” hypothesis. Research Policy, 41(8), 1309-1324.
• MacDuffie, J. P. (2013). Modularity‐as‐property, modularization‐as‐process, and ‘modularity’‐as‐frame: Lessons from product architecture initiatives in the global automotive industry. Global Strategy Journal, 3(1), 8-40.
• Mealy, George, “How to Design Modular (Software) Systems,” Proc. Nat’l. Symp. Modular Programming, Information & Systems Institute, July 1968.
• Newman, Sam. (30 Jun. 2014.). Demystifying Conway’s Law. ThoughtWorks. Retrieved from https://www.thoughtworks.com/insights/blog/demystifying-conways-law
• Parnas, D. L. (1972). On the criteria to be used in decomposing systems into modules. Communications of the ACM, 15(12), 1053-1058.
• Software Engineering Radio. Kevin Goldsmith on Architecture and Organizational Design : Software Engineering Radio. Se-radio.net. Retrieved from http://www.se-radio.net/2018/07/se-radio-episode-331-kevin-goldsmith-on-architecture-and-organizational-design/
• Van Dusen, Matthew (19 May 2016.). A principle called “Conway’s Law” reveals a glaring, biased flaw in our technology. Quartz. Retrieved from https://qz.com/687457/a-principle-called-conways-law-reveals-a-glaring-biased-flaw-in-our-technology/

Framework of the Day posts:

• Conway’s Law
• Known Unknowns
• Pace Layers
• Parable of Two Watchmakers
• Pareto Principle (aka 80/20 Rule)
• Variance Spectrum

October 9, 2018 // This post is about: architectural innovation, carliss baldwin, complexity, Conway's Law, Framework of the Day, herbert simon, innovation, john kenneth galbraith, melvin conway, mirroring, mirroring hypothesis, org chart, parable of two watchmakers, parkinson's law, rebecca henderson, tim harford

Normalization of Deviance & The Growing Organization

A really interesting concept from this blog post on the recent very-avoidable crash of a private jet:

Social normalization of deviance means that people within the organization become so much accustomed to a deviant behavior that they don’t consider it as deviant, despite the fact that they far exceed their own rules for the elementary safety. People grow more accustomed to the deviant behavior the more it occurs. To people outside of the organization, the activities seem deviant; however, people within the organization do not recognize the deviance because it is seen as a normal occurrence. In hindsight, people within the organization realize that their seemingly normal behavior was deviant.

This is a pretty good explanation for what happens inside organizations. At the small and relatively harmless end, you have the slow growth of meetings that happens in every organization (hence Percolate’s meeting rules) and in the large and dangerous end you have something like Enron or VW. The phenomena is essentially about feedback loops: The deviant behavior slowly slips in and over time becomes more and more acceptable, meaning that the intensity can also increase. In the case of meetings what starts as a one-off meeting to discuss something, becomes a weekly meeting for 15 minutes, eventually with more people involved, and eventually you have 15 people spending an hour together without any real sense for what they’re talking about or why.

This is a pretty good picture of what makes it so tough to scale organizations: It’s hardly ever the big stuff you’re watching out for, it’s almost always the small stuff that is built up over time. (As an aside, although I’m not crazy about him, this is also why Clayton Christensen’s boiling frog analogy is such a good one. It’s a perfect way to understand that it’s the slow rolling boil that kills you, not the big explosion.)

January 11, 2016 // This post is about: innovation, meetings, normalization of deviance, organizational psychology

Harbinger of Things to Come

While I don’t agree with Paul Krugman’s Bitcoin is Evil post from the weekend, his next post on Bitcoin did include this nugget I had never considered:

It occurs to me that part of the disconnect is that Bitcoin solved a major technical problem, one that people had been thinking about for about 20 years, and we nerds just can’t believe that it doesn’t also solve an economic problem. The technical problem is double spending–if I have some digital money, it’s easy enough to verify cryptographically that it’s real, but if I give it to you, how can you tell that I haven’t also given it to someone else? Until Bitcoin, the answer was to have a bank that knew which coins were valid, so you’d present my coin to the bank, which would check its database and if it’s valid, cancel it and give you a new one. Bitcoin has its decentralized blockchain which is a very clever recasting of the problem so that the state of the “bank” is whatever the majority of bitcoin miners agree that it is. Getting enough of the miners to agree is known as the Byzantine Generals problem, and has a technical history of its own.

As simple as it seems, splitting out the technical achievement from the economic one is a really interesting way to think about it. I haven’t really spent enough time thinking about Bitcoin to have a strong opinion about it specifically, but I do think the idea of a crypto-currency not attached to a specific nation is something bound to happen. In a way, I feel like Bitcoin and Google Glass have a lot in common: Early experiments in form and style that signal what’s to come.

January 1, 2014 // This post is about: Bitcoin, google glass, innovation, Technology

Setting the Stage for Innovation

There is an interesting little article on innovation and Picasso over at Medium. Basically it suggests that radical innovation happens when the market is most receptive to it:

Sgourev’s analysis of Cubism suggests that having an exceptional idea isn’t enough: if it is to catch fire, the market conditions have to be right. That’s a question of luck and timing as much as it is of genius. The closest modern analogy to Picasso’s Paris is Silicon Valley in the early days of the dotcom boom, with art dealers as venture capitalists and entrepreneurs as artists.

This reminded me a lot of Duncan Watts’ research on influence on the web, where he concluded, “large scale changes in public opinion are not driven by highly influential people who influence everyone else, but rather by easily influenced people, influencing other easily influenced people.” In fact, Watts also used a fire to explain the dynamic in his conclusion:

Some forest fires, for examples, are many times larger than average; yet no-one would claim that the size of a forest fire can be in any way attributed to the exceptional properties of the spark that ignited it, or the size of the tree that was the first to burn. Major forest fires require a conspiracy of wind, temperature, low humidity, and combustible fuel that extends over large tracts of land. Just as for large cascades in social influence networks, when the right global combination of conditions exists, any spark will do; and when it does not, none will suffice.

The challenge, of course, as Watts points out in his research, is that consistently finding and predicting this environment is all but impossible. We may understand some of the factors, but the situation is just too complex to be anywhere near accurate. As much as we give credit to innovators who capture those radical moments, we also need to appreciate the role of luck in their success.

August 14, 2013 // This post is about: influence, innovation, Picasso, watts

The Apple Org Structure

Yesterday James, my co-founder at Percolate, sent me over a really interesting nugget about how Apple structures its company about 35 minutes into this Critical Path podcast. Essentially Horace (from Asymco) argues that Apple’s non-cross-functional structure actually allows it to innovate and execute far better than a company structured in a more traditional, non-functional, way. As opposed to most other companies where managers are encourages to pick up experience across the enterprise, Apple encourages (or forces), people to stay in their role for the entirety of their career. On top of that, roles are not horizontal by product (head of iPhone) and instead are vertical by discipline (design, operations, technologies) and also quite siloed. He goes on to say that the only parallel he could think of is the military, who basically operates that way. (I know I haven’t done the best job articulating it, that’s because as I listen again I don’t necessarily think the thesis is articulated all that well.)

Below is my response back to James:

While I totally agree with what he says about the structure (that they’re organized functionally and it works for them), I’m not sure you can just conclude that’s ideal or drives innovation. The requirement of an org structure like that is that all vision/innovation comes from the top and moves down through the organization. That’s fine when you have someone like Jobs in charge, but it’s questionable what happens when he leaves (or when this first generation he brought up leaves maybe). Look at what happened when Jobs left the first time as evidence for how they lost their way. Apple is a fairly unique org in that it has a very limited number of SKUs and, from everything we’ve heard, Jobs was the person driving most/all.

My question back to Horace would be what will Apple look like in 20 years. IBM and GE are 3x older than Apple is and part of how they’ve survived, I’d say, is that they’ve built the responsibility of innovation into a bit more of a cross-functional discipline + centralized R&D. I don’t know if it matters, but if I was making a 50 year bet on a company I’d pick GE over Apple and part of it is that org structure and its ability to retain knowledge.

Military is actually a perfect example: Look at the struggles they’ve had over the last 20 years as the enemy stopped being similarly structured organizations and moved to being loosely connected networks. History has shown us over and over centralized organizations struggle with decentralized enemies. Now the good news for Apple is that everyone else is pretty much playing the same highly organized and very predictable game (with the exception of Google, who is in a functionally different business and Samsung, who because of their manufacturing resources and Asian heritage exist in a little bit of a different world).

Again, in a 10 year race Apple wins with a structure like this. But in a 50 year race, in which your visionary leader is unlikely to still be manning the helm, I think it brings up a whole lot of questions.

August 12, 2013 // This post is about: apple, innovation, management, organization, structure