You have arrived at the web home of Noah Brier. This is mostly an archive of over a decade of blogging and other writing. You can read more about me or get in touch. If you want more recent writing of mine, most of that is at my BrXnd marketing x AI newsletter and Why Is This Interesting?, a daily email for the intellectually omnivorous.

October, 2008

On Predictions and Cruft

I recently watched Kevin Kelly's great TED video on the next 5,000 days of the web and two things in it really struck me. First was Kelly's assertion that man is an extension of technology, not vice versa. This was especially interesting having just read a paper that theorized man's evolution to walk upright was a direct result of tool (aka technology) use (essentially it suggests that once tools were in play we could no longer afford to walk around on four limbs and required that we develop a larger brain to deal with the complexity of wielding these things).

That, however, wasn't what struck me most. Rather it was Kelly talking about what everyone thought the web was going to become just five thousand days ago (that's roughly when the web came to be). People generally thought (and many still generally think) that the web is, to quote Kelly, "TV only better." In the rest of the video he explains his vision for the next 5,000 days of the web, specifically around what he calls "one machine" which more and more of our world will interface with. That topic, however, is for another entry, rather I want to focus on predictions.


People frequently tout a knowledge of history as the best way to avoid its repetition, but I can't help but think about how often we've been wrong. Mostly we are pretty terrible at predicting the future. That's because we are predicting it based on what exists today and as soon as something new rolls around tomorrow, everything we considered "normal" is thrown all out of whack and our predictions no longer apply.

One way to think about this is in terms of punctuated equilibrium, which is essentially the idea that we evolve in spurts. Of course technological change is not evolutionary in the purest sense of the word, but much change happens in this kind of pattern. Think about sports, for instance, you hear stories all the time of no one believing something could be done (four-minute miles or lifting some ungodly amount of weight) and all of a sudden when one person breaks the barrier everyone else follows along. Basically we can't do what we can't imagine until we can do it or imagine it. (How's that for a bit of circular logic?)

Put another way, as Kevin Kelly explains in his entry "The Maes-Garreau Point", "it has become very hard to imagine what life will be like after we are dead. The rate of change appears to accelerate, and so the next lifetime promises to be unlike our time, maybe even unimaginable. Naturally, then, when we forecast the future, we will picture something we can personally imagine, and that will thus tend to cast it within range of our own lives." He is referring specifically to predictions far in the future, but I think this is the fallacy of predictions generally: We can't predict that which we can't predict. (Got you again.) Or, as Kelly opens the entry with, "Forecasts of future events are heavily influenced by present circumstances. That’s why predictions are usually wrong. It’s hard to transcend current assumptions. Over time, these assumptions erode, which leads to surprise."

So what's the point of all this? Well, to be quite honest I'm not sure. I'm sitting on a plane feeling a bit restless and I figured writing might do the trick. But there's more, I promise, and I will try to connect these ideas to a few other themes that have been floating around lately.

First off, there were a ton of amazing responses to my last post on the economy and postmodernism, but I want to point specifically to something that Charles wrote which struck me (partly because I read it while thinking about this topic). In his response to my entry he wrote, "I don't concur that events are as random as you assert. I've pointed out that attendants of the World Economic Forum at Davos were raising the flag of an impending crisis quite some time back, and that on page 225 in Nassim Nicholas Taleb's Black Swan the footnote could not be clearer of the statistical backflips performed by J.P. Morgan's 'Riskmetrics' and the dangerous situation Fanny Mae was in." Which got me thinking, how many people predicted this financial meltdown? The reality of the situation is that there were probably tons of people who did, but we had no reason to listen. People predict demise all the time (how many Nostradamus end of the world predictions can we listen to before the guy ceases to get front-page billing in the supermarket tabloids?). The problem is, until we are in the situation it's hard to see the clues. Or, as I said before, we can't predict that which we can't predict.

Sure, in retrospect there were any number of clues pointing to the situation at hand, but if they had been strong enough clues wouldn't we have acted upon them? A clue is only a clue if it helps solve a mystery, afterwards it becomes explanation, equally important (for our psyche) but a very different beast. This isn't to say we shouldn't look for clues or explanations, just to say that they are different things and that they're increasingly obscured when we're dealing with something that we never imagined. Or, as Vinod Khosla explained in an interview I just happen to have run across (which is specifically about energy consumption -- and written prior to the current crisis -- but is nonetheless valid):

We have to shift from extrapolating past assumptions, ignore what economists and econometrics tells us about how much oil we need and when we'll run out and what consumptions will be. Because they are based on the false assumptions. Once those assumption change and technology will drive that change, the world will be different. Our assumption was long distance calls cost money. And then the internet came along, changed that assumption, and it didn't matter what AT&T wanted, they disappeared essentially, got sold for a song.


[Editor's Note: I thought maybe I could connect these two ideas, but I gave that idea up.]

I just got done reading Neal Stephenson's "In the Beginning ... Was the Command Line", which has a very nice little section on just that subject. In relation to operating systems and specifically BeOS, Stephenson explains:

All of the fixing and patching that engineers must do in order to give us the benefits of new technology without forcing us to think about it, or to change our ways, produces a lot of code that, over time, turns into a giant clot of bubble gum, spackle, bailing wire, and duct tape surrounding every operating system. In the jargon of hackers it is called "cruft." An operating system that has many, many layers of cruft is described as "crufty." Hackers hate to do things twice, but when they see something crufty, their first impulse is to rip it out, throw it away and start anew.

That description, to me, sounds a lot like our economic system. In fact, I made the connection when I got this email from my mom about an interview she had watched with John Mack, Chairman and CEO of Morgan Stanley. Based on what he said, she wrote:

The old rule of thumb was that a country that didn't have economic dominance couldn't dominate in power (military) and influence (philosophy). That's why the 20th century was "The American Century." Now we're seeing an insistence, at least on the part of the United States, that the world financial markets adhere to common rules and regulations. This is going to force an alliance between the European, Asian and American communities that has heretofore been voluntary. This is Thomas Paine's, "If we do not hang together we will surely hang separately" writ large -- Larger than it ever has been.

Which made me feel good and excited. Ultimately I think the opportunity with all this stuff is to get rid of the cruft and get closer to a system that makes sense for a new age. I'm not suggesting that we don't learn from the past, but rather acknowledge the present when we consider how to rebuild a clearly broken system.

I think where things get extra confusing with all this stuff (and maybe it'd actually be answered if I made it through Black Swan instead of just talking about my understanding of the first few chapters and things I've read about) is how do you make decisions without relying on the past to guide you? (If this is answered in Black Swan and someone who's finished it wants to fill me in, that'd be awesome.) I don't know the answer to this question. Clearly it's going to take some leaps (and likely a few mistakes) to get us there, but its exciting to think about. As Rick wrote in reply to my last post:

I think, ultimately, we're witnessing a re-thinking of what capitalism is. I still think the core philosophy of capitalism aligns with Bataille and is a useful guide for us still – that people operate out of self interest, and if you set the rules up right around that, everyone can benefit. But I think most of the windowdressing around that – how we regulate, why, and whether wealth can really be defined only in terms of currency – are going out the window. Our very notion of Currency is insufficient as the measure of wealth, as it fails to capture so much, most notably happiness. We’ll need a new benchmark that can handle it. That, I think, will take another 100 years but is ultimately where it’s all heading.


October 18, 2008
Noah Brier | Thanks for reading. | Don't fake the funk on a nasty dunk.