Welcome to the bloggy home of Noah Brier. I'm the co-founder of Percolate and general internet tinkerer. This site is about media, culture, technology, and randomness. It's been around since 2004 (I'm pretty sure). Feel free to get in touch. Get in touch.

You can subscribe to this site via RSS (the humanity!) or .

On Predictions and Cruft

I recently watched Kevin Kelly’s great TED video on the next 5,000 days of the web and two things in it really struck me. First was Kelly’s assertion that man is an extension of technology, not vice versa. This was especially interesting having just read a paper that theorized man’s evolution to walk upright was a direct result of tool (aka technology) use (essentially it suggests that once tools were in play we could no longer afford to walk around on four limbs and required that we develop a larger brain to deal with the complexity of wielding these things).

That, however, wasn’t what struck me most. Rather it was Kelly talking about what everyone thought the web was going to become just five thousand days ago (that’s roughly when the web came to be). People generally thought (and many still generally think) that the web is, to quote Kelly, “TV only better.” In the rest of the video he explains his vision for the next 5,000 days of the web, specifically around what he calls “one machine” which more and more of our world will interface with. That topic, however, is for another entry, rather I want to focus on predictions.

Predictions

People frequently tout a knowledge of history as the best way to avoid its repetition, but I can’t help but think about how often we’ve been wrong. Mostly we are pretty terrible at predicting the future. That’s because we are predicting it based on what exists today and as soon as something new rolls around tomorrow, everything we considered “normal” is thrown all out of whack and our predictions no longer apply.

One way to think about this is in terms of punctuated equilibrium, which is essentially the idea that we evolve in spurts. Of course technological change is not evolutionary in the purest sense of the word, but much change happens in this kind of pattern. Think about sports, for instance, you hear stories all the time of no one believing something could be done (four-minute miles or lifting some ungodly amount of weight) and all of a sudden when one person breaks the barrier everyone else follows along. Basically we can’t do what we can’t imagine until we can do it or imagine it. (How’s that for a bit of circular logic?)

Put another way, as Kevin Kelly explains in his entry “The Maes-Garreau Point”, “it has become very hard to imagine what life will be like after we are dead. The rate of change appears to accelerate, and so the next lifetime promises to be unlike our time, maybe even unimaginable. Naturally, then, when we forecast the future, we will picture something we can personally imagine, and that will thus tend to cast it within range of our own lives.” He is referring specifically to predictions far in the future, but I think this is the fallacy of predictions generally: We can’t predict that which we can’t predict. (Got you again.) Or, as Kelly opens the entry with, “Forecasts of future events are heavily influenced by present circumstances. That’s why predictions are usually wrong. It’s hard to transcend current assumptions. Over time, these assumptions erode, which leads to surprise.”

So what’s the point of all this? Well, to be quite honest I’m not sure. I’m sitting on a plane feeling a bit restless and I figured writing might do the trick. But there’s more, I promise, and I will try to connect these ideas to a few other themes that have been floating around lately.

First off, there were a ton of amazing responses to my last post on the economy and postmodernism, but I want to point specifically to something that Charles wrote which struck me (partly because I read it while thinking about this topic). In his response to my entry he wrote, “I don’t concur that events are as random as you assert. I’ve pointed out that attendants of the World Economic Forum at Davos were raising the flag of an impending crisis quite some time back, and that on page 225 in Nassim Nicholas Taleb’s Black Swan the footnote could not be clearer of the statistical backflips performed by J.P. Morgan’s ‘Riskmetrics’ and the dangerous situation Fanny Mae was in.” Which got me thinking, how many people predicted this financial meltdown? The reality of the situation is that there were probably tons of people who did, but we had no reason to listen. People predict demise all the time (how many Nostradamus end of the world predictions can we listen to before the guy ceases to get front-page billing in the supermarket tabloids?). The problem is, until we are in the situation it’s hard to see the clues. Or, as I said before, we can’t predict that which we can’t predict.

Sure, in retrospect there were any number of clues pointing to the situation at hand, but if they had been strong enough clues wouldn’t we have acted upon them? A clue is only a clue if it helps solve a mystery, afterwards it becomes explanation, equally important (for our psyche) but a very different beast. This isn’t to say we shouldn’t look for clues or explanations, just to say that they are different things and that they’re increasingly obscured when we’re dealing with something that we never imagined. Or, as Vinod Khosla explained in an interview I just happen to have run across (which is specifically about energy consumption — and written prior to the current crisis — but is nonetheless valid):

We have to shift from extrapolating past assumptions, ignore what economists and econometrics tells us about how much oil we need and when we’ll run out and what consumptions will be. Because they are based on the false assumptions. Once those assumption change and technology will drive that change, the world will be different. Our assumption was long distance calls cost money. And then the internet came along, changed that assumption, and it didn’t matter what AT&T wanted, they disappeared essentially, got sold for a song.

Cruft

[Editor’s Note: I thought maybe I could connect these two ideas, but I gave that idea up.]

I just got done reading Neal Stephenson’s “In the Beginning … Was the Command Line”, which has a very nice little section on just that subject. In relation to operating systems and specifically BeOS, Stephenson explains:

All of the fixing and patching that engineers must do in order to give us the benefits of new technology without forcing us to think about it, or to change our ways, produces a lot of code that, over time, turns into a giant clot of bubble gum, spackle, bailing wire, and duct tape surrounding every operating system. In the jargon of hackers it is called “cruft.” An operating system that has many, many layers of cruft is described as “crufty.” Hackers hate to do things twice, but when they see something crufty, their first impulse is to rip it out, throw it away and start anew.

That description, to me, sounds a lot like our economic system. In fact, I made the connection when I got this email from my mom about an interview she had watched with John Mack, Chairman and CEO of Morgan Stanley. Based on what he said, she wrote:

The old rule of thumb was that a country that didn’t have economic dominance couldn’t dominate in power (military) and influence (philosophy). That’s why the 20th century was “The American Century.” Now we’re seeing an insistence, at least on the part of the United States, that the world financial markets adhere to common rules and regulations. This is going to force an alliance between the European, Asian and American communities that has heretofore been voluntary. This is Thomas Paine’s, “If we do not hang together we will surely hang separately” writ large — Larger than it ever has been.

Which made me feel good and excited. Ultimately I think the opportunity with all this stuff is to get rid of the cruft and get closer to a system that makes sense for a new age. I’m not suggesting that we don’t learn from the past, but rather acknowledge the present when we consider how to rebuild a clearly broken system.

I think where things get extra confusing with all this stuff (and maybe it’d actually be answered if I made it through Black Swan instead of just talking about my understanding of the first few chapters and things I’ve read about) is how do you make decisions without relying on the past to guide you? (If this is answered in Black Swan and someone who’s finished it wants to fill me in, that’d be awesome.) I don’t know the answer to this question. Clearly it’s going to take some leaps (and likely a few mistakes) to get us there, but its exciting to think about. As Rick wrote in reply to my last post:

I think, ultimately, we’re witnessing a re-thinking of what capitalism is. I still think the core philosophy of capitalism aligns with Bataille and is a useful guide for us still – that people operate out of self interest, and if you set the rules up right around that, everyone can benefit. But I think most of the windowdressing around that – how we regulate, why, and whether wealth can really be defined only in terms of currency – are going out the window. Our very notion of Currency is insufficient as the measure of wealth, as it fails to capture so much, most notably happiness. We’ll need a new benchmark that can handle it. That, I think, will take another 100 years but is ultimately where it’s all heading.

Sweet.

October 18, 2008

Comments

  • Josh says:

    Hey, Noah –

    I posted some thoughts inspired by yours here, for what that’s worth, and all I really have to add is that cruft sounds not only like our economic system, but like all of our systems! ;-)

  • O.S says:

    As “no plan survives contact with the enemy” – No prediction survives contact with change. Which is, of course, inevitable.

  • chartreuse says:

    A few things.

    What Kaleb was trying to say (I think cuz the dude’s writing isn’t exactly Malcolm) is the same thing the Zen Buddists say.(Which is interesting since modernism is killing religion.)

    The only thing that really matters is right now.

    Past assumptions are just that, assumptions, and have no basis in reality. Just ask a Patriots fan how they felt about the upcoming season before and after game one.

    This game we play, in politics, is the same way. Polls are just noise. Snapshots. With no grounding in what’s really going on. It’s mental foreplay before the game really starts on Nov. 4th.

    SO how do you think without relying too much on past assumptions? My suggestion. Study successful baseball sluggers.

    Really.

    Capitalism, on the other hand, has been dead a long time. It changed when the way we made money changed. Modern capitalism was built for an industrial age and that is so not what’s going on now. Banks now literally create value out of thin air. Capitalism as we know it is built for creating value out of airplane parts and other physical shit.

    That’s why knowledge-based value, such as copyrights and derivatives are in such as quandary. The system was built for it.

    One of the best books on the future of capitalism was written last year by Alvin Toffler. (I don’t remember the name but all his books are great so you won’t be disappointed no matter what you pick up!) In it he spoke extensively about creating a value system that would really work right now.

    And it won’t take 100 years. Because solutions to problems are inevitable.

    Jeez, think that’s long enough? You obviously wrote a thought provoking post. Kudos.

    I think I’ll go back to downloading the unreleased Eminem album. It seems the only way to avoid leaks is to release it right now. :)

  • Ryan says:

    Sick post. I especially like the metaphor of our Economic system as “crufty”. I experienced a similar parallel this weekend when speaking with my girlfriend, an 8th grade science teacher in one of NYC’s worst districts. We were talking about Pay for Performance, a concept in which teachers get paid for their student’s cognition which is supported by Obama but wildly unpopular with the unions.

    Her point was that the problem with Pay for Performance, despite her support of the idea, is that you inherit the bad performances of all the teachers before you in the form of the student. Ergo, the cruftiness is inherent in every aspect of the system. I assailed that once Pay for Performance worked its way through the system, this would cease to be an issue, but what to do in the interim?

    It’s frustrating as a designer to see these kind of bloated, inefficient juggernauts of systems like the Education system and the Economic system and not be able to find a simple solution. My impulse is that of a hacker: shut it down for 3 years, clean out the shit, and start anew (a shit cleaning to rival the Augean Stables, to be sure). But when you’re dealing with people, and a system that needs to continue to run at full strength even through reform…

    Well, it’s just dead depressing.

  • Noah Brier says:

    Thanks Ryan.

    Re: Pay for performance – It’s a really interesting question/example. I think basically the answer to finding newer/better solutions is all about focusing on behavior/incentives (which Pay for Performance starts to address by aligning pay with goals). Again, though, I think in this case you’d have to figure out how to deal with the inheritance question as well as how to deal with all the ways teachers would attempt to mess with the system (I can imagine a teacher purposely making things easier for students/cheating for their students). With all that said, I love solutions like this that start to let the free market do its job.

    As far as turning things around immediately, I don’t think it’s possible. Whatever happens is going to have to be something that trickles through the system. With that said, though, I think reforms in the past (especially school reforms) have focused in the wrong places (mostly with upper management rather than dealing with the day-to-day realities of teachers/unions). To me the most important thing about reforming things is honestly assessing the situation: How many schools have “fixed their problems” only to realize that because the union is so strong and teachers can’t be fired there’s no chance they’ll actually go along?

  • faris says:

    is that the one with the car analogy? that was awesome.

  • barbara says:

    I’m so glad to see someone make the cruft connection to education. Talk about a system that’s been patched beyond recognition! Having devoted my professional life to school reform for about 15 years now, I’ve finally come to accept the typical teacher’s reform credo, ‘If I just hold my breath, it will go away.’ I don’t even blame them anymore. Reform initiatives come and go. Some work quickly and get added to the ‘strategy tool-belt’, some are too complicated, take too long, don’t get enough support, etc. Most of these are designed to help kids perform better on standardized tests; very few are designed to actually help kids learn to think for themselves. And in the end, the grant money goes, and the teachers and educational administrators go. Some of the patches stick, some fall off altogether, and some wear just thin enough to keep new ideas from making it through to the heart of the problem. I’ve come to believe that the system is unfixable. I guess that makes me a hacker now. As Ryan says, the only way to make a school work is to start from scratch. That’s probably why charters have proven to be so successful. Home-schooling can work too. But both are a threat to the status quo (read teachers unions) and therefore their growth has been fairly slow. I see hope in the increasing availability of quality online courseware for all grade levels and say it’s time to blow the whole thing up. Parents still need a place for kids to go during the day while they go to work. But they don’t necessarily need teachers in the traditional sense – they need people with social work skills who are also capable of facilitating quality curriculum (online or otherwise), collabration and discussion. Isn’t that what all those ‘Teach for America’ kids are doing? The really good teachers can get into the online business and give lectures in their specialties, provide curriculum guides, focus assessments on real learning … just thinking out loud here. Cruft is the best new word I’ve learned in ages!

  • Alan Wolk says:

    Funny how a post on the economy and learning from history wound up generating conversation about education…

    I’m with most of what you’re putting out here, but Khosla give me pause. I can’t help but think he’s putting this information out there for his own purposes (the advantages of fuels made from weeds (okay, he’s got a fancier name for the plants, but if you read further, they’re just weeds.)

    That’s one of the biggest challenges we face going forward:
    Power lies in information.
    Information is everywhere, without filter, more randomly and in greater quantity than ever before,
    As a result, it’s very easy to manipulate information to your own purposes, and to mask the fact that you’re manipulating it.
    Simultaneously, it’s harder than ever to manipulate information to your own purposes, because it’s easy to fact check things online, rather than listen to rumor.
    So it’s finding that line between easily checked rumor and easily spread “truths” that is where the real power of evil lies.
    Whereas the power of good is in finding the right voices to listen to in the echo chamber of information that’s out there, focusing on them, and not becoming distracted.

    We can see this at a very basic level in the way the economic crisis is playing out, with the daily shifting from “It’s the end of the world!” to “We’ve got it under control” fueled by people who primarily get their information from other people who got their information from someone else.

    And that’s enough rambling.

  • Allan says:

    Black Swan could be chalked up in one sentence. Cross your fingers and hope you get lucky.

  • panasonic es8109s vortex says:

    Only wanna comment that you have a very decent site, I like the design and style it really stands out.

  • Leave a Comment

    Your email address will not be published. Don't sweat it.