LATEST ENTRY

SCIENCE | Noah Brier

What Makes a Science?

If you can't prove it wrong, it's probably not a science.

January 3, 2008 | RSS | EMAIL | PRINT | 5 COMMENTS

When I read Stephen Jay Gould's "Evolution as Fact and Theory" it wasn't just his explanation that impressed me. I was also quite taken by Karl Popper's explanation of the primary criterion of a science. As Gould explains:

Philosopher Karl Popper has argued for decades that the primary criterion of science is the falsifiability of its theories. We can never prove absolutely, but we can falsify. A set of ideas that cannot, in principle, be falsified is not science.

The entire creationist program includes little more than a rhetorical attempt to falsify evolution by presenting supposed contradictions among its supporters. Their brand of creationism, they claim, is "scientific" because it follows the Popperian model in trying to demolish evolution. Yet Popper's argument must apply in both directions. One does not become a scientist by the simple act of trying to falsify a rival and truly scientific system; one has to present an alternative system that also meets Popper's criterion — it too must be falsifiable in principle.

After reading that over a few times I decided to read Popper's "Science as Falsification" from 1963 in which he explains his theory in more detail. The story behind it is that he was trying to figure out why Marx's theory of history and Freud's psycho-analysis didn't sit right with him when Einstein's theory of relativity did. In digging into his own mind, he finally came to the conclusion that he believed in Einstein's theory because it was the only one that was proven by an experiment that could have easily shown it to be wrong. In other words, when Einstein's theory was proven in 1919 (by an eclipse photographed by someone named Eddington apparently) it could have just as easily been proven wrong. This stands in opposition to pseudo-science which can bend results to prove their point no matter how they turn out.

In the end this led popper to these seven conculsions for differentiating science and pseudo-science (or whatever else you want to call it):

1. It is easy to obtain confirmations, or verifications, for nearly every theory — if we look for confirmations.

2. Confirmations should count only if they are the result of risky predictions; that is to say, if, unenlightened by the theory in question, we should have expected an event which was incompatible with the theory — an event which would have refuted the theory.

3. Every "good" scientific theory is a prohibition: it forbids certain things to happen. The more a theory forbids, the better it is.

4. A theory which is not refutable by any conceivable event is non-scientific. Irrefutability is not a virtue of a theory (as people often think) but a vice.

5. Every genuine test of a theory is an attempt to falsify it, or to refute it. Testability is falsifiability; but there are degrees of testability: some theories are more testable, more exposed to refutation, than others; they take, as it were, greater risks.

6. Confirming evidence should not count except when it is the result of a genuine test of the theory; and this means that it can be presented as a serious but unsuccessful attempt to falsify the theory. (I now speak in such cases of "corroborating evidence.")

7. Some genuinely testable theories, when found to be false, are still upheld by their admirers — for example by introducing ad hoc some auxiliary assumption, or by reinterpreting the theory ad hoc in such a way that it escapes refutation. Such a procedure is always possible, but it rescues the theory from refutation only at the price of destroying, or at least lowering, its scientific status. (I later described such a rescuing operation as a "conventionalist twist" or a "conventionalist stratagem.")

Popper sums things up like this: "the scientific status of a theory is its falsifiability, or refutability, or testability." All of this got me thinking what a sham so much of marketing (and almost any business research) can be. Unfortunately I've found that very little of the research marketing (and business generally) does is for anything other than supporting their current stance. There's no science behind it and that which disproves hypotheses is simply thrown away for "better" results. Don't want to go off on a whole rant here, but just think that it's important to draw the distinction between market research and science.

As a side note, this makes me think some more about the need to release science's dark data.

PREVIOUS ENTRY | NEXT ENTRY

LEAVE A COMMENT

First name, first and last, whatever you feel like.

Required, but not displayed (so don't worry about spam).

If you've got one, flaunt it.

You can use some HTML (a's, br's, p's, oh my!) if you'd like, if you don't know what that means, don't worry about it.

REMEMBER ME?

COMMENTS

1Lee

Agreed. By Popper's standards almost all marketing research is bad science.

And it gets worse when you remember what we are trying to research.

Whilst the natural sciences of physics, biology and chemistry (mostly) study observable matter, social scientists study humans whose observable behaviour (what they say and do) can rarely be relied on to tell the full story.

As most social scientists will readily acknowledge, humans are difficult to understand because they have free will, are often irrational and have incredible imaginations which are capable of creating multiple alternative versions of reality.

January 3, 2008

2wes

Re: Jobs >> functionality is part of design, indeed. It's not just about making something look pretty. The fact that this is often forgotten leads to so many crappy websites for example ha

January 3, 2008

3Charles Frith

I'm sure that Taleb's Black Swan should feed nicely into this increasingly astute assessment of the sham largely known as marketing and business research. Its tempting to also use Poppers definitions to undermine his subsequent definitions but hey its only fun right? Happy New Year.

January 4, 2008

4jen_chan, writer SureFireWealth.com

It can get a little confusing. Ever since elementary school, we have been taught by our Science teachers that something irrefutable is the truth. But upon reading your summary, perhaps what we have been taught is slightly off the mark.

January 4, 2008

5Tangerine Toad

Marketing research is fraud? You don't say ;)
Seriously Noah, when I tell people who don't work in the business how we test TV commercials, they assume I am joking, because no one could actually place faith in anything that ridiculous.

For those of you who haven't had the pleasure: one of the more common methodologies involves inviting a people, usually in groups, to what's billed as a screening of a new sitcom. They are placed in an auditorium, where the "show" (a failed pilot) is shown on a big screen, interspersed with a variety of blatantly unfinished commercials that are being tested.

After the show is finished, the audience is asked a series of questions about the show - and about the commercials they've just seen. Now given that most of them showed up with a group of friends, they haven't actually been paying attention during commercials, so the ones that test well generally involve frequent repetition of the product's name. And still, many will spit back commercials they haven't seen, or famous commercials they remember from their youth.

Multi-million dollar decisions are based on this research.

January 4, 2008