How to Think Critically VIII: Mill's Methods

Today’s post on critical thinking concerns the five principles collectively known as Mill’s methods, first presented together in 1843 by the Enlightenment philosopher John Stuart Mill in his book A System of Logic. Each of them is intended to illuminate the flow of causality in a different way, giving us mental tools to link causes and effects. In this post, I’ll highlight past entries in my “Popular Delusions” series, and show how failure to properly use Mill’s methods has duped the practitioners of pseudoscience.

1. The method of agreement. If two or more cases of the event you’re seeking to explain have only one causal factor in common, that factor is probably the cause of the event.

Example: In my post on hauntings, I noted a way in which ghost-hunters conspicuously fail to apply the method of agreement: the vast majority of ghost sightings have nothing in common except that the people involved are in the twilight state on the edge of sleep. These conditions are most suitable for the human neurological state of sleep paralysis, which typically entails hallucination, inability to move, and a strong sense of a looming presence. This is a causal factor in common among these cases which ghost believers generally overlook. Since ghost sightings have little else in common, the method of agreement would suggest that sleep paralysis is the most probable cause of this phenomenon.

2. The method of difference. If there is one case where an event occurs and one case where it doesn’t occur, and all the causal factors except one are the same in both cases, that one factor is probably the cause of why the event happened in one instance and did not happen in another.

Example: Astrology is an excellent example of misapplication of the method of difference. Astrologers claim that the celestial bodies and constellations that are ascendant at the time of a person’s birth determine their subsequent traits of personality and character. This is the sort of claim that could be proven by the method of difference, but most astrologers are steadfastly uninterested in even doing the tests. Those few who have tried, using the method of difference to predict the personality characteristics of strangers based on their birth times, have been dismal failures.

3. The joint method of agreement and difference. If there are some cases where an event occurs and some cases where it does not occur, and all the cases where it does occur have only one causal factor in common, while all the cases where it does not occur lack that causal factor, that factor is probably the cause of the event.

Example: Faith healers are notorious for their ignorance of this method. If sick people who were prayed over frequently experienced otherwise inexplicable remissions, while people lacking such intervention never did, this would be strong evidence in favor of intercessory prayer. But all these conditions, in reality, are absent: sick people who receive prayer experience spontaneous recovery at rates no greater than chance, while spontaneous remission also occurs among people receiving no special theological intervention.

4. The method of residues. When seeking to explain the causes of a set of events, deduct all the causal factors which are known to cause some subset of those events; the remaining causal factors are probably the cause of the remaining members of the set.

Example: The method of residues is routinely ignored by proponents of homeopathy, who believe that a purported remedy can be diluted out of existence and still retain its curative power. A proper application of the method of residues would begin with the application of a chosen homeopathic remedy at full strength, then successively subtract the components of that remedy to observe how the effect diminishes. Homeopaths, however, believe the opposite: that as causes are progressively subtracted, the effects will grow greater in magnitude. This violates not just the laws of physics but the very notion of causality itself.

5. The method of concomitant variation. When two events or circumstances are observed to vary in the same proportions, such that an increase in one always accompanies an increase in the other, one is probably the cause of the other.

Example: This method is abused and misused not once, but twice, by the fanatical anti-vaccine campaigners. When they claim that the preservative chemical thimerosal causes autism in some subset of vaccinated children, they fail to take note of the fact that autism rates have been rising in recent decades even after thimerosal was removed from most vaccines. And in their steadfast denial of vaccines’ efficacy in preventing deadly disease, they steadfastly ignore the declining rates of these infectious diseases, dwindling to near zero in many cases, which accompanied the widespread use of vaccination. And as vaccination declines among the superstitious, these diseases reappear and spread in turn – an unfortunately near-perfect example of the correlation that the method of concomitant variation reminds us not to overlook.

About Adam Lee

Adam Lee is an atheist writer and speaker living in New York City. His new novel, Broken Ring, is available in paperback and e-book. Read his full bio, or follow him on Twitter.

  • http://www.myspace.com/driftwoodduo Steve Bowen

    Astrologers claim that the celestial bodies and constellations that are ascendant at the time of a person’s birth determine their subsequent traits of personality and character.

    Which is demonstrably true since as a typical aquarian I don’t believe in any of that astrological B.S:)
    Interestingly though astrologers may unwittingly be applying the method of concomitant variation but to the wrong variable. I’m sure I recall reading (and I will research this when time permits) that certain measurable character traits vary with season of birth, the theory being that oxytocin and serotonin levels etc vary with infant exposure to sunlight. Since seasons and constellations also coincide there will be an accidental correlation between character and star sign. A similar situation arises with Autism and the MMR vaccine in that signs of autism tend to be seen in infants around two years old which is also the time when (in the UK at any rate) MMR was given. Sometimes just because B follows A doesn’t mean A causes B.
    I love homeopathy! Not because it works (which it doesn’t) but the hilarious woo-woo that surrounds it is priceless. My girlfriend came home from a friend’s house who’s mother is a homeopathist. She’d been given some “tablets” for a headache which had been carefully wrapped in Bacofoil to “insulate them from cosmic rays or they might not work”. Fallacy piled upon fallacy.

  • konrad_arflane

    Which is demonstrably true since as a typical aquarian I don’t believe in any of that astrological B.S:)

    Hey, I thought that was a Capricorn trait!

    As it happens, the most unlikely thing that ever happened to me was when two people at the same party indepently guessed my sign based on nothing but around an hour’s conversation. I still don’t know how they did that.

  • Brock

    I myself have discovered that over the last couple of thousand years, the star field has shifted, I think due to the precession of the equinoxes, so that instead of twelve signs, the Zodiac actually has thirteen. I was delighted to discover that i am not a Sagitarius at all, but an Ophiuchi! No astrologer in the world has any idea how to cast my horoscope! And you’re both wrong, it’s Ophiuchis who don’t believe in astrology :-0
    Incidentally, the state of being you describe as causing people to be susceptible to seeing ghosts, has also been linked to people who report alien abductions. It was in Skeptical Inquirer a couple of years ago.

  • Jim Baerg

    I’m sure I recall reading (and I will research this when time permits) that certain measurable character traits vary with season of birth, the theory being that oxytocin and serotonin levels etc vary with infant exposure to sunlight.

    Which should be testable by comparing people born at roughly opposite latitudes at different times of the year.

  • bestonnet

    Actually there were thirteen signs of the zodiac even back in the day of Ptolemy though precession has made all popular astrology even more nonsensical than it would be if the dates were somewhat accurate.

    As for the relationship between ghosts and aliens, I think that’s pretty much it for a lot of them (and the whole UFO nonsense does seem to have advanced our understanding of psychology somewhat (along with discovered some new atmospheric phenomena)).

  • bestonnet

    Jim Baerg:

    Steve Bowen:
    I’m sure I recall reading (and I will research this when time permits) that certain measurable character traits vary with season of birth, the theory being that oxytocin and serotonin levels etc vary with infant exposure to sunlight.

    Which should be testable by comparing people born at roughly opposite latitudes at different times of the year.

    There is also the possibility that the character traits difference has to do with their age when they started school with those born around late autumn tending to be the oldest in their class.

  • lpetrich

    JS Mill’s method #5 is inferring causation from correlation. But that can be rather risky — if A is correlated with B, it does not tell us whether A causes B, B causes A, or A and B have some shared cause.

    His first three methods, and possibly the fourth, are the motivation for controlled-experiment protocols. A classic experiment is Francesco Redi’s test of the hypothesis that flies are spontaneously generated from rotting meat. He kept flies from some meat with some gauze and allowed flies to reach some other meat, and he discovered that only the meat that flies could get to would produce more flies.

    I will concede that a common anti-astrology argument is rather bad — it’s the argument from weakness of gravitational forces and other such known effects of the planets. That is entirely correct about such forces, but astrologers can then claim that there is some completely unknown force that makes the planets and stars have the effects that they do.

    But one can test for astrological effects empirically, and for the most part, the result has been abysmal failure. The only reputable claim of evidence of astrological effects has been Michel Gauquelin’s claims — and (1) his claims are rather different from the more typical astrological claims and (2) his results are rather ambiguous.

  • bestonnet

    lpetrich:

    That is entirely correct about such forces, but astrologers can then claim that there is some completely unknown force that makes the planets and stars have the effects that they do.

    If they make an existence claim it’s up to them to provide proof of the unknown force.

    If they can’t do that then weakness of gravity, etc wins.

    Oh and the Mars effect looks like it’s just data dredging (which is what we see in a lot of altmed ‘studies’).

  • NoAstronomer

    I will concede that a common anti-astrology argument is rather bad — it’s the argument from weakness of gravitational forces and other such known effects of the planets. That is entirely correct about such forces, but astrologers can then claim that there is some completely unknown force that makes the planets and stars have the effects that they do.

    Phil Plait at http://www.badastronomy.com had a good refutation of astrology on purely physical grounds, but I’m too lazy to look up the direct link.

    The main thrust of his argument was that, given the relative sizes and distances of the planets from earth, there is no conceivable attribute that could cause the effects astrologers are claiming. No matter how you assign the ‘astrological’ effect either one body (eg the Sun) swamps any possible effect from the other bodies or everything is lost in the noise from all the smaller bodies.

  • lpetrich

    Phil Plait on astrology

    He does mention how astrology is a complete empirical failure — zero statistical significance of astrological predictions, and zero ability to predict when someone was born.

    I doubt that JS Mill wrote much on statistical methods, but he was living long before it became easy to compute a variety of statistics. That aside, I note that another common astrological apologetic is

    The stars incline; they do not compel. (Astra inclinant non necessitant.)

    However, statistical testing enables one to test hypotheses of influences that incline without compelling, and as I’d mentioned, astrology fails miserably there.

  • http://www.myspace.com/driftwoodduo Steve Bowen

    Hear is at least one example of a personality trait being affected by seasonality. Of course which seasons correspond to which astrological star signs will depend on the hemisphere you live in, but although astrological divination probably originated in the Middle East, I suspect that star sign psychobabble is a more recent product of the west. It could be this kind of accidental correlation that makes astrology appear to make sense to some people (or they could be gullible idiots, but far be it for me…)

  • http://www.myspace.com/driftwoodduo Steve Bowen

    Damn that link worked earlier.

  • http://www.myspace.com/driftwoodduo Steve Bowen

    If anyone’s still interested Joinson, C. and D. Nettle (2005). Season of birth variation in sensation seeking in an adult population is the .pdf I was trying to point at. Artical 24 on this page.

  • bestonnet

    Astrology can be adequately explained by the Forer effect without having to assume that it is somehow based on seasonal personality differences (which are probably real but rather subtle).

  • Rising

    Here to agree with lpetrich.

    If you want to be a critical thinker then you need to recognise that correlation does not imply causation.

    Just look at about the millions of idiotic claims out there to see the horrible consequences of this logical fallacy.

    “Sleeping with one’s shoes on is strongly correlated with waking up with a headache.
    Therefore, sleeping with one’s shoes on causes headache”

    Or

    “The more people are taxed the more money they earn.
    Therefore taxes make people money.”

    I’d give you some more serious examples if I could- but the fallacy is committed just about every day to support idiotic arguments.

  • Stuart

    I’m sure I recall reading (and I will research this when time permits) that certain measurable character traits vary with season of birth, the theory being that oxytocin and serotonin levels etc vary with infant exposure to sunlight. Since seasons and constellations also coincide there will be an accidental correlation between character and star sign.

    There is a stronger relationship that many astrologers rely on (knowingly or not) – most developed countries have fixed school intake starting/end points for each year. So in the UK you can be fairly confident that people born in July/August are more like to be insular/solitary, less likely to take part in sports or social activities, and the opposite is true for people born in October/November, who are more likely to be leaders, sporty, socially dominant, etc.


CLOSE | X

HIDE | X