Do Rorschach Tests Work?

There’s an old joke about the psychologist who shows a patient an ink blog and asks him what he sees. “That’s a man and a woman having sex,” the man replied. He shows him a second inkblot. “That’s two men and a woman having sex,” he said. And a third inkblot. “That’s a man and two women having sex.” The psychologist says, “I think I know what your problem is; you’re clearly obsessed with sex.” “Me?” the man says, “You’re the one showing all the dirty pictures.”

Dr. Mike Drayton has an article on the BBC website about the origins and validity of the Rorschach test and points out that how a psychologist reads the test may say as much about them as it does about their client:

Criticisms of the Rorschach have centred on three things:

First, some psychologists have argued that the testing psychologist also projects his or her unconscious world on to the inkblots when interpreting responses.

For example, if the person being tested sees a bra, a male psychologist might classify this as a sexual response, whereas a female psychologist may classify it as clothing.

This seems rather obvious now that it’s been pointed out, doesn’t it?

"This comment goes out to WatchAndLearn, who has shown a remarkable ability to care about ..."

Asylum Seekers Need Not Cross at ..."
"but not (in my own views) needlessly harsh, antagonistic, etc.I think this is important."

We All Lose When Trump Picks ..."
"I want to know when I am wrong. Facts matter.I also belong to a demographic ..."

We All Lose When Trump Picks ..."

Browse Our Archives

Follow Us!


What Are Your Thoughts?leave a comment
  • Remember, of course, that we have no way of knowing whether the patient is obsessed with sex or if the psychiatrist (who was, perhaps, extremely shortsighted) had in fact inadvertently shown the poor chap his extensive collection of racy ‘French’ postcards.

  • anteprepro

    So, interpreting a Rorschach Test is itself a kind of Rorschach Test, basically.

    Mind = Blown

  • sawells

    Typo watch: “ink blog”? I think those are called “books” 🙂

  • vmanis1

    This reminds me of Piet Hein’s pithy putdown of psychoanalysis, `Dream Interpretation (Simplified)’:

    `Everything’s either concave or -vex/So whatever you dream/Will be something with sex’.

    I read that in one of his collections of Grooks some 40 years ago; it has been a handy guide ever since.

  • Sastra

    The skeptic groups have been decrying the Rorschach tests for quite a while. As the article points out, there are a lot of subjective elements in the interpretation.

    My understanding is that there is an obvious, superficial level where the tests mostly work at telling you what you already know: violent people tell you violent stories, anxious people perceive harmful, dangerous images, etc. But there’s a strong element of confirmation bias on the part of the subject themselves — and the “test” is pretty much useless at revealing anything deeply hidden (whichis what it’s supposedto do, I think.)

  • jakc

    It was obvious before it was pointed out. I assumed everyone already knew.

  • Pierce R. Butler

    Rorshach blots do, however, make a great metaphor.

    (For what? for whatever you can imagine…)

  • eric

    I always figured they “worked” in that they get a patient to open up and be comfortable talking about what’s on their mind.

    To be honest, I thought psychology had gotten beyond believing an individual’s answers had some intrinsic meaning a long time ago. Its somewhat disappointing to find out I was wrong about that.

  • StevoR

    Apparently when the Mercury astronauts wre being put through pysch tests and shown a rorsach inkblot one ha dthe doctor checking by peering suspcisouly and saying “But you’ve got it upside down!”

    Source : The Right Stuff novel by Tom Wolfe – if memory serves.

  • TGAP Dad

    Is there anyone, anywhere actually using the Rorschach clinically? I’ll bet there are more people commuting to work with Stanley Steamers than use this as a diagnostic tool.

  • matty1

    What does it indicate if you say ‘that looks like an inkblot to me Doc’?

  • StevoR

    Do Rorschach Tests Work?

    Doesn’t that depend on how you look at them?

  • StevoR

    @11. matty1 : Lack of imagination I’d guess.

    (Oddly enough I heard that comment spoken in Bugs Bunnies voice!)

  • Cuttlefish

    The danger of projective tests like the Rorschach is not that they are complete BS, but rather that they can serve as an excuse and justification for clinicians to confirm whatever view they happen to have a motivation to confirm. I’ve seen child-custody files where Rorschach responses were used to argue that this or that parent was unfit (and not just Rorschach, but also draw-a-person and sentence-completion tests), or that they had strong maternal feelings, or tremendous anxiety about the other parent’s inappropriate behavior around the child. It’s not just the shrink’s opinion, it’s the opinion based on all these scientific tests which conveniently are exactly in line with the shrink’s opinion. It need not be intentional, but it happens, and it’s wrong.

  • This is persuasive for you because, like most people, including many psychologists, you know nothing about the Rorshach. It’s use requires extensive training, most aren’t trained to use it and most don’t use it. European psychologists haven’t a clue about the research and the modern system that’s used in the US.

    The core of modern scoring and interpretation focuses on statistical analyses of objective structural properties of responses (e.g. to name a few, use of color, shading, movement, seeing dimenionality, numbers of responses at various points in the test, numbers of responses to each blot, statistical commonality or relative rarity of specific responses, to name just a few. That article clearly reflects an author who cherry picked a few criticisms with a complete lack of awareness of how the modern test is administered and scored by a trained examiner.

    The Europeans never widely adopted the modern system. When it’s taught and administered in the U.S., the Exner system is used. Contrary to the author’s suggestion, inter-rater reliability and validity are good.

    http://en.wikipedia.org/wiki/Rorschach_test#Exner_scoring_system

    The other thing to understand when we’re talking about any psychological test is that they aren’t used as stand alones, at least they aren’t by anyone who is competent. Psychological tests are, essentially, samplings of behavior from a variety of domains of mental/cognitive functioning. A report wouldn’t be based on a Rorschach alone or an MMPI-2 or any other single test. We look for convergences of findings–cross validation–across multiple tests that sample mental functioning in different ways. If you’ve got enough tests with good standardization, reliability and validity, and you see the same finding expressed across several tests and you’ve done a competent, in-depth clinical interview, repeated patterns become evident.

    Good work as an examiner requires a great deal of training–10-15 in-depth graduate courses/seminars in testing. And good testing/assessment work is surprisingly demanding, difficult work, which is why even most of those who initially trained to do it, don’t do it.

  • d cwilson

    Hrm

  • Reminds me of the old Emo Phillips joke about a psychologist who showed him an ink blot.

    He said, “It’s kind of embarrassing.”

    The doctors said, “It’s okay, Emo. Everyone sees something different. There’s no wrong answer, just tell me what you see.”

    Emo says, “Okay…to me it looks like standard pattern #3 in the Rorschach series to test obsessive compulsiveness.”

    And the doctor got all depressed, so I said, “Okay, it’s a butterfly.”

  • Artor

    All this makes me wonder why the London Olympics decided to use an abstract cartoon of Lisa Simpson blowing Bart as their logo.

    http://www.celebitchy.com/82229/2012_olympic_logo_looks_like_lisa_simpson_doing_something_inappropriate/

    It’s as clear as day, isn’t it?

  • Dennis N

    Depends on what you mean by “works”.

    Rorshach blots do, however, make a great metaphor.

    They inspire great comic book characters, as well.

  • And, by the way, research is extensive and ongoing.

    http://scholar.google.com/scholar?q=rorschach&btnG=&hl=en&as_sdt=0%2C14&as_ylo=2008

    Cuttlefish: if you’ve seen a single response interpreted to mean something about parenting, then the psychologist testifying was a total hack or an attorney’s whore. That would represent jaw-dropping professional incompetence. The idea that you can use one response or one piece of data from any part of any test to make valid comprehensive statements about a person isn’t a sign of problems with the test–it’s a sign of incompetence of the examiner.

    This principle is basic to tests and measurements theory, which is what we study before we even begin training with specific tests. Tests deal with correlation-based probabilities. Repeated patterns and repeated cross validation are necessary to say anything with reasonable confidence. Think about flipping a coin. If you flip a coin once and it comes up heads, would you then say the coin has a head on both sides? You could flip it twice and get heads both times and you’d still be wrong if you said both sides have heads. Flip the coin twenty times and a more reasonably accurate picture will emerge. Tests used properly are repeated samplings, not only within the test, but across tests.

    That said, I’m sure what you’re describing is true. People who aren’t qualified examiners see an opportunity to make a great deal of money in court by portraying themselves as experts. And others see that they can make a good deal of money selling their opinion–whatever opinion is needed. The last place I’d look to judge test validity is a courtroom.

  • robnyny

    It’s pretty much the same with polygraph examinations. The operator poses the questions and reads the results, while knowing what results the police want. And then basically no one can ever read the results again. No double blinding, every kind of bias, and very little scientific vallidity.

  • The Lorax

    @3, I will never refer to them as “books” for as long as I live.

    Ink blogs, forever!

  • robnyny

    I think there are two things you would have to show about inkblot tests:

    First you would have to show that a group of psychologists examining a transcript of responses to a test would reach a general consensus, with each psychologist examining blindly and independently (with whatever other patient records they need).

    Second, demonstrate that the consensus is in fact clinically correct. This could be done by comparing the consensus from the first test to the patient’s actual records, or by having psychologissts match the correct test (from a group of 1 real and 9 spurious results) to a known patient.

    I always get the feeling when research on something has been going on for decades and there are still no conclusive results, it’s because that’s the right answer.

  • Reginald Selkirk

    Is there something abnormal about being obsesses with sex?

  • kermit.

    So what does it say about the patients when one reads the bible and sees directives to be kind to the poor, and another sees a God-given duty to enslave the Children of Ham? When one sees a command to be peaceful and another sees license to conquer non-believers and “convert” at the point of a sword?

    About 30 years back my wife and I were watching a doumentary about a Yanomamo tribe in the Amazon. At one point several little boys about 9 or so had taken one’s little sister, tied her to a pole set upright, and were shooting her with sharp-stick arrows (no rubber tips!) from kid-sized bows. She was of course crying and screaming, and no adults interfered. The narrator talked about the boys “practicing to become warriors and hunters”; my sweetie and I looked at each other and said that it looked more like they were training to be rapists and wifebeaters. I study martial arts and I’m normally all over that “training to be a warrior” stuff, but that’s not what I was seeing. Maybe my Rorschach skills need work.

  • Stacy

    Apparently when the Mercury astronauts wre being put through pysch tests and shown a rorsach inkblot one ha dthe doctor checking by peering suspcisouly and saying “But you’ve got it upside down!

    I’m sure the astronaut was joking, but the doctor’s response was understandable. There actually is a right way to hold/look at a Rorschach inkblot; there’s an up and down. I believe (Dr X could confirm) the same ten images are always used.

    In his book Big Secrets, William Poundstone talks about the test, how it’s administered, what clinicians look for, and how to “pass” a Rorschach (the book is old though, some info may be out of date.)

  • grumpyoldfart

    The first time I ever heard about those tests I thought to myself, “What a load of crap.” Fifty years later I feel the same way.

  • geraldmcgrew

    DrX,

    You’ve come into this thread and given a well-argued, substantive response to the knee-jerk pooh-poohing by the “skeptic” crowd, who were all anxious to stick their noses in the air and collectively express their disdain. IOW, you’ve rained on their parade.

    The more time I spend among self-described skeptics, the more I’m surprised at how often they display behaviors very, very similar to creationists and denialists in general. This thread is a good example. How different is…

    Creationist: “C-14 dating is unreliable and flawed!”

    [expert provides an explanation of the methodology and resources for further learning]

    Creationist: [ignores the above] “C-14 dating is unreliable and flawed!”

    …from…

    Skeptic: “Rorschach tests are unreliable and flawed!”

    [expert provides an explanation of the methodology and resources for further learning]

    Skeptic: [ignores the above] “Rorschach tests are unreliable and flawed!”

    …? Looks like two sides of the same coin to me.

  • Ichthyic

    My understanding is that there is an obvious, superficial level where the tests mostly work at telling you what you already know: violent people tell you violent stories, anxious people perceive harmful, dangerous images, etc. But there’s a strong element of confirmation bias on the part of the subject themselves — and the “test” is pretty much useless at revealing anything deeply hidden (whichis what it’s supposedto do, I think.)

    the problem really is with people deciding to think of these “tests” as being diagnostic to begin with.

    they were not originally designed to be.

    the purpose of them was to encourage directed dialogue, when patients were unable or unwilling to open up at all.

    they simply help give direction to pursue further dialogue.

    that’s it, really.

    whether it’s the general public, or any specific clinician that purports to suggest that Rorshac tests are somehow specifically diagnostic, these are post-hoc claims and not the original intent of the tests.

    so, in essence, all this debate is about post-hoc claims of the diagnostic value, or lack thereof.

    it’s pointless, really.

  • Gerald McGrew,

    I don’t have a problem with skeptics. It’s perfectly fine to disbelieve when one hasn’t seen evidence or heard a persuasive argument. Overall, I find the commenters here to be interested in evidence and open to persuasion.

    Robnyny:

    First you would have to show that a group of psychologists examining a transcript of responses to a test would reach a general consensus, with each psychologist examining blindly and independently (with whatever other patient records they need).

    This is called inter-scorer reliability and this research has been conducted. Inter-scorer reliability is high for trained Rorschach examiners. Scoring is a large part of learning to administer and interpret the Rorschach. While inter-scorer reliability isn’t 1.0 (i.e., perfect), it is more than sufficiently high that a properly trained group will independently come up with scores for which the small amount of variation is inconsequential to the interpretation. Interscorer reliability is not 1.0 for any psych test–not even IQ tests)

    @robinny:

    Second, demonstrate that the consensus is in fact clinically correct. This could be done by comparing the consensus from the first test to the patient’s actual records, or by having psychologissts match the correct test (from a group of 1 real and 9 spurious results) to a known patient.

    This is called validity and the validity research is vast, relying on many methods to determine what can and cannot be predicted based on scores. Once reliability of measurement is established, all subsequent research is validity research.

    The research has gone on for decades because new predictive hypotheses are generated based on what’s already been demonstrated and researchers test those new hypotheses. It isn’t because nothing is known. It’s a reflection of the fact that a great deal is known and this particular test provides very rich data samples along with other reliable, valid tests. The MMPI in its several forms is also subject of extensive, ongoing research for the same reason.

  • geraldmcgrew

    Dr X,

    I don’t have a problem with skeptics.

    I don’t either. It’s more about recognizing that skeptics, as a group, are no more immune to things like groupthink, knee-jerk reactions, and emotion-based responses than any other group.

    It’s perfectly fine to disbelieve when one hasn’t seen evidence or heard a persuasive argument.

    Sure, but it’s something else entirely to take the position “It’s all crap” without knowing anything about it, or even worse, based on “This one time…” stories, as seen in this thread. A “skeptic” should say “I don’t know” until they do some objective research, rather than immediately go to the “It’s crap” position.

    Further, if you look at the progress of this thread, the substantive points in your posts have been largely ignored while the “It’s all crap” sentiment has continued, just as I illustrated via the analogy with creationists. That’s the exact opposite of skepticism IMO.

    Overall, I find the commenters here to be interested in evidence and open to persuasion.

    In some cases yes. But as a whole, I don’t see much difference from most other blog comment communities.

  • @StevoR.

    Apparently when the Mercury astronauts wre being put through pysch tests and shown a rorsach inkblot one ha dthe doctor checking by peering suspcisouly and saying “But you’ve got it upside down!

    I’m sure the astronaut was joking, but the doctor’s response was understandable. There actually is a right way to hold/look at a Rorschach inkblot; there’s an up and down. I believe (Dr X could confirm) the same ten images are always used.

    The Exner system, the system in use did not exist at the time of the Mercury program, but my guess based on what I learned about the earlier systems is that this is a fabrication. If we were talking about anytime in the last 30 years I’d call it gross examiner incompetence or a fabrication.

    Turning the card is called ‘rotation.’ The card is passed to an examinee right side up. Some examinees will spontaneously rotate the card and the rotations are noted on the score sheet with curved arrows showing direction, multiple rotations and final position. The examiner makes no comment when someone rotates a card. If an examinee asks if it’s alright to turn the card, the examiner response is “if you like.”

  • lofgren

    It’s more about recognizing that skeptics, as a group, are no more immune to things like groupthink, knee-jerk reactions, and emotion-based responses than any other group.

    If you aren’t aware of this, you shouldn’t call yourself a skeptic. At its most basic, scientific skepticism is the study of the brain’s biases. At the forefront of every skeptic’s mind should be precisely this fact.

    If you think that skeptics are more immune to these biases, then you have already failed as a skeptic.

  • busterggi

    33 responses and not one Watchmen reference.

    Tres disappointing.

  • geraldmcgrew

    lofgren,

    If you aren’t aware of this, you shouldn’t call yourself a skeptic.

    I don’t call myself a skeptic. I don’t call myself anything. I’m just me, doing the best I can.

  • By the way, if anyone even cares, the correct content code for a bra is Cg (clothing) unless it is mentioned specifically with reference to sexual activity. The original content category definitions already pointed toward this, but sometimes there are hard calls, specific items where more scoring errors occur. In his 2002 edition, Exner discussed this study and made the correct scoring explicit.

    What this study means is that men were more prone to making a content error on this item than women. For most items you will see virtually no errors. You might get 2 or 3 tough calls per protocol out of maybe a couple of hundred different types of scores you’re calling. One point of ongoing research would be to identify items that might come up for inclusion in future texts. inconsequential. And, in any case, a couple of extra Sx content scores wouldn’t affect interpretation. Most of what belongs in the category is quite clear.

    Finding the scoring errors occur more frequently on certain items is like finding out that certain physicians more frequently make a particular kind of error interpreting X-rays or blood work. Finding that a particular kind of error occurs more frequently doesn’t show that X-rays are worthless. It does mean that the causes of the errors should be identified and methods should be developed to prevent them. The bra thing is the kind of item that would now be brought up in a Exner Rorschach class.

  • lofgren

    33 responses and not one Watchmen reference.

    I believe you mean 18 responses.

    I don’t call myself a skeptic.

    That was more of a universal you. If it’s confusing, substitute “one” and make the whole thing third person singular.

  • itinerant

    Dr. X., you’re obviously a psychologist or a psychiatrist. Yes, most of the lay criticisms of the Rorschach (basing conclusions on single responses, very idiosyncratic interpretations) are like lay criticisms of science; considered and addressed many years ago. So Ed’s point has been put to rest many years ago, and isn’t a real issue.

    However, it’s only fair to note the ongoing substantial controversy in the current psychological community about the validity of the Rorschach, for example, overpathologizing, productivity bias (people who give many responses versus those who give just a few) the bias against minority groups(common responses that were not part of Exner’s normative base), and the fact that the incremental validity of the Rorschach (what it adds to diagnosis) isn’t very great after more defensible personality and psychopathology measures are used.

    For those interested in learning more about the actual psychological controversies, rather than the lay, ‘fake controversy’, here’s a decent place to start, taking a skeptical slant:

    http://works.bepress.com/james_wood/

  • satanaugustine

    Dr. X –

    You know a great deal about Rorschach testing. What, exactly, is it supposed to test? Based on the way that you describe it and the Wiki link you provided, it seems like it started out as a completely BS diagnostic method which has been retrofitted so as to make it more scientific. I don’t understand what the point of such an endeavor would be. It seems like a waste of time. What is the theoretical framework behind Rorschach testing? Why should inkblots offer any insight whatsoever into the functioning of another individuals mind?

  • Azkyroth, Former Growing Toaster Oven

    This is persuasive for you because, like most people, including many psychologists, you know nothing about the Rorshach. It’s use requires extensive training, most aren’t trained to use it and most don’t use it

    Not this shit again.

    When I see people talking nonsense on physics, I correct them. I don’t haughtily explain that they CANNOT POSSIBLY UNDERSTAND.