Is Fact Checking Biased?

The advent of instant media has made it much harder for politicians to say one thing to one group and something different to another group, or to twist the facts or tell baldfaced lies – or at least to do those things without getting caught. It seems like with each election, fact checking sites become more prominent and play a greater role.

But the increasing prominence of fact checking has also led to the politicization of fact checking – i.e., to groups like the new Conservative Fact Check website accusing mainstream fact checking sites of liberal bias. Conservative Fact Check’s “about” section states that:

CFC is dedicated to providing a conservative alternative to enormously liberal-biased fact checking sites like, and

How does CFC back up its claim that the regular fact checking sites are biased?

We assembled this chart by sorting PolitiFact’s complete list of “Pants On Fire” rulings. We filtered out any rulings of chain email or anonymous Facebook posts (although these were virtually all conservative viewpoints), as well as politicians or organizations who had only one “Pants On Fire” ruling (but again, even these were mostly conservatives).

The results are, sadly, not surprising. … To have any semblance of fairness, PolitiFact should play it 50/50 and present an equal number of lies from both sides. They clearly are not concerned with any pretense.

Actually, that last bit is only true if both sides lie with the same frequency.

I mean, this would be like saying that if a teacher is fair, she has to give every student the same number of demerits without considering that some students might be tardy more often than other students. Isn’t that sort of thing generally something conservatives oppose?

And I should point out that CFC doesn’t actually take the time to dispute any of PolitiFact’s “Pants On Fire” rulings. Or even to discuss them. It simply counts them. See, you find evidence of liberal bias by showing that PolitiFact’s analysis of various political claims is incorrect, not by pointing out that PolitiFact has called conservatives “Pants On Fire” liars more often than it has liberals. CFC doesn’t even attempt this.

This is the bit where CFC really lost me, though:

PolitiFact (and other supposedly unbiased fact-checking sites) paint Mitt Romney as a serial liar. They also unfairly tarnish Michele Bachmann as a liar, when anybody who follows her already understands that many of her statements aren’t meant to be truthful in the first place — she simply says what she feels.

And this would be why I can’t trust sites like CFC. Bachmann’s not a liar because we shouldn’t expect her to tell the truth in the first place? What?

So let’s take a look at the assembled graphic:

Actually, that image is a pretty good reason why I trust sites like PolitiFact and they call out lies on both sides. I mean my goodness! According to this image, PolitiFact has called Barack Obama a “Pants On Fire” liar more often than it has Rush Limbaugh! Bill Clinton, Nancy Pelosi, and other liberals are on the list too. Sure, the list has more conservatives than liberals, but that by itself is not evidence of bias unless we have some guarantee that conservatives and liberals lie at the same rate, and we don’t.

You know what would be evidence of bias? If PolitiFact only called conservatives “Pants On Fire” liars and never even investigated the whether things liberals said were true or false. I’m not naive enough to think that everyone on “my side” is always truthful or that everyone on “the other side” is always lying. I want a fact checker that checks both sides, because I don’t want to be taken in by lies told by my own side any more than I want to be taken in by lies told by the other side.

After the second presidential debate last month, a friend who knew I was progressive and an Obama supporter sent me a link to a fact check from In the spirit of fairness, I took a look at it. It claimed to debunk “the top ten lies” told during the debate, and you know what? Every single one was something Obama (0r in once case, the moderator) said. There was no attempt to fact check Romney. None. I responded to my friend by telling her that I don’t trust fact check articles or sites that only examine one side.

The reality is that I tend to trust sites like and PolitiFact because they check both sides. And somehow I don’t think this new Conservative Fact Check site will do that. Because, you know, we’re not actually supposed to assume that what Bachmann says ought to be true in the first place, am I right?

Note: After reading this page on the Conservative Fact Check website, I’m wondering if it’s a parody. However, this page on Free Republic indicates that it’s probably legit. Further, whether or not this particular site is legit, I have over the past election cycle seen many conservatives accuse and PolitiFact of liberal bias. 

My Kindergartener Knows What It Means to Be Transgender (and the Sky Hasn't Fallen)
Did Ted Cruz Actually Ejaculate into a Cup? Some Thoughts on How We Cover Politics
When Men Wax Poetic about My Womb
The Real Travesty of the "Hero Mom" Story
About Libby Anne

Libby Anne grew up in a large evangelical homeschool family highly involved in the Christian Right. College turned her world upside down, and she is today an atheist, a feminist, and a progressive. She blogs about leaving religion, her experience with the Christian Patriarchy and Quiverfull movements, the detrimental effects of the "purity culture," the contradictions of conservative politics, and the importance of feminism.

  • Sam

    Attacking fact-checkers has been popular lately, especially during the past election. I received an e-mail from a relative saying “Snopes lies” with no actual evidence, and had another acquaintance claim online that the Heritage Foundation is “at least as neutral as Fact Check”.

  • from two to one

    This is the same sentiment of when my father-in-law told me that CNN was as far left as FOX News was far right. Umm…no.

  • Jayn

    What’s that line, reality has a liberal bias? Or maybe liberals have a reality bias…

    • Liberated Liberal

      Bingo. Liberals are more open to facts, and are also open to changing their minds if they find themselves to be wrong. Fact-checking, when done right, should have no bias. The only reason the right is screaming about the evils of fact-checking is because the Right Wing is made up of a pack of rabid liars.

  • Eamon Knight

    “Reality has a well-known liberal bias” — Stephen Colbert

    Also can’t decide if these guys are legit or a parody — not quite over the top enough to call Poe. I mean, WingNutDaily regularly runs material that’s nuttier than that.

    • MichaelD

      “not quite over the top enough to call Poe”

      Uh, then it wouldn’t be a Poe. By definition a Poe is satire so close to real thing that they are indistinguishable. So if you can tell them apart without being told or winked at it aint a poe.

  • smrnda

    There’s also a problem that people with a higher profile, who get more media attention are going to be called out on lies more often. Mitt Romney hit the top of the list, but I don’t think he’s really any less honest or reality based than many other people on the list; it’s just that he got the nomination and ended up having to open his mouth and when he had nothing to say, just made it up, or switched position when necessary. If Gingrich had gotten the nomination instead of Romney, maybe their rankings would have been different just since Romney would have had less air time.

    To defend Michelle Bachman as someone who just ‘says what she feels’ and it ‘isn’t meant to be truthful’ is no compliment to her, and no compliment to anyone who would say such a thing. What you feel ought to be connected to what actually happens to be real. If you aren’t going to make statements based in reality, I am just going to ignore you.

    Perhaps a bigger problem is that fact-checking should be done by candidates before they speak, though it doesn’t fit the format of rapid sound-bytes that news programs want. If I was running for office and someone asked me a question, you’d bet that I’d look up information right there, in front of everyone, before I provided an answer. Would this be tolerated? Given that I think it’s logical, I hope so.

    • Steve

      Exactly. Not that most of the top people on that list are Republican Presidential candidates. If you put ten of them together with all the media exposure you get a lot of stupid things said. In comparison, there was only one Democratic candidate.

  • Bryan White

    Your criticisms of the CFC site’s chart and argument are spot on, unfortunately.

    But just beneath the numbers on the chart we do have evidence of a liberal bias by PolitiFact. That evidence comes from a combination of two things: The higher rate at which Republicans receive a “Pants on Fire” rather than a “False” rating and the way PolitiFact fails to draw any objective distinction between the “Pants on Fire” rating and the “False” rating. Unless somebody has some way of objectively distinguishing between the two based on objective criteria, the “Pants on Fire” rating qualifies as a largely or entirely subjective judgment where Republicans fare considerably worse than Democrats.

    That result should surprise nobody. Journalists as a profession align center-left. They’re about in the middle on economic issues and further left on social issues. On what basis do we assume that journalists do not allow their ideological biases to affect their reporting? We can’t reasonably make that assumption without evidence. The default position is skepticism of journalistic objectivity. And while applying that skepticism look for the organizations and individuals that do the best job of reporting an analyzing the news.

    • Libby Anne

      Your paper explains that PolitiFact makes the following distinction:

      FALSE – The statement is not accurate.
      PANTS ON FIRE – The statement is not accurate and makes a ridiculous claim.

      And says this about it:

      Do Republicans lie more? Perhaps, but the proportion of “Pants on Fire” ratings should prove about the same for each party regardless of which party lies more if subjective impressions determine the rating.

      You are arguing that the ratio of ridiculous claims to statements that are simply false should be the same for each party. I don’t think that this argument actually makes sense. Why should the ratio of ridiculous claims to false statements be the same? Why is it implausible that Republicans are more likely to add hysterical claims about the end of the world as we know it than are Democrats?

      If you want to show bias here, you really need to show where PolitiFact doesn’t give a statement made by a Democrat a “Pants on Fire” rating when it should because it involves a “ridiculous claim,” or that some of the “Pants on Fire” ratings given to Republicans aren’t warranted because they statements involved are not actually “ridiculous claims.” You don’t do that.

  • Christine

    What I find most disturbing about the right-wing fact checking is the way they split hairs over what was and wasn’t said. I remember reading one getting upset because “no, he never said that Obama caused the recession” (or something similar). It was apparently inappropriate to call someone for heavily suggesting something, because he didn’t actually say it, so you can’t say he was wrong.

  • Jason Dick

    If anything, the fact check sites often lean to the right, not because they’re conservative-leaning in any sense, but because they do feel a need to paint both sides as somehow equal in an attempt to appear unbiased. For one really stark example of this, the PolitiFact “Lie of the Year” for 2011 was, “Republicans voted to end Medicare,” even though this is actually true, albeit the vote was to end Medicare slowly.

  • Bix

    I’m just floored by the Bachmann comment. It’s unfair to hold her to objective standards because it’s not really about truth anyway? Well, that’s a great example of goalpost moving.

    Like Eamon, I’m tempted to call a Poe. I kind of hope it is just bad parody, and if that’s the case, I hope they start doing it better.

    • Leigha

      I can’t help but wonder if the Bachmann thing is related to her gender. Most conservatives, after all, constantly say that men are logical and women are emotional, so of COURSE Bachmann would be expected to say things based on emotion, not logic. She’s not even capable of logic, obviously, because no women are. (It pains me that people actually believe that. Unfortunately, such statements would logically elicit an emotional response in most women, thus “proving” them correct.)

  • machintelligence

    Here is why Mitt Romney topped the charts:,29845/
    Please note that it’s from The Onion, but it is a fun read.

  • jose

    lol, they have climate denialist stuff right there at the front page. Can’t wait to see them support Paul Broun and fact-check that the big bang is indeed a lie from the pit of hell.

    I’m afraid these guys fact check the same way Fox News does fair and balanced reporting.

  • lucrezaborgia

    Snopes is liberal???

    • Julanar

      Snopes isn’t really a political fact-checking site. It’s a site that debunks urban legends – REALLY ridiculous claims. Naturally, some of them are political; most aren’t, but they rarely take sides in any of their discussions. Check it out sometime. (

      • lucrezaborgia

        Oh I read them a lot and constantly use them to debunk all the infographic crap that gets passed around on FB.

  • Anat

    It would be interesting to compare what happens in an election where both major parties hold primaries between multiple candidates. Perhaps then we will hear more lies from both because of vying for attention on both sides.

  • Bryan White

    “Why should the ratio of ridiculous claims to false statements be the same?”

    Because, based on the proposition that the journalists use no objective criteria to judge between the two ratings (context is important), the rulings are subjective. Ideological neutrality would result in equal treatment of Republican and Democratic Party claims.

    “Why is it implausible that Republicans are more likely to add hysterical claims about the end of the world as we know it than are Democrats?”

    How do you objectively determine what is and what isn’t a “hysterical” claim? You’re welcome to come up with criteria and see if PolitiFact is using them. I looked and I couldn’t see anything that was remotely consistent. Until then, it seems to make sense to take PolitiFact at its word: that statements it labels “Pants on Fire” are statements it feels are worthy of being subjected to ridicule. That’s a subjective judgment on its face. And the subjectivity appears more than just skin deep.

    “(blah blah blah) … You don’t do that.”

    Of course I don’t do that. I clearly explain that *all* “Pants on Fire” ratings are unfair. So it’s pointless after arguing that to argue that some of them are fair and some are not. They’re objectively unfair if they are subjectively determined. Fair rulings will use objective criteria, I hope you’d agree. The study measures bias because the unfair rulings are applied much more often to Republicans.

    For what it’s worth, I’ve done plenty of work showing how PolitiFact has made mistakes and used inconsistent standards, and it appears that Republicans tend to get worse treatment in that respect, also. But the advantage of the study I performed was the unequivocal nature of the raw numbers and the obvious lack of objectivve standards according to PolitiFact’s definitions.

    Bottom line, to claim that a rating that something is “ridiculous” is unfair will require an argument that appeals to some standard. PolitiFact does not provide any such standard. Nor does it describe any such standard either implicitly or through self-descriptions of its process.

    • phantomreader42

      So, you’re arguing that there is no such thing as a ridiculous claim? Did you look at any of the examples? Are you seriously going to pretend that the GOP’s hysterical screeching and nonsensical conspiracy ravings should not be called “ridiclous”?

      • Bryan White

        “So, you’re arguing that there is no such thing as a ridiculous claim?”

        Am I?

        I’m arguing that calling a claim “ridiculous” is subjective on its face. Hypothetically I can define it however I like and make it objective. For example, I could say that numerical errors greater than 50 percent will receive a “Pants on Fire” rating. That quantifies the term “ridiculous” and keeps it from being subjective. I’m saying that PolitiFact provides no evidence that it does anything like that, and that it doesn’t make sense to assume that PolitiFact has objective criteria in support of the ruling. If there’s an objective standard then PolitiFact should explain what it is.

  • Ibis3

    I think it’s been established that this site is a prank.

  • Ed Darrell

    Happy to discover that FactCheck came clean on their bias many months ago — most people just missed it.

    I covered it here:

  • Checkout

    Dear Writer.

    Let me summarize your argument. Factcheck and other liberal labeled organizations are not biased because they do point out some errors on the left?

    If factcheck was in fact biased to the left the would WANT to call out some untruths of the left. Not to do so would make their true (accused) nature evident. It’s a simple trick of deception. In World War Two in order to keep the axis ignorant that their secret codes were broken the allies would let the axis decode messages that were not critical to their victory. As I believe factcheck could possibly be doing now. So your chart is weak evidence to your argument, a better chart would show the number of factcheck alerts called on critical issues and how many of those were actually accurate.

    • phantomreader42

      So, Checkout, do you have even the slightest speck of evidence that “Factcheck and other liberal labeled organizations” are WRONG about anything? Are they factually inaccurate? Yes or no? Or do you just not CARE about accuracy? Or worse yet, do you see the very act of telling the truth as showing bias against conservatives?

  • Dan

    My EQ dropped reading this..

  • Dan

    My IQ dropped reading this..* see