Confirmation Bias: Everybody’s Problem

In a recent study, researchers found that both conservatives and progressives tend to have less faith in scientific findings that go against their political presuppositions. This comes as no surprise to me. I was always suspicious of arguments that appreciation of science is tied to a given political perspective. In fact, scholarship that reinforces the idea that individuals of a certain political ideology are superior to others always seemed self-serving to me. Perhaps that is why I never bought into theories such as Right-Wing Authoritarianism which argued, among other things, that political conservatives are more willing to oppress others than political progressives. In my previous work, I showed that these findings are largely tied to the group that faces prejudice or oppression. When respondents are asked about conservative Christians, then those who do not tend to have Right-Wing Authoritarianism, who are more likely to be political progressives, are more likely to support measures that take away the rights of others.

The findings of this study on having faith in science are tied to the well-known concept of confirmation bias. We tend to use this bias to determine the evidence we will accept. So if there is scientific research that supports our political beliefs, then we will give those scientific results more weight than scientific research challenging those beliefs. This is true with our experiences, logical thinking, advice from peers and any other information we gain during the course of our life. Confirmation bias does not know political party, racial identity or religious affiliation. It affects us all.

There is a little game I like to play from time to time to remind me about the power of confirmation bias. I will look at a current political event and then imagine it with the political parties switched. Then I consider if I have evidence that the political groups would merely switch roles. For example, remember not too long ago Republicans were complaining about the amount of vacation time the President Obama was taking. Of course, the Democrats were working hard to defend him. What if President Obama were a Republican? I know that then it would be the Democrats complaining about the amount of vacation time and Republicans would defend the president. I know this because that is precisely what happened with President George Bush. Or consider the advisability of the filibuster. Right now, the Democrats consider the filibuster to be pure gold while the Republicans hate it. Just a few months ago the roles were reversed, and they were calling the Republicans the “party of No.” I am often amazed at how individuals quickly change their values once their political party is under the gun. We can thank confirmation bias for our ability to be morally flexible.

It is one thing when we see the effects of confirmation bias in politics. After all, part of what politics is about is fighting for one’s own social group. Naturally, political activists are not going to be unbiased when trying to gain more resources for their groups. But confirmation bias is much more problematic when we consider the sciences. Science is supposed to be the institution whereby we gain important information about our world. Can we gain that information when we have the reality of confirmation bias? In theory we should be able to overcome that bias with a community of scientists from different perspectives. Competing ideas should drive out the lesser ones and leave the theories that best describe our physical and social reality. I have yet to read or hear of a way that we can ensure that any single individual can avoid confirmation bias. But, if we provide our work to academic peers, some of whom have different scientific perspectives than our own and will be especially harsh on our ideas, then we can at least partially neutralize that bias as a community.

The problem is that this is what works in theory but not in reality. Science is a social institution, and like all social institutions, it tends to be shaped by the social biases of the individuals in it. In theory, the sciences could attract individuals from contrasting social places in our society and thus likely have distinct perspectives. In reality, scholars tend to come from a very similar social position and tend to bring in quite similar presuppositions about reality and society. They do not challenge the overarching social and political paradigm of other scholars. If they dare to make such challenges, then they can find themselves stigmatized for having the wrong political ideas, not for their inability to conduct scientific research as I observed with the controversy over Mark Regnerus’s work.

It is not hard to understand why we have confirmation bias. We naturally want to confidently assert the correctness of our beliefs and that those who disagree with us are mistaken. We may even feel offended that others would dare to have beliefs that do not comport to the reality we “know” to be true. So, just as the research on political bias and scientific results suggest, we set higher standards for evidence that challenges our beliefs or even move the goalpost when others produce evidence that meets those standards. If we cannot have complete confidence in the scientific community’s ability to overcome confirmation bias, then can we trust our own ability? I suggest that we cannot have that complete confidence in ourselves. A more rational approach is to recognize that confirmation bias is in all of us, and yes I definitely include myself, and then attempt to take steps to reduce its effect.

How can we reduce the possible effects of confirmation bias in our lives? I am just like everyone else in that I am vulnerable to confirmation bias. But it is something that I have thought a lot about. As such, I have tried to take steps to at least limit its potential effects in my life. Maybe I have succeeded and maybe I have failed. Nevertheless, I will share some of the ideas I have developed in an effort to deal with the reality of confirmation bias.

The obvious first step is the step every drug abuser knows, which is to recognize that we have a problem. I come to my conclusions as best as I can and, as often seen in my blogs, will argue for those conclusions. But I acknowledge in the back of my mind that I may be wrong. For example, I believe in God. I will tend to look for evidence that confirms my theistic presuppositions. Consistently recognizing that tendency within myself helps me to see when I am giving evidence for my previous beliefs undue weight. It helps me, but that does not mean that I am immune to the effects of selective attention to evidence.

There are questions that the scientific method cannot help us answer. But to the degree that we can use this type of approach to deal with the confirmation bias in our lives, the better off we will be. To this extent, when I settle on research questions to pursue, I seek out research designs that allow for the possibility of the opposite of my expectation. For example, when I explored the possibility of academic bias I suspected that bias was highest against political conservatives. But I asked about a wide variety of social groups so that if there was bias against political progressives then I would capture that effect. That strategy paid off since my inclusion of religious groups allowed me to find out that bias was highest against religious, and not political, conservatives. My care to use research designs that allow for findings that are contrary to my presuppositions comes in part because I recognize the power of confirmation bias and that I must find ways to account for it. This does not mean that the scientific method can completely neutralize confirmation bias since we can become too confident in its accuracy when we do not recognize confirmation bias in ourselves. Only when we recognize confirmation bias in our own lives are we able to use a scientific method perspective to deal with that bias.

This leads to a second way we can try to reduce confirmation bias. As much as possible, we should seek out arguments that contradict our own. And these arguments should be the best of the arguments of the other side, not the worst. We definitely should not create straw man arguments to tear down so that we can feel good about ourselves. In my case, political ideology is not nearly as salient to my social identity as my religious faith. Being a Christian on a state campus is an excellent way to hear arguments that run counter to my faith. When I conducted research on atheists, I found an excellent opportunity to read atheist material and hear the best arguments from atheists. Of course, I bring a confirmation bias into those readings and cannot pretend to be an objective observer. But at least exposing myself to different ideas provides me with an opportunity to allow those ideas to challenge my presuppositions.

It is fair to ask whether I ever change my ideas due to gaining new information. If change is not possible, then there is evidence that my confirmation bias is so strong that I cannot see how it shapes my perspectives. Of course, I have made changes on minor issues that are not important to me. But that does not show much as the true test is whether I can change when I have an emotional or identity investment on the issues. Early in my career, I conducted research on interracial romance, probably in part because I was part of such romances. So I was heavily invested in the notion that interracial romance was a social good. It was a time in which interracial relationships were not accepted in many sectors in society. In fact, one of the arguments I heard against them was that they were less stable. My inclination was to believe this to be a myth perpetuated by those troubled with racial bigotry. Such a belief fit well into my social identity as one who wanted to see more acceptance of interracial romance. However, as I read scientific research, I came to the conclusion that interracial relationships indeed are less stable than same-race relationships. As much as I wanted to deny what I thought to be a racist myth, research indicated empirical support for that “myth.” Even though it seemed to empower those who I knew were wrong about interracial marriage, I had to change my mind on an issue due to exposing myself to new evidence. That was difficult to do given my presuppositions about interracial romance. It did not remove my commitment to battle against the voices at that time that resisted interracial romance but it did force me to make sure that my arguments against those voices were based on accurate research.

Another way of dealing with confirmation bias is to be aware of the non-rational, emotional aspect of our lives. I tend to be more cognitive than emotional. But all of us are emotional to some degree, and we make decision based on those emotions. Confirmation bias works by using those emotions to reinforce our presuppositions. So while we believe that we have logically come to certain conclusions, the reality is often that we are using logic to reinforce the conclusions we emotionally want. I wish there was an easy way to turn off our emotions, but there is no off switch. The emotional hit we get from reinforcing the ideas we feel good about is going to be there and the best thing we can do is recognize it.

One way we can challenge this tendency is to simply allow life to bring us to a point where those emotions are not driving us the way they once did. I recently blogged about a time I was challenged in my Christian faith. I was not challenged because of any logical argument but because certain circumstances made me emotionally desirous of dropping my faith. Although I grew up a Christian and found emotional comfort in it up to that point, I now had emotional reasons to leave my previous beliefs. But what began as a challenge to my previous Christian beliefs turned out to be a hidden blessing. It forced me to deeply consider whether my religious presuppositions were merely due to the way I was socialized or whether there were justifiable reasons for my Christian beliefs. In the end, I come to the conclusion that the evidence for my beliefs was generally stronger than the evidence against them. I made minor alterations but did not jettison them. Indeed, I felt better about my beliefs since I knew that they were not entirely based on my emotional desires.

Combating confirmation bias on a personal level will not eliminate it in the individual battling it, much less the larger social institutions such as academia. Just as individuals have to be self-aware about the potential of confirmation bias in themselves, there is a need for institutional solutions. In Compromising Scholarship I offered potential answers for dealing with academic social biases. Many of my suggestions centered upon intentional efforts to make academia more comfortable for those who tend to experience the effects of those biases. Changing a community is more difficult than changing an individual. As important as it is to deal with confirmation bias in the sciences, I am skeptical that enough scholars will be convinced in the importance of tackling this important issue. Confirmation bias is too powerful for many scholars to overcome it.

This blog entry started out with an explanation on why political conservatives and political progressives do not tend to have faith in scientific findings that go against their political perspectives. Obviously pure confirmation bias on the part of those conservatives and progressives is a factor. Ironically, confirmation bias among academics may also play a role in the lack of scientific faith as well. Since conservatives and progressives become very skeptical of scientists who conduct research that works against their political interest, they are likely to observe elements of confirmation bias among those scientists, which may contribute to their lack of faith. Perhaps the main reason why academics should consider dealing with confirmation bias in their own ranks is that it may help them win over those who have lost their scientific faith.

Science and the Questioning of Our Biases

You never know when an article will catch your attention. I ran across this article concerning how individuals will not change their mind even when facing information that directly contradicts their argument. Our presuppositions are so strong that even when we receive evidence that they are wrong, we hold onto those assumptions. This tendency has been called confirmation bias and it is a part of our human nature. Understanding this bias is important to fully understand how science works.

Confirmation bias is so strong that often we do not recognize when it affects us. We just think that our beliefs are the way things are without seriously questioning if we are wrong. Thus, we have to make a conscious effort to combat confirmation bias to engage in critical thinking. I have learned how hard it is to question one’s beliefs. Even though I grew up going to church, I remember a time when I was struggling with my faith. Things had not worked out the way I thought they should and I questioned whether following God was what I wanted to do. But I did not want to make an emotional decision based on my life’s current circumstances. So I thought deeply about the apologetic information I had collected and the arguments against theism I had heard over the last few years. I was in graduate school at the time so I had learned a lot of anti-theistic arguments by then. After a lot of thinking and questioning of my faith, I came to the conclusion that theism was the most logical alternative. Once I made that decision then some more thinking led me to reaffirm that Christianity was the most logical decision. I came to those decisions but for a time was emotionally spent because of how difficult it was to look at those issues.

Although that was a very painful time for me, I am glad that I went through that period of questioning. Going through it allowed me to at least temporarily test the presuppositions I had developed. This questioning is not a guarantee that I overcame confirmation bias. I am responsible for continuing to seek out new information that may alter the previous conclusions I have drawn. But the emotional pain I experienced from reconsidering my religious beliefs showed me just how hard it is to confront our personal assumptions about reality. That difficulty goes a long way to explain the findings in the linked article about individuals holding on to beliefs even when confronted with disconfirming evidence.

This gets us back to thinking about the institution of science. What is true for the individual can be even truer in a community that reinforces previously accepted corporate assumptions. Thomas Kuhn argued that scientists develop paradigms based on previous ideas. This paradigm is accepted and defended until it reaches a point when there is so much disconfirming evidence that it becomes nearly logically impossible to believe in the paradigm. But usually the paradigm goes relatively unchallenged and scientists attempt to defend it against all attacks. According to Kuhn, the normal state of affairs is scientists fighting to maintain the paradigm against all attacks instead of looking for all possible answers to research questions. We like to think of scientists as individuals open to all possible answers. But the result of the article reinforces the reality that scientists, like others, when looking for answers to research questions tend to look for those that fit their presuppositions.

There is no easy way to deal with confirmation bias. The best we can do is to struggle to make sure that our ideas are driven by evidence and not merely what we want to find to be true. I have seen this occur in my life in more than in dealing with my faith but also in looking at the results of research. One of the topics I used to study was interracial romantic relationships. Being interracially married, I was eager to disprove the myths some of my Christian brothers and sisters had about interracial couples. One of the “myths” was that interracial marriages are more likely to end up in divorce. However, as I looked at the research I saw evidence that interracial marriages are indeed slightly less stable than same-race marriages. I could look at the enduring racism in our society as a reason for this relative instability, but more than one study indicated that this instability was real. The only honest approach was to alter my message to accept what science was telling me.
Not all of us are scientists or scholars. But all of us have opportunities to deal with confirmation bias in our lives. It may be in how we approach our politics, how we think about our interpersonal relationships, or how we consider our beliefs about the supernatural. I fear that most individuals will not take advantage of our opportunities to deal with this bias since it is painful and difficult to confront our presuppositions. The advent of the internet allows us to read only the ideas that reinforce our own ideas. So instead of the internet exposing us to philosophies that differ from our own, it merely allows us to remain sheltered in our own ideological cocoon. And when we do encounter those we disagree with, we do so in comment sections of web articles, or on Facebook we engage in “debates” that provide more heat than light. On the internet we do not have to show any common respect to individuals who remain anonymous to us and so discussions with others in online forums often prove to be unfruitful.

When I was in graduate school I was taught many different skills. I was taught statistics and social science methodology. I learned how to read the classic sociological theorists and contemporary academic literature. I was trained in how to construct a research question and organize an empirical project to study that question. But there were no lessons on controlling my confirmation bias and making sure it does not get in the way of my work. Professors did talk about critical thinking but much of that training merely reinforced the ideas professors had previously accepted as the results of his/her “critical thinking.” Most scholars are not any more prepared to deal with their presuppositions about reality than anyone else.

In an ideal world science would be what we expect it to be. It would be an open search for truth. But given the reality we have today, it is a mistake to expect scientists to be able to consistently overcome their confirmation bias. It is beyond the scope of this blog to discuss instances where it was clear that this bias played a major role in how science is done. However, I have seen enough in my chosen field of sociology to know that scientific results can be the outcome of presuppositions rather than rigorous study. Given that I know this to be true in sociology, I have to wonder how much this is also true in other scientific disciplines. Given the power of confirmation bias, I must wonder what our limits are in accumulating scientific knowledge.