Chuck the Checklists

I have been a fan of the CRAP test, but a recent article by John Warner in Inside Higher Ed has me rethinking this. The article points out that knowledge and motivation are essential to getting at the truth, and no checklist can effectively substitute for those things.

Here is an excerpt:

Experienced academics don’t actually use the CRAAP test. The CRAAP test is an approximation of a much more sophisticated process rooted in that domain knowledge. Giving it to students as a substitute for knowledge may not be doing them any favors.

In fact, many students in the Stanford study applied checklists like the CRAAP test and came up with incorrect assumptions about the reliability of the sources anyway.

This is when I begin to panic and think that we may have an unsolvable problem on our hands, but Caufield and the Stanford researchers point us in a different direction.

Rather than training students to evaluate sources – as has been our practice for generations – we need to adapt and help them not understand sources, but the “web” itself.

And a little later, by way of conclusion, the author poses the following questions:

[W]hat if checklists are only appropriate for things we’ve already learned, but are prone to forget, like medical professionals practicing the proper pre-surgery practices? What is the impact of checklists on people who have not yet developed disciplinary instincts, or who have yet to experience the full complexities of their field?

What happens both in terms of motivation and curiosity when we substitute checklists for literacy and knowledge?

What do you think? I ask this question, as you know, as someone who has used the CRAAP or CRAP Test in the past. And, for those who have missed it, I should emphasize that the CRAPpy Song, which teaches the CRAP or CRAAP Test, by my colleague Chad Bauman, is now back online after a long absence from YouTube!