In an age where one can pretend to be an expert on the internet when one is not, and one can self-publish works that really aren’t scholarly work, it is good to have some reminders as to why careful critical review of scholarly work is important. BW3
Expertise and How to Detect It
Last week a friend pointed me to a web site where a guy, claiming expertise in something else (cryptography, I think, but it doesn’t matter) also claimed to have established beyond dispute and for the first time in modern scholarly studies the “true” meaning of a particular Greek word used by Paul. Moreover, on this basis the guy claims a radically different understanding of what Paul had to say on the topic with which this Greek word is associated. So, what did I think?
Well, I have to say that it’s curious that someone with no training in a given field, lacking in at least some of the linguistic competence required (both relevant classical language and key modern scholarly languages), thinks himself able to find something that has eluded the entire body of scholars in that field who labor year-upon-year to try to discover anything new and interesting. It’s also curious that, as is typical, the guy doesn’t submit his findings to scholarly review for publication in peer-reviewed journals or with a peer-reviewed publisher, but flogs his thinking straight out on his web site, complete with bold claims about its unique validity. We mere scholars in the field, by contrast, do submit our work for critique by others competent in the subject. We present at symposia and conferences where other scholars can engage our views. We strive to get published in peer-reviewed journals and with respected publishers. Even after publication, we hope for critical engagement by other scholars.
Now, of course, I believe in freedom of speech and thought, and I wouldn’t press for a gag on the sort of dubious stuff that I criticize here. But in scholarly life the peer-testing of claims/results is absolutely crucial, and it’s really considered rather unscholarly (and so of little credibility) to present as valid/established claims that haven’t gone through such testing. People (specifically those not clearly qualified in a field) have always been able to make bold claims about a subject of course, asserting their idiosyncratic “take” over against whatever view(s) is/are dominant in the subject. But before the World Wide Web I guess it was much more difficult to get such unqualified opinion circulated. Now, however, “the Web” and the “Blogosphere” make it so easy.
But, frankly, when I’m shown something that hasn’t been through the rigorous scholarly review process (often, it appears, peer-review deliberately avoided), and comes from someone with no prior reputation for valid contributions in the subject, I’m more than a bit skeptical. If the work is really soundly based, then why not present it for competent critique before making such claims?
Peer-review typically doesn’t mean quashing any new view. Instead, it means that a submitted piece of work is studied to see if any relevant evidence or important other analysis is overlooked, or if there is something quirky and apparently wrong in method or assumptions. I’ve certainly had articles accepted for publication in cases where the reviewers weren’t necessarily convinced but did agree that my argument couldn’t be faulted on data or method, and so my article deserved to get publication and thus a wider “hearing” by scholars.
So, how does some innocent peruser of the Web who isn’t an expert in a given field judge a claim about something in that field? Well, is it being made by someone who appears to have the requisite training for that subject? Is it from someone with an established reputation in that subject? (And the Web now makes it fairly simple to check up on people.) Or, if it’s from an emergent scholar, is the claim published in a peer-reviewed journal or from a respected published (who uses peer-review)? If not, then I’d advise you not to bet more than a tuppence on it.
Think of the Web/Internet as something like a postal service. You can send all sorts of things through the post (and much more via the Internet that wouldn’t easily or legally get into the post!). So, simply because something is “published” on the Web doesn’t mean anything by itself. The key questions concern the qualifications of the person authoring the material, and whether it’s been adequately reviewed and had critique by those competent in the field.