Algorithmythicism at #AARSBL19

Algorithmythicism at #AARSBL19 February 18, 2020
Continuing my recap of the Digital Humanities session I chaired in November. Let me begin with links to three updates on Claire Clivaz’s Mark 16 project (highlighted in my first post from this session). The most pressing is probably the one with news and upcoming events.
Daniel Stökl Ben Ezra of the Ecole Pratique des Hautes Etudes and Hayim Lapin of the University of Maryland – College Park presented on “Automatic Transcriptions of Medieval Hebrew Manuscripts and Crowdsourcing Their Corrections.” They mentioned that it is unfortunate SBL does not allow submissions that include authors that are not present. In the natural sciences it is different. Astronomers have been living in this world for half a century. When it comes to digital rendering of Hebrew texts, we need to have data points at the level of the stroke, letter, etc.
(See Patrick Sahle’s text wheel.) This project is at the intersection of several other projects: eRabbinica, Sofer Mahir (automatic transcription of Tannaitic treatises), Tikkoun Sofrim (automatic transcription of Tanhuma), BiblIA, Scripta numerique. Their approach allows fast transcription of huge amounts of data. They were originally at 3 errors per 100 characters, and are now at 1.8. Computers and humans can both include extra space, other things not needed, because of not knowing the language and thus not knowing what is significant. Computer can learn to recognize letters extended to fill the line! Some alphabets have curved, straight, and oblique lines. The initial OCR had 20,000 mistakes. It turns out that the general public can be helpful in eliminating them. Questions I had included: Can the correctors help the software learn to do better? Risk of “correctors” who do not know language making software worse at recognition? Do large numbers of participants in such a project offset negative impact of any individual who is incompetent or malicious?
The presenters doubted this could be done with Latin scripts in Israel. The number of users grew to 373, and the number of lines checked by the most active users was 11,552. The hyperengaged are 10-20% of the total, and they do 80-90% of the work. Almost 90,000 transcribed lines resulted from this project. They noted there were fewer users on the sabbath. News coverage caused peaks. System malfunction also led to low points in activity. The quality of crowd corrections was pretty good for those who were experienced. The character error ratio was .5, typically special characters. Users improved over time. They mentioned collatEx. A text service must provide instructions about how it should be cited, machine and human readable. The project is aiming to be open source. Another question I had was whether users transcribe accurately even if a text is grammatically wrong?
A. Marshall King V of the University of Notre Dame presented a test case on Deuteronomy exploring big data, intertextuality, and the Bible. This represents a personal project with no funding, working on tools to investigate shared citation. Looking at existing sources such as Biblia Patristica and Biblindex, what they do they do well, but some questions were not answered by the configurations of the data provided there. For instance there was no help with weighing, i.e. evaluation of which verses have been cited most in a time period. Other interesting questions that digitized texts ought to help answer but do not currently are: Which groups cite particular verses most? Which verses are cited by the largest number of authors? Which verse is most cited? Which of the ten commandments is most cited? It takes “years of analysis for a day of synthesis.” King wondered why has it taken so long for network theory to be applied to this, specifically Willem Vorster’s work on network of traces.
This was where the question occurred to me: Could an AI calculate a probability that something is an allusion to an earlier text?
The raw frequency with which commandments are cited by later authors is fascinating. Coveting is the most mentioned. Honoring parents is cited by the most authors. The parting of the ways and the different ways of delineating the commandments affect things. Searching for citation of Deut. 32 yields very different results among Jewish and Christian authors. This project relied on existing lists of citations. But in theory one might automatically detect citation.
Benjamin White from Clemson University presented on the idea of “dirty texts” and how it relates to stylometric analysis. Stylometry, for those not familiar with it, is the attempt to quantify the writing style of an author such as Paul, with a view to determining the authenticity or actual authorship of letters. The previous year White tackled whether named coauthors might have impacted the stylome. It turned out that there is no reason to hang multiple authorship over stylometric analysis. Inclusion of Timothy doesn’t change the stylome. He talked about the work Paul Duwalla of Duquesne University has done on canonization and the idea of a pure text, and what Maciej Eder has done on the “dirty corpus.” The work of the Computational Stylistics Group was mentioned. In this year’s presentation White focused on clustering of tests and signal interference through scribal transmission process. This represents a form of metacriticism of the use of stylometric analysis to determine authenticity. We don’t always consider the way using different manuscripts, or an eclectic critical edition, could potentially produce different results in a study like this. On the other hand, when classical speeches are analyzed, even random replacement of 40% of letters did not prevent 70% accuracy being achieved, suggesting that this computer-assisted method should still work in principle, and work better than human judgment.
In the case of Paul’s letters, however, one has to also account for the very different results that have emerged from past stylometric analysis by computer scientists, mathematicians, and NT scholars of Pauline epistles. Ironically, the consensus of scholars, that there is a corpus of seven letters that is the most likely to be authentic, has the least stylistic support! Some of those who undertook studies in the past have used the Textus Receptus, some used Nestle Aland, and White wondered whether this could be the cause of the divergences. (P46 could not be included because it is incomplete.) The Stylo package lets you run a variety of tests at once. It then provides a depiction of the average result. One can choose the increments used. Most frequent word vectors ranged  from 10-50, avoiding the accusation of cherry-picking results you like from among different outcomes. White showed us his Codex Sinaiticus cluster analysis, talking about a “Bootstrap Consensus Tree.” Comparing the results from Sinaiticus vs. Family 35 we could clearly see the cluster differences represented visually. 1 Timothy and Titus align differently in the two results! Word frequency changes between the manuscripts, but sometimes they merely flip places. As words become more infrequent, they skip around in the ranking a lot more. White focused a lot on function words, as it necessary to try to get the content and subject of a particular text not to determine the result, since the same author often wrote about many different subjects, by definition using different vocabulary when doing so.
Looking at weighting tables and network analysis of connections, similarities, and degrees of difference, a key question is which texts are most centrally related to the group. Interestingly, 2 Timothy emerged as the middle of the network, then Philippians, then 2 Corinthians. If one adds other early Christian literature, similar patterns emerge as the network expands.
The key takeaway result is that the Pauline epistles are not especially dirty. They fall within the margin of error for the industry standard for stylometric analysis, regardless of which manuscripts are used. The study has nonetheless helped make clear what has led to past discrepancies. Hugh Houghton said he would have preferred that a modern critical edition be used because of the scribal tendency towards harmonization. Another key point to take away is that to get 80% correct attribution, you need a minimum number of words (2000), and that cuts out half the Pauline epistles. And so, on the one hand, excuses sometimes offered for ignoring stylometric analysis are invalid. On the other hand, the method applied to much of the Pauline corpus is by definition invalid.
The connections of this part of the session with mythicism include not only the connection with evaluating authenticity, something that mythicists handle in sweeping generalizations and other ham-fisted ways, but also the use of technology to spot intertextual connections. My evaluation of Thomas Brodie’s work, in which he argues that the Gospels were created by taking earlier texts and transforming them in ways that can and are supposed to be detected, is that it is intertextuality and parallelomania run amok. A machine could find even more parallels than I did in my article on this topic. And so on the one hand a machine might be able to quantify the differences and similarities with respect not only to vocabulary but also structure in ways that might be useful. But it would also find parallels where it will make more sense to interpret them as due to the fact that human vocabulary involves a limited number of words, and they are used in predictable ways by us all, and sometimes converge on the same word choice even in ways that are unpredictable and even striking but in no reasonable scenario can be due to direct borrowing. The use of computer technology in this area has great promise, as long as we remember what we need to remember about machine learning and algorithmic processes in general, namely that they offer us information which it is still up to us to use wisely. At least at the present, our machines have intelligence but do not possess wisdom. Thankfully some humans do.
In an exciting coincidence, after I wrote this post but before it was scheduled to appear, an article appeared on precisely the topic I have been discussing: “On the Feasibility of Automated Detection of Allusive Text Reuse.”
And of course, don’t miss my earlier post about this session at SBL 2019 if you haven’t already read it:


Browse Our Archives

Follow Us!

TRENDING AT PATHEOS Progressive Christian
What Are Your Thoughts?leave a comment
  • Chris H

    I think you hit on, there, what is one of my biggest pet peeves with mythicism (that and archetypal criticism *gags*) as a whole: the treatment of source evaluation with sweeping generalization leading to gross parallelomania (side note, long live Sandmel’s title of the same name).

    The evaluations of Verenna and Carrier, which continue a rather common trend, of the Pauline corpus I think exemplify this the most. One of the largest instances I find of this is where Carrier argues that the ἀρχόντων in 1 Cor. 2 are demonic figures, based on his connections to the Asc. of. Isa. and Gnostic literature. He is, unfortunately, not alone and it has been a common trend in scholarship. But if compared with other Jewish literature of the time, including that which bears striking similarity to Paul’s writing, we find that the ἀρχόντων are most likely human figures, in fact (as Wesley Carr, “The Rulers of this Age – 1 Corinthians ii. 6-8,” New Testament Studies 23.1 [1976], pp. 20-35, along with commentators like Fee, Moo, Heil, Keener, Collins and others have all shown). It is based on this backwards of reading of later accrued meanings to terminology back into Paul, in this sweeping generalization.

    It is the human desire to find similarities, which, I think, had led Carrier and others to more project their ability to spot these imaginary parallels, rather than finding something meaningful. In fact, this projection, for Carrier, is so rampant that in his book he even supposes (without a shred of viable evidence) that Paul may have even had access to an early version of the Asc. of. Isa. Of course, Carrier’s conclusions and writing on this topic primarily stem from the work of the woefully under-qualified Verenna and Doherty, who are severely outdated, especially the latter whose information on the Asc. of Isa. (which is where Carrier predominantly receives his own) is now some 30 years out of date, which Carrier does not rectify.

    I think another case of this can be seen in the Dutch Radical revivalist literature of Darrell Doughty and Robert M. Price as of recently as well, particularly relevant to your discussion on the Pauline Epistles in fact. Price has an extreme desire to create parallels and does so quite frequently in his book The Amazing Colossal Apostle in a sweeping fashion. He does this to try and show that the story of Paul’s conversion is made up, and then also with large sections of the Pauline letters, which he regards as all inauthentic and being piecemeal hodgepodges of other works and authors being assembled together (in an A. Dirk Loman and Doughty type of fashion).

    We can also see this whole issue, imo, in the case of the controversy over possible interpolations in Pauline texts, such as 1 Thess. 2:13-16, where the stylistic arguments are often biased in their input of information (as Luckensmeyer, Hurd, Theissen and Merz, Ludemann, Keener, Stevens, and others have all shown to be quite intricately flawed). Most of these arguments limit the scope of the Pauline data they are taking in, and also of the Hellenistic Jewish context as well. So their stylistic arguments against the authenticity of this passage is entirely flawed from the get go (the same problem pervades mythicist/agnostic literature arguing this same point, such as Godfrey, Carrier, Wells, and Price).