At a recent gathering of local New Testament scholars here in Indianapolis, the question of whether the Gospel of John’s Christology might be adoptionist came up. In my own work on the Gospel of John, I looked closely at the few and often neglected academic studies of this topic, and found myself persuaded that the Gospel of John can be plausibly interpreted in this way. My very first academic publication, in Princeton’s journal of graduate research Koinonia Journal, addressed this topic. Since it is rather niche and not the dominant view on the internet nor in print, it seemed like a good way to test the capacity and limitations of AI to perform literature reviews.
For those unfamiliar with the term, a literature review is what academic (especially graduate students) do prior to trying to break new ground with their research. For a researcher, the question of what has already been done is the crucial foundation for trying to say something new. If you are not a professional researcher, your own investigation is probably asking much the same question: what conclusions have others drawn on this topic? Thus for any and all interested parties, the “lit review” gets you at what you need to know. So how do AI tools do in providing this?
The Test
I gave ChatGPT, Perplexity, and Grok the same prompt: Perform a literature review on the question of whether the Christology of the Gospel of John is rightly categorized as adoptionist. Perplexity and Grok got right to work. It is difficult to know whether the scratchpad in which an LLM supposedly “shows its work” is evidence of its reasoning process. It is entirely possible that it is simply the LLM generating more of the kind of output that LLMs generate, showing what would be plausible text if a human were working through the process in imitation of human text, rather than a genuine presentation of its reasoning. Because machine learning algorithms are a black box, we cannot know.
At any rate, Grok showed its work by writing “The request is about Christology in the Gospel of John and its adoptionist categorization. First, I’m figuring out what Christology means and the adoptionist concept.” Not a promising start, but things got worse as it turned to Wikipedia, Brittanica, gotquestions, and Desiring God websites. You can hopefully see why I’m skeptical that LLMs are genuinely showing us their reasoning. It is not as though an LLM had not been trained on all these texts already.
It supposedly searched for “adoptionism in the Gospel of John” and got 10 weak generic web results. Nothing academic, nothing likely to reflect relevant scholarship. When it did eventually refer to New Testament scholars, they were D. A. Carson and Leon Morris.
Perplexity drew heavily on The Gospel Coalition and cited Mike Bird. Readers may recall that I expressed disappointment with Mike Bird’s book on this topic. It showed a complete lack of awareness of the relevant scholarship on the Gospel of John’s Christology and adoptionism.
The Winner
ChatGPT did a much better job. It asked a follow-up question about the scope of my interest, which has become a standard feature in its Deep Research functionality. It then worked on the question for quite some time after the other two platforms had produced their output. The most striking and exciting moment was when I happened to notice what it was currently working on. It was looking for “McGrath 1996 adoption…”
Eventually when it was done, it had found that article of mine via James Tabor’s blog. It also seemed to be looking for my thesis on Durham University’s repository of theses. Again, whether this genuinely reflected something akin to a reasoning process is hard to say. Can you see the sidebar the same way I can?
ChatGPT didn’t go to my article first, I should point out. It initially got things in the right order, starting with Reginald Fuller’s 1976 “The Incarnation in Historical Perspective.” Then it proceeded to Francis Watson. While I am honored that it turned to me next, it would have been more chronologically appropriate for Charles Talbert to be next, then me, and then Pamela Kinlaw.
Ultimately ChatGPT too emphasized that this is not the dominant view of John’s Christology. Unlike the others, it provided and interacted with the right sources. This was the bibliography:
Brown, Raymond E. An Introduction to New Testament Christology. New York: Paulist Press, 1994.
Ehrman, Bart D. How Jesus Became God: The Exaltation of a Jewish Preacher from Galilee. New York: HarperOne, 2014.
Fuller, Reginald H. “The Incarnation in Historical Perspective.” In Theology and Culture: Essays in Honor of A. T. Mollegen, 57–66. Anglican Theological Review Supp. Series 7, 1976.
Kinlaw, Pamela E. The Christ is Jesus: Metamorphosis, Possession, and Johannine Christology. SBL/Brill, 2005.
McGrath, James F. “Johannine Christianity – Jewish Christianity?” Koinonia 8.1 (1996): 1–20.
Watson, Francis. “Is John’s Christology Adoptionist?” In The Glory of Christ in the New Testament: Studies in Christology, ed. L. D. Hurst & N. T. Wright, 113–124. Oxford: Clarendon, 1987.
The Reason
My impression is that it was able to do so well due to the source that it cited, Michael Kok’s blog and his post citing these and other relevant sources.
Its conclusion was this:
In light of the textual evidence and scholarly study, the Christology of the Gospel of John is overwhelmingly characterized as incarnational (high), not adoptionist (low). Historical-critical scholarship has largely confirmed what the Gospel’s patristic reception already suggested: John presents Jesus as the pre-existent Son who became human, rather than a human who was elevated to Sonship. Key debates in the literature – such as the timing of “the Word became flesh” or the nature of Jesus’ anointing by the Spirit – have probed whether any adoptionist interpretation is tenable. The consensus outcome of these debates is that any adoptionist reading of John is at best forced and at worst contradicted by the Gospel’s central claims. As Francis Watson concluded after thorough analysis, John’s Gospel does not espouse an adoptionist Christology. Rather, it consciously asserts a view of Christ that transcends that category.
Kok’s judgment becomes ChatGPT’s judgment. He and it may be right or wrong, just as I may be right or wrong about this. In the context of the present discussion, which is about the technology more than the Christological question per se, the most interesting thing that deserves further investigation is why and how ChatGPT did so much better than its competitors. Ultimately the black box character of machine learning algorithms means that we cannot know for certain. What is to be noted is that it could only offer major academic publications inasmuch as these were mentioned online in places that are not paywalled. My books that touch on this topic are never mentioned.
The Takeaway
What is to be highlighted and emphasized is that an LLM performing a literature review will only do as well as the best available human-created material available online and not behind a paywall. It may do significantly worse, but it simply cannot do better by virtue of what the technology is and does, and what it has access to. Thus it needs to be emphasized that outsourcing your literature review to an LLM tout court is seriously unwise. This point applies to those doing academic literature reviews prior to their own research, as well as those simply asking an LLM to answer a question (such as the one I focused on here about the Christology in the Gospel of John).
The fact that we call these systems “artificial intelligence” doesn’t mean that they can do what you do. They have an exponentially larger set of data points by an order of magnitude that the human mind finds unfathomable. It is easy to be impressed and the reaction is appropriate. Deferring to the technology, however, reflects a serious misunderstanding. This is a system trained on human text that can often connect you with answers found in that human text more quickly than you would have managed on your own. Even in that best case scenario (note again the much more disappointing outputs of Grok and Perplexity) it does what it does because it was trained on human text.
Ultimately it is never ever the AI that is providing the information you need. It is human beings, with the AI sometimes connecting you effectively with that human-generated content, even as the more familiar and older AI tool of Google search has long done.
I have shared the chat with ChatGPT so that you can see it for yourself.
The Fun
As I was working on this, I also read an article that was praising the ability of Claude to create games in response to prompts from users. Like the author of that article, I was impressed by what it was able to do, and am eager to explore the potential uses of this for educators.
I asked Claude to create a game in which one solves riddles from Jesus. This is a link to what it came up with. Try out the game and let me know what you think. It may seem “rudimentary” until you consider that it was created by an AI in a matter of seconds in response to a vague half-baked idea that I provided. If you try it out, you will see that it is not without issues. Yet it had some impressive features, including accepting “kingdom of God” as a response when the hint text from the Gospel of Matthew made me wonder whether only “kingdom of heaven” would be accepted.
Since I am teaching Women in Early Christianity this semester, I also asked Claude for a game about women in the teaching of Jesus. It offered a board game, and while I am not sure it is a marketable product or something that students would find fun, to have a game on a specific topic generated in seconds is itself impressive. Here too I think the point is that AI can be a useful tool in exploring ideas and possibilities, and realizing them in at least rough draft form very quickly. It is not poised to replace human beings or to offer what we alone can.
The Rest
For more from me on this topic (with lots of links to others):
Of related interest elsewhere:
Follow Liza Long’s Substack
Here are two recent podcasts from Tripp Fuller:
The Tech Takeover: Reimagining Connection in a Digital World
The New Colonialism: Power, Data, and the Transformation of Human Experience
From AI in Society:
The AI Recruiter Will See You Now
From Inside Higher Ed:
Cheating, and Assessment, and AI
AI Companies Want to Take Educators’ Jobs
From Existential Comics: Residential Philosopher for AI Ethics
From this post, hopefully I have made clear some of the shortcomings of AI as well as good reason to be skeptical of hyped marketing claims about the capacity of AI to replace humans. Let me thus end with a video that illustrates how humans using AI tools can accomplish amazing things.
Ad-Free & Subscriber Log-In