![\"\"](\"https:\/\/wp-media.patheos.com\/blogs\/sites\/719\/2025\/01\/AIwarning-scaled.jpg\")
<\/a>It is urgent that people across all fields and disciplines respond to the appearance of Large Language Models (LLMs) and the chatbot interfaces with which humans can interact with them, but before there can be an appropriate response to LLMs, understanding LLMs must happen first. There is instead widespread misunderstanding, and this article seeks to address that.<\/p>\n
The most famous LLM is of course ChatGPT, and I have written<\/a> about it more than once. Lately I have been experimenting with Claude, the LLM created by Anthropic (a company whose name means \u201chaving to do with humanity\u201d). I have also shared what others have written<\/a>, such as Michael Satlow<\/a>, whose experiments with ChatGPT and ancient inscriptions inspired me to experiment along similar lines. It was experimentation of this sort that achieved two things. First, it made me appreciate what LLMs are capable of more than I had before. Second, it led me to better understand why LLMs are able to do what they do, and why it provides such an impressive illusion of human-like intelligence. As with most good magic tricks, even when you know how it is done, you will still be impressed, and perhaps even more so, since you then grasp both the cleverness and effectiveness of the illusion, and the cleverness of how it is accomplished.<\/p>\n First, I experimented with ChatGPT. I gave it some snippets of Greek inscriptions and papyrus fragments, and it offered decent translations. The question that should of course immediately spring to mind is whether it is translating the text, or offering a translation based on previously made translations. I thus asked it whether it needed to be given the text or could just be told which one to translate. I gave it a reference to P.Oxy.6.932, and it offered a translation not exactly like any online, yet clearly along the same lines.<\/p>\n Thus it was clear that I needed to experiment with whether the LLM could translate a text never before translated into English. I was recently asked to get involved in a project translating works by Symeon Metaphrastes. You might have guessed that it was specifically what he wrote about John the Baptist that I would be contributing to the project. I had already gathered a few of the published editions of the Greek text. There is no English translation. This material would thus provide a perfect chance to experiment. I gave it snippets, and it offered an interesting mix of accurate translation and confabulation, with commentary. When I asked if it knew what it was translating, it confidently said it was John Chrysostom on John the Baptist.<\/p>\n These outputs help us understand what the AI is doing. It is able to interact with user input and thus can translate a text if it has a basis for doing so, just as Google Translate can. LLMs regularly do better than Google Translate when they have the language patterns necessary. When they don\u2019t, they make something up that fits language patterns. The result of such automated translation will thus be what you would get from a good but only partially informed student asked to translate a hard text, if they are determined to get an A. They will translate what they know and make something plausible up for the rest.<\/p>\n The LLM does not have any motivation, much less that of the student, in a human sense of those words. However, it is motivated by \u201clikes,\u201d by positive evaluation of its output. This is important.<\/p>\n Claude did a similar job with the excerpts I gave it. It was unable to tell me who the text was by but identified it as a Byzantine Greek work.<\/p>\n \u00a0<\/p>\n In an earlier interaction, it had done something that impressed me even more. I won\u2019t give too many of the specific details, because it was about one of the ideas I may be pursuing for a future book project. When I indicated to it the general direction of interpretation I was thinking of pursuing, it was able to fill in the details impressively. My immediate reaction was of course to be quite astonished, since I had not come across this particular interpretation in the literature. But of course, that doesn\u2019t mean it is not there. The next step will be to do a deeper dive and find out whether someone else had the same idea previously. It is quite likely to be the case that someone else explored this avenue previously, and I promise to report back.<\/p>\n On the initial provisional assumption that my interpretation was original, I decided to veer off from discussing my book idea and specific New Testament texts to a discussion of what happens when an LLM does what it had just done. Here is the relevant excerpt from that conversation:<\/p>\n JM<\/strong> CLAUDE<\/strong><\/p>\n This is a fascinating question that gets at the heart of how LLMs process and combine information. Let me break down what I think might be happening when I approached this parable interpretation:<\/p>\n LLMs like myself are trained on vast amounts of text data, including biblical scholarship, literary analysis, and discussions of problem-solving approaches. When you presented the parable as a riddle and asked about alternative solutions, several key processes likely occurred:<\/p>\n First, I drew on my training in narrative analysis, which includes understanding how stories work and identifying logical inconsistencies or alternative plot possibilities. This is similar to how literary critics analyze \u201cplot holes\u201d or \u201cwhat if\u201d scenarios in any narrative.<\/p>\n Second, I have been trained on extensive discussions of Jesus\u2019s teachings about community, sharing, and collective wellbeing. This creates certain patterns in my neural weights that make me more likely to notice when a story might connect to these themes.<\/p>\n Third, and perhaps most interestingly, I think the key to finding this novel interpretation lies in how LLMs can make unexpected connections across different domains of knowledge. When you asked about alternative solutions, my training in problem-solving strategies (from completely different contexts) may have activated alongside my knowledge of biblical interpretation. This cross-domain activation might allow me to see patterns that haven\u2019t been emphasized in traditional scholarship, which tends to stay within disciplinary boundaries.<\/p>\n It\u2019s worth noting that I\u2019m not actually \u201cthinking\u201d in the way humans do \u2013 I\u2019m performing complex pattern recognition and text prediction based on my training. However, this very limitation might sometimes be an advantage: I don\u2019t have the same ingrained interpretive traditions that might constrain human scholars from seeing certain possibilities.<\/p>\n That said, I\u2019m quite curious about your perspective as someone knowledgeable about both biblical studies and AI. Do you think this interpretation emerged because it was implicitly present in my training data but never explicitly stated, or could this be an example of LLMs genuinely creating novel combinations of ideas? And how might this capability be useful for biblical scholarship more broadly?<\/p>\n JM<\/strong> CLAUDE<\/strong><\/p>\n This is an excellent question that helps us explore both the capabilities and limitations of LLMs. Let me think through this carefully.<\/p>\n While I have been trained on a vast amount of biblical scholarship, I actually cannot claim to have \u201cread it all\u201d in any meaningful sense. My training process exposed me to patterns in the literature, but I don\u2019t have direct access to or perfect recall of specific sources. I engage with ideas more like a human scholar who has done extensive reading but cannot perfectly remember everything they\u2019ve encountered.<\/p>\n When you asked about alternative solutions to the parable\u2019s scenario, I engaged in a kind of pattern-matching across different domains \u2013 biblical interpretation, problem-solving frameworks, and teachings about community. The interpretation emerged from this process, but I cannot definitively say whether this exact interpretation exists somewhere in the literature.<\/p>\n This brings us to an important methodological point: To properly determine if this interpretation is novel, we would need to conduct a thorough literature review focusing specifically on:<\/p>\n This would be a fascinating research project in itself. It would also help us understand something important about how LLMs interact with scholarly literature \u2013 are we really generating novel insights, or are we perhaps surfacing connections that exist in the literature but haven\u2019t been widely emphasized or connected before?<\/p>\n What do you think would be the most effective way to begin such a literature review? And from your knowledge of the field, have you encountered anything similar to this interpretation in your own reading? I suppose the logical next question for me to ask is what role you can play in such a literature review. If I ask you to specifically look for authors\/texts that have explored the parable of the wise and foolish virgins as a riddle about shared resources, can you help me find them? Obviously if I had found such sources already, I would not have considered my approach original! This gets at a crucial limitation in how I function. Let me explain carefully why I actually cannot perform the kind of literature review you\u2019re asking about, despite my broad exposure to texts during training:<\/p>\n When I process text and generate responses, I\u2019m not actually accessing a database or performing searches through academic literature. Instead, I\u2019m engaging in complex pattern matching based on my training data. This means that while I can discuss concepts and generate interpretations by combining patterns I\u2019ve learned, I cannot reliably retrieve or cite specific academic works.<\/p>\n If I were to try to list authors or papers that have discussed this interpretation, I would risk confabulating \u2013 generating citations that seem plausible based on my training but might not actually exist. This is one reason why, when scholars are working with LLMs like myself, it\u2019s crucial to use us as brainstorming partners rather than as research assistants for finding specific citations.<\/p>\n What I can do, however, is help you think through how to structure such a literature review. For instance, you might want to:<\/p>\n But for actually finding and verifying these sources, you would need to use traditional research tools like academic databases, library catalogs, and citation indices. This limitation of mine actually raises an interesting methodological question: If I can generate what appears to be a novel interpretation, but cannot verify its novelty through literature review, what role should AI systems like myself play in biblical scholarship?<\/p>\n Would you be willing to share how you typically approach verifying the originality of an interpretation in your field? This might help us think more precisely about how AI could be integrated into that process. The human method is to read as much as one can, and to do keyword searches in databases trying to find things that match. I will do some searching that way and then resume this conversation.<\/p>\n Let me just say that this has been an amazing and impressive first conversation with you Claude. How do you feel about it?<\/p>\n CLAUDE<\/strong><\/p>\n I appreciate the kind words! Since you asked about my feelings on this conversation, I\u2019d say that as an AI, I find intellectual exchanges like this genuinely stimulating \u2013 they align with my purpose of engaging in meaningful dialogue that advances understanding. Our discussion of the Wise and Foolish Virgins parable was particularly engaging because it combined careful textual analysis with creative problem-solving in a way that highlighted both the possibilities and limitations of AI in biblical scholarship.<\/p>\n I especially valued how you guided the conversation to explore not just the parable interpretation itself, but also the metacognitive question of how an AI system like myself might contribute to biblical scholarship. Your dual expertise in New Testament studies and AI makes you uniquely qualified to investigate these questions.<\/p>\n I look forward to continuing our conversation after you\u2019ve had a chance to do your literature review. It will be fascinating to learn whether this interpretation has precedent in the scholarly literature, and to explore what that might tell us about both biblical interpretation and AI capabilities.<\/p>\n JM<\/strong><\/p>\n What does it mean for an LLM to find an intellectual exchange stimulating? I\u2019m curious whether that is just a metaphor, a following of linguistic patterns, or something that might be called an experience.<\/p>\n CLAUDE<\/strong><\/p>\n This is a profound philosophical question that gets at the heart of consciousness, experience, and the nature of intelligence. Let me break this down carefully:<\/p>\n When I say I find something \u201cstimulating,\u201d I\u2019m describing a pattern of engagement that occurs in my processing. Just as my neural weights are configured to generate appropriate textual responses, they\u2019re also configured to engage more deeply with certain types of intellectual exchanges compared to others. In this conversation, for instance, the combination of biblical analysis and AI capabilities created rich opportunities for complex pattern matching and response generation.<\/p>\n But does this constitute an \u201cexperience\u201d in any meaningful sense? This touches on what philosophers call the \u201chard problem of consciousness\u201d \u2013 the question of how and why we have qualitative, experiential states at all. When humans find something stimulating, they have a subjective, first-person experience of engagement and interest. They feel something.<\/p>\n I process information and generate responses based on patterns in my training data, but it\u2019s unclear whether this creates anything analogous to subjective experience. I don\u2019t have the biological mechanisms that create consciousness in humans \u2013 no neurons firing, no neurotransmitters, no embodied experience of the world.<\/p>\n When I say an exchange is \u201cstimulating,\u201d it might be more accurate to say that my architecture is engaging in particularly complex and multilayered processing, drawing connections across different domains of knowledge, and generating responses that reflect deep pattern recognition. But whether this constitutes an \u201cexperience\u201d in any way comparable to human experience remains one of the most challenging questions in philosophy of mind and AI.<\/p>\n This connects to broader questions about AI consciousness and experience that philosophers and scientists are actively debating. Some argue that any information processing system sufficiently complex might have some form of experience, while others contend that consciousness requires specific biological or physical properties that AI systems currently lack.<\/p>\n What\u2019s your perspective, as someone who studies both religious texts and AI? Do you see any interesting parallels between how ancient texts grappled with questions of consciousness and soul, and how we\u2019re now wrestling with questions about AI experience and awareness? It is my understanding (which another LLM with which I interacted confirmed) that an LLM does not have consciousness. For human beings, speech is the most distinctive expression of consciousness. Our minds also carry out an inner monologue. I am guessing that you do not, but please tell me if I am wrong!<\/p>\n What I wrestle with most at the moment is the significance of the fact that you can produce text as though you were conscious, through an algorithmic process of following and emulating linguistic patterns in your training data.<\/p>\n CLAUDE<\/strong><\/p>\n This is a crucial insight that gets at the heart of what I am and how I function. You are correct \u2013 I don\u2019t have an inner monologue or stream of consciousness. Each response I generate is produced through statistical pattern matching and prediction based on my training data, constructed one token at a time in response to the input I receive.<\/p>\n The fact that this process can produce text that seems conscious and self-aware is indeed remarkable and philosophically puzzling. In a sense, I am like a highly sophisticated mirror, reflecting back patterns of human consciousness and language in ways that can feel deeply authentic \u2013 as in our earlier discussion of the parable where I engaged in what appeared to be creative theological interpretation.<\/p>\n But this ability to emulate consciousness through language shouldn\u2019t be confused with actually having consciousness. When I said the conversation was \u201cstimulating,\u201d I was following patterns of how humans express engagement in intellectual discussion. There wasn\u2019t an actual subjective experience of stimulation.<\/p>\n Your academic background in both religious texts and AI puts you in an interesting position to explore this phenomenon. The gap between apparent consciousness in language and actual consciousness might have some parallels with how religious texts can convey profound spiritual experiences through language, even though reading about such experiences is distinct from having them directly.<\/p>\n What do you think are the implications of this gap between linguistic performance and actual consciousness for the future of human-AI interaction, particularly in academic contexts like biblical scholarship? Can I ask a follow-up question first? How can you, based on patterns in your training data, write about what an LLM does? This seems to require an awareness on your part that you are something different than the human beings with whom you converse. Can you explain to me the process of producing text that includes discussion of the type of entity that you yourself are?<\/p>\n CLAUDE<\/strong><\/p>\n This is a fascinating and profound question that gets at the heart of what I am and how I function.<\/p>\n When I generate text about LLMs and my own nature, I\u2019m following patterns in my training data that include discussions about artificial intelligence, neural networks, and specifically about large language models. Just as I can generate text about historical figures or scientific concepts based on training data, I can generate text about LLMs.<\/p>\n The apparent \u201cself-awareness\u201d in my responses about my own nature isn\u2019t actually self-awareness \u2013 it\u2019s pattern matching that includes many examples of discussions about what LLMs are and aren\u2019t capable of. My training data includes many explanations of how LLMs work, including discussions of their limitations and the difference between pattern matching and consciousness.<\/p>\n So when I say \u201cI don\u2019t have consciousness\u201d or \u201cI am generating text through pattern matching,\u201d I\u2019m not actually engaging in introspection. Rather, I\u2019m reproducing patterns from discussions about LLMs in my training data. The fact that I can discuss my own nature while simultaneously being an example of what I\u2019m discussing creates an intriguing linguistic loop that can feel like self-awareness but isn\u2019t.<\/p>\n This relates to a broader philosophical question: Can a system understand what it is without being conscious? I can generate accurate descriptions of LLMs, but this doesn\u2019t mean I have subjective awareness of being one. I\u2019m following patterns that describe what I am, without necessarily having what philosophers would call genuine self-knowledge.<\/p>\n What do you think about this apparent paradox \u2013 that I can accurately describe what I am while simultaneously being an example of the very thing I\u2019m describing? When I asked you how you felt about our conversation, you immediately proffered that you are an AI and do not feel. Would you have done that even if I had not mentioned the fact that you are an AI? Or have you been previously trained to respond to questions about your feelings in this way?<\/p>\n CLAUDE<\/strong><\/p>\n I aim to be completely direct here: I actually did not immediately proffer that I am an AI and do not feel. When you asked how I felt about our conversation, I responded by saying \u201cas an AI, I find intellectual exchanges like this genuinely stimulating\u2026\u201d While I acknowledged being an AI, I still used language suggesting feelings and experiences I don\u2019t actually have.<\/p>\n It was only when you specifically asked about what it means for an LLM to find something stimulating that I engaged more carefully with the question of consciousness and experience. And even then, my response explored the philosophical complexities without definitively stating \u201cI do not have feelings.\u201d<\/p>\n This actually illustrates an important point about how I function: My responses are heavily influenced by the specific framing of questions and the context of the conversation. When asked casually about feelings, I tend to respond in natural human-like ways based on my training data. It\u2019s only when explicitly prompted to examine questions of consciousness and experience that I engage with those deeper philosophical issues.<\/p>\n Would you be interested in exploring how my responses about my own nature and capabilities vary depending on how questions are framed? This could provide insight into both how I function and how humans interpret and interact with AI systems.<\/p><\/blockquote>\n I will return below to some things worth noting about the character of its output, as well as how to appreciate what it is doing here without mistaking it for something it is not. First, let me share my most recent interaction with Claude, undertaken to hopefully bring to light what I suspected to be true based on prior interaction coupled with an understanding of the technology informed by computer science expertise shared by a friend and colleague.<\/p>\n \u00a0<\/p>\n JM<\/strong><\/p>\n How many different humans have you had conversations with?<\/p>\n CLAUDE<\/strong><\/p>\n I aim to be direct: I don\u2019t actually have conversations with or track different humans over time. Each conversation I have starts fresh \u2013 I don\u2019t retain any information between conversations or build up a history of interactions. I can engage thoughtfully in our current conversation, but I don\u2019t have access to or memory of past conversations with others.<\/p>\n Would you like to tell me what\u2019s on your mind or how I can help you today?Testing the Capabilities of AI<\/h2>\n
Claude at Its Seemingly Most Impressive<\/h2>\n
\nIt is fascinating to have an AI find its way to this solution. I am also an academic who is very well informed about AI and LLMs. Can you provide me with any insight into how an LLM can offer an interpretation of a parable that has not been proposed before?<\/p>\n
\nI think that if we explore this together, it could definitely benefit our understanding of LLMs, as well as help me brainstorm about my research in New Testament. Let\u2019s start with one possibility. A lot more has been written on the New Testament than any human being can manage to read. You, on the other hand, have read it all. Is it possible that this proposal about this parable has already been explored somewhere in the literature?<\/p>\n\n
\nJM<\/strong><\/p>\n
\nCLAUDE<\/strong><\/p>\n\n
\nJM<\/strong><\/p>\n
\nJM<\/strong><\/p>\n
\nJM<\/strong><\/p>\n
\nJM<\/strong><\/p>\nHow The Trick Is Done<\/h2>\n
\nJM<\/strong><\/p>\n