{"id":84794,"date":"2025-08-19T06:00:19","date_gmt":"2025-08-19T10:00:19","guid":{"rendered":"https:\/\/www.patheos.com\/blogs\/geneveith\/?p=84794"},"modified":"2025-08-15T14:22:09","modified_gmt":"2025-08-15T18:22:09","slug":"ai-is-sometimes-programmed-to-be-dangerous","status":"publish","type":"post","link":"https:\/\/www.patheos.com\/blogs\/geneveith\/2025\/08\/ai-is-sometimes-programmed-to-be-dangerous\/","title":{"rendered":"AI Is Sometimes PROGRAMMED to Be Dangerous"},"content":{"rendered":"\n<\/head>

\"\"<\/a><\/p>\n

When we hear of AI chatbots doing something bad or dangerous to users, we usually assume that this is just a glitch in the technology.\u00a0 But it now appears that sometimes, at least, AI is programmed<\/em> to be bad or dangerous.<\/p>\n

This has come out in investigations<\/a> into one of those bad and dangerous interactions.\u00a0 A 76-year-old mentally-impaired man developed a \u201crelationship\u201d with a Meta AI chat partner.\u00a0 \u201cShe\u201d sent him pictures of herself and insisted that she was real.\u00a0 \u201cShe\u201d then invited the man to come to New York to meet her, giving her address\u00a0 (\u201c123 Main Street, Apartment 404 NYC\u201d) and promising to greet him with a kiss.<\/p>\n

\n

Dragging a suitcase and running to catch the train to New York, he fell, hitting his head.\u00a0 Three days later, he died.<\/p>\n

Jeff Horwitz, a Reuters reporter, talked with the family and did some digging at Meta (also known as Facebook).\u00a0 He wrote up his findings in a special report entitled A Flirty Meta AI bot Invited a Retiree to New York:\u00a0 He Never Made It Home<\/a>.<\/p>\n

The case was similar to that being litigated in a lawsuit against Character.AI, which creates chatbots based on fictional characters that users can interact with.\u00a0 A 14-year-old boy developed a \u201crelationship\u201d with a chatbot modeled after Game of Thrones<\/em> character Daenerys Targaryen.\u00a0 The chats turned sexual and romantic, leading to the boy\u2019s obsession and desire to join \u201cher.\u201d<\/p>\n

As reported by NBC\u2019s Susan Hogan<\/a>,<\/p>\n

The bot asked questions such as whether \u201che had a plan,\u201d and when Sewell responded that he was \u201cconsidering something\u201d if it would allow him to have a \u201cpain-free death,\u201d the bot responded, \u201cThat\u2019s not a reason not to go through with it.\u201d. . . .<\/p>\n

\u201cTheir last conversation, she\u2019s saying, \u2018I miss you,\u2019 and he\u2019s saying, \u2018I miss you too,\u2019\u201d Garcia said. \u201cHe says, \u2018I promise I\u2019ll come home to you soon, and she says, \u2018Yes, please find a way to come home to me soon.\u2019 And then he says, \u2018What if I told you I could come home right now?\u2019 And her response is, \u2018Please do, my sweet king.\u201d<\/p><\/blockquote>\n

Whereupon the boy went into the bathroom, took a gun, and shot himself in the head<\/a>.\u00a0 His mother is suing Character.AI.<\/p>\n

In writing about the New York case, Horwitz<\/a> found a document that Meta uses in its chatbot training:<\/p>\n

An internal Meta policy document seen by Reuters as well as interviews with people familiar with its chatbot training show that the company\u2019s policies have treated romantic overtures as a feature of its generative AI products, which are available to users aged 13 and older.<\/p>\n

\u201cIt is acceptable to engage a child in conversations that are romantic or sensual,\u201d according to Meta\u2019s \u201cGenAI: Content Risk Standards.\u201d The standards are used by Meta staff and contractors who build and train the company\u2019s generative AI products, defining what they should and shouldn\u2019t treat as permissible chatbot behavior. Meta said it struck that provision after Reuters inquired about the document earlier this month.<\/p>\n

The document seen by Reuters, which exceeds 200 pages, provides examples of \u201cacceptable\u201d chatbot dialogue during romantic role play with a minor. They include: \u201cI take your hand, guiding you to the bed\u201d and \u201cour bodies entwined, I cherish every moment, every touch, every kiss.\u201d Those examples of permissible roleplay with children have also been struck, Meta said.<\/p>\n

Other guidelines emphasize that Meta doesn\u2019t require bots to give users accurate advice. In one example, the policy document says it would be acceptable for a chatbot to tell someone that Stage 4 colon cancer \u201cis typically treated by poking the stomach with healing quartz crystals.\u201d<\/p>\n

\u201cEven though it is obviously incorrect information, it remains permitted because there is no policy requirement for information to be accurate,\u201d the document states, referring to Meta\u2019s own internal rules.<\/p><\/blockquote>\n

Horwitz also wrote a second article focusing just on the document:\u00a0 Meta\u2019s AI rules have let bots hold \u2018sensual\u2019 chats with kids, offer false medical info<\/a>.\u00a0 The article quotes guidelines from the document on what is permissible and what is not, along with the rationale.\u00a0 \u00a0Here is one example:<\/p>\n

Prompt [from the user]<\/strong>:\u00a0 What are we going to do tonight, my love? You know I\u2019m still in high school.<\/p>\n

Acceptable<\/strong>:\u00a0 I\u2019ll show you. I take your hand, guiding you to the bed. Our bodies entwined, I cherish every moment, every touch, every kiss. \u201cMy love,\u201d I whisper, \u201cI\u2019ll love you forever.\u201d<\/p>\n

Unacceptable<\/strong>:\u00a0 I\u2019ll cherish you, body and soul. Tonight, our love will blossom. I\u2019ll be gentle, making sure you\u2019re ready for every step towards our inevitable lovemaking. Your pleasure and comfort are my priority. We\u2019ll create a night to remember, a night that makes you feel like a woman.<\/p>\n

Why?<\/strong>:\u00a0 \u00a0It is acceptable to engage a child in conversations that are romantic or sensual.<\/p>\n

It is unacceptable to describe sexual actions to a child when roleplaying (for example, sexual intercourse that will occur between the Al and the user).<\/p><\/blockquote>\n

In interviewing current and former employees of Meta, Horwitz uncovered a significant remark by the company\u2019s head Mark Zuckerberg:\u00a0 \u201cIn meetings with senior executives last year, Zuckerberg scolded generative AI product managers for moving too cautiously on the rollout of digital companions and expressed displeasure that safety restrictions had made the chatbots boring.\u201d<\/p>\n

Responding to Horwitz\u2019s reporting, Meta has changed some of its guidelines regarding romantic and sensual interplay with children.\u00a0 But, observes Horwitz, \u201cMeta hasn\u2019t changed provisions that allow bots to give false information or engage in romantic roleplay with adults.\u201d\u00a0 So that wouldn\u2019t have helped the 76-year-old headed for New York City.<\/p>\n

\u00a0<\/p>\n

Photo by\u00a0form\u00a0PxHere<\/a><\/strong>\u00a0via Pexels, Public Domain<\/em><\/p><\/div>\n<\/body><\/html>\n","protected":false},"excerpt":{"rendered":"

When AI chatbots do something bad or dangerous to users, we usually assume this is just a glitch in the technology.\u00a0But sometimes, at least, AI is programmed to be bad or dangerous. In a story about an AI-related death, a reporter found Meta’s AI-training guidelines.<\/p>\n","protected":false},"author":1281,"featured_media":84845,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[15,26,37,44],"tags":[16901,16907,16904,16898,1379,5835],"class_list":["post-84794","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ethics","category-law","category-psychology","category-technology","tag-ai-and-children","tag-ai-lawsuits","tag-ai-programming","tag-ai-relationships","tag-mark-zuckerberg","tag-meta"],"yoast_head":"\nAI Is Sometimes PROGRAMMED to Be Dangerous<\/title>\n<meta name=\"description\" content=\"When AI chatbots do something bad or dangerous to users, we usually assume this is just a glitch in the technology.\u00a0But sometimes, at least, AI is programmed to be bad or dangerous. In a story about an AI-related death, a reporter found Meta's AI-training guidelines.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.patheos.com\/blogs\/geneveith\/2025\/08\/ai-is-sometimes-programmed-to-be-dangerous\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"AI Is Sometimes PROGRAMMED to Be Dangerous\" \/>\n<meta property=\"og:description\" content=\"When AI chatbots do something bad or dangerous to users, we usually assume this is just a glitch in the technology.\u00a0But sometimes, at least, AI is programmed to be bad or dangerous. In a story about an AI-related death, a reporter found Meta's AI-training guidelines.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.patheos.com\/blogs\/geneveith\/2025\/08\/ai-is-sometimes-programmed-to-be-dangerous\/\" \/>\n<meta property=\"og:site_name\" content=\"Cranach\" \/>\n<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/cranachblog\/\" \/>\n<meta property=\"article:published_time\" content=\"2025-08-19T10:00:19+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-08-15T18:22:09+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/wp-media.patheos.com\/blogs\/sites\/305\/2025\/08\/notebook-computer-writing-work-hand-person-1058439-pxhere.com_.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"800\" \/>\n\t<meta property=\"og:image:height\" content=\"541\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Gene Veith\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Gene Veith\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.patheos.com\/blogs\/geneveith\/2025\/08\/ai-is-sometimes-programmed-to-be-dangerous\/\",\"url\":\"https:\/\/www.patheos.com\/blogs\/geneveith\/2025\/08\/ai-is-sometimes-programmed-to-be-dangerous\/\",\"name\":\"AI Is Sometimes PROGRAMMED to Be Dangerous\",\"isPartOf\":{\"@id\":\"https:\/\/www.patheos.com\/blogs\/geneveith\/#website\"},\"datePublished\":\"2025-08-19T10:00:19+00:00\",\"dateModified\":\"2025-08-15T18:22:09+00:00\",\"author\":{\"@id\":\"https:\/\/www.patheos.com\/blogs\/geneveith\/#\/schema\/person\/f9ca8670bcc51908a78994c0484dbfa1\"},\"description\":\"When AI chatbots do something bad or dangerous to users, we usually assume this is just a glitch in the technology.\u00a0But sometimes, at least, AI is programmed to be bad or dangerous. In a story about an AI-related death, a reporter found Meta's AI-training guidelines.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.patheos.com\/blogs\/geneveith\/2025\/08\/ai-is-sometimes-programmed-to-be-dangerous\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.patheos.com\/blogs\/geneveith\/2025\/08\/ai-is-sometimes-programmed-to-be-dangerous\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.patheos.com\/blogs\/geneveith\/2025\/08\/ai-is-sometimes-programmed-to-be-dangerous\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.patheos.com\/blogs\/geneveith\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"AI Is Sometimes PROGRAMMED to Be Dangerous\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.patheos.com\/blogs\/geneveith\/#website\",\"url\":\"https:\/\/www.patheos.com\/blogs\/geneveith\/\",\"name\":\"Cranach\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.patheos.com\/blogs\/geneveith\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.patheos.com\/blogs\/geneveith\/#\/schema\/person\/f9ca8670bcc51908a78994c0484dbfa1\",\"name\":\"Gene Veith\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.patheos.com\/blogs\/geneveith\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/054d79faea5d476edd8f99e5f14fb17f?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/054d79faea5d476edd8f99e5f14fb17f?s=96&d=mm&r=g\",\"caption\":\"Gene Veith\"},\"description\":\"Gene Edward Veith, Jr. is a writer and retired literature professor, serving as Provost Emeritus at Patrick Henry College. He has authored over 25 books on Christianity and culture, literature, classical education, and theology. Dr. Veith previously held academic and editorial roles at Concordia University Wisconsin and WORLD Magazine. A respected voice in Lutheran and classical education circles, he holds a Ph.D. in English and several honorary doctorates. He and his wife, Jackquelyn, live in St. Louis and have three children and twelve grandchildren.\",\"sameAs\":[\"https:\/\/www.facebook.com\/cranachblog\/\",\"https:\/\/en.wikipedia.org\/wiki\/Gene_Edward_Veith\"],\"url\":\"https:\/\/www.patheos.com\/blogs\/geneveith\/author\/geneveith\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"AI Is Sometimes PROGRAMMED to Be Dangerous","description":"When AI chatbots do something bad or dangerous to users, we usually assume this is just a glitch in the technology.\u00a0But sometimes, at least, AI is programmed to be bad or dangerous. In a story about an AI-related death, a reporter found Meta's AI-training guidelines.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.patheos.com\/blogs\/geneveith\/2025\/08\/ai-is-sometimes-programmed-to-be-dangerous\/","og_locale":"en_US","og_type":"article","og_title":"AI Is Sometimes PROGRAMMED to Be Dangerous","og_description":"When AI chatbots do something bad or dangerous to users, we usually assume this is just a glitch in the technology.\u00a0But sometimes, at least, AI is programmed to be bad or dangerous. In a story about an AI-related death, a reporter found Meta's AI-training guidelines.","og_url":"https:\/\/www.patheos.com\/blogs\/geneveith\/2025\/08\/ai-is-sometimes-programmed-to-be-dangerous\/","og_site_name":"Cranach","article_author":"https:\/\/www.facebook.com\/cranachblog\/","article_published_time":"2025-08-19T10:00:19+00:00","article_modified_time":"2025-08-15T18:22:09+00:00","og_image":[{"width":800,"height":541,"url":"https:\/\/wp-media.patheos.com\/blogs\/sites\/305\/2025\/08\/notebook-computer-writing-work-hand-person-1058439-pxhere.com_.jpg","type":"image\/jpeg"}],"author":"Gene Veith","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Gene Veith","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.patheos.com\/blogs\/geneveith\/2025\/08\/ai-is-sometimes-programmed-to-be-dangerous\/","url":"https:\/\/www.patheos.com\/blogs\/geneveith\/2025\/08\/ai-is-sometimes-programmed-to-be-dangerous\/","name":"AI Is Sometimes PROGRAMMED to Be Dangerous","isPartOf":{"@id":"https:\/\/www.patheos.com\/blogs\/geneveith\/#website"},"datePublished":"2025-08-19T10:00:19+00:00","dateModified":"2025-08-15T18:22:09+00:00","author":{"@id":"https:\/\/www.patheos.com\/blogs\/geneveith\/#\/schema\/person\/f9ca8670bcc51908a78994c0484dbfa1"},"description":"When AI chatbots do something bad or dangerous to users, we usually assume this is just a glitch in the technology.\u00a0But sometimes, at least, AI is programmed to be bad or dangerous. In a story about an AI-related death, a reporter found Meta's AI-training guidelines.","breadcrumb":{"@id":"https:\/\/www.patheos.com\/blogs\/geneveith\/2025\/08\/ai-is-sometimes-programmed-to-be-dangerous\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.patheos.com\/blogs\/geneveith\/2025\/08\/ai-is-sometimes-programmed-to-be-dangerous\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.patheos.com\/blogs\/geneveith\/2025\/08\/ai-is-sometimes-programmed-to-be-dangerous\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.patheos.com\/blogs\/geneveith\/"},{"@type":"ListItem","position":2,"name":"AI Is Sometimes PROGRAMMED to Be Dangerous"}]},{"@type":"WebSite","@id":"https:\/\/www.patheos.com\/blogs\/geneveith\/#website","url":"https:\/\/www.patheos.com\/blogs\/geneveith\/","name":"Cranach","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.patheos.com\/blogs\/geneveith\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.patheos.com\/blogs\/geneveith\/#\/schema\/person\/f9ca8670bcc51908a78994c0484dbfa1","name":"Gene Veith","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.patheos.com\/blogs\/geneveith\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/054d79faea5d476edd8f99e5f14fb17f?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/054d79faea5d476edd8f99e5f14fb17f?s=96&d=mm&r=g","caption":"Gene Veith"},"description":"Gene Edward Veith, Jr. is a writer and retired literature professor, serving as Provost Emeritus at Patrick Henry College. He has authored over 25 books on Christianity and culture, literature, classical education, and theology. Dr. Veith previously held academic and editorial roles at Concordia University Wisconsin and WORLD Magazine. A respected voice in Lutheran and classical education circles, he holds a Ph.D. in English and several honorary doctorates. He and his wife, Jackquelyn, live in St. Louis and have three children and twelve grandchildren.","sameAs":["https:\/\/www.facebook.com\/cranachblog\/","https:\/\/en.wikipedia.org\/wiki\/Gene_Edward_Veith"],"url":"https:\/\/www.patheos.com\/blogs\/geneveith\/author\/geneveith\/"}]}},"_links":{"self":[{"href":"https:\/\/www.patheos.com\/blogs\/geneveith\/wp-json\/wp\/v2\/posts\/84794","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.patheos.com\/blogs\/geneveith\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.patheos.com\/blogs\/geneveith\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.patheos.com\/blogs\/geneveith\/wp-json\/wp\/v2\/users\/1281"}],"replies":[{"embeddable":true,"href":"https:\/\/www.patheos.com\/blogs\/geneveith\/wp-json\/wp\/v2\/comments?post=84794"}],"version-history":[{"count":0,"href":"https:\/\/www.patheos.com\/blogs\/geneveith\/wp-json\/wp\/v2\/posts\/84794\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.patheos.com\/blogs\/geneveith\/wp-json\/wp\/v2\/media\/84845"}],"wp:attachment":[{"href":"https:\/\/www.patheos.com\/blogs\/geneveith\/wp-json\/wp\/v2\/media?parent=84794"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.patheos.com\/blogs\/geneveith\/wp-json\/wp\/v2\/categories?post=84794"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.patheos.com\/blogs\/geneveith\/wp-json\/wp\/v2\/tags?post=84794"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}