What papers you should read on AI risk and related issues

In this comment thread, there was one point where I ended up just quoting Eliezer Yudkowsky at length, and more points where I was tempted to do so. But I think it would be more productive to write a post telling people what papers they should read if they have a serious interest in this stuff. Here’s my list, which is similar but not identical to lists that other people like Luke Muehlhauser might give:

  1. Eliezer Yudkowsky. 2008. Artificial Intelligence as a Positive and Negative Factor in Global Risk. I don’t agree with Eliezer on everything, but at the very least if you’re going to criticize his views you should read the main academic paper where he lays them out.
  2. Luke Muehlhauser and Anna Salamon. 2012. Intelligence Explosion: Evidence and Import.  One thing that really impresses me about Luke is that he seems to have mastered the art of identifying claims that are relatively non-contentious but also very important if true. I agree with virtually everything in this paper, aside from a few nitpicks. One of the best parts of this paper is the discussion of the advantages digital minds would have over meat minds.
  3. Kaj Sotala. 2012. Advantages of Artificial Intelligences, Uploads, and Digital Minds. The title is self-explanatory. I’m including a second paper with this title just because I think the issue is so important.
  4. Nick Bostrom. 2012. The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents. A more rigorous explanation of why a superintelligent AI is potentially dangerous. TLDR; intelligence doesn’t imply benevolence, and it may not imply malevolence as an end goal, but it does imply certain instrumental goals unless they would conflict with its other goals.
  5. Anders Sandberg and Nick Bostrom. 2008. Whole Brain Emulation: A Roadmap. This is the thing that originally made me go “oh crap, this technology isn’t just theoretically possible, it could happen this century.”
"Atomsk - Yes, I think the way I feel about it is normal. I think ..."

Let’s talk about violent pornography
"The Scientific Method works by testing a hypothesis for implications, contradictions, and ridiculous/false results. You ..."

Pulling some devastating punches: a review ..."
"A bit OT: Found this article and it is imo closely related to the issue ..."

Let’s talk about violent pornography
"Just one thing for now, because it takes quite a bit of time to think ..."

Let’s talk about violent pornography

Browse Our Archives

What Are Your Thoughts?leave a comment