What papers you should read on AI risk and related issues

In this comment thread, there was one point where I ended up just quoting Eliezer Yudkowsky at length, and more points where I was tempted to do so. But I think it would be more productive to write a post telling people what papers they should read if they have a serious interest in this stuff. Here’s my list, which is similar but not identical to lists that other people like Luke Muehlhauser might give:

  1. Eliezer Yudkowsky. 2008. Artificial Intelligence as a Positive and Negative Factor in Global Risk. I don’t agree with Eliezer on everything, but at the very least if you’re going to criticize his views you should read the main academic paper where he lays them out.
  2. Luke Muehlhauser and Anna Salamon. 2012. Intelligence Explosion: Evidence and Import.  One thing that really impresses me about Luke is that he seems to have mastered the art of identifying claims that are relatively non-contentious but also very important if true. I agree with virtually everything in this paper, aside from a few nitpicks. One of the best parts of this paper is the discussion of the advantages digital minds would have over meat minds.
  3. Kaj Sotala. 2012. Advantages of Artificial Intelligences, Uploads, and Digital Minds. The title is self-explanatory. I’m including a second paper with this title just because I think the issue is so important.
  4. Nick Bostrom. 2012. The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents. A more rigorous explanation of why a superintelligent AI is potentially dangerous. TLDR; intelligence doesn’t imply benevolence, and it may not imply malevolence as an end goal, but it does imply certain instrumental goals unless they would conflict with its other goals.
  5. Anders Sandberg and Nick Bostrom. 2008. Whole Brain Emulation: A Roadmap. This is the thing that originally made me go “oh crap, this technology isn’t just theoretically possible, it could happen this century.”
  • Alexander Kruel

    A concise introduction (from first principles (only moderate assumptions (no presuppositions about the workings or motives of AI))): A Primer On Risks From AI

  • Darren

    Thanks for the usefull links, some I had seen, some not.

  • Pingback: yellow october


CLOSE | X

HIDE | X