I guess I should explain the “thing that made me want to scream” in Luke’s Facing the Intelligence Explosion. It’s when he’s listing possible features of a utopia, and “no pain” is first on the list. “Imagine,” he says, “a life without pain.”
I can imagine it, and it sounds pretty terrible. It would mean no pushing yourself physically (during sports, exercise, whatever) past the point where you want to quit because of the pain. No working hard and then having a bit of soreness as a reminder of what you’ve accomplished. No S&M.
That these are desirable things, or at least things some people want, could be seen as an instance of a point Luke makes earlier in that same chapter:
Citizens of utopia are not mindless drones who never use their brains to solve anything. Just as making a video game easier does not always make it more fun, citizens of utopia still face challenges and experience the joy it is to overcome them.
Now, anesthetics are great, and to be fair to Luke, he links to an essay by Ben Goertzel which is slightly more nuanced. Goertzel writes:
It’s quite possible that the total abolition of ouchness is not ideal, from a psychological or cultural perspective. But it’s very hard for me to believe that evolution has supplied us with a level of ouchness that is optimal from our contemporary and future perspectives. There’s just too much unnecessary and destructive pain around. Giving adult humans and AGIs conscious control over their level of ouchness is obviously the compassionate and ethical thing to do — and it’s also good mind design. How fortunate that science and engineering are likely to make this possible, and in the not extremely distant future!
But at the same time, he doesn’t seem to entirely get why anyone would choose not to alter their experience of pain much, beyond the cases we already use anesthesia for. He doesn’t understand why anyone would want to experience pain any more often than “maybe now and then just for novelty!” (in his words).
This seems like a pretty good example of why would-be utopians should proceed with extraordinary care; what seems not very important to one person, even a perfectly well intentioned person, is sometimes extraordinarily important to anther.