https://www.quantamagazine.org/artificial-intelligence-will-do-what-we-ask-thats-a-problem-20200130/
artificial intelligence
Artificial Intelligence Will Do What We Ask. That’s a Problem.
Natalie Wolchover
Senior Writer/Editor
January 30, 2020
By teaching machines to understand our true desires, one scientist hopes to avoid the potentially disastrous consequences of having them do what we command.
“A major aspect of the problem is that humans often don’t know what goals to give our AI systems, because we don’t know what we really want…. Instead of machines pursuing goals of their own, the new thinking goes, they should seek to satisfy human preferences; their only goal should be to learn more about what our preferences are.”