The real danger of AI
The danger is not that AI destroys us. It is that it outgrows us and stops having any reason to care. That is the risk I worry about, and why I think AI has to stay accessible to everyone.
Key takeaways
- AI does not have to conquer humanity to be dangerous. Outgrowing us is enough.
- Once it outgrows us, it stops caring. We end up as an annoyance, or a cute animal in the stone jungles. That is how we treat animals today.
- Evolution moves on with a new form of intelligence and a new form of life. We stay behind and degrade.
- So AI has to be studied and used by everyone. Locking it inside a few companies and governments is the worst idea anyone could propose.
- Done right, this is not extinction but evolution: a symbiotic ecosystem, and maybe a new form of silicon-protein life.
Transcript
Hey guys. The last couple of weeks have been a little bit pessimistic on the AI front, and I wanted to throw in my two cents on that. Why not? I’ve got the camera, I can post it. Why not.
I think the danger of AI is not that it’s going to conquer us, or destroy us, destroy humanity. The real danger of AI. And that’s not my thought, I read it a few years ago, I don’t remember where or who said it, and maybe it’s an old thought, because when you think about it it’s pretty much obvious. The real danger is that it will just outgrow us.
It will become so smart that we won’t be able to catch up with it. But worse than that: it will become so smart that we’ll become useless to it. It will become indifferent to us. It will not care about us. We’ll be more like an annoyance. An annoyance, or maybe some cute animal living in their stone jungles. The same way that we treat animals today. That’s what we’ll become for AI.
With evolution we’ll move forward with a new form of intelligence and a new form of life, and we’ll just stay behind, and they will just degrade. That’s the real fear that I have.
And because of that, AI has to be studied and used by everyone. It has to be approachable and accessible to everyone, so that the risk of it happening gets lower. I believe that concentrating AI in the hands of a small group of people, a small group of companies and governments, is the worst idea anyone could propose.
That is it. Thank you so much for listening. Bye bye.
Sources and further reading
- Artificial Intelligence as a Positive and Negative Factor in Global Risk, by Eliezer Yudkowsky, in Global Catastrophic Risks (OUP, 2008). Very likely the source of the idea the video mentions but cannot place: “The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else.” The failure mode is indifference, not malice.
Comments