5 ms·
I'm getting kind of sick of people who haven't read 2% of the stuff imagining what they think we've never talked about over the last 16 years. https://intellig
by Eliezer 10y ago
I'm getting kind of sick of people who haven't read 2% of the stuff imagining what they think we've never talked about over the last 16 years.
https://intelligence.org/files/IEM.pdf https://intelligence.org/files/IEM.pdf
- idlewords 10y agoReading 2% of your stuff is like reading 400% of my stuff, and I'm verbose as hell. Edit.
- devilsavocado 10y agoI haven't had time to read the entire thing, so I've skimmed over lots, but there doesn't seem to be much mention of complexity theory in the way I believe idlewords is talking about. I'm very curious: What happens if the algorithms for general artificial intelligence, and the ability for an AI to improve itself are all NP-Hard problems? Is that covered? It might be similar to the "intelligence combustion" scenario outlined. But that appears to be a scenario where do not need to be fearing superintelligence.
- Eliezer 10y agoAs it says in the paper, the evolution of hominids doesn't look like this; population genetics says that if hominid brains got larger, the marginal fitness returns on neurons went up.
- Eliezer 10y agoOr in a more intuitive sense, humans didn't seem to need brains 1000x as large to get one unit of improvement in practical effectiveness over chimpanzee cognition. But yes, this stuff is complicated, hence the paper not being two paragraphs long.
- devilsavocado 10y agoThank you, that is a very intuitive explanation. You've probably read Scott Aaronson's Why Philosophers Should Care About Computational Complexity? [0]. This seems like the perfect area to apply a lot of the questions he brings up. That's what I was looking for as I skimmed through the paper. Maybe that's what idlewords was talking about as well. [0] http://www.scottaaronson.com/papers/philos.pdf http://www.scottaaronson.com/papers/philos.pdf