3 ms·

There used to be a subreddit called Sneerclub where people would make fun of Eliezer and some of his buddies. Here's a discussion of a basic factual error he m

There used to be a subreddit called Sneerclub where people would make fun of Eliezer and some of his buddies. Here's a discussion of a basic factual error he made on how training AI works, even though this topic is supposedly his life's work: https://www.reddit.com/r/SneerClub/comments/131rfg0/ey_gets_sneered_on_by_one_of_the_writers_of_the/?utm_source=share&utm_medium=mweb3x&utm_name=mweb3xcss&utm_term=1&utm_content=share_button https://www.reddit.com/r/SneerClub/comments/131rfg0/ey_gets_... I enjoyed the comment that his understanding of how AI training works is like "thinking that you need to be extremely careful when solving the equations for designing a nuclear bomb, because if you solve them too quickly then they'll literally explode."

0xDEAFBEAD 3y ago
Read the mesa-optimization paper I linked elsewhere in this thread: https://arxiv.org/pdf/1906.01820.pdf https://arxiv.org/pdf/1906.01820.pdf Eliezer's point is that if AI researchers aren't looking for anomalous behavior that could indicate a potential danger, they won't find it.
- staticman2 3y ago
  The issue isn't whether "his point" as you put it is correct. If I said people should safety test the space shuttle to make sure the phlogiston isn't going to overheat, I may be correct in my belief that people should "safety test" the space shuttle but I'm still a crank because phlogeston isn't a real thing.
  - 0xDEAFBEAD 3y ago
    See my comment here: https://news.ycombinator.com/item?id=38336374 https://news.ycombinator.com/item?id=38336374 The reason AI alignment is challenging is because we're trying to make accurate predictions about unusual scenarios that we have essentially zero data about. No one can credibly claim expertise on what would constitute evidence of a worrisome anomaly. Jeremy Howard can't credibly say that a sudden drop in the loss function is certainly nothing to worry about, because the entire idea is to think about exotic situations that don't arise in the course of ordinary machine learning work. And the "loss" vs "loss function" thing is just silly gatekeeping, I worked in ML for years -- serious people generally don't care about minor terminology stuff like that.
    - staticman2 3y ago
      That's not what the conversation was about- you're just doing the thing Howard said where you squint and imagine he was saying something other than he did.

0xDEAFBEAD 3y ago

Read the mesa-optimization paper I linked elsewhere in this thread: https://arxiv.org/pdf/1906.01820.pdf https://arxiv.org/pdf/1906.01820.pdf Eliezer's point is that if AI researchers aren't looking for anomalous behavior that could indicate a potential danger, they won't find it.

staticman2 3y ago
The issue isn't whether "his point" as you put it is correct. If I said people should safety test the space shuttle to make sure the phlogiston isn't going to overheat, I may be correct in my belief that people should "safety test" the space shuttle but I'm still a crank because phlogeston isn't a real thing.
- 0xDEAFBEAD 3y ago
  See my comment here: https://news.ycombinator.com/item?id=38336374 https://news.ycombinator.com/item?id=38336374 The reason AI alignment is challenging is because we're trying to make accurate predictions about unusual scenarios that we have essentially zero data about. No one can credibly claim expertise on what would constitute evidence of a worrisome anomaly. Jeremy Howard can't credibly say that a sudden drop in the loss function is certainly nothing to worry about, because the entire idea is to think about exotic situations that don't arise in the course of ordinary machine learning work. And the "loss" vs "loss function" thing is just silly gatekeeping, I worked in ML for years -- serious people generally don't care about minor terminology stuff like that.
  - staticman2 3y ago
    That's not what the conversation was about- you're just doing the thing Howard said where you squint and imagine he was saying something other than he did.