4 ms·

It's not wrong, but I find the underlying corrolay pretty creepy that actually trying to understand those problems and fix errors at edge cases is also a fool's

by xg15 2y ago

It's not wrong, but I find the underlying corrolay pretty creepy that actually trying to understand those problems and fix errors at edge cases is also a fool's errand, because why try to understand a specific behavior if you can just (try to) finetune it away? So we'll have to get used for good to a future where AI is unpredictable, usually does what you want, but has a 0.1% chance of randomly going haywire and no one will know how to fix it? Also, the focus on hardware seems to imply that it's strictly a game of capital - who has access to the most compute resources wins, the others can stop trying. Wouldn't this lead to massive centralization?

latentsea 2y ago
>So we'll have to get used for good to a future where AI is unpredictable, usually does what you want, but has a 0.1% chance of randomly going haywire and no one will know how to fix it? Just like humans. I don't think is a solvable problem either.
- layer8 2y ago
  It’s not “just like”, because humans can be held accountable. Also, I suspect that the distribution of failure modes is actually substantially different between LLMs and humans.
  - Androider 2y ago
    Add more humans and LLMs to correct for errors. If humans sometimes go crazy and try to randomly end the world at a rate of 0.1%, requiring two humans to turn two keys synchronously to end the world reduces the error rate to 0.01%. So, to avoid depressed AIs ending the world randomly, have a stable of multiple AIs with different provenance (one from Anthropic, one from OpenAI, one from Google...) require a majority agreement to reduce the error rate. Adjust thresholds depending on criticality of the task at hand.
    - qbit42 2y ago
      Let's hope they're not correlated.
  - latentsea 2y ago
    Late to reply, but my point was that even with the ability to be held accountable on pain of unemployment, imprisonment or even death, sometimes even these disincentives are not enough to stop a certain small percentage of humans from going haywire. It's all just something we have to live with.

by xg15 2y ago

latentsea 2y ago

>So we'll have to get used for good to a future where AI is unpredictable, usually does what you want, but has a 0.1% chance of randomly going haywire and no one will know how to fix it? Just like humans. I don't think is a solvable problem either.

layer8 2y ago
It’s not “just like”, because humans can be held accountable. Also, I suspect that the distribution of failure modes is actually substantially different between LLMs and humans.
- Androider 2y ago
  Add more humans and LLMs to correct for errors. If humans sometimes go crazy and try to randomly end the world at a rate of 0.1%, requiring two humans to turn two keys synchronously to end the world reduces the error rate to 0.01%. So, to avoid depressed AIs ending the world randomly, have a stable of multiple AIs with different provenance (one from Anthropic, one from OpenAI, one from Google...) require a majority agreement to reduce the error rate. Adjust thresholds depending on criticality of the task at hand.
  - qbit42 2y ago
    Let's hope they're not correlated.
- latentsea 2y ago
  Late to reply, but my point was that even with the ability to be held accountable on pain of unemployment, imprisonment or even death, sometimes even these disincentives are not enough to stop a certain small percentage of humans from going haywire. It's all just something we have to live with.