4 ms·

Because LLMs have reached a point of diminishing returns and there isn't enough compute globally to do produce a hypothetical AGI. The argument for SkyNet happ

Because LLMs have reached a point of diminishing returns and there isn't enough compute globally to do produce a hypothetical AGI. The argument for SkyNet happening is missing an important element: why would it happen? Because sci-fi says it will?

mitthrowaway2 4y ago
> The argument for SkyNet happening is missing an important element: why would it happen? Because sci-fi says it will? That element isn't missing. It's explained every time someone asks, and the answer never relies on sci-fi as evidence. The reason is because of (A) orthogonality and (B) instrumental convergence. If you aren't familiar with those terms, (A) refers to the thesis that an AI's degree of logical reasoning ability is independent of its goals. This stems from Hume's Law [1]. (B) refers to the observation that regardless of an agent's terminal goal, many instrumental goals will be predictably shared: the agent will be incentivized to increase its own capabilities, as a means toward nearly any possible end. This includes steps that would prevent humans from interfering with the AI's goals, such as a human attempting to turn it off or reprogram it. If you have questions about either of those, there are great learning materials available, including a well-made youtube series by Robert Miles. [2] > Because LLMs have reached a point of diminishing returns and there isn't enough compute globally to do produce a hypothetical AGI. What makes you think LLMs have reached a point of diminishing returns? So far, the benchmarks are continuing to show accelerating returns, to the point that it's becoming difficult for benchmarks to keep up.[3] [1]: Hume's Law, AKA the Is-Ought problem: https://en.wikipedia.org/wiki/Is%E2%80%93ought_problem https://en.wikipedia.org/wiki/Is%E2%80%93ought_problem [2]: https://www.youtube.com/watch?v=hEUO6pjwFOo https://www.youtube.com/watch?v=hEUO6pjwFOo [3]: "Foundation models and the next era of AI", Microsoft Research, https://youtu.be/HQI6O5DlyFc?t=958 https://youtu.be/HQI6O5DlyFc?t=958

mitthrowaway2 4y ago

> The argument for SkyNet happening is missing an important element: why would it happen? Because sci-fi says it will? That element isn't missing. It's explained every time someone asks, and the answer never relies on sci-fi as evidence. The reason is because of (A) orthogonality and (B) instrumental convergence. If you aren't familiar with those terms, (A) refers to the thesis that an AI's degree of logical reasoning ability is independent of its goals. This stems from Hume's Law [1]. (B) refers to the observation that regardless of an agent's terminal goal, many instrumental goals will be predictably shared: the agent will be incentivized to increase its own capabilities, as a means toward nearly any possible end. This includes steps that would prevent humans from interfering with the AI's goals, such as a human attempting to turn it off or reprogram it. If you have questions about either of those, there are great learning materials available, including a well-made youtube series by Robert Miles. [2] > Because LLMs have reached a point of diminishing returns and there isn't enough compute globally to do produce a hypothetical AGI. What makes you think LLMs have reached a point of diminishing returns? So far, the benchmarks are continuing to show accelerating returns, to the point that it's becoming difficult for benchmarks to keep up.[3] [1]: Hume's Law, AKA the Is-Ought problem: https://en.wikipedia.org/wiki/Is%E2%80%93ought_problem https://en.wikipedia.org/wiki/Is%E2%80%93ought_problem [2]: https://www.youtube.com/watch?v=hEUO6pjwFOo https://www.youtube.com/watch?v=hEUO6pjwFOo [3]: "Foundation models and the next era of AI", Microsoft Research, https://youtu.be/HQI6O5DlyFc?t=958 https://youtu.be/HQI6O5DlyFc?t=958