3 ms·

> Automating AI research is terrifying. what why?

• kridsdale1 2mo ago
  Skynet.
• reasonableklout 2mo ago
  It depends what they mean by automating. But the classic argument goes: * Each new generation of models has emergent capabilities we did not anticipate. * We already have trouble monitoring and controlling the current generation (see HuggingFace incident). * The more we let models shape their successors, the more out-of-distribution each generation's learning environment becomes. * If not done carefully, we risk creating extraordinarily intelligent and powerful models with unintended behaviors, like deceptiveness or power-seeking.