3 ms·

What’s your evidence that scaling won’t improve AI?

• janalsncm 2y ago
  The question isn’t whether scaling will improve AI. The question is whether the return is worth it. You can build a bigger pogo stick to jump higher, but no pogo stick will get you to the moon. Chess is a pretty good example. You could theoretically train an LLM on just chess games. The problem is there are more chess positions than atoms in the universe. So you can’t actually do it in practice. And chess is a much more constrained environment than life. At any chess position there are only ~35 moves on average. Life has tons of long-tail situations which have never been seen before. And for chess we already have superhuman intelligence. It doesn’t require trillion-dollar training clusters, you can run a superhuman chess bot on your phone. So there are clear questions of optimality as well: VC money should be aware of the opportunity cost in investing money under “infinite scaling” assumptions.