4 ms·

That's apples to oranges. No human ingests that many tokens of speech; individually we learn from far fewer tokens.

• famouswaffles 2y ago
  No human brain is the near blank slate an untrained transformer is either. "We only need x tokens of ingested speech" to learn language doesn't hit the same when you have billions of years of the brain baking in an oven to get to that point. But i agree, it's not a direct comparison.
  • Jensson 2y ago
    > No human brain is the near blank slate an untrained transformer is either. Why can't we start out a transformer at a better state and then teach it language in few tokens? Seems like a problem with architecture.
    • famouswaffles 2y ago
      Because we don't know or understand what a better state would be and so by far the most successful attempts are with architectures with as clean a slate/free of bias as possible. There's nothing wrong with blank slates. It's not a problem, whatever that means. It just is.