3 ms·
No human brain is the near blank slate an untrained transformer is either. "We only need x tokens of ingested speech" to learn language doesn't hit the same wh
by famouswaffles 2y ago
No human brain is the near blank slate an untrained transformer is either.
"We only need x tokens of ingested speech" to learn language doesn't hit the same when you have billions of years of the brain baking in an oven to get to that point.
But i agree, it's not a direct comparison.
- Jensson 2y ago> No human brain is the near blank slate an untrained transformer is either. Why can't we start out a transformer at a better state and then teach it language in few tokens? Seems like a problem with architecture.
- famouswaffles 2y agoBecause we don't know or understand what a better state would be and so by far the most successful attempts are with architectures with as clean a slate/free of bias as possible. There's nothing wrong with blank slates. It's not a problem, whatever that means. It just is.