3 ms·
> No human brain is the near blank slate an untrained transformer is either. Why can't we start out a transformer at a better state and then teach it language
by Jensson 2y ago
> No human brain is the near blank slate an untrained transformer is either.
Why can't we start out a transformer at a better state and then teach it language in few tokens? Seems like a problem with architecture.
- famouswaffles 2y agoBecause we don't know or understand what a better state would be and so by far the most successful attempts are with architectures with as clean a slate/free of bias as possible. There's nothing wrong with blank slates. It's not a problem, whatever that means. It just is.