4 ms·
> we don't have any good idea how this tech works. Do you mean the specifics of GPT-4, or Transformers in general ?
by qorrect 4y ago
> we don't have any good idea how this tech works.
Do you mean the specifics of GPT-4, or Transformers in general ?
- MacsHeadroom 4y agoPresumably he means all types of machine learning in general.
- famouswaffles 4y agohe's likely talking bout the internals. sure we know how to train them but nobody knows what the models learn exactly. how those billions of parameters shape the output on inference. a few months ago, just this year some researchers discovered what might be the neuron that largely decides when to use an in gpt-2. yes 2. that's what he means. https://clementneo.com/posts/2023/02/11/we-found-an-neuron https://clementneo.com/posts/2023/02/11/we-found-an-neuron