3 ms·

Can someone provide a layman's explanation?

• periheli0n 5y ago
  A transformer network is a variant of associative memory. You give it a query and it returns a value that it has learned to associate with the query during training.
  • svcrunch 5y ago
    > You give it a query and it returns a value that it has learned to associate with the query during training. The zero-shot scenario they describe does not work like this. They explicitly mention that it's not trained with any queries (which is what makes it a very promising technique).
    • periheli0n 5y ago
      Sorry for the imprecise phrasing. During training, patterns are stored in the network parameters. The query will naturally be similar to one or several of the patterns stored, which are then returned (in hopfield-speak "completed").