3 ms·

Does anyone know how this fits with (or not) Chomsky's ideas of language processing?

• convolvatron 4y ago
  the idea that some linguistic facilities are innate? or the government binding model of grammar or something else? for the first two, I think this orthogonal
• IIAOPSW 4y ago
  |-----long timing loop / top of parse tree-----| | | |-shorter / child node -| |-shorter / child node-| | | | | |highest freq| |highest freq| |highest freq| |highest freq|
  • politician 4y ago
    Similarly, ChatGPT (gpt2-medium) has 12 heads of attention. GPT-3 has 96.
    • IIAOPSW 4y ago
      I don't know how superimposed waves in finely tuned timing loops with non-linear interference translates into heads of attention, and honestly I suspect a lot of the things difficult to do with heads of attention (and other approaches of the past) come for free in a resonance based system.
• YeGoblynQueenne 4y ago
  What are Chomsky's ideas on language processing? He's a linguist, not an NLP person.