5 ms·
Thanks for reading! Absolutely, I included a few references that explore that approach at the bottom of section 4 (last two paragraphs).
by benanne 2y ago
Thanks for reading! Absolutely, I included a few references that explore that approach at the bottom of section 4 (last two paragraphs).
- magicalhippo 2y agoExcellent, thanks, will check them out. Had just finished watching the Physics of Language Models[1] talk, where they show how GPT2 models could learn non-trivial context-free grammars, as well as effectively do dynamic programming to an extent, so though it would be interesting to see how they performed in the spectral fine-graining task. [1]: https://physics.allen-zhu.com/home https://physics.allen-zhu.com/home
- magicalhippo 2y ago> I included a few references that explore that approach at the bottom of section 4 Man, reading on mobile phone just ain't the same. Somehow managed to not catch then end of that section. The first reference, "Generating Images with Sparse Representations", is very close to what I had in mind.