4 ms·
tl;dr - They model operations (e.g. multiply, add, cosine, sqrt) as tokens, and use a transformer to predict a recursive function from (incomplete) integer sequ
by ShamelessC 5y ago
tl;dr - They model operations (e.g. multiply, add, cosine, sqrt) as tokens, and use a transformer to predict a recursive function from (incomplete) integer sequences (and float sequences, separately).
This is not dissimilar from e.g. DALL-E where they model images as NLP-style tokens and use a transformer to predict image tokens (operations, respectively) from text tokens (list of ints/floats, respectively).
This demo lets you interact with the model. Pretty cool stuff. Refreshing to see symbolic problems being tackled with deep learning.
Yannic Kilcher video with one of the authors:
https://youtu.be/1HEdXwEYrGM https://youtu.be/1HEdXwEYrGM
arxiv:
https://arxiv.org/abs/2201.04600 https://arxiv.org/abs/2201.04600
- verdverm 5y agoIf you find this interesting, my phd work is an interesting read https://seminars.math.binghamton.edu/ComboSem/worm-chiu.pge_gecco2013.pdf https://seminars.math.binghamton.edu/ComboSem/worm-chiu.pge_... It is a deterministic algo for SR that I've always thought would benefit from RL. The biggest problem is SR research has been and still is the lack of a widely used dataset and reproducible results. There is a researcher who beats everyone's results within a year by publishing to Springer without code or data...
- verdverm 5y agoFast Function extraction is another interesting Algo for SR http://trent.st/ffx/ http://trent.st/ffx/