4 ms·
This really makes me think if it would be feasible to make an llm trained exclusively on toki pona (https://en.wikipedia.org/wiki/Toki_Pona https://en.wikipedia
by algoth1 6mo ago
This really makes me think if it would be feasible to make an llm trained exclusively on toki pona (https://en.wikipedia.org/wiki/Toki_Pona https://en.wikipedia.org/wiki/Toki_Pona)
- MarkusQ 6mo agoThere isn't enough training data though, is there? The "secret sauce" of LLMs is the vast amount of training data available + the compute to process it all.
- algoth1 6mo agoI think you could probably feed a copy of a toki pona grammar book to a big model, and have it produce ‘infinite’ training data
- eden-u4 6mo agoThere are not enough samples in that book to generate new "infinite" data.
- MarkusQ 6mo agoThis is essentially a distillation on the bigger model; you'd wind up surfacing a lot of artifacts from the host model, amplifying them in the same way repeated photocopying introduces errors. https://dailyai.com/2025/05/create-a-replica-of-this-image-dont-change-anything-ai-trend-takes-off/ https://dailyai.com/2025/05/create-a-replica-of-this-image-d...
- mudkipdev 6mo agoPeople have made toki pona translation models before, not exclusively trained though