3 ms·
It is a goalpost that is easy to move. By "works" I mean learning from a continuous single (meaning batch-1) stream of data. The fact that it produces full word
by volotat 12d ago
It is a goalpost that is easy to move. By "works" I mean learning from a continuous single (meaning batch-1) stream of data. The fact that it produces full words and full coherent phrases instead of a random stream of characters that would any typical LM produce if trained under the same training regime.
- ilaksh 12d agoI would be okay if you shared it as a potential idea and possibly interesting early result, but the language you are actually using to characterize it is misleading or delusional. Please get a model to the point where it seems like it has some natural language understanding and then share again with reasonable characterization.
- volotat 12d agoFor sure. As it will pass through the whole corpus I will share the weights, run it through established benchmarks for small models and share all of this as an update. I am also planning on making a Youtube video explaining in detail how it works on a deeper level and the whole reasoning behind why it is built the way it is. But no promises here.
- deleted 12d ago[deleted]
- bigbadfeline 12d agoIs "AGI" the language that bothers you? Well, one has to keep his eyes on the prize and I see a bright idea which could lead to AGI, so, why not describe it as such? I also see the inspiration and hard work necessary to move that idea further along, so fingers crossed.