volotat

4 ms·

1.
by volotat 5d ago
I got you, will add later to the repository.
2.
by volotat 5d ago
For sure. As it will pass through the whole corpus I will share the weights, run it through established benchmarks for small models and share all of this as an update. I am also planning on making a Youtube video explaining in detail how it
3.
by volotat 5d ago
I have no practical means of scaling it up at any compatible scale. I will not make any money on it either way as well. So there is absolutely no reason for hoarding it. And as I said I did use Claude in the process, so Anthropic already ha
4.
by volotat 5d ago
It is a goalpost that is easy to move. By "works" I mean learning from a continuous single (meaning batch-1) stream of data. The fact that it produces full words and full coherent phrases instead of a random stream of characters t
5.
by volotat 5d ago
There is no special algorithm, the finding is that slowing down the LR or the trunk, while keeping the LR of the experts is enough to eliminate most of the forgetting in the network. You can see in that experiment where chess data was the o
6.
by volotat 5d ago
The held-out scores reported in the Readme IS the unseen dataset.
7.
by volotat 5d ago
It interleaves random streams of 32K characters long each when reading the whole corpus, but each such stream reads continuously as you would expect. This is a necessary step to prevent just normal, not catastrophic, forgetting. I have not
8.
by volotat 5d ago
The model is way too small and undertrained to make any generalization claims. I want to wait until it reads the whole corpus I gave and then test it on some simple established benchmarks to see how it will behave.
9.
by volotat 5d ago
I also like how it is very organic. It naturally grows and deletes unused elements, so in addition to traditional backprop there is also a natural selection happening in the background. Each new expert has 16 parents by the way, lol.
10.
Show HN: Mini-AGI – Dynamic continual learning model trained on 8GB VRAM(github.com)
235 points by volotat 5d ago|51 comments
11.
Show HN: Anagnorisis – local recommendation engine (v0.4.10 update)(github.com)
3 points by volotat 1mo ago|0 comments
12.
by volotat 8mo ago
I built this project as an alternative to cloud-based media services. The system uses embedding models (CLAP for audio, SigLIP for images, Jina v3 for text) to enable semantic search across your local files. You can search for "relaxin
13.
Show HN: AI coding assistant that helps to build things, not to ruin them(github.com)
2 points by volotat 1y ago|0 comments
14.
Can humans speak the language of machines?(medium.com)
1 points by volotat 1y ago|0 comments
15.
Show HN: Anagnorisis. A Vision for Better Information Management(medium.com)
2 points by volotat 1y ago|0 comments
16.
Show HN: Anagnorisis, local data-management with trainable recommendation engine(github.com)
2 points by volotat 1y ago|0 comments