3 ms·
I've been working on an ML model capable of robust continuous learning, resistant to catastrophic forgetting without relying on replay, an external memory syste
by jballanc 6mo ago
I've been working on an ML model capable of robust continuous learning, resistant to catastrophic forgetting without relying on replay, an external memory system, or unbounded parameter growth. Last week I confirmed the first non-toy, 580M parameter version soundly beat LoRA, EWC, and full fine tuning. This week I'm scaling up to 4.4B parameters...
- Findeton 6mo agoDo you have a public repo for that? I'm also trying to do that although I'm using "replay"/distillation and hopfield memory banks.
- jballanc 6mo agoNo public repo yet, but coming soon. Just filed for a patent on the technique and am preparing a paper. Posted the first figure I have for the paper here: https://dev.to/jballanc/what-would-you-do-with-an-ai-model-capable-of-continuous-learning-31c8 https://dev.to/jballanc/what-would-you-do-with-an-ai-model-c...