3 ms·
The model is way too small and undertrained to make any generalization claims. I want to wait until it reads the whole corpus I gave and then test it on some si
by volotat 13d ago
The model is way too small and undertrained to make any generalization claims. I want to wait until it reads the whole corpus I gave and then test it on some simple established benchmarks to see how it will behave.
- jacquesm 13d agoWhat kind of hardware are you using for training? nm, I found it: > RTX 3070 Laptop GPU with 8 GB Super impressive.
- dinfinity 13d agoSeems a bit premature to make an HN post about then, imho. It's an interesting idea, but it doesn't really do anything interesting yet. I looked at the output in the training run and it is a far, far cry from intelligence. Worse than GPT-2 as it stands. I do hope it will perform well when scaled and trained, though; best of luck.