3 ms·
The ability to self-evaluate seems to be improved by adding highly evaluated output to its training data. I'm not sure how far that technique can be pushed but
by CuriouslyC 3y ago
The ability to self-evaluate seems to be improved by adding highly evaluated output to its training data. I'm not sure how far that technique can be pushed but it's very promising given the performance of this 70b model. I bet progress in the self-evaluation technique could let this trick go quite far.
It's hard to say how well this would work on a MoE model, but best case scenario is something decently better than GPT4 that can run on 2x3090/4090 or 1x48gb 5090.
- TOMDM 3y agoI hope Nvidia gives us a 48GB 5090, but I can just as easily see them wanting to keep the data center divide going by keeping the consumer cards low on VRAM. Here's hoping AMD forces them to keep pushing the boundary.
- CuriouslyC 3y agoThat's not a problem, you can get used 24gb 3090 TIs for a decent price and run them together for WAY cheaper than a flagship 5090 will run. It'll generate tokens slower, but probably still faster than you can read.
- fbdab103 3y agoGiven Intel seeming to make a stronger push, I think it more likely that Intel delivers a high RAM (if slower) card into the fray.
- joshspankit 3y agoEven if they did, 48GB would quickly become “the new 12GB” as models expanded.