5 ms·
Token cap will probably be the biggest problem here. After validation. After getting the changes to disk, documented, actually compiling, etc… But the bigges
by theK 4y ago
Token cap will probably be the biggest problem here.
After validation.
After getting the changes to disk, documented, actually compiling, etc…
But the biggest problem is that transferring the nuance that is external to the code base is typically really tiresome and lengthy grunt work and again token cap.
- anonzzzies 4y agoThey went from 4K to 32k in a few months. Wait a few years.
- Jensson 4y agoGPT-3 is 3 years old, not a few months.
- anonzzzies 4y agoSure but the token jump to 32k is new isn’t it? Even 8k to 32k is significant and that was this year.
- Jensson 4y agoYeah, but the jump to 32k didn't take a few months, it was years in the making. Otherwise you could extrapolate with "yesterday we had 4k, today we have 32k, tomorrow we will have 256k", that isn't how we do it. If we follow the same exponential pace 256k would have to wait 3 years, and even that is unlikely.
- osigurdson 4y agoI think it is better to assume that we are at the beginning of the s-curve, not the end.
- osigurdson 4y agoWow. I never expected to be downvoted for such a comment. Downvotes please elucidate as to why we are in the asymptomatic phase with LLMs.
- hnfong 4y ago"Luddites" refusing to accept that we might be onto something that is going to change humanity forever... The sad thing is that real luddites would go out and actively sabotage AI development because they think it's a real threat. Yet these people just makes bold and false claims in online forums and continue to move the goalposts once they're proven wrong. Sad. Pathetic. (and obviously, I don't mind being downvoted. Whatever! :)
- anonzzzies 4y agoSeems very man of these here for some reason, trying to find whatever reasons and ways (gonna be licensing mostly) to spread FUD.
- theK 4y agoFew years, yes. I’m with you “the change is coming” but we still need to transfer millions of tokens in and out to cater for context and out of repo intricacies.
- anon7725 4y agoLarge enterprises will just pay to fine tune over their documents/code/etc.
- riku_iki 4y agowe don't know what that jump means, did they increased model proportionally, or slapped some projection layer with quality degradation?..