5 ms·
Genuinely curious, what does it mean in this context to perform better? (hopefully that doesn't come across as snarky as text sometimes does).
by gwoolhurme 4y ago
Genuinely curious, what does it mean in this context to perform better? (hopefully that doesn't come across as snarky as text sometimes does).
- jerrygenser 4y agoSome of these large models are able to do zero shot learning and perform tasks they weren't explicitly trained on since the training objective is very general. Being able to perform more advanced types of zero shot learning tasks would be comparable and further the accuracy on those tasks can be evaluated
- dmix 4y agoAny chance this could improve coding LLMs like Co-pilot? Or would that sort of thing be limited to source code feeds (not that Github has a shortage).
- refulgentis 4y agoThe next big step for coding LLMs will be context window increases, leaked docs have OpenAI pricing for up to 16K I believe, 4x the current maximum. Now you're talking "write a class" instead of this line and maybe sometimes a method
- williamcotton 4y agoIt can already reliably write a todo web browser application. With 16k and some other techniques, I’m guessing it could write a custom CMS database backed web application.
- flangola7 4y agoNot 16k, 32k. 8x the current window.
- refulgentis 4y agoNice
- dmix 4y agoWhat is 16k referring to here
- refulgentis 4y agoI’ve begun to grok it as “the amount of ram I have to play in before I have to start sharding work” more literally and correctly, it’s the maximum number of tokens in the input and output, combined, where a token is 4/3 of a word So we’re shifting from 5K words maximum to 40K (per sibling comment, who pointed out 32K context leaked as well)
- MattRix 4y agoA minor correction: a token is 3/4 of a word. ie. it’s slightly smaller than a word, not larger.