4 ms·
Along those lines, how is that position not randomania? You’re basically saying that the signal to noise ratio is zero so its utility is zero. I guess you must
by pizza 3y ago
Along those lines, how is that position not randomania?
You’re basically saying that the signal to noise ratio is zero so its utility is zero. I guess you must think all of the comments here where someone mentioned how it helped them solve a problem, also have a 0 (or at least dismissible) signal to noise ratio?
What signal to noise would be needed for you to say it’s not devoid of any information? What would the minimum accuracy need to be? 80%? 95? 99? 100?
The thing is there are many questions where it is a short path to a solution that can then be verified. The most useful way to use it productively is to induce it to give you pointers to things you had no idea existed. Then dereference those pointers via traditional means (ie google, man pages). Be your own null ptr (hallucinations) segfault handler. Critical thinking a hundred percent preserved.
It’s a little bemusing how people take issue with next token prediction trained with a log likelihood loss. In particular they think that getting a high accuracy extremely low perplexity LLM is unimpressive because of the low accuracy high perplexity autocorrect on their phone. No- the llm’s emergent generalization capabilities as nll drops really is a capital p Phenomenon.
But I feel that I don’t need to try to change anyone’s mind anymore. Everyone had a threshold logp where they’ll change their mind about this, whether it’s attained in the next week or long from now. Just state the nonzero perplexity that you would accept from a person thinking, and wait for the loss curve to meet it.
- hanselot 3y agoDon't bargain on that. For some the threshold is way too high given how far they've sunk. As always, the ones most vocal about it haven't even bothered trying out guanaco 70b 2bit. Probably don't know what ggml, gptq, cformers or any of the others are. They still base their assumptions on the openai gold standard.