3 ms·
I think the amazing thing about the LLM is that it's not a random text generator so it's not 99% meaningless. I give it only 50% meaningless.
by ninetyninenine 2y ago
I think the amazing thing about the LLM is that it's not a random text generator so it's not 99% meaningless.
I give it only 50% meaningless.
- mensetmanusman 2y agoWith nearly unlimited energy, if you set up an indefinite while-loop asking it to continue to expand and make new theories on the same token vector it would approach >99% meaninglessness.
- ninetyninenine 2y agoYeah that's because the only thing changing is some random seed. It's like looking at f(0) = 1 and f(0.00001) = 1.00001 and saying that the function can't produce anything novel because the answer is always less than 2. Hint: try f(99999). Of course it's all meaningless because most of the input and output is virtually identical. Vary the input heavily and then you will approach 50%. Of course this is assuming the token vector is truly random in terms of subject matter.
- mensetmanusman 2y agoIt’s not just a random seed though, this would be attempting to squeeze everything out of the LLM network with all the seeds :)
- ninetyninenine 2y agoIf you wanted to squeeze everything out of the network it's not just varying the seeds. It's varying the token vector.
- mensetmanusman 2y agoAgreed, I doubt the full squeeze would be 50% correct on new hypotheses though. Would be a fun PhD thesis if I went back to grad school :D