49 ms·
If that were the case, outputs would be consistently nonsense - the number of possible variations of text like "colorless green ideas sleep furiously" is so muc
by int_19h 4y ago
If that were the case, outputs would be consistently nonsense - the number of possible variations of text like "colorless green ideas sleep furiously" is so much larger than the meaningful subset, the probability of hitting the latter by chance would be zero for all practical purposes.
- cageface 4y agoOnly if the words were chosen simply at random in sequence and of course they're not this simplistic. They're constrained by the attention models so they do much better than this but they're still random. You can control the degree of randomness with the temperature knob.
- int_19h 4y agoThis part about "constrained by the attention model" is doing a lot of implicit work here to dodge the question why GPT-4 can verifiably reason about things in text.
- cageface 4y agoIt also demonstrably is either flat out wrong about a lot of things or completely invents things that don't exist. It's a random process that sometimes generates content with actual informational value but the randomness is inherent in the algorithm.