4 ms·
Author here, would love feedback / thoughts / corrections!
by andreyk 5y ago
Author here, would love feedback / thoughts / corrections!
- skybrian 5y agoAnother limitation to be aware of is that it generates text by randomly choosing the next word from a probability distribution. If you turn that off, it tends to go into a loop. The random choices improve text generation from an artistic perspective, but if you want to know why it chose one word rather than another, the answer is sometimes that it chose a low-probability word at random. So there is a built-in error rate (assuming not all completions are valid), and the choice of one completion versus another is clearly not made based on meaning. (It can be artistically interesting anyway since a human can pick the best completions based on their knowledge of meanings.) On the other hand, going into a loop (if you always choose the highest probability next word) also demonstrates pretty clearly that it doesn’t know what it’s saying.
- andreyk 5y agoGood point, thanks!
- mojuba 5y agoProve us you are not GPT-3 ;)
- andreyk 5y agoWhy would I write an article about my own limitations :P