3 ms·
> They clearly go far beyond this and generate completely novel ideas that are not in the training data. There's a case where this is trivially false. Language
by AlexandrB 3y ago
> They clearly go far beyond this and generate completely novel ideas that are not in the training data.
There's a case where this is trivially false. Language. LLMs are bound by language that was invented by humans. They are unable to "conceive" of anything that cannot be described by human language as it exists, whereas humans create new words for new ideas all the time.
- pixl97 3y agoUh, I believe you're really confused on things like ChatGPT versus LLMs in general. You don't have to feed human language to an LLM for them to learn things. You can feed wifi data waveforms for example and they can 'learn' insights from that. Furthermore you're thinking here doesn't even begin to explain multimodal models at all.
- DiogenesKynikos 3y agoI just asked ChatGPT to make up a Chinese word for hungry+angry. It came up with a completely novel word that actually sounds okay: 饥怒. It then explained to me how it came up with the word. You can't claim that that isn't understanding. It just strikes me that we've moved the goalposts into every more esoteric corners: sure, ChatGPT seems like it can have a real conversation, but can it do X extremely difficult task that I just thought up?