4 ms·
he lost me at "a language model’s vocabulary is limited to the words that exist within the model’s training texts, which means a LLM can only refer to objects a
by obiefernandez 3y ago
he lost me at "a language model’s vocabulary is limited to the words that exist within the model’s training texts, which means a LLM can only refer to objects and relations that we humans have already discerned, named, and written about."
This is trivially demonstrated to be a false statement, as GPT is capable of synthesizing entirely novel words based on very little input guidance.
- epups 3y agoI find it exhausting when articles are written about ChatGPT which can be trivially demonstrated false by simply testing their assumptions. Noam Chomsky pulled a similar one a while ago.
- mahathu 3y agoWhats up with the number of mind-numbingly stupid takes in this thread? Are you actually this dumb or is there some elaborate in-joke i missed? Just because LLMs can (poorly) come up with new words when you instruct them to doesn't mean they're actually "thinking" in novel concepts, its just another indirectly grounded label for existing concepts. https://dai.fmph.uniba.sk/~retova/CSCTR/materials/CSCTR_07sem_Barsalou_perc_sym_sys_1999.pdf https://dai.fmph.uniba.sk/~retova/CSCTR/materials/CSCTR_07se...