4 ms·
I agree with your general point, but I think “the movie was like candy“ could be a perfectly reasonable simile to make. Maybe the movie was flashy but had no su
by rsfern 5y ago
I agree with your general point, but I think “the movie was like candy“ could be a perfectly reasonable simile to make. Maybe the movie was flashy but had no substance?
I’m not sure if I expect modern language models to work at that level of abstraction though? I guess you need a larger generated fragment to assess if the hypothetical sentence was statistical nonsense or part of some larger conceptual sequence
- toxik 5y agoI think if you read what I wrote more carefully, you'll see I never claimed that the model cannot generate "candy" as the successor word, but that it wouldn't do so simply out of having seen the words colocated. The relative sequence order matters to these models, and they do model grammar and semantic roles.
- rsfern 5y agoYeah, I agree with your assessment. I thought you had written that sentence as an example of some Markova chain non-sequitur, and i was just musing that it doesn’t seem like a super unlikely sentence. “The movie was like ticket” maybe makes the distinction clearer?