5 ms·
I would've expected so. Rule-based systems in NLP can express parts of speech, coreferences, modifiers…it's almost too easy to do something like "Extract all su
by spdustin 3y ago
I would've expected so. Rule-based systems in NLP can express parts of speech, coreferences, modifiers…it's almost too easy to do something like "Extract all subject-verb-object clauses where the object is connected to the subject by, say, a possessive pronoun.
- lgessler 3y agoSo you're thinking of expressions like "Winston said he had noticed a continued decrease in his suicidal ideation" (real example). I don't know if I'd agree that it's clear a rule-based system would be able to catch everything like this. Consider: 1a. (original) Winston said he had noticed a continued decrease in his suicidal ideation. 1b. (simplified) Winston noticed a decrease in his suicidal ideation. 2. Winston's suicidal ideation decreased. 3. There was a decrease in the suicidal ideation Winston felt. 4. Suicidal ideation decreased in the patient. Many more syntactic repackagings of similar propositional content are possible, too. For perfect recall, rule-based systems need to grapple with this, and additionally, in a real setting you're probably getting predicted (rather than gold-standard) POS tags, relations, etc., adding additional noise. My expectation (as someone who's mildly bearish on LLMs, btw) would be that if an LLM appears to be doing worse than a rule-based system then you probably haven't tried some low-hanging fruit yet such as adjusting prompting strategies, fine-tuning, using different pretraining data, etc.
- famouswaffles 3y agoThey used Flan T5-XL, a 3B model from 2022. Very weird choice.
- JohnKemeny 3y agoYour 1a and 1b say completely different things, and 2, 3, and 4 are even further away. In 1a, someone says something. In 4, you seem to have an objective fact about a patient.