2 ms·
LLMs will definitely get better -- and people will adapt too. But natural language contains inherent ambiguity, and that's not going to change. A large part of
by zamfi 3y ago
LLMs will definitely get better -- and people will adapt too. But natural language contains inherent ambiguity, and that's not going to change.
A large part of the paper talks about how the challenges of programming and developing ML systems don't go away just because you're using natural language and an LLM.
For example, users incorrectly extrapolating from a single failed/successful prompt to other contexts: that's not something that will be solved by better LLMs, really. It's solved by getting users to recognize when they do/don't have enough data to believe what they're seeing.
Disclaimer: I wrote the paper.
- digdugdirk 3y agoThanks for the official TL/DR. ... And the laughs. Paper is great, too. What's it like trying to write a paper while the field is shifting so rapidly? Is there anything you would have liked to include that had to be cut so you could get it out the door?
- zamfi 3y agoOh, that's not a full TL;DR -- but sounds like you've read the paper. :) Shifting field: yes, we spend a lot of time trying to figure out what is going to change and what isn't, at least in HCI (my field). LLM capabilities are changing by the...day? So anything that's very focused on LLM capabilities won't be relevant for long. Humans don't change quite as fast! Keep in mind this paper was first written in September 2022, reflecting work done prior to then...practically an eternity in which we've since seen ChatGPT take the world by storm, and now GPT-4. But I think the lessons there are still very relevant, and will be for quite some time.