3 ms·
The real question is how long it'll take until they're not brittle
by ramon156 1y ago
The real question is how long it'll take until they're not brittle
- yahoozoo 1y ago“Treat it like a junior developer” … 5 years later … “Treat it like a junior developer”
- agile-gift0262 1y agowhile True: print("This model that just came out changes everything. It's flawless. It doesn't have any of the issues the model from 6 months ago had. We are 1 year away from AGI and becoming jobless") sleep(timedelta(days=180).total_seconds)
- TeMPOraL 1y agoUsable LLMs are 3 years old at this point. ChatGPT, not Github Copilot, is the marker.
- LtWorf 1y agoUsable for fun yes.
- kubb 1y agoOr will they ever be reliable. Your question is already making an assumption.
- diggan 1y agoThey're reliable already if you change the way you approach them. These probabilistic token generators probably never will be "reliable" if you expect them to 100% always output exactly what you had in mind, without iterating in user-space (the prompts).
- kubb 1y agoI also think they might never become reliable.
- diggan 1y agoBut what does that mean? If you tell the LLM "Say just 'hi' without any extra words or explanations", do you not get "hi" back from it?
- TeMPOraL 1y agoThat's literally the wrong way to use LLMs though. LLMs think in tokens, the less they emit the dumber they are, so asking them to be concise, or to give the answer before explanation, is extremely counterproductive.
- diggan 1y agoI was trying to make a point regarding "reliability", not a point about how to prompt or how to use them for work.
- TeMPOraL 1y agoThis is relevant. Your example may be simple enough, but for anything more complex, letting the model have its space to think/compute is critical to reliability - if you starve it for compute, you'll get more errors/hallucinations.
- diggan 1y agoYeah I mean I agree with you, but I'm still not sure how it's relevant. I'd also urge people to have unit tests they treat as production code, and proper system prompts, and X and Y, but it's really beyond the original point of "LLMs aren't reliable" which is the context in this sub-tree.
- kubb 1y ago
- vFunct 1y agoIts perfectly reliable for the things you know it to be, such as operations within its context window size. Don't ask LLMs to "Write me Microsoft Excel". Instead, ask it to "Write a directory tree view for the Open File dialog box in Excel". Break your projects down into the smallest chunks you can for the LLMs. The more specific you are, the more reliable it's going to be. The rest of this year is going to be companies figuring out how to break down large tasks into smaller tasks for LLM consumption.
- dist-epoch 1y agoI remember when people were saying here on HN that AIs will never be able to generate picture of hands with just 5 fingers because they just "don't have common sense"
- guappa 1y ago∞