3 ms·
The trust isn't there because it's a gimmicky toy and not useful for anything serious. You have to put up with: 1. Incorrect information. 2. Political procli
by freitzkriesler2 3y ago
The trust isn't there because it's a gimmicky toy and not useful for anything serious.
You have to put up with:
1. Incorrect information.
2. Political proclivities of the company hosting it
3. Spying
Anyone who tells me they use LLMs for anything other then a curious experiment make me suspect. I had one of my PMs tell me he had an LLM make a meeting summary from the minutes so I grilled him on it since I was in that same meeting. The LLM doesn't have context of what's important and will harp upon things it thinks are necessary. That Pm now writes up the summaries because what the LLM made was useless.
- shermantanktop 3y agoUseless? Or not useful enough? These anecdotes represent a bad taste that may linger for a long time, even while the actual performance of LLMs moves from curiosity to utility to indispensability. I’m allergic to FOMO-based thinking but I am convinced we’re already at the “useful” stage for many use cases, when combined with human judgment.
- jprete 3y agoOne of the things LLMs can't imitate is having serious stake in the game, and therefore responsibility for the outcome. The PM has a stake - if they get this stuff wrong frequently, their job is at risk. The LLM provider is probably not getting paid much for the work - nothing, if it's a freely available model - and since the fine print says "do not rely on factual accuracy", the provider will claim no responsibility for its mistakes. (And the fine print will continue to say that for the foreseeable future, because even though LLMs might get more accurate, the providers will never want liability for errors unless the user is paying through the nose.)
- shermantanktop 3y agoIf the PM in question copy-pastes LLM output into a status report, they won't last long. If they read it, realize the LLM output missed a key point, disagree with the emphasis in a section, and failed to mention an underlying trend or concern---at that point the PM is engaging their own brain, fixing the issues, but is likely way faster to get to a good place. At that point the LLM is acting like a semi-competent intern.
- freitzkriesler2 3y agoUseless as in it harped upon information everyone was already aware of and completely ignored the mitigation strategies that were discussed. The PM ended up going back and listening to the recording because the strategies were what was the most important part of the discussion.