4 ms·
I was recently in a call (consulting capacity, subject matter expert) where HR is driving the use of Microsoft Copilot agents, and the HR lead said "You can avo
by toomuchtodo 1y ago
I was recently in a call (consulting capacity, subject matter expert) where HR is driving the use of Microsoft Copilot agents, and the HR lead said "You can avoid hallucinations with better prompting; look, use all 8k characters and you'll be fine." Please, proceed. Agree with sibling comment wrt cargo culting and simply ignoring any concerns as it relates to technology limitations.
- TZubiri 1y ago"Can I get that in writing?" They know it's wrong, they won't put it in an email
- NikolaNovak 1y agoMy problem is the "avoid" keyword: * You can reduce risk of hallucinations with better prompting - sure * You can eliminate risk of hallucinations with better prompting - nope "Avoid" is that intersection where audience will interpret it the way they choose to and then point as their justification. I'm assuming it's not intentional but it couldn't be better picked if it were :-/
- deleted 1y ago[deleted]
- horizion2025 1y agoEssentially a motte-and-bailey. "mitigate" is the same. Can be used when the risk is only partially eliminated but you can be lucky (depending on perspective) the reader will believe the issue is fully solved by that mitigation.
- toomuchtodo 1y agoTIL. Thanks for sharing. https://en.wikipedia.org/wiki/Motte-and-bailey_fallacy https://en.wikipedia.org/wiki/Motte-and-bailey_fallacy
- gerdesj 1y ago"Essentially a motte-and-bailey" A M&B is a medieval castle layout. Those bloody Norsemen immigrants who duffed up those bloody Saxon immigrants, wot duffed up the native Britons, built quite a few of those things. Something, something, Frisians, Romans and other foreigners. Everyone is a foreigner or immigrant in Britain apart from us locals, who have been here since the big bang. Anyway, please explain the analogy. (https://en.wikipedia.org/wiki/Motte-and-bailey_castle https://en.wikipedia.org/wiki/Motte-and-bailey_castle)
- Sabinus 1y agoFrom your link: "Motte" redirects here. For other uses, see Motte (disambiguation). For the fallacy, see Motte-and-bailey fallacy.
- horizion2025 1y agohttps://en.wikipedia.org/wiki/Motte-and-bailey_fallacy https://en.wikipedia.org/wiki/Motte-and-bailey_fallacy Essentially: you advance a claim that you hope will be interpreted by the audience in a "wide" way (avoid = eliminate) even though this could be difficult to defend. On the rare occasions some would call you on it, the claim is such it allows you to retreat to an interpretation that is more easily defensible ("with the word 'avoid' I only meant it reduces the risk, not eliminates").
- gerdesj 1y agoI'd call that an "indefensible argument". That motte and bailey thing sounds like an embellishment.
- kiitos 1y agowhat a great reference! thank you! another prolific example of this fallacy, often found in the blockchain space, is the equivocation of statistical probability, with provable/computational determinism -- hash(x) != x, no matter how likely or unlikely a hash collision may be, but try explaining this to some folks and it's like talking to a wall
- beeflet 1y agoThe solution is to sanitize text that goes into the prompt by creating a neural network that can detect prompts
- datadrivenangel 1y agoThis adds latency and the risk of false positives... If every MCP response needs to be filtered, then that slows everything down and you end up with a very slow cycle.
- singlow 1y agoI was sure the parent was being sarcastic, but maybe not.
- horizion2025 1y agoIsn't that just another guardrail that can be bypassed much the same as the guard rails are currently quite easily bypassed? It is not easy to detect a prompt. Note some of the recent prompt injection attack where the injection was a base64 encoded string hidden deep within an otherwise accurate logfile. The LLM, while seeing the Jira ticket with attached trace , as part of the analysis decided to decode the b64 and was led a stray by the resulting prompt. Of course a hypothetical LLM could try and detect such prompts but it seems they would have to be as intelligent as the target LLM anyway and thereby subject to prompt injections too.
- wrs 1y agoYep. https://gandalf.lakera.ai/baseline https://gandalf.lakera.ai/baseline
- Huppie 1y agoThis is genius, thank you.
- dotancohen 1y agoIt took me days to complete!
- DonHopkins 1y ago"You will get a better Gorilla effect if you use as big a piece of paper as possible." -Kunihiko Kasahara, Creative Origami. https://www.youtube.com/watch?v=3CXtLeOGfzI https://www.youtube.com/watch?v=3CXtLeOGfzI
- dstroot 1y agoHR driving a tech initiative... Checks out.