7 ms·
I get that this is basically fraud and spam, but this should really highlight the dangers of letting an unattended LLM do anything for your company at all. It c
by packetlost 3y ago
I get that this is basically fraud and spam, but this should really highlight the dangers of letting an unattended LLM do anything for your company at all.
It can, and will, fuck up dramatically sooner or later.
- sandworm101 3y ago>> unattended LLM do anything for your company at all. It can, and will, fuck up dramatically sooner or later. So, just like any other random employee?
- hereme888 3y agonot THAT badly, lol
- TrololoTroll 3y agoPeople on this forum often "joke" about dropping the production database as a rite of passage for noobs
- jader201 3y ago> as a rite of passage for noobs I’ve been in the field for nearly 30 years. I’m far from incapable of such screwups.
- deusum 3y agoBeing a pro means you can fix anything you break - preferably before anyone noticies
- sophacles 3y agoI would hope that your experience has at least decreased the time between "first hearing about wierdness" and "realizing you accidnetally dropped prod". It's why pay generally increases with experience :D.
- markusde 3y agoThe difference is, a junior employee knows that killing prod is bad. An LLM doesn't know anything.
- SketchySeaBeast 3y agoAnd we only do it once (I didn't kill the db, but I did kick off a process thinking I was in a test environment).
- Anamon 3y agoWere you the guy sending test push notifications from Firebase to all users of Xperia phones last year? :D
- blihp 3y agoDon't be so sure that all, or even most, junior employees know any such thing. I've seen junior employees fired for doing silly things in prod before[1] [1] Of course whatever more senior bozo granted the junior the rights to blow up the thing(s) they did should have been fired instead. That's not the way things work in the corporate world.
- gopher_space 3y agoI like getting juniors into situations where they can blow up a db since it's the perfect introduction to backups.
- TrololoTroll 3y agoThem knowing that it is bad isn't much of a consolation for the dead production The magic that happen in someone's mind that leads to their actions matters very little for everyone else. Their actions and the consequences of their actions are what everyone else actually cares about
- altairprime 3y agoNo. Random employees have a well-understood distribution of mostly normal human errors of certain types and estimated severity, relative to unattended LLM which has a poorly-understood distribution of errors in both type and severity. (“SolidGoldMagikarp”.)
- dylan604 3y agocopy&paste errors are exactly what human employees are good at. this could very easily be the result of a bad copy&paste by a human into a form. especially if the copy&paste text is in a language not understood by the human employee. to them, it might look just like one of the other hundreds of search term word salad used as titles
- deleted 3y ago[deleted]
- altairprime 3y agoWhether it’s human or not is irrelevant to the point: human beings fail much more predictably. When the same search term salad is presented hundreds of times for copy paste, a human would notice and have an opportunity to ask a supervisor. A chatbot automation would not notice the repetition unless it had been coded to detect repetition, and/or to reject the ChatGPT refusal message. Ironically, it was probably an automation coded by ChatGPT.
- csours 3y agoThe employee generally knows they fucked up and can escalate the issue. Discussion on whether or not this actually happens will follow in comments below.
- em-bee 3y agoor if they don't know at the time, they may eventually realize it later and react accordingly.
- ben_w 3y agoSometimes. https://news.bbc.co.uk/2/hi/7702913.stm https://news.bbc.co.uk/2/hi/7702913.stm
- Vicinity9635 3y ago[dead]
- nitwit005 3y agoI brought up translation as a risk with a friend. If you pay someone for a translation these days, there is a chance they will just feed it to some AI to cut costs. You'll have no way to validate yourself if you don't speak the language.
- csours 3y agoThe shape of managing the work approaches the work in terms of fractal complexity
- emporas 3y agoJust a chance? I routinely translate hundred pages of pdfs to greek, in 3 minutes. The translation is far from perfect depending the text and it still needs a human in the loop for corrections, but i couldn't imagine translating a 300 pages pdf to greek by hand. There is also the translaxy bot on poe.com which i use to translate english or modern greek to ancient greek. Out of this world good translation. I mean, are humans still employed to translate text? Like an employee doing that job, and only that?
- mtlmtlmtlmtl 3y agoThis meme is getting old.
- Dig1t 3y agoidk I do think it's worth pointing out sometimes that the ways these models mess up are very similar to the ways that humans mess up. It's funny you can almost always look at an obvious failure of an LLM and think of an equivalent way that a human might make the same (or a similar) mistake. It doesn't make the failure any less of a failure, but it is thought-provoking and worthwhile to point it out. Obviously this particular case is not the failure of the LLM but the failure of the spammer who tried to use it.
- mtlmtlmtlmtl 3y agoIt's certainly useful to draw carefully thought out comparisons between human and AI performance at various tasks. But this meme is not that. It's literally just a meme that's posted reflexively to any and all posts that unfavourably compare AI to humans, without any thought or analysis added.
- klyrs 3y agoSometimes I read comments like this and feel a swell of gratitude that I don't work with braindead novices that make LLM-like mistakes. Are your coworkers actually that bad?
- add-sub-mul-div 3y agoBut a human can only mess up so many times per second. Even if it wasn't AI, if it was just a pill that allowed them to type unhumanly fast, once they have the power to scale up their incompetence (or predation) they're a new kind of danger.
- skywhopper 3y agoNo, not at all. People can be held accountable for the decisions they make. You can have a relationship of trust between people. LLMs do not have these properties.
- shadowgovt 3y agoRelationships of trust between users and the LLMs they choose to use definitely exist.
- johnnyanmac 3y agowell no one has 5 years of experience as an LLM prompter, so the trust will be low in the short term. With current lawsuits, trust in the LLM is probably low for at least a year or two, with companies trusting employees to NOT use them for their work.
- mstolpm 3y agoWhy is it that LLMs are so often compared to employees and their responsibilities? In my opinion, it is an employee that actively USES the LLM as a tool and this employee (or his/her employer) is responsible for the results.
- add-sub-mul-div 3y agoIt's a dumb/lazy/specious talking point. You can kill someone with a pencil just like you can kill someone with a gun, but the gun scales up the danger so we treat it and regulate it differently. You can kill someone with a bike, a car, or an airplane, but the risks go up at each step so we treat and regulate the respective drivers differently. If AI gives every individual the power to suddenly scale up the bullshit they can cause by 3+ orders of magnitude, that is a qualitatively different world that needs new considerations.
- wegfawefgawefg 3y agoOne of the biggest recent "mass shootings" was some guy at a walmart with a $200 bow and arrow kit.
- add-sub-mul-div 3y agoWhere/when was that? Link?
- wegfawefgawefg 3y agoNorway, https://en.wikipedia.org/wiki/Kongsberg_attack https://en.wikipedia.org/wiki/Kongsberg_attack The way it hit headlines in the USA was somewhat misleading. Look for American news posts if you want to feel out the weird position. You can see the video of him just going about it in a store, and in the entrance.
- add-sub-mul-div 3y agoFive people were killed by stabbing, none by bow and arrow. While it may have been the biggest recent mass killing in that country, it's still small compared to what can be achieved with greater weapons, which was the point.
- EGreg 3y agoWhy do people not understand that LLMs can do things at scale, next year they can form swarms, etc. Swarms of LLMs are not comparable to an employee, they have far better coordination and can carry out long-term conspiracies far better than any human collective. They can amass reputation and karma (as is happening on this very site, and Reddit, etc. daily) and then deploy it in coordinated ways against any number of opponents, or to push public opinion towards a specific goal. It's like comparing a CPU to a bunch of people in an office calculating tables.
- PheonixPharts 3y ago> they have far better coordination I think LLMs are still underutilized, but to this point, it's been repeatedly shown that even the most state of the art LLMs are incapable of generalization, which is very necessary for coordinating large scale conspiracies against humanity.
- EGreg 3y agoI dunno, sentiment recognition and coordinated downvoting seems pretty simple for AIs ;-)
- tomxor 3y ago>> unattended LLM do anything for your company at all. It can, and will, fuck up dramatically sooner or later. > So, just like any other random employee? Right, might as well just replace it all with a roll of the dice in that case. Wait do we have to quantify our comparisons? no, no, sorry, I almost forgot this was the internet for a second.
- nitwit005 3y agoThat's a testable assertion isn't it? Do you observe any product with that extreme level of silliness, which weren't intentional? People generally review their product catalogues.
- jerf 3y agoTo err is human. To fuck up a million times per second, you need a computer. Granted, here at the beginning of 2024, an LLM can not quite attain that fuck up velocity. But take heart! Many of the smartest people on Earth are working on solving that exact problem even as you read this.
- dan_bez 3y ago"Fuck up velocity" goes straight into my vocabulary.
- bcrosby95 3y agoFPS? Fuckups per second?
- scotty79 3y agoMaybe FuPS? So it's easier to tell from other two FPS-es.
- sirspacey 3y agoFups FTW!
- mmaunder 3y agoThat’s it. FUPS is a frequency measurement of the rate at which an AI produces “fuckups” per second, where a “fuckup” is defined as an individual non-unique production error that has a measurable negative impact on society.
- theturtletalks 3y agoI imagine this is how the jerk unit (rate at which acceleration changes) got made up
- dan_bez 3y ago
- johnnyanmac 3y agoyes, but humans have contracts and plausible deniability and all that jazz from companies. A human can't go on a shooting spree that will end up getting the employer sued for that very reason. Robot as of now, not so much.
- willcipriano 3y agoOnly if your employee is prone to episodes where they call all your customers speaking in tongues.
- ChatGTP 3y agoHumans can also be held accountable for fuck ups, which makes them less desirable therefore less likely. A bot doesn't care about this.
- Laaas 3y agoWhy is it fraud? Maybe it's a legitimate item.
- autoexec 3y agoA legitimate item from the totally legit company "FOPEAS" that's being sold for $100 less at vidaxl.com and is still probably made from formaldehyde-soaked wood and covered in lead paint.
- dns_snek 3y agoAnd pay no attention to the fact that the seller is registered in China and sells everything from furniture to underwear, UV lamps, and I kid you not, "effective butt lifting massage cream".
- sophacles 3y agoWalmart, cosco, and a hundred other stores sell a wide range of stuff too. (on their websites, even if it is available direct from the manufacturer's website or other websites). Is the problem "registered in China"?
- dns_snek 3y agoNo, the problem is that this stuff is absolute junk sold by sellers who face zero accountability even if they put rat poison in your skin care cream, who can keep returning to the platform by making up new nonsense brand names like "FOPEAS" that don't even have a website, as fake and low effort as it might have been if they at least tried to pretend. This issue is highly specific to Amazon and has been documented in great detail.
- autoexec 3y agoCan you get one of these at Walmart? https://www.amazon.com/complete-information-provided-provide-context/dp/B0C5X1GXPH https://www.amazon.com/complete-information-provided-provide... Cats with "Exceptional Read/Write Speeds" aren't sold at cosco either Good old not-a-scam FOPEAS has you covered though!
- PheonixPharts 3y agoI don't find this any different than seeing an exposed jinja template: "{{product_name}} is perfect people who work in {{customer_industry}}" or the typical recruiter "Dear {{candidate}} I read your profile carefully and think you'd be perfect for {{job_title}} because of your experience at {{random_co_from_resume}}" If anything, I think it's kind of cool that we're seeing LLMs actually used for something very practical, even if it is spammy (I mean I don't think template engines are evil just because they make spam easier).
- madeofpalk 3y agoThose emails from recruiters is also spam.
- packetlost 3y agoI don't think LLMs are evil either, but I think the real risks are extremely underplayed. This is a mostly innocuous example, but there are a lot of people trying to get LLMs into more places where the just aren't ready for yet. The difference between a template is that the behavior is generally deterministic. Even if someone fucks it up, it means it's (usually) trivial to fix.
- CobrastanJorji 3y agoIs this a dramatic fuckup? Because it quite possibly successfully created tens of thousands of listings more or less successfully. This one will probably generate no sales, but were there any consequences for this mistake?
- packetlost 3y agoThe difference is the failure is non-deterministic and not predictable in any real capacity
- PKop 3y agoThis isn't the only one https://news.ycombinator.com/item?id=38974931 https://news.ycombinator.com/item?id=38974931
- Analemma_ 3y agoWhat dangers? Nobody will see any consequences for this: not Amazon-- they're a monopoly, they don't give a shit-- and not the seller-- who probably won't see any impact whatsoever on their sales or reputation, and will just recreate under a new shell name if they do. The fact that LLMs drive the cost of junk text production to zero is a tremendous opportunity when there is no penalty for messing up. It's the same think as bulk spam mailing: if it's free, there's no reason not to keep trying even if only one a million is a success.
- BHSPitMonkey 3y agoFrequent run-ins with listings like this will definitely build (even more of) a reputation in some users' minds that Amazon is a spam-filled and unproductive place to look for things, but yes—it would take a lot to actually threaten their market position.
- quickthrower2 3y agoWhen the LLM spits out “clinically proven” then you are in trouble
- deleted 3y ago[deleted]