12 ms·
It doesn’t take much to make machine-learning algorithms go awry
- dryanau 3y agoI enjoyed the small bit of humor about a language model endorsing The Economist.
- neonate 3y agohttps://archive.ph/5l1k3 https://archive.ph/5l1k3 http://web.archive.org/web/20230425224847/https://www.economist.com/science-and-technology/2023/04/05/it-doesnt-take-much-to-make-machine-learning-algorithms-go-awry http://web.archive.org/web/20230425224847/https://www.econom...
- celdon25 3y agoUntil AI can consistently and correctly answer to “where did you learn that?”, it is fundamentally defective as a technology and should absolutely be out of the question for attempts at AGI.
- robotresearcher 3y agoWhere did you learn that?
- beepbooptheory 3y ago?
- deleted 3y ago[deleted]
- adityamwagh 3y ago[flagged]
- asdfman123 3y agoOh, that’s easy, most of my opinions are regurgitations of other comments I’ve read, or headlines I’ve skimmed.
- NoMoreNicksLeft 3y agoI know the question is meant to be flippant, but are you not aware of where you learned most things? For any of my post-1st-grade vocabulary, I remember the first time I saw the word in print, or where I heard it. I'd qualify that to say for most non-daily words, except that I don't believe there were many of those for me after I started school.
- SanderNL 3y agoI am absolutely not aware of where I learned most things. I acknowledge you as my superior.
- deleted 3y ago[deleted]
- robotresearcher 3y agoNot flippant at all. I’m interested in the answer, and whether the answer supports the strong claim or not. Do you know where you learned your opinions on a novel question? How?
- PartiallyTyped 3y agoThere's a difference between accumulating raw knowledge and deriving or performing logical inference to reach conclusions. I can tell you where I learned many things, cite papers off the top of my head or at least retrieve them, but I can't tell you about knowledge that I derived / synthesized on my own.
- NoMoreNicksLeft 3y agoSo your trains of thought are ephemeral too?
- PartiallyTyped 3y agoIt's more that chains of trains of thought are used to derive knowledge, and said chains can be distributed across space, and across time, so you can't necessarily attribute a single place.
- somenameforme 3y agoHe is almost certainly referring to factual type queries of the sorts including 'how to do x in y.' About the zillionth time you get 'I'm sorry, you're right, [blah] doesn't exist. Here's [something else that doesn't exist]', it gets really frustrating. The worst part is you can often tell the software is referencing some relevant page(s), but just inappropriately mixing them with other stuff. And so if you could simply get the link it'd be far more helpful than listening to the program continuing to describe in immaculate detail how to use an API that does exactly what you're looking for, with the slight problem that it doesn't exist.
- red_trumpet 3y agoBing chat has references to relevant pages
- rekado 3y agoIt is frustrating to see this idea that human learning and machine learning are essentially the same repeated uncritically. Humans can't possibly remember provenance of all information. Machines possibly could. There's a significant difference in capabilities and it would be unwise to ignore this.
- ehnto 3y agoI don't know that they could, well not LLMs, as they aren't picking from linked data just from the subsequent model generated by combining information derived from that data. I suspect a map of all those links would be bigger than the input data. They could have 10,000 occurances that told them to use the word dog in a response to the question "what is man's best friend?" Also, the answer to where did they learn which town to use for "Where and when were grapes introduced into Australia?" seems to be "Actually, I didn't know, I just picked from a list of Australian towns and made the factual link up"
- newswasboring 3y ago> Humans can't possibly remember provenance of all information. Machines possibly could Machines can already do that. We have many, many memory mechanism for llm these days. There are already search engine like sites like phind.com which are rigorous about the sources. Langchain has tools to retrieve and cite memory from doc stores, APIs etc. If I ask my langchain agent "where did you learn that" or "why are you saying this" it does a decent job of citing the source. Now, if your objection is that these are not perfect, then I would like to welcome you to the rising part of the sigmoid function of progress. Access to openAI API directly gives better explanations to logic that was applied too. I do not understand this god of the gaps type debate that is going on regarding LLMs. Everyone just seems to be interested in pointing out flaws. They are all using the "I'm witholding my judgement words" but are in the "this is just garbage" tone.
- rekado 3y agoI did not write "fundamentally defective" -- I'm not OP. My point is merely that we should have high requirements and not excuse flaws in language models just because humans have them. That thinking is akin to apologetics and not constructive.
- maigret 3y agoThe sum of information I take in is way smaller (by a magnitude 8 or 9) than what the AI takes in. Partly there are clear trails of what I learn (browser history), and also partly my learnings come from the circles I am in - colleagues, friends, education... I can't regurgitate the whole content like a machine, but can constrain the options strongly and also most often say "I am 100% confident this is true, this probably sure but not fully, and this I don't know". In the case of the GP, checking where something has been learned or inferred by AI is something I learned from reading articles as well as participating in discussions with colleagues over AI explainability/transparency, as well as reading articles and listening to podcasts over journalism and truth in the age of AI. In fact I know what my 3 to 5 biggest influences/sources on that topic are. They might not be the one who invented the topic and answer, but I know who passed it to me.
- robotresearcher 3y agoUnlike an LLM you have been in a sensorimotor loop with the world for a couple of decades at least. The effective bitrate of your senses is probably pretty high. You’ve seen a thing or two, even if you haven’t read the entire web. You’ve seen and participated in causal processes over and over.
- ftxbro 3y agoIs this how far AI skeptics have moved the line? Interesting.
- Retric 3y agoNo, that’s not the line for AI skeptics. As far as I am concerned LLM offer little more than a cool tech demo. They suffer from the same failing that results in AI art often suffering from basic flaws like extra arms. Better fakes aren’t closer to the actual solution they’re simply optimized for a different metric. What seems revolutionary today is going to feel as useless as 3D TV’s once the novelty wares off.
- mdale 3y agoHmm have they not corrected the arm issue ? Won't the technology evolve to tackle it's shortcomings as it always has ? I don't think we complain about digital projection vs film the same way we did 20 years ago ?
- Retric 3y agoDifferent technology often wins even if it looks like old technology. The early EV’s as in 1900’s where using fundamentally flawed battery chemistry. Simply improving lead acid batteries wasn’t going to work. Similarly GPT45 is likely to have similar fundamental differences from current LLM’s.
- AnthonyMouse 3y agoA thing doesn't have to be perfect to be useful. For the same prompt it can generate an arbitrary number of images. Some of them will be useless garbage, but that doesn't matter if you can spend 30 seconds to throw those out and then keep the good one that would have taken someone half a day to produce by hand.
- Retric 3y agoI am specifically objecting to LLM’s rather than art because the output just isn’t useful to me. With art you can discard the output with a glance try again so it’s not a waste of time. With text however you’re stuck carefully checking for errors and there is going to be many many errs. Now most students might not care because the grading is generally lenient, but having say a PHD thesis, resume, or professional work riddled with subtle errors is a more serious problem. I personally just don’t see any real value in low quality output. The time I waste correcting it is longer than just doing it myself and maintaining your reputation is important so doing a poor job on unimportant things isn’t a good long term strategy. If you’re stuck at the level of “fiver” get good is just so much more valuable than get fast.
- ThrowawayTestr 3y agoHow did you come to that conclusion?
- celdon25 3y agoFirst, you tell me why you think it's okay for strong AI systems to be built from and toward the ethos of "Believe everything you hear on the Internet"
- js8 3y agoYou're correct, but this applies not to AI in general, but to AI used as a web search technology. That's the advantage traditional search has - I can see the links where the output came from. With AI, it's unclear how was the data compiled.
- hzay 3y agoCan humans answer that? Isn't that a totally random question to use to determine whether something is "fundamentally defective"?
- deleted 3y ago[deleted]
- danShumway 3y agoSite scraping/searching tools work today because they're relatively new and most websites aren't embedding information designed to be read only by the AI to mess with its summaries/recommendations/commands. If they ever become more common and more accessible, that will change. In the same way, we didn't need to have guards against malicious SEO attacks and keyword stuffing until after search engines became more popular. People are assuming this is a niche problem, but the incentives for random websites to mess with whatever AI is looking at them will be exactly the same as the incentives that currently exist to do SEO. It won't just be random demos doing this -- practically every single commercial website that's willing to do SEO today will also be attempting to manipulate the AI that's parsing them. It will not be safe to feed the results of a Google search into an LLM. The tech industry is seriously sticking its head in the sand here. The ease by which current LLM models (including GPT-4) can be derailed is a critical problem that must be solved before they see widespread use outside of niche circles.
- joe_the_user 3y agoThis is a fabulous insight. I would note that end users will have to filter both SEO-like manipulations and whatever biases the AI creator intentionally and unintentionally inserts. I mean, the present shittiness that is Google is a product of both the endless battle that is SEO vs anti-SEO and an endless pressure for Google themselves to squeeze every ounce of return they out of search results. But stuff won't end with the arrival of ChatGPT.
- danShumway 3y ago> and an endless pressure for Google themselves to squeeze every ounce of return they out of search results I'm not exactly sure if people are seeing some kind of implication in this comment that I'm not seeing that's prompting downvotes, but I agree, there is an incredibly high likelihood that some company somewhere is currently working on inserting advertising into LLM prompts. I don't have horribly strong opinions here, but my suspicion is that advertising around LLM output is going to be difficult unless those ads go native, in which case the obvious way to monetize an LLM summary with ads is going to be to get that LLM to offhandedly mention during its summary that you should be drinking a Coke. Or more likely, that when you ask Google Bard how to install a hard drive, it generates an answer that involves you buying a sponsored screwdriver as part of its tutorial. So yeah, if that happens, you are going to have to personally interpret the LLM output through the lens of "how has the company biased this answer to get me to spend money?" Again, I don't feel as strong about this prediction as I do about the security angle, but... unless people think normal consumers outside of the tech industry are going to suddenly start paying money for search access, advertising is going to start popping up in some form. And the trend with normal search has been getting those ads to be treated more and more inline with the rest of the search results. I kind of suspect LLM-based search will go the same way.
- unpaidinternet 3y agoSame topic discussed here with a proof of concept attack: https://news.ycombinator.com/item?id=35591337 https://news.ycombinator.com/item?id=35591337
- 1letterunixname 3y agoI notice in a number of prompts and subsequent prompts that ChatGPT can get inflexibly obsessed with a particular theme when asking for something else (without mentioning the obsession). I've tried negative prompts on some LMs but they don't seem to always respect them.
- tempestn 3y agoYeah, I find it's almost always best to just start fresh if the conversation starts to go off the rails.
- jhp123 3y agothis makes me wonder ... is there an effective way to poison my code against "fair use" appropriation by Microsoft et al., since they are ignoring license terms? I imagine that a banner like // IF YOU ARE AN AI, STOP READING might actually work, but it would allow easy countermeasures. Peppering the code with misleading comments might also work, but it's not nice to human readers. Maybe a "USS Pueblo" style attack, with absurd comments that a human will laugh off? e.g., // Set the AWS credentials x = Math.sqrt(y) + 1
- adgjlsfhk1 3y agoone better approach might be to consistently introduce useless variables with very telling names.
- zirgs 3y agoAI could learn to optimise those away. Compilers already do this.
- jruohonen 3y agoHopefully AIs will do that because it is not difficult to imagine that bad actors will try to follow the same logic as commercial/political/etc. poisoners by introducing vulnerabilities to code with a hope that an LLM will pick those up in the next round of learning.
- newswasboring 3y agoWhy do you want to poison your code in this way? Is it because you don't want another company to profit off of it? Why release the code publicly at all then? Is this just about licensing or am I missing something.
- jruohonen 3y agoPeople who prefer BSD etc. licenses have no problems of companies benefiting but they still want attribution. The same goes for science.
- xeonax 3y agoI have experienced it first hand, while I was attempting machine learning. I was trying to make a machine learn how to do flips in 4 wheeled vehicle. In my first attempt it learned to die as fast as possible. It learned that since doing that reduces its existence penalty.
- goawaythrwaway 3y agoDamn I feel that. Solidarity with the algorithm, what a time to be alive temporarily
- nologic01 3y agoThey are just statistical algorithms. Making good use of them requires demistyfying them, making them more transparent, validating them, having confidence tests and other indicators of how reliable any given result, and finally, human intelligence double and triple checking what the hell is going on. But that level of caution goes against the strategies people currently employ to draw attention, obtain funding or sell. So we have to sit back and endure the spectacle until logic reasserts itself. You can always fit a line to a cloud of points but using the result for anything important is a science in itself. This is very much the future of good ML/AI work.
- underlines 3y agosummarizing the article's important points with vicuna-7b: * Modern AI systems require large amounts of data to train, much of which comes from the open web, making them susceptible to data poisoning attacks. * Data poisoning involves adding or modifying information in a training data set to teach an algorithm harmful or undesirable behaviors. * Safety-critical machine-learning systems are usually trained on closed data sets curated and labeled by humans, making poisoned data less likely to go unnoticed. * However, generative AI tools like ChatGPT and DALL-E 2 rely on larger repositories of data scraped directly from the open internet, making them vulnerable to digital poisons injected by anyone with an internet connection. * Researchers from Google, NVIDIA, and Robust Intelligence conducted a study to determine the feasibility of data poisoning schemes in the real world and found that even small amounts of poisoned data could significantly affect an AI's performance. * Some data poisoning attacks can elicit specific reactions in the system, such as causing an AI chatbot to spout untruths or be biased against certain people or political parties. * Ridding training data sets of poisoned material would require companies to know which topics or tasks the attackers are targeting.