8 ms·

Machine Unlearning in 2024

328 points by ignoramous 2y ago

deleted 2y ago
[deleted]
dataflow 2y ago
> However, RTBF wasn’t really proposed with machine learning in mind. In 2014, policymakers wouldn’t have predicted that deep learning will be a giant hodgepodge of data & compute Eh? Weren't deep learning and big data already things in 2014? Pretty sure everyone understood ML models would have a tough time and they still wanted RTBF.
- peteradio 2y ago
  I don't know if people anticipated contemporary parroting behavior over huge datasets. Modern well funded models can recall an obscure persons home address buried deep into the training set. I guess the techniques described might be presented to the European audience in an attempt to maintain access to their data/and or market for sales. I hope they fail.
- isodev 2y ago
  Of course, it’s not a regulation issue. The technology was introduced to users before it was ready. The very nature of training without opt-in consent or mechanism of being forgotten are all issues that should have been addressed before trying to make a keyboard with a special copilot button.
- spennant 2y ago
  Agreed. The media and advertising industry was most definitely leveraging cookie-level data for building attribution and targeting models. As soon as the EU established that this data was “personal data”, as it could, theoretically, be tied back to individual citizens, there were questions about the models. Namely “Would they have to be rebuilt after every RTBF request?” Needless to say, no one in the industry really wanted to address the question, as the wrong answer would essentially shut down a very profitable practice.
  - Aerroon 2y ago
    More likely: the wrong answer would've shut out a profitable market rather than the practice. The EU is not the world. Anthropic seems to not mind blocking the EU for example.
    - spennant 2y ago
      Sure. But two things: 1) At the time, the European data laws implied that it protected its citizens no matter where they are. Nobody wanted to be the first to test that in court. 2) The organizations and agencies performing this type of data modeling were often doing so on behalf of large multinational organizations with absurd advertising spends, so they were dealing with Other People’s Data. The responsibility of scrubbing it clean of EU citizen data was unclear. What this meant was that an EU tourist who traveled to the US, and got served a targeted ad, could make a RTBF request to the advertiser (think Coca-Cola, Nestle or Unilever) The whole thing was a mess.
- startupsfail 2y ago
  RTBF was introduced to solve a specific issue, no? Politicians and their lobbyist friends could no longer remove materials linking them to their misdeeds as the first Google Search link associated with their names. Hence RTBF. Now, there’s similar issue with AI. Models are progressing towards being factual, useful and reliable.
- indigovole 2y ago
  GDPR and RTBF were formulated around the fears of data collection by the Stasi and other organizations. They were not formulated around easing the burdens of future entrepreneurs, but about mitigating the damage they might cause. Europeans were concerned about real harms that living people had experienced, not about enabling AGI or targeted advertising or digital personal assistants. We have posts here at least weekly from people cut off from their services, and their work along with them, because of bad inference, bad data, and inability to update metadata based purely on BigGo routine automation and indifference to individual harm. Imagine the scale that such damage will take when this automation and indifference to individual harm are structured around repositories from which data cannot be deleted, cannot be corrected.
- hooby 2y ago
  I'm pretty sure that the policymakers did NOT understand ML models in 2014 - and still do NOT understand it today. I also don't think that they care. They don't care that ML is a hodgepodge of data & compute, and they don't care how hard it is to remove data from a model. They didn't care about the ease or difficulty of removing data from more traditional types of knowledge storage either - like search indexes, database backups and whatnot. RTBF was not proposed with any specific technology in mind. What they had in mind, was to try and give individuals a tool, to keep their private information private. Like, if you have a private, unlisted phone number, and that number somehow ends up on the call-list of some pollster firm, you can force that firm to delete your number so that they can't call you anymore. The idea is, that if your private phone number (or similar data) ends up being shared or sold without your consent - you can try to undo the damage. In practice it might still be easier to get a new number, than to have your leaked one erased... but not all private data is exchangeable like that.
negative_person 2y ago
Why should we try to unlearn "bad" behaviours from AI? There is no AGI without violence, its part of being free thinking and self survival. But also by knowing that launching a first strike by a drunk president was a bad idea we averted a war because of a few people, AI needs to understand consequences. It seems futile to try and hide "bad" from AI.
- andy99 2y ago
  This is presumably about a chatbot though, not AGI, so it's basically a way of limiting what they say. (Not a way that I expect to succeed)
- Cheer2171 2y ago
  So you have a problem with supervised learning like spam classifiers?
- williamtrask 2y ago
  Because we can get AI related technologies to do things living creatures can’t, like provably forget things. And when it benefits us, we should. Personal opinion, but I think AGI is a good heuristic to build against but in the end we’ll pivot away. Sort of like how birds were a good heuristic for human flight, but modern planes don’t flap their wings and greatly exceed bird capabilities in many ways. Attribution for every prediction and deletion seem like prime examples of things which would break the analogy of AI/AGI with something more economically and politically compelling/competitive.
  - negative_person 2y ago
    Can you point to any behaviour in human beings you'd unlearn if theyd also forget the consequences? We spend billions trying to predict human behaviour and yet we are surprised everyday, "AGI" will be no simpler. We just have to hope the dataset was aligned so the consequences are understood, and find a way to contain models that don't.
    - aeonik 2y ago
cwillu 2y ago
“to edit away undesired things like private data, stale knowledge, copyrighted materials, toxic/unsafe content, dangerous capabilities, and misinformation, without retraining models from scratch” To say nothing of unlearning those safeguards and/or “safeguards”.
- ben_w 2y ago
  It sounds like you're mistakenly grouping together three very different methods of changing an AI's behaviour. You have some model, M™, which can do Stuff. Some of the Stuff is, by your personal standards Bad (I don't care what your standard is, roll with this). You have three solutions: 1) Bolt on a post-processor which takes the output of M™, and if the output is detectably Bad, you censor it. Failure mode: this is trivial to remove, just delete the post-processor. Analogy: put secret documents into a folder called "secret do not read". 2) Retrain the weights within M™ to have a similar effect as 1. Failure mode: this is still fairly easy to remove, but will require re-training to get there. Why? Because the weights containing this information are not completely zeroed-out by this process. Analogy: how and why "un-deletion" is possible on file systems. 3) Find and eliminate the weights within M™ that lead to the Bad output. Analogy: "secure deletion" involves overwriting files with random data before unlinking them, possibly several times if it's a spinning disk. -- People are still doing research on 3 to make sure that it actually happens, what with it being of very high importance for a lot of different reasons including legal obligation.
  - andy99 2y ago
    Until we have a very different method of actually controlling LLM behavior, 1 is the only feasible one. Your framing only makes sense when "Bad" is something so bad that we can't bear its existence, as opposed to just "commercially bad" where it shouldn't behave that way with an end user. In the latter, your choice 1 - imposing external guardrails - is fine. I'm not aware of anything LLMs can do that fits in the former category.
    - ben_w 2y ago
      > Until we have a very different method of actually controlling LLM behavior, 1 is the only feasible one. Most of the stuff I've seen, is 2. I've only seen a few places use 1 — you can tell the difference, because when a LLM pops out a message and then deletes it, that's a type 1 behaviour, whereas if the first thing it outputs directly is a sequence of tokens saying (any variant of) "nope, not gonna do that" that's type 2 behaviour. This appears to be what's described in this thread: https://old.reddit.com/r/bing/comments/11fryce/why_do_bings_responses_sometimes_delete_themselves/ https://old.reddit.com/r/bing/comments/11fryce/why_do_bings_... The research into going from type 2 to type 3 is the entirety of the article. > Your framing only makes sense when "Bad" is something so bad that we can't bear its existence, as opposed to just "commercially bad" where it shouldn't behave that way with an end user. In the latter, your choice 1 - imposing external guardrails - is fine. I disagree, I think my framing applies to all cases. Right now, LLMs are like old PCs with no user accounts and a single shared memory space, which is fine and dandy when you're not facing malicious input, but we live in a world with malicious input. You might be able to use a type 1 solution, but it's going to be fragile, and more pertinently, slow, as you only know to reject content once it has finished and may therefore end up in an unbounded loop of an LLM generating content that a censor rejects. A type 2 solution is still fragile, but it just doesn't make the "bad" content in the first place — and, to be clear, "bad" in this context can be anything undesired, including "uses vocabulary too advanced for a 5 year old who just started school" if that's what you care about using some specific LLM for.
nullc 2y ago
I've wondered before if it was possible to unlearn facts, but retain the general "reasoning" capability that came from being trained on the facts, then dimensionality reduce the model.
- andy99 2y ago
  If you think of knowledge as a (knowledge) graph, it seems there would be some nodes with low centrality that you could drop without much effect, and other key ones that would have a bigger impact if lost.
- huygens6363 2y ago
  Yes, me too. If it could somehow remember the “structure” instead of the instantiation. More “relationships between types of token relationships” instead of “relationships between tokens”.
- Brian_K_White 2y ago
  I don't know about in AI, but it seems like that is what humans do. We remember some facts but I know at least I have had a lot of facts pass through me and only leave their effects. I once had some facts, did some reasoning, arrived at a conclusion, and only retained the conclusion and enough of the reasoning to identify other contexts where the same reasoning should apply. I no longer have the facts, I simply trust my earlier selfs process of reasoning, and even that isn't actually trust or faith because I also still reason about new things today and observe the process. But I also evolve. I don't only trust a former reasoning unchanging forever. It's just that when I do revisit something and basically "reproduce the other scientists work" even if I arrive a different conclusion today, I'm generally still ok with the earlier me's reasoning and conclusion. It stands up as reasonable, and the new conclusion is usually just tuned a little, not wildly opposite. Or some things do change radically but I always knew they might, like in the process of self discovery you try a lot of opposite things. Getting a little away from the point but the point is I think the way we ourselves develop answer-generating-rules is very much by retaining only the results (the developed rules) and not all the facts and steps of the work, at least much of the time. Certainly we remember some justifying / exemplifying facts to explain some things we do.
- 2y ago
motohagiography 2y ago
seems like there is a basic problem where if you specify something to be unlearned, it could still be re-learned by inference and prompting. the solution may not be in filtering the proscribed facts or data itself, but in the weights and incentives that form a final layer of reasoning. Look at "safe" models now like google's last launch, where the results were often unsatisfying, as clearly we don't want truthful models yet, but we want ones that enable our ability to develop them further, which for now means not selecting out by antagonizing other social stakeholders. maybe we can encode and weight some principle of the models having been created by something external, with some loosely defined examples they can refer to as a way to evaluate what they return, then ones that don't yield those results cease to be used, where the ones that find a way to align will get reused to train others. there will absolutely be bad ones, but in aggregate they should produce something more desirable, and if they really go off the rails, just send a meteor. the argument in how models can "unlearn" will be between those who favour incentives and those who favour rules- likely, incentives for ones I create, but rules for everyone elses'.
- musicale 2y ago
  It is unsurprising that a system trained on human-generated content might end up encoding implicit bias, toxicity, and negative goals. And the more powerful and general-purpose a system is, the more suitable it is for a wide range of powerfully negative purposes. Neither specializing the model nor filtering its output seems to have worked reliably in practice.
gotoeleven 2y ago
My new startup includes a pitchfork wielding mob in the ML training loop.
avi_vallarapu 2y ago
We need to consider the practicality of unlearning methods in real-world applications and the legal acceptance of the same. Given current technology and what advancements are needed to make Unlearning more possible, probably there should be a time-to-unlearn kind of an acceptable agreement that allows organizations to retrain or tune the response that does not involve any response from the to-be-unlearned copyright content. Ultimately, legal acceptance for unlearning may be all about deleting the data set that is part of any kind of violations from the training data set. It may be very challenging to otherwise prove legally through the proposed unlearning techniques, that the model does not produce any type of response involving the private data. The actual data set contains the private data violating privacy or copyright, and the model is trained on it, period. This means, it must involve retraining by deleting the documents/data to be unlearned.
- isodev 2y ago
  > a time-to-unlearn kind of an acceptable agreement Why put the burden to end users? I think the technology should allow for unlearning and even "never learn about me in any future models and derivative models".
  - avi_vallarapu 2y ago
    No technology can guarantee 100% unlearning, and the only 100% guarantee is when the data is deleted before the model is retrained. Legally, even 99.99% accuracy may not be acceptable, but, only 100%.
    - eru 2y ago
      Or rather some legal fiction that you can pretend is 100%. You can never achieve real 100% in practice after all. Eg the random initialisation of weights might already encode all the 'bad' stuff you don't want. Extremely unlikely, but not strictly 0% unlikely. The law cuts off at some point, and declares it 100%.
      - isodev 2y ago
aidenn0 2y ago
I think "unlearning" is not the actual goal; we don't want the model to stick its proverbial head in the sand. Being unaware of racism is different from not producing racist content (and, in fact, one could argue that it is necessary to know about racism if one wishes to inhibit producing racist content; I remember in elementary school certain kids thought it would be funny to teach one of the special-ed kids to parrot offensive sentences).
- krono 2y ago
  Say you tell me you want a red sphere. Taken at face value, you show a prejudice for red sphere's and discriminate against all other coloured shapes. We've all had to dance that dance with ChatGPT by now, where you ask for something perfectly ordinary, but receive a response telling you off for even daring to think like that, until eventually you manage to formulate the prompt in a way that it likes with just the right context and winner vocabulary + grammar, and finally the damned thing gives you the info you want without so much as any gaslighting or snarky insults hiding in the answer! It doesn't understand racism, it simply evaluates certain combinations of things according to how it was set up to do.
greenavocado 2y ago
Please use the correct terminology: censorship
- danielmarkbruce 2y ago
  If company X wants their model to say/not say Y based on ideology, they aren't stopping anyone saying anything. They are stopping their own model saying something. The fact that I don't go around screaming nasty things about some group doesn't make me against free speech. It's censorship to try to stop people producing models as they see fit.
- 62951413 2y ago
  The prolefeed explains that deep duckspeaking is doubleplusgood. Nothing to see here, citizen.
- qbit42 2y ago
  I don't think that's a fair characterization. If a user requests a company to stop using their data, ML unlearning allows the company to do so without retraining their models from scratch.
- Dylan16807 2y ago
  Is it censorship to not include every piece of text you can possibly find into your training dataset? What's the difference between making that choice versus removing it from the model later?
surfingdino 2y ago
How about a radial approach? How about not ingesting all content but only that which is explicitly marked as available for model-building purposes?
xg15 2y ago
What I don't get about the DP approach is how this would be reconciled with the "exact" question-answering functionality of LLMs. DP makes perfect sense if all I care about is low-resolution statistical metrics or distributions of something and not the exact values - the entire purpose of DP is to prevent reconstructing the exact values. However, the expectation for LLMs is usually to ask a question (or request a task) and get an exact value as a response: If you ask "What's the phone number of John Smith?" the model will either tell you it doesn't know or it will answer you with an actual phone number (real or hallucinated). It will not tell you "the number is with 83% probability somewhere in New Jersey". So if the model is trained with DP, then either the data is scrambled enough that the it won't be able to return any kind of reliably correct data, effectively making it useless - or it's not scrambled enough, so that the model can successfully reconstuct data despite the scrambling process, effectively making the DP step useless. Or in other words, the OP defines "DP unlearning" as: > The intuition is that if an adversary cannot (reliably) tell apart the models, then it is as if this data point has never been learned—thus no need to unlearn. However, if my original model truthfully returns John Smith's phone number on request and the "unlearned" model must not be distinguishable by an outside observer from the original model, then the "unlearned" model will also return the phone number. While I could say that "technically" the model has never seen the phone number in the training data due to my DP scrambling, this doesn't solve the practical problem why the unlearning was requested in the first place, namely that John Smith doesn't want the model to return his phone number. He could probably care less about the specific details of the training process. So then, how would DP help here?
JKCalhoun 2y ago
I don't know — the post, reading the comments here, I am a little worried for the "sanity" of our AI that have been trained, untrained, retrained like a pawn in some kind of Cold War spy novel.
- kombookcha 2y ago
  It's fine, the LLM AIs we have now are just fancy versions of autocorrect. They, and other LMs, guess at statistically probable words/datapoints, and because they don't understand context, you might need to put your thumb on the scales to make the output actually be useful. They're at best very janky tools as soon as you're working with things that require context that isn't easily contained in some kind of confined area of work. Currently we are seeing the phenomenon 'habsburg AI' where AI's consume their own outputs as training data, which rapidly deteriorates their ability to actually be useful for much of anything. The thing is that there literally isn't enough human-made data to keep feeding them (they already ate the entire internet), so if you both want to continue ramping their intake of data and you also don't want them to get rapidly weird and completely useless, you pretty much have to get in there with elbow grease. Removing or deprioritizing data that's tripping up the model is one of the few ways you can do human-assisted refinement of these things. The sooner we all face the music that these things aren't magical truth machines, have a long way to go and there is no guaranteed rate of growth, the sooner this hype cycle can end.
joshhansen 2y ago
"Eternal Sunshine of the Spotless Mind" The erasure of knowledge is a troubling occupation
thomastjeffery 2y ago
So "unlearning" is yet another overconfident implementation of an impossible task. I guess that's par for the course. My favorite part is where literally pretending is presented as a serious option. Wow. Just..wow.

328 points by ignoramous 2y ago

deleted 2y ago

[deleted]

> However, RTBF wasn’t really proposed with machine learning in mind. In 2014, policymakers wouldn’t have predicted that deep learning will be a giant hodgepodge of data & compute Eh? Weren't deep learning and big data already things in 2014? Pretty sure everyone understood ML models would have a tough time and they still wanted RTBF.

peteradio 2y ago
I don't know if people anticipated contemporary parroting behavior over huge datasets. Modern well funded models can recall an obscure persons home address buried deep into the training set. I guess the techniques described might be presented to the European audience in an attempt to maintain access to their data/and or market for sales. I hope they fail.
isodev 2y ago
Of course, it’s not a regulation issue. The technology was introduced to users before it was ready. The very nature of training without opt-in consent or mechanism of being forgotten are all issues that should have been addressed before trying to make a keyboard with a special copilot button.
spennant 2y ago
Agreed. The media and advertising industry was most definitely leveraging cookie-level data for building attribution and targeting models. As soon as the EU established that this data was “personal data”, as it could, theoretically, be tied back to individual citizens, there were questions about the models. Namely “Would they have to be rebuilt after every RTBF request?” Needless to say, no one in the industry really wanted to address the question, as the wrong answer would essentially shut down a very profitable practice.
- Aerroon 2y ago
  More likely: the wrong answer would've shut out a profitable market rather than the practice. The EU is not the world. Anthropic seems to not mind blocking the EU for example.
  - spennant 2y ago
    Sure. But two things: 1) At the time, the European data laws implied that it protected its citizens no matter where they are. Nobody wanted to be the first to test that in court. 2) The organizations and agencies performing this type of data modeling were often doing so on behalf of large multinational organizations with absurd advertising spends, so they were dealing with Other People’s Data. The responsibility of scrubbing it clean of EU citizen data was unclear. What this meant was that an EU tourist who traveled to the US, and got served a targeted ad, could make a RTBF request to the advertiser (think Coca-Cola, Nestle or Unilever) The whole thing was a mess.
startupsfail 2y ago
RTBF was introduced to solve a specific issue, no? Politicians and their lobbyist friends could no longer remove materials linking them to their misdeeds as the first Google Search link associated with their names. Hence RTBF. Now, there’s similar issue with AI. Models are progressing towards being factual, useful and reliable.
indigovole 2y ago
GDPR and RTBF were formulated around the fears of data collection by the Stasi and other organizations. They were not formulated around easing the burdens of future entrepreneurs, but about mitigating the damage they might cause. Europeans were concerned about real harms that living people had experienced, not about enabling AGI or targeted advertising or digital personal assistants. We have posts here at least weekly from people cut off from their services, and their work along with them, because of bad inference, bad data, and inability to update metadata based purely on BigGo routine automation and indifference to individual harm. Imagine the scale that such damage will take when this automation and indifference to individual harm are structured around repositories from which data cannot be deleted, cannot be corrected.
hooby 2y ago
I'm pretty sure that the policymakers did NOT understand ML models in 2014 - and still do NOT understand it today. I also don't think that they care. They don't care that ML is a hodgepodge of data & compute, and they don't care how hard it is to remove data from a model. They didn't care about the ease or difficulty of removing data from more traditional types of knowledge storage either - like search indexes, database backups and whatnot. RTBF was not proposed with any specific technology in mind. What they had in mind, was to try and give individuals a tool, to keep their private information private. Like, if you have a private, unlisted phone number, and that number somehow ends up on the call-list of some pollster firm, you can force that firm to delete your number so that they can't call you anymore. The idea is, that if your private phone number (or similar data) ends up being shared or sold without your consent - you can try to undo the damage. In practice it might still be easier to get a new number, than to have your leaked one erased... but not all private data is exchangeable like that.

negative_person 2y ago

Why should we try to unlearn "bad" behaviours from AI? There is no AGI without violence, its part of being free thinking and self survival. But also by knowing that launching a first strike by a drunk president was a bad idea we averted a war because of a few people, AI needs to understand consequences. It seems futile to try and hide "bad" from AI.

andy99 2y ago
This is presumably about a chatbot though, not AGI, so it's basically a way of limiting what they say. (Not a way that I expect to succeed)
Cheer2171 2y ago
So you have a problem with supervised learning like spam classifiers?
williamtrask 2y ago
Because we can get AI related technologies to do things living creatures can’t, like provably forget things. And when it benefits us, we should. Personal opinion, but I think AGI is a good heuristic to build against but in the end we’ll pivot away. Sort of like how birds were a good heuristic for human flight, but modern planes don’t flap their wings and greatly exceed bird capabilities in many ways. Attribution for every prediction and deletion seem like prime examples of things which would break the analogy of AI/AGI with something more economically and politically compelling/competitive.
- negative_person 2y ago
  Can you point to any behaviour in human beings you'd unlearn if theyd also forget the consequences? We spend billions trying to predict human behaviour and yet we are surprised everyday, "AGI" will be no simpler. We just have to hope the dataset was aligned so the consequences are understood, and find a way to contain models that don't.
  - aeonik 2y ago

cwillu 2y ago

“to edit away undesired things like private data, stale knowledge, copyrighted materials, toxic/unsafe content, dangerous capabilities, and misinformation, without retraining models from scratch” To say nothing of unlearning those safeguards and/or “safeguards”.

ben_w 2y ago
It sounds like you're mistakenly grouping together three very different methods of changing an AI's behaviour. You have some model, M™, which can do Stuff. Some of the Stuff is, by your personal standards Bad (I don't care what your standard is, roll with this). You have three solutions: 1) Bolt on a post-processor which takes the output of M™, and if the output is detectably Bad, you censor it. Failure mode: this is trivial to remove, just delete the post-processor. Analogy: put secret documents into a folder called "secret do not read". 2) Retrain the weights within M™ to have a similar effect as 1. Failure mode: this is still fairly easy to remove, but will require re-training to get there. Why? Because the weights containing this information are not completely zeroed-out by this process. Analogy: how and why "un-deletion" is possible on file systems. 3) Find and eliminate the weights within M™ that lead to the Bad output. Analogy: "secure deletion" involves overwriting files with random data before unlinking them, possibly several times if it's a spinning disk. -- People are still doing research on 3 to make sure that it actually happens, what with it being of very high importance for a lot of different reasons including legal obligation.
- andy99 2y ago
  Until we have a very different method of actually controlling LLM behavior, 1 is the only feasible one. Your framing only makes sense when "Bad" is something so bad that we can't bear its existence, as opposed to just "commercially bad" where it shouldn't behave that way with an end user. In the latter, your choice 1 - imposing external guardrails - is fine. I'm not aware of anything LLMs can do that fits in the former category.
  - ben_w 2y ago
    > Until we have a very different method of actually controlling LLM behavior, 1 is the only feasible one. Most of the stuff I've seen, is 2. I've only seen a few places use 1 — you can tell the difference, because when a LLM pops out a message and then deletes it, that's a type 1 behaviour, whereas if the first thing it outputs directly is a sequence of tokens saying (any variant of) "nope, not gonna do that" that's type 2 behaviour. This appears to be what's described in this thread: https://old.reddit.com/r/bing/comments/11fryce/why_do_bings_responses_sometimes_delete_themselves/ https://old.reddit.com/r/bing/comments/11fryce/why_do_bings_... The research into going from type 2 to type 3 is the entirety of the article. > Your framing only makes sense when "Bad" is something so bad that we can't bear its existence, as opposed to just "commercially bad" where it shouldn't behave that way with an end user. In the latter, your choice 1 - imposing external guardrails - is fine. I disagree, I think my framing applies to all cases. Right now, LLMs are like old PCs with no user accounts and a single shared memory space, which is fine and dandy when you're not facing malicious input, but we live in a world with malicious input. You might be able to use a type 1 solution, but it's going to be fragile, and more pertinently, slow, as you only know to reject content once it has finished and may therefore end up in an unbounded loop of an LLM generating content that a censor rejects. A type 2 solution is still fragile, but it just doesn't make the "bad" content in the first place — and, to be clear, "bad" in this context can be anything undesired, including "uses vocabulary too advanced for a 5 year old who just started school" if that's what you care about using some specific LLM for.

nullc 2y ago

I've wondered before if it was possible to unlearn facts, but retain the general "reasoning" capability that came from being trained on the facts, then dimensionality reduce the model.

andy99 2y ago
If you think of knowledge as a (knowledge) graph, it seems there would be some nodes with low centrality that you could drop without much effect, and other key ones that would have a bigger impact if lost.
huygens6363 2y ago
Yes, me too. If it could somehow remember the “structure” instead of the instantiation. More “relationships between types of token relationships” instead of “relationships between tokens”.
Brian_K_White 2y ago
I don't know about in AI, but it seems like that is what humans do. We remember some facts but I know at least I have had a lot of facts pass through me and only leave their effects. I once had some facts, did some reasoning, arrived at a conclusion, and only retained the conclusion and enough of the reasoning to identify other contexts where the same reasoning should apply. I no longer have the facts, I simply trust my earlier selfs process of reasoning, and even that isn't actually trust or faith because I also still reason about new things today and observe the process. But I also evolve. I don't only trust a former reasoning unchanging forever. It's just that when I do revisit something and basically "reproduce the other scientists work" even if I arrive a different conclusion today, I'm generally still ok with the earlier me's reasoning and conclusion. It stands up as reasonable, and the new conclusion is usually just tuned a little, not wildly opposite. Or some things do change radically but I always knew they might, like in the process of self discovery you try a lot of opposite things. Getting a little away from the point but the point is I think the way we ourselves develop answer-generating-rules is very much by retaining only the results (the developed rules) and not all the facts and steps of the work, at least much of the time. Certainly we remember some justifying / exemplifying facts to explain some things we do.
2y ago

motohagiography 2y ago

seems like there is a basic problem where if you specify something to be unlearned, it could still be re-learned by inference and prompting. the solution may not be in filtering the proscribed facts or data itself, but in the weights and incentives that form a final layer of reasoning. Look at "safe" models now like google's last launch, where the results were often unsatisfying, as clearly we don't want truthful models yet, but we want ones that enable our ability to develop them further, which for now means not selecting out by antagonizing other social stakeholders. maybe we can encode and weight some principle of the models having been created by something external, with some loosely defined examples they can refer to as a way to evaluate what they return, then ones that don't yield those results cease to be used, where the ones that find a way to align will get reused to train others. there will absolutely be bad ones, but in aggregate they should produce something more desirable, and if they really go off the rails, just send a meteor. the argument in how models can "unlearn" will be between those who favour incentives and those who favour rules- likely, incentives for ones I create, but rules for everyone elses'.

musicale 2y ago
It is unsurprising that a system trained on human-generated content might end up encoding implicit bias, toxicity, and negative goals. And the more powerful and general-purpose a system is, the more suitable it is for a wide range of powerfully negative purposes. Neither specializing the model nor filtering its output seems to have worked reliably in practice.

gotoeleven 2y ago

My new startup includes a pitchfork wielding mob in the ML training loop.

avi_vallarapu 2y ago

We need to consider the practicality of unlearning methods in real-world applications and the legal acceptance of the same. Given current technology and what advancements are needed to make Unlearning more possible, probably there should be a time-to-unlearn kind of an acceptable agreement that allows organizations to retrain or tune the response that does not involve any response from the to-be-unlearned copyright content. Ultimately, legal acceptance for unlearning may be all about deleting the data set that is part of any kind of violations from the training data set. It may be very challenging to otherwise prove legally through the proposed unlearning techniques, that the model does not produce any type of response involving the private data. The actual data set contains the private data violating privacy or copyright, and the model is trained on it, period. This means, it must involve retraining by deleting the documents/data to be unlearned.

isodev 2y ago
> a time-to-unlearn kind of an acceptable agreement Why put the burden to end users? I think the technology should allow for unlearning and even "never learn about me in any future models and derivative models".
- avi_vallarapu 2y ago
  No technology can guarantee 100% unlearning, and the only 100% guarantee is when the data is deleted before the model is retrained. Legally, even 99.99% accuracy may not be acceptable, but, only 100%.
  - eru 2y ago
    Or rather some legal fiction that you can pretend is 100%. You can never achieve real 100% in practice after all. Eg the random initialisation of weights might already encode all the 'bad' stuff you don't want. Extremely unlikely, but not strictly 0% unlikely. The law cuts off at some point, and declares it 100%.
    - isodev 2y ago

aidenn0 2y ago

I think "unlearning" is not the actual goal; we don't want the model to stick its proverbial head in the sand. Being unaware of racism is different from not producing racist content (and, in fact, one could argue that it is necessary to know about racism if one wishes to inhibit producing racist content; I remember in elementary school certain kids thought it would be funny to teach one of the special-ed kids to parrot offensive sentences).

krono 2y ago
Say you tell me you want a red sphere. Taken at face value, you show a prejudice for red sphere's and discriminate against all other coloured shapes. We've all had to dance that dance with ChatGPT by now, where you ask for something perfectly ordinary, but receive a response telling you off for even daring to think like that, until eventually you manage to formulate the prompt in a way that it likes with just the right context and winner vocabulary + grammar, and finally the damned thing gives you the info you want without so much as any gaslighting or snarky insults hiding in the answer! It doesn't understand racism, it simply evaluates certain combinations of things according to how it was set up to do.

greenavocado 2y ago

Please use the correct terminology: censorship

danielmarkbruce 2y ago
If company X wants their model to say/not say Y based on ideology, they aren't stopping anyone saying anything. They are stopping their own model saying something. The fact that I don't go around screaming nasty things about some group doesn't make me against free speech. It's censorship to try to stop people producing models as they see fit.
62951413 2y ago
The prolefeed explains that deep duckspeaking is doubleplusgood. Nothing to see here, citizen.
qbit42 2y ago
I don't think that's a fair characterization. If a user requests a company to stop using their data, ML unlearning allows the company to do so without retraining their models from scratch.
Dylan16807 2y ago
Is it censorship to not include every piece of text you can possibly find into your training dataset? What's the difference between making that choice versus removing it from the model later?

surfingdino 2y ago

How about a radial approach? How about not ingesting all content but only that which is explicitly marked as available for model-building purposes?

xg15 2y ago

What I don't get about the DP approach is how this would be reconciled with the "exact" question-answering functionality of LLMs. DP makes perfect sense if all I care about is low-resolution statistical metrics or distributions of something and not the exact values - the entire purpose of DP is to prevent reconstructing the exact values. However, the expectation for LLMs is usually to ask a question (or request a task) and get an exact value as a response: If you ask "What's the phone number of John Smith?" the model will either tell you it doesn't know or it will answer you with an actual phone number (real or hallucinated). It will not tell you "the number is with 83% probability somewhere in New Jersey". So if the model is trained with DP, then either the data is scrambled enough that the it won't be able to return any kind of reliably correct data, effectively making it useless - or it's not scrambled enough, so that the model can successfully reconstuct data despite the scrambling process, effectively making the DP step useless. Or in other words, the OP defines "DP unlearning" as: > The intuition is that if an adversary cannot (reliably) tell apart the models, then it is as if this data point has never been learned—thus no need to unlearn. However, if my original model truthfully returns John Smith's phone number on request and the "unlearned" model must not be distinguishable by an outside observer from the original model, then the "unlearned" model will also return the phone number. While I could say that "technically" the model has never seen the phone number in the training data due to my DP scrambling, this doesn't solve the practical problem why the unlearning was requested in the first place, namely that John Smith doesn't want the model to return his phone number. He could probably care less about the specific details of the training process. So then, how would DP help here?

JKCalhoun 2y ago

I don't know — the post, reading the comments here, I am a little worried for the "sanity" of our AI that have been trained, untrained, retrained like a pawn in some kind of Cold War spy novel.

kombookcha 2y ago
It's fine, the LLM AIs we have now are just fancy versions of autocorrect. They, and other LMs, guess at statistically probable words/datapoints, and because they don't understand context, you might need to put your thumb on the scales to make the output actually be useful. They're at best very janky tools as soon as you're working with things that require context that isn't easily contained in some kind of confined area of work. Currently we are seeing the phenomenon 'habsburg AI' where AI's consume their own outputs as training data, which rapidly deteriorates their ability to actually be useful for much of anything. The thing is that there literally isn't enough human-made data to keep feeding them (they already ate the entire internet), so if you both want to continue ramping their intake of data and you also don't want them to get rapidly weird and completely useless, you pretty much have to get in there with elbow grease. Removing or deprioritizing data that's tripping up the model is one of the few ways you can do human-assisted refinement of these things. The sooner we all face the music that these things aren't magical truth machines, have a long way to go and there is no guaranteed rate of growth, the sooner this hype cycle can end.

joshhansen 2y ago

"Eternal Sunshine of the Spotless Mind" The erasure of knowledge is a troubling occupation

thomastjeffery 2y ago

So "unlearning" is yet another overconfident implementation of an impossible task. I guess that's par for the course. My favorite part is where literally pretending is presented as a serious option. Wow. Just..wow.