3 ms·

While this post seems aimed more at compliance or sensitive data issues, “unlearning” aka forgetting itself may be essential for better or more human like AI ag

While this post seems aimed more at compliance or sensitive data issues, “unlearning” aka forgetting itself may be essential for better or more human like AI agents. You are as much defined by what you have forgotten as what you have learned.

version_five 3y ago
I've heard than some mental illness (maybe schizophrenia) is theorized to be in part due to an inability to forget things. Certainly normal human existence involves preferential remembering and forgetting. For example not dwelling on pain or failure etc. Gradient descent obviously has none of that, it has not feelings or goals, and so there would be no preferential remembering or forgetting other than to do better next word prediction or rlhf or whatever. So it would be interesting to think about what a model should remember or forget in order to align it with our goals (because it doesn't have any of it's own). Also, don't volunteer for Google. They have lots of money, they can pay for this stuff and if you know how to do it you have lots of options.
- pessimizer 3y ago
  Yeah, the prize for doing the best on this challenge should be a piece of the company.
- B1FF_PSUVM 3y ago
  > I've heard than some mental illness (maybe schizophrenia) is theorized to be in part due to an inability to forget things. My totally amateur pet theory is that paranoia is threat pattern recognition gone bonkers. Actually, most of what we do with brains is pattern recognition, and if there isn't enough good input they will make shit up.
  - bitcurious 3y ago
    > My totally amateur pet theory is that paranoia is threat pattern recognition gone bonkers. Supposedly OCD is your brain’s cause-effect loop being too potent. As in, you have a random fear I.e. “stove is on, fire will burn down house,” you go to check the stove, and the act of checking (regardless of its being on) creates the feeling that you saved your house from burning down, so now you feel compelled to check every time. Paraphrased from the Huberman podcast.
    - __loam 3y ago
      I have OCD. This is basically accurate in that it's an anxiety disorder that affects your ability to judge whether a worst case scenario is likely or not, but the thoughts are more like "What if I drive across the median and kill a family of 5 in a minivan" or "What if I pull my genitals out in this meeting and try to fuck my boss". The compulsions are usually not as logically coherent as needing to check if the stove is on or the house will burn down (though there are people who experience ocd like this). Usually my compulsions happen as a way to try and stop thinking those thoughts, like pulling over and playing internet chess for half an hour. The problem is that the compulsions steal a lot of time and don't typically address the underlying obsessive thought.
- inglor_cz 3y ago
  In 1999 I dated a girl whose father was a military doctor. He was the only person whom I ever knew to have an almost total recall. Like, you asked him what happened on this particular day five years before, and he told you exactly what. If there were records (like photos), they would match his words. He was also quite unassuming and very "normal". No trace of mental dysbalance.
Barrin92 3y ago
it is certainly better for all AI agents, not just human like systems, because it is almost the definition of intelligence. Forgetting the irrelevant stuff and having a system that does as much as possible with as little information as possible is what learning is, compression in other terms.
vintermann 3y ago
I always though it funny that in short term, LMMs are (or can be) much better at forgetting than humans can. If a person tells you, "I'm going to forget you ever said that", you know they won't actually forget. An LMM, on the other hand, can just drop the context, and they're literally going to forget it ever happened. They already train chatbot agents with a special "end the conversation" token, it would be easy to train them with a "drop the context" token too.

version_five 3y ago

I've heard than some mental illness (maybe schizophrenia) is theorized to be in part due to an inability to forget things. Certainly normal human existence involves preferential remembering and forgetting. For example not dwelling on pain or failure etc. Gradient descent obviously has none of that, it has not feelings or goals, and so there would be no preferential remembering or forgetting other than to do better next word prediction or rlhf or whatever. So it would be interesting to think about what a model should remember or forget in order to align it with our goals (because it doesn't have any of it's own). Also, don't volunteer for Google. They have lots of money, they can pay for this stuff and if you know how to do it you have lots of options.

pessimizer 3y ago
Yeah, the prize for doing the best on this challenge should be a piece of the company.
B1FF_PSUVM 3y ago
> I've heard than some mental illness (maybe schizophrenia) is theorized to be in part due to an inability to forget things. My totally amateur pet theory is that paranoia is threat pattern recognition gone bonkers. Actually, most of what we do with brains is pattern recognition, and if there isn't enough good input they will make shit up.
- bitcurious 3y ago
  > My totally amateur pet theory is that paranoia is threat pattern recognition gone bonkers. Supposedly OCD is your brain’s cause-effect loop being too potent. As in, you have a random fear I.e. “stove is on, fire will burn down house,” you go to check the stove, and the act of checking (regardless of its being on) creates the feeling that you saved your house from burning down, so now you feel compelled to check every time. Paraphrased from the Huberman podcast.
  - __loam 3y ago
    I have OCD. This is basically accurate in that it's an anxiety disorder that affects your ability to judge whether a worst case scenario is likely or not, but the thoughts are more like "What if I drive across the median and kill a family of 5 in a minivan" or "What if I pull my genitals out in this meeting and try to fuck my boss". The compulsions are usually not as logically coherent as needing to check if the stove is on or the house will burn down (though there are people who experience ocd like this). Usually my compulsions happen as a way to try and stop thinking those thoughts, like pulling over and playing internet chess for half an hour. The problem is that the compulsions steal a lot of time and don't typically address the underlying obsessive thought.
inglor_cz 3y ago
In 1999 I dated a girl whose father was a military doctor. He was the only person whom I ever knew to have an almost total recall. Like, you asked him what happened on this particular day five years before, and he told you exactly what. If there were records (like photos), they would match his words. He was also quite unassuming and very "normal". No trace of mental dysbalance.

Barrin92 3y ago

it is certainly better for all AI agents, not just human like systems, because it is almost the definition of intelligence. Forgetting the irrelevant stuff and having a system that does as much as possible with as little information as possible is what learning is, compression in other terms.

vintermann 3y ago

I always though it funny that in short term, LMMs are (or can be) much better at forgetting than humans can. If a person tells you, "I'm going to forget you ever said that", you know they won't actually forget. An LMM, on the other hand, can just drop the context, and they're literally going to forget it ever happened. They already train chatbot agents with a special "end the conversation" token, it would be easy to train them with a "drop the context" token too.