3 ms·

> This isn’t a failure of neural networks. It’s a feature. It’s why you’re not flooded with every single memory and experience you’ve ever had every moment. Th

> This isn’t a failure of neural networks. It’s a feature. It’s why you’re not flooded with every single memory and experience you’ve ever had every moment. This is an interesting point, and made me think on whether this "reversal curse" is something we experience with our own, human neural networks. I think it is. Like, I can imagine being given a character in a movie, being able to tell you what actor played them, but given the actor and the movie, not being able to tell you what character they played, or vice-versa. So the pairing exists in my brain somewhere, but I only have an accessible pointer to one side. I think we run into cases like this all the time, actually.

dj_mc_merlin 3y ago
Another similar bug/feature of the brain is how we can immediately know if we like a movie or a book, but if asked what movies or books we like, we often blank or name like 3 pieces. Some information is only designed to be retrieved in certain ways in our brain. The fact that neural networks have similar but different limitations isn't that concerning, just something to keep in mind. Another funny limitation: when a word is "on the tip of your tongue", what often happened is you thought of it, but your brain rejected it, so now it's on "cooldown" before it appears in your mind again. Usually this mechanism helps but a bug in your brain causes it to be harmful. If you start thinking of something else it "refreshes the cache" and the word comes to you.. we're really not that much less buggy than the machine. disclaimer: there's other theories about why the phenomenon happens
- thehappypm 3y ago
  Interestingly, I think the “favorite movies” scenario is also an artifact of our training dataset (our experiences). You spend hours watching a movie, so you have a lot of data about it, and have a lot to build an opinion. But comparing movies? Not something we think about for hours and hours on end. But, people who are movie critics or movie buffs train themselves to do it.
- azeirah 3y ago
  Yeah this becomes very noticeable when you're deliberately learning facts with Anki. For example when trying to learn immunology, "This bacterium causes neural bacteriumeritis", score 100% on the set of cards after a few sessions. Add "What are the possible causes of neural bacteriumeritis?" and sometimes I'd draw a blank. There's a lot of nuance in how we learn facts and their relationships. It reminds me a lot of data structures in programming to be honest. It's a lot easier to retrieve a person's age from a dictionary indexed by first-name+last-name than from a list with 10000 objects with people in them.
rdedev 3y ago
Reading that article I can't help but see similarities between what we have in the human brain. We can easily form and recall short term memories, even using them to logically reason out facts, but loose such memories if it's not frequent enough. So it seems like the context prompt is similar to short term memory and what's missing now is a good way to transfer the short term to long term. There is a huge amount of assumptions going on here so take it with a huge amount of salt. On another note I should have read the actual paper mentioned in the post critically instead of just skimming it. I completely missed the footnote about the reversal curse not being a problem if everything is present in the initial prompt

dj_mc_merlin 3y ago

Another similar bug/feature of the brain is how we can immediately know if we like a movie or a book, but if asked what movies or books we like, we often blank or name like 3 pieces. Some information is only designed to be retrieved in certain ways in our brain. The fact that neural networks have similar but different limitations isn't that concerning, just something to keep in mind. Another funny limitation: when a word is "on the tip of your tongue", what often happened is you thought of it, but your brain rejected it, so now it's on "cooldown" before it appears in your mind again. Usually this mechanism helps but a bug in your brain causes it to be harmful. If you start thinking of something else it "refreshes the cache" and the word comes to you.. we're really not that much less buggy than the machine. disclaimer: there's other theories about why the phenomenon happens

thehappypm 3y ago
Interestingly, I think the “favorite movies” scenario is also an artifact of our training dataset (our experiences). You spend hours watching a movie, so you have a lot of data about it, and have a lot to build an opinion. But comparing movies? Not something we think about for hours and hours on end. But, people who are movie critics or movie buffs train themselves to do it.
azeirah 3y ago
Yeah this becomes very noticeable when you're deliberately learning facts with Anki. For example when trying to learn immunology, "This bacterium causes neural bacteriumeritis", score 100% on the set of cards after a few sessions. Add "What are the possible causes of neural bacteriumeritis?" and sometimes I'd draw a blank. There's a lot of nuance in how we learn facts and their relationships. It reminds me a lot of data structures in programming to be honest. It's a lot easier to retrieve a person's age from a dictionary indexed by first-name+last-name than from a list with 10000 objects with people in them.

rdedev 3y ago

Reading that article I can't help but see similarities between what we have in the human brain. We can easily form and recall short term memories, even using them to logically reason out facts, but loose such memories if it's not frequent enough. So it seems like the context prompt is similar to short term memory and what's missing now is a good way to transfer the short term to long term. There is a huge amount of assumptions going on here so take it with a huge amount of salt. On another note I should have read the actual paper mentioned in the post critically instead of just skimming it. I completely missed the footnote about the reversal curse not being a problem if everything is present in the initial prompt