4 ms·

To my intuition, all of these ways of building memory in “text space” seem super hacky. It seems intuitive to me that memory would be best stored in dense embe

To my intuition, all of these ways of building memory in “text space” seem super hacky. It seems intuitive to me that memory would be best stored in dense embedding space that can preserve full semantic meaning for the model rather than as some hacked on process of continually regenerating summaries. And similarly, the model needs to be trained in a setting where it is aware of the memory and how to use it. Preferably that would be from the very beginning (ie. the train on text).

SubiculumCode 3y ago
It seems to me that sparse encodings would be more efficient and practical for medium term memory. Isn't rhe problem wirh dense embedding is memory usage?
- whimsicalism 3y ago
  > Isn't rhe problem wirh dense embedding is memory usage? Not necessarily, given that a dense embedding can encode the equivalent of many many words or even higher order concepts not easily expressed in word space.
nottheengineer 3y ago
It does seem hacky, but then again the whole concept of conversational LLMs is. You're just asking it to add an extra word to a given conversation and after a bit, it spits out an end token that tells your application to hand control back to the user. I think latent space and text space aren't as far apart as you think. LLMs are pretty stupid, but very good at speech. They are good at writing code because that's very similar, but fall apart in things that need some actual abstract thinking, like math. Those text space hacks do tend to work and stuff like "think step by step" has become common because of that. LoRAs are closer to what you mean and they're great at packing a lot of understanding into very little data. But adjusting weights for a single conversation just isn't feasible yet, so we're exploring text space for that purpose. Maybe someome will transfer the methods we discover in text space to embedding space to make them more efficient, but that's for the future.
- famouswaffles 3y ago
  >They are good at writing code because that's very similar, but fall apart in things that need some actual abstract thinking, like math. Pretty odd assertion. LLMs are not "good at speech, bad at abstract thinking". What do these have to do with speech ? https://general-pattern-machines.github.io/ https://general-pattern-machines.github.io/ https://arxiv.org/abs/2212.09196 https://arxiv.org/abs/2212.09196 It doesn't even hold with your example because GPT-4 is pretty good at Math, nowhere near "falling apart".
  - riku_iki 3y ago
    > GPT-4 is pretty good at Math, nowhere near "falling apart". its good at tasks which were included into training dataset in some variations.
    - PaulHoule 3y ago
      Kinda able to do some math tasks some of the time whereas you can use techniques from the arithmetic textbook to get the right answer all of the time with millions of times less CPU even including the overhead of round-tripping to ASCII numerals which is shockingly large compared to what a multiply costs. Kinda "the problem" with LLMS is that they successfully seduce people by seeming to get the right answer to anything 80% of the time.
      - famouswaffles 3y ago
        Math is a lot more than just Arithmetic.
        PaulHoule 3y ago
        Yeah but if you can only do arithmetic right X% of the time you aren't going to get other answers right as often as would really be useful. That said, LLMs have a magic ability to "short circuit" and get the right answer despite not being able to get the steps right. I remember scoping out designs for NLP systems about 5 years ago and frequently conclude that "that won't work" because information was lost at an early stage but in retrospect by short circuiting a system like that can outperform its parts but it still faces a ceiling on how accurate the answers are because the reasoning is not sound.
fnordpiglet 3y ago
I would note that almost everything in computing we use today is super hacky stuff sufficiently abstracted and error handled such that it seems like it’s not at all a hack.
- bigyikes 3y ago
  I would note that all of biology is hacks on top of hacks! Have you seen a human being? They have this useless fucking organ that tends to burst and kill them.
  - elevaet 3y ago
    This is getting way off topic, but the appendix may have an important role as a refuge for the gut microbiome: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3551545/#:~:text=theorized%20that%20the%20vermiform%20appendix,reservoir%20(3%2C%204) https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3551545/#:~:tex....
tinco 3y ago
Why do you have the intuition that a dense embedding space could preserve full semantic meaning? From what I understand from embeddings is that they are inherently lossy. At least with a textual summary you could have an agent verify the summary actually accurately represents the information that it's meant to summarize.
- chefandy 3y ago
  From a technical perspective, sure: it's clear why it functions like that, and there's no technical reason it shouldn't. From a user interface perspective— likely what most people would judge with intuition— that doesn't matter. Processes that mimick familiar interaction patterns cause dissonance when they don't match our mental model of those interactions, and familiar conversation is about as familiar as you get. People we interact with know the things we explicitly tell them or guide them through figuring out, and people intuitively interact with these applications as they would with people because they've deliberately adopted the voice of an individual person. Additionally, for SaaS products, we're used to them maintaining state automatically. (As annoying as our input can be, this is why dev teams that try to solve problems for non-dev users have designers.)
eganist 3y ago
> rather than as some hacked on process of continually regenerating summaries. incidentally, this isn't far off from how the human brain is believed to work (at least with long term memories). https://news.northwestern.edu/stories/2012/09/your-memory-is-like-the-telephone-game https://news.northwestern.edu/stories/2012/09/your-memory-is...
- sudokuist 3y ago
  No one knows how the brain works and how it is connected with the body. Did you know your gut is directly connected with cognition? An unhealthy digestive system has been linked with several neurodegenerative diseases. Also, walking and cardio in general is known to create new neurons and prevent cognitive degeneration. It's always funny to me when people on online forums confidently proclaim to know what cognition and thinking is all about and that furthermore it can be reduced to symbol shuffling on computers. No one has any clue how to make computers intelligent and anyone that claims to know is either doing marketing for some "AI" company or thoroughly confused about what counts as biological intelligence.
  - eganist 3y ago
    You're 100% right, no one knows how the brain works. And all the elements you described are probably relevant — including things you didn't mention, such as personality changes tied to heart transplants, etc. But that's probably reading a little too deeply and seriously into what I said.
    - sudokuist 3y ago
      [flagged]
  - btilly 3y ago
    Nobody knows, but this model works well enough to, for instance, treat PTSD. Read through https://www.ptsd.va.gov/understand_tx/emdr.asp https://www.ptsd.va.gov/understand_tx/emdr.asp to verify that.
  - 3y ago
digitcatphd 3y ago
Most things right now seem so. Seems like we are going through many rounds of iteration and guessing what works long term and what is a short term fix is frustrating
deleted 3y ago
[deleted]
3abiton 3y ago
But there are hardware limitations for storing them in memory
galaxyLogic 3y ago
Consider what happens if we use this method in our head. Recursively summarize the discussion so far. It will improve our memory. It seem 'hacky" to summarize things in your head but I think that is a big part of how memory works.
- coldtea 3y ago
  That's what memory is. Whether sensory information (what you remember for a place) or the points from a meeting (where we only remember the big picture items, not what was said minute to minute) it's all summarized/compressed.
- whimsicalism 3y ago
  This would be more equivalent to having some little spool in our head that writes down our summarized thoughts and then when we need to remember something we take the spool out of our head and look at it to reorient ourselves.
  - jonas21 3y ago
    Isn't that what people are doing when they take notes and read them later to refresh their memory?
amelius 3y ago
I'm wondering how well the data is preserved if you put it in a buffer like that. It reminds me of the game where someone tells A some story, then A tells it to B, B to C, etc. and then of course the end result is that the story became completely different.
skybrian 3y ago
It seems to me that there's a major advantage, though: you can read it and fix any mistakes.
kristopolous 3y ago
Sure but airplanes don't look like birds
- ChatGTP 3y ago
  They don't fly like them either...

SubiculumCode 3y ago

It seems to me that sparse encodings would be more efficient and practical for medium term memory. Isn't rhe problem wirh dense embedding is memory usage?

whimsicalism 3y ago
> Isn't rhe problem wirh dense embedding is memory usage? Not necessarily, given that a dense embedding can encode the equivalent of many many words or even higher order concepts not easily expressed in word space.

nottheengineer 3y ago

It does seem hacky, but then again the whole concept of conversational LLMs is. You're just asking it to add an extra word to a given conversation and after a bit, it spits out an end token that tells your application to hand control back to the user. I think latent space and text space aren't as far apart as you think. LLMs are pretty stupid, but very good at speech. They are good at writing code because that's very similar, but fall apart in things that need some actual abstract thinking, like math. Those text space hacks do tend to work and stuff like "think step by step" has become common because of that. LoRAs are closer to what you mean and they're great at packing a lot of understanding into very little data. But adjusting weights for a single conversation just isn't feasible yet, so we're exploring text space for that purpose. Maybe someome will transfer the methods we discover in text space to embedding space to make them more efficient, but that's for the future.

famouswaffles 3y ago
>They are good at writing code because that's very similar, but fall apart in things that need some actual abstract thinking, like math. Pretty odd assertion. LLMs are not "good at speech, bad at abstract thinking". What do these have to do with speech ? https://general-pattern-machines.github.io/ https://general-pattern-machines.github.io/ https://arxiv.org/abs/2212.09196 https://arxiv.org/abs/2212.09196 It doesn't even hold with your example because GPT-4 is pretty good at Math, nowhere near "falling apart".
- riku_iki 3y ago
  > GPT-4 is pretty good at Math, nowhere near "falling apart". its good at tasks which were included into training dataset in some variations.
  - PaulHoule 3y ago
    Kinda able to do some math tasks some of the time whereas you can use techniques from the arithmetic textbook to get the right answer all of the time with millions of times less CPU even including the overhead of round-tripping to ASCII numerals which is shockingly large compared to what a multiply costs. Kinda "the problem" with LLMS is that they successfully seduce people by seeming to get the right answer to anything 80% of the time.
    - famouswaffles 3y ago
      Math is a lot more than just Arithmetic.
      - PaulHoule 3y ago
        Yeah but if you can only do arithmetic right X% of the time you aren't going to get other answers right as often as would really be useful. That said, LLMs have a magic ability to "short circuit" and get the right answer despite not being able to get the steps right. I remember scoping out designs for NLP systems about 5 years ago and frequently conclude that "that won't work" because information was lost at an early stage but in retrospect by short circuiting a system like that can outperform its parts but it still faces a ceiling on how accurate the answers are because the reasoning is not sound.

fnordpiglet 3y ago

I would note that almost everything in computing we use today is super hacky stuff sufficiently abstracted and error handled such that it seems like it’s not at all a hack.

bigyikes 3y ago
I would note that all of biology is hacks on top of hacks! Have you seen a human being? They have this useless fucking organ that tends to burst and kill them.
- elevaet 3y ago
  This is getting way off topic, but the appendix may have an important role as a refuge for the gut microbiome: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3551545/#:~:text=theorized%20that%20the%20vermiform%20appendix,reservoir%20(3%2C%204) https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3551545/#:~:tex....

tinco 3y ago

Why do you have the intuition that a dense embedding space could preserve full semantic meaning? From what I understand from embeddings is that they are inherently lossy. At least with a textual summary you could have an agent verify the summary actually accurately represents the information that it's meant to summarize.

chefandy 3y ago
From a technical perspective, sure: it's clear why it functions like that, and there's no technical reason it shouldn't. From a user interface perspective— likely what most people would judge with intuition— that doesn't matter. Processes that mimick familiar interaction patterns cause dissonance when they don't match our mental model of those interactions, and familiar conversation is about as familiar as you get. People we interact with know the things we explicitly tell them or guide them through figuring out, and people intuitively interact with these applications as they would with people because they've deliberately adopted the voice of an individual person. Additionally, for SaaS products, we're used to them maintaining state automatically. (As annoying as our input can be, this is why dev teams that try to solve problems for non-dev users have designers.)

eganist 3y ago

> rather than as some hacked on process of continually regenerating summaries. incidentally, this isn't far off from how the human brain is believed to work (at least with long term memories). https://news.northwestern.edu/stories/2012/09/your-memory-is-like-the-telephone-game https://news.northwestern.edu/stories/2012/09/your-memory-is...

sudokuist 3y ago
No one knows how the brain works and how it is connected with the body. Did you know your gut is directly connected with cognition? An unhealthy digestive system has been linked with several neurodegenerative diseases. Also, walking and cardio in general is known to create new neurons and prevent cognitive degeneration. It's always funny to me when people on online forums confidently proclaim to know what cognition and thinking is all about and that furthermore it can be reduced to symbol shuffling on computers. No one has any clue how to make computers intelligent and anyone that claims to know is either doing marketing for some "AI" company or thoroughly confused about what counts as biological intelligence.
- eganist 3y ago
  You're 100% right, no one knows how the brain works. And all the elements you described are probably relevant — including things you didn't mention, such as personality changes tied to heart transplants, etc. But that's probably reading a little too deeply and seriously into what I said.
  - sudokuist 3y ago
    [flagged]
- btilly 3y ago
  Nobody knows, but this model works well enough to, for instance, treat PTSD. Read through https://www.ptsd.va.gov/understand_tx/emdr.asp https://www.ptsd.va.gov/understand_tx/emdr.asp to verify that.
- 3y ago

digitcatphd 3y ago

Most things right now seem so. Seems like we are going through many rounds of iteration and guessing what works long term and what is a short term fix is frustrating

deleted 3y ago

[deleted]

3abiton 3y ago

But there are hardware limitations for storing them in memory

galaxyLogic 3y ago

Consider what happens if we use this method in our head. Recursively summarize the discussion so far. It will improve our memory. It seem 'hacky" to summarize things in your head but I think that is a big part of how memory works.

coldtea 3y ago
That's what memory is. Whether sensory information (what you remember for a place) or the points from a meeting (where we only remember the big picture items, not what was said minute to minute) it's all summarized/compressed.
whimsicalism 3y ago
This would be more equivalent to having some little spool in our head that writes down our summarized thoughts and then when we need to remember something we take the spool out of our head and look at it to reorient ourselves.
- jonas21 3y ago
  Isn't that what people are doing when they take notes and read them later to refresh their memory?

amelius 3y ago

I'm wondering how well the data is preserved if you put it in a buffer like that. It reminds me of the game where someone tells A some story, then A tells it to B, B to C, etc. and then of course the end result is that the story became completely different.

skybrian 3y ago

It seems to me that there's a major advantage, though: you can read it and fix any mistakes.

kristopolous 3y ago

Sure but airplanes don't look like birds

ChatGTP 3y ago
They don't fly like them either...