4 ms·

I have an entire life worth of context and I still remember projects I worked on 15 years ago.

adastra22 1y ago
Not with pixel perfect accuracy. You vaguely remember, although it may not feel like that because your brain fills in the details (hallucinates) as you recall. The comparisons are closer than you might think.
- vidarh 1y ago
  The comparison would be apt if the LLM was trained on your codebase.
  - jimbokun 1y ago
    Isn’t that the problem? I don’t see any progress on incrementally training LLMs on specific projects. I believe it’s called fine tuning, right? Why isn’t that the default approach anywhere instead of the hack of bigger “context windows”?
    - adastra22 1y ago
      Because fine-tuning can be used to remove restrictions from a model, so they don't give us plebs access to that.
    - gerhardi 1y ago
      I’m not well versed enough on this but wouldn’t it be a problem with custom training that the specific project training codebases probably would likely have a lot of the implemented stuff, relevant for the domain, only once and in one way, compared to how the todays popular large models have been trained maybe with countless different ways to use common libraries for whatever various tasks with whatever Github ripped material fed in?
- krainboltgreene 1y ago
  You have no idea if I remember with pixel perfect accuracy (whatever that even means). There are plenty of people with photographic memory. Also, you're a programmer you have no foundation of knowledge on which to make that assessment. You might as well opine on quarks or martian cellular life. My god the arrogance of people in my industry.
  - johnisgood 1y ago
    > There are plenty of people with photographic memory. I thought it was rare.
  - adastra22 1y ago
    Repeated studies have shown that perfect "photographic memory" does not in fact exist. Nobody has it. Some people think that they do though, but when tested under lab conditions those claims don't hold up. I don't believe these people are lying. They are self-reporting their own experiences, which unfortunately have the annoying property of being generated by the very mind that is living the experience. What does it mean to have an eidetic memory? It means that when you remember something you vividly remember details, and can examine those details to your heart's content. When you do so, it feels like all those details are correct. (Or so I'm told, I'm the opposite with aphantasia.) But it turns out if you actually have a photo reference and do a blind comparison test, people who report photographic memories actually don't do statistically any better than others in remembering specific fine details, even though they claim that they clearly remember. The simpler explanation is that while all of our brains are provide hallucinated detail to fill the gaps of memories, their brains are wired up to present those made up details feel much more real than they do to others. That is all.
    - HarHarVeryFunny 1y ago
      > Repeated studies have shown that perfect "photographic memory" does not in fact exist. This may change your mind! https://www.youtube.com/watch?v=jVqRT_kCOLI https://www.youtube.com/watch?v=jVqRT_kCOLI
      - 1y ago

Not with pixel perfect accuracy. You vaguely remember, although it may not feel like that because your brain fills in the details (hallucinates) as you recall. The comparisons are closer than you might think.

vidarh 1y ago
The comparison would be apt if the LLM was trained on your codebase.
- jimbokun 1y ago
  Isn’t that the problem? I don’t see any progress on incrementally training LLMs on specific projects. I believe it’s called fine tuning, right? Why isn’t that the default approach anywhere instead of the hack of bigger “context windows”?
  - adastra22 1y ago
    Because fine-tuning can be used to remove restrictions from a model, so they don't give us plebs access to that.
  - gerhardi 1y ago
    I’m not well versed enough on this but wouldn’t it be a problem with custom training that the specific project training codebases probably would likely have a lot of the implemented stuff, relevant for the domain, only once and in one way, compared to how the todays popular large models have been trained maybe with countless different ways to use common libraries for whatever various tasks with whatever Github ripped material fed in?
krainboltgreene 1y ago
You have no idea if I remember with pixel perfect accuracy (whatever that even means). There are plenty of people with photographic memory. Also, you're a programmer you have no foundation of knowledge on which to make that assessment. You might as well opine on quarks or martian cellular life. My god the arrogance of people in my industry.
- johnisgood 1y ago
  > There are plenty of people with photographic memory. I thought it was rare.
- adastra22 1y ago
  Repeated studies have shown that perfect "photographic memory" does not in fact exist. Nobody has it. Some people think that they do though, but when tested under lab conditions those claims don't hold up. I don't believe these people are lying. They are self-reporting their own experiences, which unfortunately have the annoying property of being generated by the very mind that is living the experience. What does it mean to have an eidetic memory? It means that when you remember something you vividly remember details, and can examine those details to your heart's content. When you do so, it feels like all those details are correct. (Or so I'm told, I'm the opposite with aphantasia.) But it turns out if you actually have a photo reference and do a blind comparison test, people who report photographic memories actually don't do statistically any better than others in remembering specific fine details, even though they claim that they clearly remember. The simpler explanation is that while all of our brains are provide hallucinated detail to fill the gaps of memories, their brains are wired up to present those made up details feel much more real than they do to others. That is all.
  - HarHarVeryFunny 1y ago
    > Repeated studies have shown that perfect "photographic memory" does not in fact exist. This may change your mind! https://www.youtube.com/watch?v=jVqRT_kCOLI https://www.youtube.com/watch?v=jVqRT_kCOLI
    - 1y ago