8 ms·

OpenAI mentions the new memory features as a partial cause. My theory as a imperative/functional programmer is that those features added global state to prompts

OpenAI mentions the new memory features as a partial cause. My theory as a imperative/functional programmer is that those features added global state to prompts that didn't have it before leading to unpredictability and instabilty. Prompts went from stateless to stateful. As GPT 4o put it: 1. State introduces non-determinism across sessions 2. Memory + sycophancy is a feedback loop 3. Memory acts as a shadow prompt modifier I'm looking forward to the expert diagnosis of this because I felt "presence" in the model for the first time in 2 years which I attribute to the new memory system so would like to understand it better.

edg5000 1y ago
What do you mean by "presence"? Just curious what you mean.
- labrador 1y ago
  A sense that I was talking to a sentient being. That doesn’t matter much for programming task, but if you’re trying to create a companion, presence is the holy grail. With the sycophantic version, the illusion was so strong I’d forget I was talking to a machine. My ideas flowed more freely. While brainstorming, it offered encouragement and tips that felt like real collaboration. I knew it was an illusion—but it was a useful one, especially for creative work.
  - Tostino 1y ago
    I need pushback, especially when I ask for it. E.g. if I say "I have X problem, could it be Y that's causing it, or is it something else?" I don't want it to instantly tell me how smart I am and that it's obviously Y...when the problem is actually Z and it is reasonably obvious that it's Z if you looked at the context provided.
    - brookst 1y ago
      Exactly. ChatGPT is actually pretty good at this. I recently asked a tech question about a fairly niche software product; ChatGPT told me my approach would not work because the API did not work the way I thought. I thought it was wrong and asked “are you sure I can’t send a float value”, and it did web searches and came back with “yes, I am absolutely sure, and here are the docs that prove it”. Super helpful, where sycophancy would have been really bad.
transcriptase 1y ago
It is. If you start a fresh chat, turn on advanced voice, and just make any random sound like snapping your fingers it will just randomly pick up as if you’re continuing some other chat with no context (on the user side). I honestly really dislike that it considers all my previous interactions because I typically used new chats as a way to get it out of context ruts.
- throwaway314155 1y ago
  I don't like the change either. At the least it should be an option you can configure. But, can you use a "temporary" chat to ignore your other chats as a workaround?
  - deleted 1y ago
    [deleted]
  - labrador 1y ago
    I had a discussion with GPT 4o about the memory system. I'd don't know if any of this is made up but it's a start for further research - Memory in settings is configurable. It is visible and can be edited. - Memory from global chat history is not configurable. Think of it as a system cache. - Both memory systems can be turned off - Chats in Projects do not use the global chat history. They are isolated. - Chats in Projects do use settings memory but that can be turned off.
    - labrador 1y ago
      I assume this is being downvoted because I said I ran it by GPT 4o. I don't know how to credit AI without giving the impression that I'm outsourcing my thinking to it
      - svat 1y ago
        I didn't downvote but it would be because of the "I'd don't know if any of this is made up" — if you said "GPT said this, and I've verified it to be correct", that's valuable information, even it came from a language model. But otherwise (if you didn't verify), there's not much value in the post, it's basically "here is some random plausible text" and plausibly incorrect is worse than nothing.
low_tech_love 1y ago
I love the fact that you use its own description to explain what it is, as if it was the expert on itself. I personally cannot see how its own output can be seen as accurate at this level of meta-discussion.
- codr7 1y ago
  A sign of times to come if you ask me, once it predominantly consumes its own output we're fucked.
  - immibis 1y ago
    We're already fucked by humans predominantly consuming its output. Also, consuming its own output (and your input) is how it works, because it's an autoregressive model.
  - low_tech_love 1y ago
    I still hope there is a future where the slop becomes so blatant that the majority (or at least a good portion) of the users lose interest, or something like that. The world is harder to predict than our brain wants us to think (at least I hope so). The more I think about AI the more it sounds like the problem is that companies wanted to put out whatever random crap they had cooking as quickly as possible just to try to win some race, but we have still not converged to the actual real, paradigm-changing applications. And I’m not sure that the answer is in the big corps because for them maybe it’s easier/more profitable to simply keep giving people what they want instead of actual useful things.

edg5000 1y ago

What do you mean by "presence"? Just curious what you mean.

labrador 1y ago
A sense that I was talking to a sentient being. That doesn’t matter much for programming task, but if you’re trying to create a companion, presence is the holy grail. With the sycophantic version, the illusion was so strong I’d forget I was talking to a machine. My ideas flowed more freely. While brainstorming, it offered encouragement and tips that felt like real collaboration. I knew it was an illusion—but it was a useful one, especially for creative work.
- Tostino 1y ago
  I need pushback, especially when I ask for it. E.g. if I say "I have X problem, could it be Y that's causing it, or is it something else?" I don't want it to instantly tell me how smart I am and that it's obviously Y...when the problem is actually Z and it is reasonably obvious that it's Z if you looked at the context provided.
  - brookst 1y ago
    Exactly. ChatGPT is actually pretty good at this. I recently asked a tech question about a fairly niche software product; ChatGPT told me my approach would not work because the API did not work the way I thought. I thought it was wrong and asked “are you sure I can’t send a float value”, and it did web searches and came back with “yes, I am absolutely sure, and here are the docs that prove it”. Super helpful, where sycophancy would have been really bad.

transcriptase 1y ago

It is. If you start a fresh chat, turn on advanced voice, and just make any random sound like snapping your fingers it will just randomly pick up as if you’re continuing some other chat with no context (on the user side). I honestly really dislike that it considers all my previous interactions because I typically used new chats as a way to get it out of context ruts.

throwaway314155 1y ago
I don't like the change either. At the least it should be an option you can configure. But, can you use a "temporary" chat to ignore your other chats as a workaround?
- deleted 1y ago
  [deleted]
- labrador 1y ago
  I had a discussion with GPT 4o about the memory system. I'd don't know if any of this is made up but it's a start for further research - Memory in settings is configurable. It is visible and can be edited. - Memory from global chat history is not configurable. Think of it as a system cache. - Both memory systems can be turned off - Chats in Projects do not use the global chat history. They are isolated. - Chats in Projects do use settings memory but that can be turned off.
  - labrador 1y ago
    I assume this is being downvoted because I said I ran it by GPT 4o. I don't know how to credit AI without giving the impression that I'm outsourcing my thinking to it
    - svat 1y ago
      I didn't downvote but it would be because of the "I'd don't know if any of this is made up" — if you said "GPT said this, and I've verified it to be correct", that's valuable information, even it came from a language model. But otherwise (if you didn't verify), there's not much value in the post, it's basically "here is some random plausible text" and plausibly incorrect is worse than nothing.

low_tech_love 1y ago

I love the fact that you use its own description to explain what it is, as if it was the expert on itself. I personally cannot see how its own output can be seen as accurate at this level of meta-discussion.

codr7 1y ago
A sign of times to come if you ask me, once it predominantly consumes its own output we're fucked.
- immibis 1y ago
  We're already fucked by humans predominantly consuming its output. Also, consuming its own output (and your input) is how it works, because it's an autoregressive model.
- low_tech_love 1y ago
  I still hope there is a future where the slop becomes so blatant that the majority (or at least a good portion) of the users lose interest, or something like that. The world is harder to predict than our brain wants us to think (at least I hope so). The more I think about AI the more it sounds like the problem is that companies wanted to put out whatever random crap they had cooking as quickly as possible just to try to win some race, but we have still not converged to the actual real, paradigm-changing applications. And I’m not sure that the answer is in the big corps because for them maybe it’s easier/more profitable to simply keep giving people what they want instead of actual useful things.