3 ms·

I don’t see this as a big deal in practice. Conversations contain a bunch of junk anyway, so removing it from the context is usually good. In my repo, I have a

I don’t see this as a big deal in practice. Conversations contain a bunch of junk anyway, so removing it from the context is usually good. In my repo, I have a notes directory. I ask the AI to write a markdown file with what it learned, what work has been done, and what remains. In the next conversation, I can ask another model to pick it up from there. Sometimes I edit the note first.

leoedin 3mo ago
My experience with long running sessions is that they lose track of what's going on. The signal to noise ratio is often very poor - some models are particularly verbose and spew a lot of crap. The output of an LLM session is either modification of code or a plan or summary - that has the value, not the session text itself.
heisig 3mo ago
Oh, it very much is a big deal. In a setting where users can switch models easily and with no downsides, market forces will give us better and cheaper AI over time. If switching models is painful (as in, losing part of your context), the providers can create vendor lock-in, enshittify the user experience, and drive up cost. Even if there are workarounds right now, those AI providers have every incentive to make freedom-seekers more miserable over time. It is disheartening that some AI companies are now setting the stage for enshittification. I hope we can collectively dodge that bullet.
- dspillett 3mo ago
  > It is disheartening that some AI companies are now setting the stage for enshittification. Now?! I find it very hard to beleive that plans for enshittification haven't existed from quite early on. At some point they are going to have to do something about https://isaiprofitable.com/ https://isaiprofitable.com/ and once they are extracting from you to fill that hole they won't want you jumping ship.
- skybrian 3mo ago
  The coding agent I use (Shelley on exe.dev) has a pulldown menu where you choose the LLM when starting a new conversation. There is no lock-in. Conversations are not precious. Although, I do use ChatGPT more nowadays because their subscriptions work with Shelley. Sometimes I try other models after I hit the weekly cap on the subscription.
cube00 3mo ago
> Conversations contain a bunch of junk anyway This is what brought me around to doing more agentic coding. I set the task, require tests and the strict linting must pass and then leave it to blow smoke up its own ass about what's going on. I see glimpses scrolling past of all the conversational language that used to frustrate me so much when using a chat interface and I can just let it flow on past. I come back when its made everything pass, my life is better now. The previous match was still too clever for clippy’s type analysis, so I’ve simplified it into a direct borrowed-pattern form and I’m validating once more. I’m replacing that wrapper with a direct, non-transparent error variant so the enum stays explicit and doesn’t rely on a generic anyhow bridge.
- springtimesun 3mo ago
  I have been doing this with opencode. Spend a fair bit of time composing and scoping a ticket. tmux on the server where the local inference lives, start a non-interactive session that starts with “read issue #123, do all your work in a feature branch” and ends with “push and open a PR”. Like not watching someone else load your dishwasher, you appreciate the end result a lot more if you don’t watch the process, even if you rearrange things before you start it.
  - cube00 3mo ago
    Local inference can't arrive soon enough. On my inferior hardware the models still struggle to build the correct JSON to be able to call tools. Once either I get better hardware or the local models improve I'll happily leave it chugging away for a few hours to arrive at a solution because I know I'm not going to get token shock from all the looping if I'm running local inference.
    - springtimesun 3mo ago
      I have spent way too much on hardware. I’m sure I will look back on that after the eventual price normalization with some regret. I do think we will look back at this shortage and name it as the thing that accelerated innovation at getting quality out of smaller models. Hang tough. The other day someone posted a solution to get a model running off of SSDs. It won’t be fast, but it will be coming.
hashstring 3mo ago
You loose inspection, export, replay and audit. None of the popular providers have real moat, which is very much problematic for OpenAI and Anthropic; their operating margins are deeply red and they do not have the same reserves that FAANG/MANGA has. FAANG/MANGA does the same to kill fair market competition in the long run. This is one of the ways in which model providers are attempting to artificially create moat where there is none. There are only functional objections to this. This is fundamentally antitrust material. However, this is “acceptable” in contemporary USA because the FTC has been gutted to follow Trumponomics.
esafak 2mo ago
Conversations provide a window into the minds of your coworkers and their agents. You could be using them to read between the lines when debugging.

leoedin 3mo ago

My experience with long running sessions is that they lose track of what's going on. The signal to noise ratio is often very poor - some models are particularly verbose and spew a lot of crap. The output of an LLM session is either modification of code or a plan or summary - that has the value, not the session text itself.

heisig 3mo ago

Oh, it very much is a big deal. In a setting where users can switch models easily and with no downsides, market forces will give us better and cheaper AI over time. If switching models is painful (as in, losing part of your context), the providers can create vendor lock-in, enshittify the user experience, and drive up cost. Even if there are workarounds right now, those AI providers have every incentive to make freedom-seekers more miserable over time. It is disheartening that some AI companies are now setting the stage for enshittification. I hope we can collectively dodge that bullet.

dspillett 3mo ago
> It is disheartening that some AI companies are now setting the stage for enshittification. Now?! I find it very hard to beleive that plans for enshittification haven't existed from quite early on. At some point they are going to have to do something about https://isaiprofitable.com/ https://isaiprofitable.com/ and once they are extracting from you to fill that hole they won't want you jumping ship.
skybrian 3mo ago
The coding agent I use (Shelley on exe.dev) has a pulldown menu where you choose the LLM when starting a new conversation. There is no lock-in. Conversations are not precious. Although, I do use ChatGPT more nowadays because their subscriptions work with Shelley. Sometimes I try other models after I hit the weekly cap on the subscription.

cube00 3mo ago

> Conversations contain a bunch of junk anyway This is what brought me around to doing more agentic coding. I set the task, require tests and the strict linting must pass and then leave it to blow smoke up its own ass about what's going on. I see glimpses scrolling past of all the conversational language that used to frustrate me so much when using a chat interface and I can just let it flow on past. I come back when its made everything pass, my life is better now. The previous match was still too clever for clippy’s type analysis, so I’ve simplified it into a direct borrowed-pattern form and I’m validating once more. I’m replacing that wrapper with a direct, non-transparent error variant so the enum stays explicit and doesn’t rely on a generic anyhow bridge.

springtimesun 3mo ago
I have been doing this with opencode. Spend a fair bit of time composing and scoping a ticket. tmux on the server where the local inference lives, start a non-interactive session that starts with “read issue #123, do all your work in a feature branch” and ends with “push and open a PR”. Like not watching someone else load your dishwasher, you appreciate the end result a lot more if you don’t watch the process, even if you rearrange things before you start it.
- cube00 3mo ago
  Local inference can't arrive soon enough. On my inferior hardware the models still struggle to build the correct JSON to be able to call tools. Once either I get better hardware or the local models improve I'll happily leave it chugging away for a few hours to arrive at a solution because I know I'm not going to get token shock from all the looping if I'm running local inference.
  - springtimesun 3mo ago
    I have spent way too much on hardware. I’m sure I will look back on that after the eventual price normalization with some regret. I do think we will look back at this shortage and name it as the thing that accelerated innovation at getting quality out of smaller models. Hang tough. The other day someone posted a solution to get a model running off of SSDs. It won’t be fast, but it will be coming.

hashstring 3mo ago

You loose inspection, export, replay and audit. None of the popular providers have real moat, which is very much problematic for OpenAI and Anthropic; their operating margins are deeply red and they do not have the same reserves that FAANG/MANGA has. FAANG/MANGA does the same to kill fair market competition in the long run. This is one of the ways in which model providers are attempting to artificially create moat where there is none. There are only functional objections to this. This is fundamentally antitrust material. However, this is “acceptable” in contemporary USA because the FTC has been gutted to follow Trumponomics.

esafak 2mo ago

Conversations provide a window into the minds of your coworkers and their agents. You could be using them to read between the lines when debugging.