3 ms·

I know dozens of people who are in a similar state right now, following the November 2025 moment when Claude Code (and Codex) got really good. I wouldn't worry

by simonw 8mo ago

I know dozens of people who are in a similar state right now, following the November 2025 moment when Claude Code (and Codex) got really good. I wouldn't worry about it just yet - this is all very novel, and there's a lot of excitement involved in figuring out what it can do and trying different things. If you're still addicted to it in three months time I'd start to be concerned. For the moment though you're building a valuable mental model of how to use it and what it can do. That's not wasted time.

zof3 8mo ago
Your mental model point is very true. We all had to learn how to google at some point — explaining how to use these tools to someone outside the bubble feels like explaining how googling works to someone. Much of it is intuitive understanding from experience. My question would be how much we think the processes will change as the models do. Much advice from two years ago is no longer relevant or realistic. Where do we think it will go next? Does anyone have a really good way to explain to their relatives and friends how using an agent is different from simply using Google? Just saying ‘fundamentally different’ doesn’t go very far; the best I’ve found is sitting down and giving a demonstration. It’s also difficult to explain the enormous gap between frontier models and the free ones many people are accustomed to using. Is there a tangible comparison to a normie real-life ‘thing’ that anyone has used successfully?
- simonw 8mo ago
  I have no idea how to explain agents to a non-technical audience - there is SO much that can go wrong with them, it's still very much a technical power-user technology in my opinion. ChatGPT and Claude can both execute code now, so a safe subset is to show people how to upload files there and have them do useful things with the data.
- ahsisjb 8mo ago
  > Much of it is intuitive understanding from experience. You’re mistaking domain expertise with tool expertise. You can’t teach a non dev how to use an LLM effectively for dev without teaching them to be an experienced dev. Once you have that knowledge, LLMs aren’t that hard to use.
lkm0 8mo ago
I'm seeing the limits when Claude makes some statements that are extremely wrong but incredibly hard to spot unless you're in the field, recently telling me that "some people say" that rydberg atoms and neutral atoms are different enough to be in different quantum computing categories (they're the same). The stakes are lowering somehow, because I know I can't trust it for anything but fun side-projects. For serious research it's still me and reading papers.
- gnat 8mo ago
  I'm not trying to convert you, just want to share process tips that I see working for me and others. We're using agents, not a chat, because they can do complex work in pursuit of a goal. 1. Make artifacts. If you're doing research into a tech, or a hypothesis, then fire off subagents to explore different parts of the problem space, each reporting back into a doc. Then another agent synthesizes the docs into a conclusion/report. 2. Require citations. "Use these trusted sources. Cite trusted sources for each claim. Cite with enough context that it's clear your citations supports the claim, and refuse to cite if the citation doesn't support the claim." 3. Review. This lets you then fire off a subagent to review the synthesis. It can have its own prompt: look for confirming and disconfirming evidence, don't trust uncited claims. If you find it making conflation mistakes, figure out at what stage and why, and adjust your process to get in front of them. 4. Manage your context. LLM only has a fixed context size ("chat length") and facts & instructions at the front of that tend to be better hewn to than things at the end. Subagents are a way of managing that context to get more from a single run. Artifacts like notebooks or records of subagent output move content outside the context so you can pick up in a new session ("chat") and continue the work. It's less fun that just having a chat with ChatGPT. I find that I get much better quality results using these techniques. Hope this helps! If you're not interested in doing this (too much like work, and you already have something that works), it's no skin off my nose. All the best!
  - lkm0 8mo ago
    Thanks for the thoughtful reply! I definitely want to try a more complex setup when I have more time on my hands

by simonw 8mo ago

zof3 8mo ago

Your mental model point is very true. We all had to learn how to google at some point — explaining how to use these tools to someone outside the bubble feels like explaining how googling works to someone. Much of it is intuitive understanding from experience. My question would be how much we think the processes will change as the models do. Much advice from two years ago is no longer relevant or realistic. Where do we think it will go next? Does anyone have a really good way to explain to their relatives and friends how using an agent is different from simply using Google? Just saying ‘fundamentally different’ doesn’t go very far; the best I’ve found is sitting down and giving a demonstration. It’s also difficult to explain the enormous gap between frontier models and the free ones many people are accustomed to using. Is there a tangible comparison to a normie real-life ‘thing’ that anyone has used successfully?

simonw 8mo ago
I have no idea how to explain agents to a non-technical audience - there is SO much that can go wrong with them, it's still very much a technical power-user technology in my opinion. ChatGPT and Claude can both execute code now, so a safe subset is to show people how to upload files there and have them do useful things with the data.
ahsisjb 8mo ago
> Much of it is intuitive understanding from experience. You’re mistaking domain expertise with tool expertise. You can’t teach a non dev how to use an LLM effectively for dev without teaching them to be an experienced dev. Once you have that knowledge, LLMs aren’t that hard to use.

lkm0 8mo ago

I'm seeing the limits when Claude makes some statements that are extremely wrong but incredibly hard to spot unless you're in the field, recently telling me that "some people say" that rydberg atoms and neutral atoms are different enough to be in different quantum computing categories (they're the same). The stakes are lowering somehow, because I know I can't trust it for anything but fun side-projects. For serious research it's still me and reading papers.

gnat 8mo ago
I'm not trying to convert you, just want to share process tips that I see working for me and others. We're using agents, not a chat, because they can do complex work in pursuit of a goal. 1. Make artifacts. If you're doing research into a tech, or a hypothesis, then fire off subagents to explore different parts of the problem space, each reporting back into a doc. Then another agent synthesizes the docs into a conclusion/report. 2. Require citations. "Use these trusted sources. Cite trusted sources for each claim. Cite with enough context that it's clear your citations supports the claim, and refuse to cite if the citation doesn't support the claim." 3. Review. This lets you then fire off a subagent to review the synthesis. It can have its own prompt: look for confirming and disconfirming evidence, don't trust uncited claims. If you find it making conflation mistakes, figure out at what stage and why, and adjust your process to get in front of them. 4. Manage your context. LLM only has a fixed context size ("chat length") and facts & instructions at the front of that tend to be better hewn to than things at the end. Subagents are a way of managing that context to get more from a single run. Artifacts like notebooks or records of subagent output move content outside the context so you can pick up in a new session ("chat") and continue the work. It's less fun that just having a chat with ChatGPT. I find that I get much better quality results using these techniques. Hope this helps! If you're not interested in doing this (too much like work, and you already have something that works), it's no skin off my nose. All the best!
- lkm0 8mo ago
  Thanks for the thoughtful reply! I definitely want to try a more complex setup when I have more time on my hands