11 ms·

This is a pretty common position: "I don't worry about getting left behind - it will only take a few weeks to catch up again". I don't think that's true. I'm

by simonw 10mo ago

This is a pretty common position: "I don't worry about getting left behind - it will only take a few weeks to catch up again". I don't think that's true. I'm really good at getting great results out of coding agents and LLMs. I've also been using LLMs for code on an almost daily basis since ChatGPT's release on November 30th 2022. That's more than three years ago now. Meanwhile I see a constant flow of complaints from other developers who can't get anything useful out of these machines, or find that the gains they get are minimal at best. Using this stuff well is a deep topic. These things can be applied in so many different ways, and to so many different projects. The best asset you can develop is an intuition for what works and what doesn't, and getting that intuition requires months if not years of personal experimentation. I don't think you can just catch up in a few weeks, and I do think that the risk of falling behind isn't being taken seriously enough by much of the developer population. I'm glad to see people like antirez ringing the alarm bell about this - it's not going to be a popular position but it needs to be said!

systemf_omega 10mo ago
> Using this stuff well is a deep topic. Just like the stuff LLMs are being used for today. Why wouldn't "using LLMs well" be not just one of the many things LLMs will simplify too? Or do you believe your type of knowledge is somehow special and is resistant to being vastly simplified or even made obsolete by AI?
- simonw 10mo ago
  An interesting trend over the past year is that LLMs have learned how to prompt each other. Back in ~2024 a lot of people were excited about having "LLMs write the prompt!" but I found the results to be really disappointing - they were full of things like "You are the world's best expert in marketing" which was superstitious junk. As of 2025 I'm finding they actually do know how to prompt, which makes sense because there's a ton more information about good prompting approaches in the training data as opposed to a couple of years ago. This has unlocked some very interesting patterns, such as Claude Code prompting sub-agents to help it explore codebases without polluting the top level token window. But learning to prompt is not the key skill in getting good results out of LLMs. The thing that matters most is having a robust model of what they can and cannot do. Asking an LLM "can you do X" is still the kind of thing I wouldn't trust them to answer in a useful way, because they're always constrained by training data that was only aware of their predecessors.
- leonidasv 10mo ago
  Unless we figure out how to make 1 billion+ tokens multimodal context windows (in a commercially viable way) and connect them to Google Docs/Slack/Notion/Zoom meetings/etc, I don't think it will simplify that much. Most of the work is adjusting your mental model to the fact that the agent is a stateless machine that starts from scratch every single time and has little-to-no knowledge besides what's in the code, so you have to be very specific about the context of the task in some ways. It's different from assigning a task to a co-worker who already knows the business rules and cross-implications of the code in the real world. The agent can't see the broader picture of the stuff it's making, it can go from ignoring obvious (to a human that was present in the last planning meeting) edge cases to coding defensively against hundreds of edge cases that will never occur, if you don't add that to your prompt/context material.
jeroenhd 10mo ago
So far every new AI product and even model update has required me to relearn how to get decent results out of them. I'm honestly kind of sick of having to adjust my work flow every time. The intuition just doesn't hold. The LLM gets trained and retrained by other LLM users so what works for me suddenly changes when the LLM models refresh. LLMs have only gotten easier to learn and catch up on over the years. In fact, most LLM companies seem to optimise for getting started quickly over getting good results consistently. There may come a moment when the foundations solidify and not bothering with LLMs may put you behind the curve, but we're not there yet, and with the literally impossible funding and resources OpenAI is claiming they need, it may never come.
- christophilus 10mo ago
  Really? Claude Code upgrades for me have been pretty seamless- basically better quality output, given the same prompts, with no discernible downsides.
Mawr 10mo ago
I don't see how your position is compatible with the constant hype about the ever-growing capabilities of LLMs. Either they are improving rapidly, and your intuition keeps getting less and less valuable, or they aren't improving.
- simonw 10mo ago
  They're improving rapidly, which means your intuition needs to be constantly updated. Things that they couldn't do six months go might now be things that they can do - and knowing they couldn't do X six months ago is useful because it helps systematize your explorations. A key skill here is to know what they can do, what they can't do and what the current incantations are that unlock interesting capabilities. A couple I've learned in the past week: 1. Don't give Claude Code a URL to some code and tell it to use that, because by default it will use its WebFetch tool but that runs an extra summarization layer (as a prompt injection defense) which loses details. Telling it to use curl sometimes works but a guaranteed trick is to have it git clone the relevant repo to /tmp and look at the code there instead. 2. Telling Claude Code "use red/green TDD" is a quick to type shortcut that will cause it to write tests first, run them and watch them fail, then implement the feature and run the test again. This is a wildly effective technique for getting code that works properly while avoiding untested junk code that isn't needed. Now multiply those learnings by three years. Sure, the stuff I figure out in 2023 mostly doesn't apply today - but the skills I developed in learning how to test and iterate on my intuitions from then still count and still keep compounding. The idea that you don't need to learn these things because they'll get better to the point that they can just perfectly figure out what you need is AGI science fiction. I think it's safe to ignore.
  - crakhamster01 10mo ago
    I feel like both of these examples are insights that won't be relevant in a year. I agree that CC becoming omniscient is science fiction, but the goal of these interfaces is to make LLM-based coding more accessible. Any strategies we adopt to mitigate bad outcomes are destined to become part of the platform, no? I've been coding with LLMs for maybe 3 years now. Obviously a dev who's experienced with the tools will be more adept than one who's not, but if someone started using CC today, I don't think it would take them anywhere near that time to get to a similar level of competency.
mmcnl 10mo ago
Why can't both be true at the same time? Maybe their problems are more complex than yours. Why do you assume it's a skill issue and ignore the contextual variables?
- simonw 10mo ago
  On the rare occasions that I can convince them to share the details of the problems they are tackling and the exact prompts they are using it becomes very clear that they haven't learned how to use the tools yet.
  - UncleEntity 10mo ago
    I'm kind of curious about the things you're seeing since I find the best way is to have them come up with a plan for the work they're about to do and then make sure they actually finish it because they like to skip stuff if it requires too much effort. I mean, I just think of them like a dog that'll get distracted and go off doing some other random thing if you don't supervise them enough and you certainly don't want to trust them to guard your sandwich.
rubslopes 10mo ago
I don't disagree, knowing how to use the tools is important. But I wanted to add that great prompting skill nowadays are far far less necessary for top-tier models that it was years ago. If I'm clear about what I want and how I want it to behave, Claude Opus 4.5 almost always nails it first time. The "extra" that I do often, that maybe newcomers don't, is to setup a system where the LLM can easily check the results of its changes (verbose logs in terminal and, in web, verbose logs in console and playwright).
coffeemug 10mo ago
Strongly disagree. Claude Code is the most intuitive technology I've ever used-- way easier than learning to use even VS Code for example. It doesn't even take weeks. Maybe a day or two to get the hang of it and you're off to the races.
- simonw 10mo ago
  Don't underestimate the number of developers who aren't comfortable with tools that live in the terminal.
  - HDThoreaun 10mo ago
    Well these people are left behind either way. Competent devs can easily learn to use coding assistants in a day or two
  - coffeemug 10mo ago
    I actually don't use it in the terminal, I use the vs code extension. It's a better experience (bringing up the file being edited, nicer diffs, etc.) But both are trivial to pick up.
- johnsmith1840 10mo ago
  The difference is AI tooling lies to you. Day 0 you think it's perfect but the more you use ai tools you realize using them wrong can give you gnarly bugs. It's intuitive to use but hard to master
  - coffeemug 10mo ago
    It took me a couple of days to find the right level of detail to prompt it. Too high level, and the codebase gets away from me/the tooling goes off the rails. Too low level, and I may as well do it myself. Maybe also learn the sorts of things Claude Code isn't good at yet. But once I got in the groove it was very easy from there. I think the whole process took 2-3 days.
    - 10mo ago
furyofantares 10mo ago
I think I'm also very good at getting great results out of coding agents and LLMs, and I disagree pretty heavily with you. It is just way easier for someone to get up to speed today than it was a year ago. Partly because capabilities have gotten better and much of what was learned 6+ months ago no longer needs to be learned. But also partly because there is just much more information out there about how to get good results, you might have coworkers or friends you can talk to who have gotten good results, you can read comments on HN or blog posts from people who have gotten good results, etc. I mean, ok, I don't think someone can fully catch up in a few weeks. I'll grant that for sure. But I think they can get up to speed much faster than they could have a year ago. Of course, they will have to put in the effort at that time. And people who have been putting it off may be less likely to ever do that. So I think people will get left behind. But I think the alarm to raise is more, "hey, it's a deep topic and you're going to have to put in the effort" rather than "you better start now or else it's gonna be too late".
biophysboy 10mo ago
What are your tips? Any resources you would recommend? I use Claude code and all the chat bots, but my background isn't programming, so I sometimes feel like I'm just swimming around.
Humorist2290 10mo ago
It needs to be said that your opinion on this is well understood by the community, respected, but also far from impartial. You have a clear vested interest in the success of _these_ tools. There's a learning curve to any toolset, and it may be that using coding agents effectively is more than a few weeks of upskilling. It may be, and likely will be, that people make their whole careers about being experts on this topic. But it's still a statistical text prediction model, wrapped in fancy gimmicks, sold at a loss by mostly bad faith actors, and very far from its final form. People waiting to get on the bandwagon could well be waiting to pick up the pieces once it collapses.
- mattmanser 10mo ago
  I have a lot of respect from Simon and read a lot of his articles. But I'm still seeing clear evidence it IS a statistical text prediction model. You ask it the right niche thing and it can only pump out a few variations of the same code, that's clearly someone else's code stolen almost verbatim. And I just use it 2 or 3 times a day. How are SimonW and AntiRez not seeing the same thing? How are they not seeing the propensity for both Claude + ChatGPT to spit out tons of completely pointless error handling code, making what should be a 5 line function a 50 line one? How are they not seeing that you constantly have to nag it to use modern syntax. Typescript, C#, Python, doesn't matter what you're writing in, it will regularly spit out code patterns that are 10 years out of date. And woe betide you using a library that got updated in the last 2 years. It will constantly revert back to old syntax over and over and over again. I've also had to deal with a few of my colleagues using AI code on codebases they don't really understand. Wrong sort, id instead of timestamp. Wrong limit. Wrong json encoding, missing key converters. Wrong timezone on dates. A ton of subtle, not obvious, bugs unless you intimately know the code, but would be things you'd look up if you were writing the code. And that's not even including the bit where the AI obviously decided to edit the wrong search function in a totally different part of the codebase that had nothing to do with what my colleague was doing. But didn't break anything or trigger any tests because it was wrapped in an impossible to hit if clause. And it created a bunch of extra classes to support this phantom code, so hundreds of new lines of code just lurking there, not doing anything but if I hadn't caught it, everyone thinks it does do something.
csmpltn 10mo ago
So where’s all of this cutting edge amazing and flawless stuff you’ve built in a weekend that everybody else couldn’t because they were too dumb or slow or clueless?
- wild_egg 10mo ago
  This is such a tired response at this point. People are under zero obligation to release their work to the public. Simon actually publishes and writes about a remarkable amount of the side projects he builds with AI. The rest of us just build tons of cool stuff for personal use or for $JOB. Releasing stuff to the public is, in general, a massive amount of extra work for very little benefit. There are loads of FOSS maintainers trapped spending as much time managing their communities as they do their actual projects and many of us just don't have time for that.
  - Anamon 10mo ago
    The response may be tired when asked in this personal way, but in general, it's a fair question. Nobody is forced to share their work. But with all the high praises, we'd expect to see at least some uptick in the software world. But there is no surge in open source projects. No surge in app store entries. And for the bigger companies claiming high GenAI use, they're not iterating faster or building more. They are continually removing features and their software is getting worse, slower, less robust, and less secure. Software quality has been on a step downwards curve as far as quality and capabilities are concerned, for years before LLM coding had its breakthrough. For all the promises I'd have expected to, three years later, at least notice the downward trajectory easing off. But it hasn't been happening.
  - rgoulter 10mo ago
    > The rest of us just build tons of cool stuff for personal use or for $JOB. Releasing stuff to the public is, in general, a massive amount of extra work for very little benefit. There are loads of FOSS maintainers trapped spending as much time managing their communities as they do their actual projects and many of us just don't have time for that. I wouldn't worry about this. There are many examples of people sharing a project they've used LLMs to help write, and the result was not a huge amount of attention & expectation of burden. Perhaps "I don't share it because I'm worried people will love it too much" even suggests the opposite: you can concretely demonstrate the kinds of things you've been able to build using LLMs. > This is such a tired response at this point. Lack of specificity & concrete examples frequently mean all that's left for discussion is emotion for hype and anti-hype, though. In this thread, the discussion was: pro: use LLMs or get left behind conserve: okay, I'll start using LLMs when they're good pro: no no they won't be that good, it takes effort to get to use them conserve: do you have any examples? pro: why should we have to share examples? I like LLMs. But making big claims while being reticent about concrete claims and demonstrations is irksome.
hollowturtle 10mo ago
I can't buy it because for many people like you it's always the other that uses the tools wrong, proving the contrary for skeptics that keep getting bad results from llms it simply is impossible with this narrative as the base of the discourse, eg "you're not using it well". I don't even get why you need to praise yourself so much being really good at using these tools, if not for building some tech influencer status around here... same thing I believe antirez is trying to do(who knows why)
- kevin42 10mo ago
  Have you considered that maybe you aren't using it well? It's something that can and should be learned. It's a tool, and you can't expect to get the most out of a tool without really learning how to use it. I've had this conversation with a few people so far, and I've offered to personally walk through a project of their choosing with them. Everyone who has done this has changed their perspective. You may not be convinced it will change the world, but if you approach it with an open mind and take the time to learn how to best use it, I'm 100% sure you will see that it has so much potential. There are tons of youtube videos and online tutorials if you really want to learn.
  - hollowturtle 10mo ago
    > Have you considered that maybe you aren't using it well? Here we go, as I said, and again and again and again it's always out fault we're not using well. It is impossible to counter argument. Btw to reply to your question, yes many times and proved to be useful in very small specialized tasks and a couple of migrations. I really like how LLMs are helping me in my day to day, but still so far away from all this astroturfing
camel-cdr 10mo ago
How many thing you learned working with LLMs in 2022 are relevant today? How many things you learned now are relevant in the future?
- y1n0 10mo ago
  This question misses the point. Everything you learn today informs how you learn in the future.
tehnub 10mo ago
I guess this applies to the type of developer who needs years, not weeks, to become proficient in say Python?
deleted 10mo ago
[deleted]
noosphr 10mo ago
I've been building Ai apps since gpt 3 so 5 years now. The pro AI people don't understand what quadratic attention means and the anti-ai people don't understand how much information can be contained in a tb of weights. At the end of the day both will be hugely disappointed. >The best asset you can develop is an intuition for what works and what doesn't, and getting that intuition requires months if not years of personal experimentation. Intuition does not translate between models. Whatever you think dense llms were good at deepseek completely upended it in an afternoon. The difference between major revisions of model families is substantial enough that intuition is a drawback not an asset.
- simonw 10mo ago
  What does quadratic attention mean? I've so far found that intuition travels between models of a similar generation remarkably well. The conformance suite trick (find a 9,200 test existing conformance suite and tell an agent to build a fresh implementation that passes all those tests) I first found with GPT-5.2 turned out to work exactly as well against Claude Opus 4.5, for example.
  - noosphr 10mo ago
    https://arxiv.org/abs/2209.04881 https://arxiv.org/abs/2209.04881
    - simonw 10mo ago
      To save anyone else the click, this is the paper "On The Computational Complexity of Self-Attention" from September 2022, with authors from NYU and Microsoft. It argues that the self-attention mechanism in transformers works by having every token "attend to" every other token in a sequence, which is quadratic - n^2 against input - which should limit the total context length available to models. This would explain why the top models have been stuck at 1 million tokens since Gemini 1.5 in February 2024 (there has been a 2 million token Gemini but it's not in wide use, and Meta claimed their Llama 4 Scout could do 10 million but I don't know of anyone who's seen that actually work.) My counter-argument here is that Claude Opus 4.5 has a comparatively tiny 200,000 token window which turns out to work incredibly well for the kinds of coding problems we're throwing at it, when accompanied by a cleverly designed harness such as Claude Code. So this limit from 2022 has been less "disappointing" than people may have expected.
tymscar 10mo ago
I think you are right in saying that there is some deep intuition that takes months, if not years, to hone about current models, however, the intuition some who did nothing but talk and use LLMs nonstop two years ago would be just as good today as someone who started from scratch, if not worse because of antipatterns that don’t apply anymore, such as always starting a new chat and never using a CLI because of context drift. Also, Simon, with all due respect, and I mean it, I genuinely look in awe at the amount of posts you have on your blog and your dedication, but it’s clear to anyone that the projects you created and launched before 2022 far exceed anything you’ve done since. And I will be the first to say that I don’t think that’s because of LLMs not being able to help you. But I do think it’s because what makes you really, really good at engineering you kept replacing slowly but surely with LLMs more and more by the month. If I look at Django, I can clearly see your intelligence, passion, and expertise there. Do you feel that any of the projects you’ve written since LLMs are the main thing you focus on are similar? Think about it this way: 100% of you wins against 100% of me any day. 100% of Claude running on your computer is the same as 100% of Claude running on mine. 95% of Claude and 5% of you, while still better than me (and your average Joe), is nowhere near the same jump from 95% Claude and 5% me. I do worry when I see great programmers like you diluting their work.
- simonw 10mo ago
  My great regret from the past few years is that experimenting with LLMs has been such a huge distraction from my other work! My https://llm.datasette.io/ https://llm.datasette.io/ tool is from that era though, and it's pretty cool.
  - tymscar 10mo ago
    I do think your datasettes work is fantastic and I genuinely hope you take my previous message the right way. I’m not saying you do something bad, quite the opposite, I feel like we need more of you and I’m afraid because of LLMs we get less of you.
    - beaker52 10mo ago
grayhatter 10mo ago
Show me what you've made with AI? What's the impressive thing that can convince me it's equivalent, or better than anything created before, or without it? I understand you've produced a lot of things, and that your clout (which depends on the AI ferver) is based largely because of how refined a workflow you've invented. But I want to see the product, rather than the hype. Make me say; I wish I was good enough to create this! Without that, all I can see is the cost, or the negative impact. edit: I've read some of your other posts, and for my question, I'd like to encourage you to pick only one. Don't use the scatter shot approach that LLMs love, giving plenty of examples, hoping I'll ignore the noise for the single that sounds interesting. Pick only one. What project have you created that you're truly proud of? I'll go first, (even though it's unfinished): Verse
- tsimionescu 10mo ago
  There are so many projects named Verse (or similar) that you really need to be more specific.
  - grayhatter 10mo ago
    The point wasn't to level set, or for it to feel like I'm promoting it. Only so that I couldn't back out later, or not have any skin in the game. But if you really couldn't figure out which I meant. It's the one on my github, and also hosting on srctree, which I link to from the site in my profile, and from my github.
yeasku 10mo ago
[dead]
lunar_mycroft 10mo ago
The core of your argument is that using LLMs is a skill that takes a significant amount of time to master. I'm not going to argue against that (although I have some doubts) because I think it's ultimately irrelevant. The question isn't "is prompting a skill that you'll need to be an effective software developer in the future" but "what other skills will you need to do so", and regardless of the answer you don't need to start adopting LLMs right away. Maybe AI gets good enough at writing code that it's users' knowledge of computer science and software development becomes irrelevant. In that case, approximately everyone on this site is just screwed. We're all in the business of selling that specialized knowledge, and if it's no longer required then companies aren't going to pay us to operate the AI, they're going to pay PMs, middle managers, executives, etc. But even that won't be particularly workable long term, because all their customers will realize they no longer need to pay the companies for software either. In this world, the price of software goes to zero (and hosting likely gets significantly more commoditized than it is now). Any time you put into learning to use LLMs for software development doesn't help you keep making money selling software, and actually stops you from picking up a new career. If, on the other hand, CS and software engineering knowledge is still needed, companies will have to keep/restart hiring or training new developers. In terms of experience using AI, it is impossible for anyone to have less experience than these new developers. We will, however, have much more experience and knowledge of the aforementioned non-LLM skills that we're assuming (in this scenario) are still necessary for the job. In this scenario you might be better off if you'd started learning to prompt a bit earlier, but you'll still be fine if you didn't.
elvis10ten 10mo ago
While intuition takes a while, I think it can be learned in less than a month or two. This has been my experience. When something gets good enough, someone will create some really good resource on it. Allowing the dust to settle, to me is a more efficient strategy than constantly trying to “keep up”. Maybe also not waiting too long to do so. This wouldn’t work of course if a person was trying to be some AI thought leader.
- helloplanets 10mo ago
  I'd say that it's a different type of learning process, where even a good resource doesn't help as much as it would with a traditional programming language. Sort of like you can't get very good at writing by just reading a ton of instructional books about it.
  - elvis10ten 10mo ago
    Even CRUD programming: you can’t get very good at it with just reading.
stefanlindbohm 10mo ago
Maybe it’s just two different ways to reach the same result. You need to spend time to be great at prompting to get high-quality code from LLM’s, which might just be equivalent to the fact you need to spend time to write high-quality code without LLM’s too. From where I’m standing, I don’t see any massive difference on overall productivity between anyone all in on vibe coding than those who aren’t. There’s not more features, higher quality, etc from teams/companies out there than before on any high-level metrics/observations. Maybe it will come, but there’s also no evidence it will. I do, however, see great gains within certain specific tasks using LLM’s. Smaller scope code gen, rubber ducking, etc. But this seems much less difficult to get good at using (and I hope for tooling that help facilitate the specific types of use cases) and on the whole amounts to marginal gains. It seems fine to be a few years late to catch up, worst case.
noodletheworld 10mo ago
> I don't think you can just catch up in a few weeks, and I do think that the risk of falling behind isn't being taken seriously enough by much of the developer population. This is nonsense. This field moves so fast the things you did more than a year ago aren't relevant anymore. Claude code came out last year. Anyone using random shit from before that is not using it any more. It is completely obsolete in all but a handful of cases. To make matters worse “intuition” about models is wasted learning, because they change, significantly, often. Stop spreading FUD. You can be significantly less harmful to people who are trying to learn by sharing what you actually do instead of nebulously hand waving about magical BS. Dear readers: ignore this irritating post. Go and watch Armin Ronacher on youtube if you want to see what a real developer doing this looks like, and why its hard.
- simonw 10mo ago
  You're accusing me of spreading harmful advice here, when you're the one telling people that they don't need to worry about not investing in their skills because "This field moves so fast the things you did more than a year ago aren't relevant anymore." One of us is right here. I hope for your sake and the people that listen to you that it's you. I don't think it is.
  - noodletheworld 10mo ago
    Simon, you're literally fear mongering. You're making wild claims, and absolutely failing to back them up with evidence. That is FUD. People should invest, they should try things. …but its far faaaaaar less clear cut that dropping everything and focusing on AI right now is so absolutely important. The difference between what you get prompting and a totally naïve user of claude code gets is marginal. People are not being left behind if they try it a bit and find its ok, not great, and come back later. It is not a deep topic. writing AI tools is a deep topic, but most people arent doing that. Youre in the wrong here. Stop making people scared. I quote antirez here, since you clearly arent interested in listening to me: > I have a single suggestion for you, my friend. Whatever you believe about what the Right Thing should be, you can't control it by refusing what is happening right now. > Skipping AI is not going to help you or your career. Think about it. Test these new tools, with care, with weeks of work, not in a five minutes test where you can just reinforce your own beliefs. > Find a way to multiply yourself, and if it does not work for you, try again every few months.
febusravenga 10mo ago
> Using this stuff well is a deep topic. These things can be applied in so many different ways, and to so many different projects. The best asset you can develop is an intuition You're basically saying that using LLMs is like using magic. Telling people to use intuition is basically telling that i don't know how it works and why, but works for me sometimes. That's why we programmers hate it - we have safe space where there's no intuition - namely programming languages & runtimes with deterministic behavior. And we're shoehorned back into mess of magic/intuition and wishfullthinking. (yes, i try llm, i have some results, i'm frustrated mostly by people AI-slopping _everything_ around me)
- hahahahhaah 10mo ago
  Intuition is the wrong word IMO. Tacit knowledge is the thing. Knowledge that is hard to communicate and needs experience. Problem with AI is it isn't woodwork. The material keeps changing!
- theshrike79 10mo ago
  It's like any other power tool. It requires skill to use it safely and efficiently. Anyone can use a band saw to cut things. Then go look what Jimmy DiResta makes with one and you see the difference. The chance of an inexperienced person cutting off their finger with a bandsaw is also way over zero, there are things you should not and must not do with it. As with any power tool.
- simonw 10mo ago
  Oddly enough I wrote about the magic analogy and why I stopped using is a few years ago (pre-ChatGPT, even): https://simonwillison.net/2022/Oct/5/spell-casting/ https://simonwillison.net/2022/Oct/5/spell-casting/ I am eternally frustrated that "intuition" is the key skill people need to work effectively with LLMs, because it's something I can't teach people! If I could figure out how to download my intuition into other people's heads I would do that. Instead I have to convince people that intuition is key, and the only way to get it is to invest in experimenting.
matsemann 10mo ago
I learned Django 15 years after its inception. After 5 years of experience I'm probably not too far behind someone doing the exact same work as me but for 15 years. Or would you say people shouldn't learn Django now? As it's useless as they're already far behind? They shouldn't study computer science, as it will be too late? Every profession have new people continuously entering the workforce, that quickly get up to speed on whatever is in vogue. Honestly, what you've spent years learning and experimenting with, someone else will be able to learn in months. People will figure out the best ways of using these tools after lots of attempts, and that distilled knowledge will be transferred quickly to others. This is surely painful to hear for those having spent years in the trenches, and is perhaps why you refuse to acknowledge it, but I think it's true.
- simonw 10mo ago
  I would not say that about a framework like Django - though I would encourage people not to under-invest in understanding web fundamentals since once you have those Django, Rails, Next.js etc are all quick to pick up. I would say that about LLMs. That's why I'm ringing the alarm bells here. LLM skills are not the same as framework or library usage skills. They aren't clearly documented or predictable - they're really weird! If you assume learning to use coding agents is the same category of challenge as learning to use something like Django you'll get burned by that assumption.
  - matsemann 10mo ago
    But what makes it impossible to catch up? Does it matter if I wait a year and then start? It's not a linear thing, at some point I will catch up with those that started before me, as the logarithmic curve of learning flattens out. And then, why does it matter that I started a year later?
water-drummer 10mo ago
> Using this stuff well is a deep topic. These things can be applied in so many different ways, and to so many different projects. The best asset you can develop is an intuition for what works and what doesn't, and getting that intuition requires months if not years of personal experimentation. You feel that way because it took you years or months to reach that point. But after reaching that point, do you really think that it's equally—if not more—difficult to put what you learned into words compared to, let's say, programming or engineering? See, the thing about these tools is that they're designed to be operated via natural language, which is something most people (with a certain level of education) are quite comparable to each other at; consequently, the skill ceiling is considerably lower compared to something like programming. I am not saying there's no variance in people's ability to articulate, but that the variance is considerably less than what we get when comparing people's ability to write code or solve engineering problems. So, whatever you learned by trial and error was just different ways or methods to get around the imperfections of the existing LLMs—not ways to use them skillfully according to their design goals. Their design goal is to achieve whatever task is given to them, as long as the intent is clear. These workarounds and tricks that you learned aren't something you build an intuition for. What you build an intuition for is finding new workarounds, but once you've found them, they're quite concrete and easy to describe to someone else who can simply use them to achieve the same results as you. Tools that are designed to be operable via natural language aren't designed to be more thorough—it's actually the opposite. If you want more control, you have programming languages and search engines; thoroughness is where you get that high skill ceiling. The skill ceiling for using these tools is going to get narrower and narrower. The workarounds that you figure out may take skill to discover, but they don't take much skill to replicate. If you share your "tips and tricks" with someone, then yeah, it will take them a week to start getting the same results as you because the skill ceiling is low and the workarounds are concrete/require less thinking.
- simonw 10mo ago
  The more I see of how different people use LLMs the more convinced I am that communication skills differ wildly between different people. Clear, unambiguous communication is a key skill to unlock LLMs. I suspect it's a lot less common than you think!
nitwit005 10mo ago
> Using this stuff well is a deep topic. It might be now, but the intent of these tools is clearly not to have to learn a bunch of work arounds to get the tool to do what you want. If these tools do improve, that inefficiency would presumably reduce, or go away entirely, which means you wouldn't see an advantage to your head start.
- ej88 10mo ago
  I would rather learn how to use these tools effectively now and ship more and of higher quality now. If these tools improve to the point where anyone can pick it up - that's great! I enjoyed my head start while it lasted. If these tools continue to require experience and a skillset to use, that's great too - I'll continue to learn and pull ahead.

by simonw 10mo ago

systemf_omega 10mo ago

> Using this stuff well is a deep topic. Just like the stuff LLMs are being used for today. Why wouldn't "using LLMs well" be not just one of the many things LLMs will simplify too? Or do you believe your type of knowledge is somehow special and is resistant to being vastly simplified or even made obsolete by AI?

simonw 10mo ago
An interesting trend over the past year is that LLMs have learned how to prompt each other. Back in ~2024 a lot of people were excited about having "LLMs write the prompt!" but I found the results to be really disappointing - they were full of things like "You are the world's best expert in marketing" which was superstitious junk. As of 2025 I'm finding they actually do know how to prompt, which makes sense because there's a ton more information about good prompting approaches in the training data as opposed to a couple of years ago. This has unlocked some very interesting patterns, such as Claude Code prompting sub-agents to help it explore codebases without polluting the top level token window. But learning to prompt is not the key skill in getting good results out of LLMs. The thing that matters most is having a robust model of what they can and cannot do. Asking an LLM "can you do X" is still the kind of thing I wouldn't trust them to answer in a useful way, because they're always constrained by training data that was only aware of their predecessors.
leonidasv 10mo ago
Unless we figure out how to make 1 billion+ tokens multimodal context windows (in a commercially viable way) and connect them to Google Docs/Slack/Notion/Zoom meetings/etc, I don't think it will simplify that much. Most of the work is adjusting your mental model to the fact that the agent is a stateless machine that starts from scratch every single time and has little-to-no knowledge besides what's in the code, so you have to be very specific about the context of the task in some ways. It's different from assigning a task to a co-worker who already knows the business rules and cross-implications of the code in the real world. The agent can't see the broader picture of the stuff it's making, it can go from ignoring obvious (to a human that was present in the last planning meeting) edge cases to coding defensively against hundreds of edge cases that will never occur, if you don't add that to your prompt/context material.

jeroenhd 10mo ago

So far every new AI product and even model update has required me to relearn how to get decent results out of them. I'm honestly kind of sick of having to adjust my work flow every time. The intuition just doesn't hold. The LLM gets trained and retrained by other LLM users so what works for me suddenly changes when the LLM models refresh. LLMs have only gotten easier to learn and catch up on over the years. In fact, most LLM companies seem to optimise for getting started quickly over getting good results consistently. There may come a moment when the foundations solidify and not bothering with LLMs may put you behind the curve, but we're not there yet, and with the literally impossible funding and resources OpenAI is claiming they need, it may never come.

christophilus 10mo ago
Really? Claude Code upgrades for me have been pretty seamless- basically better quality output, given the same prompts, with no discernible downsides.

Mawr 10mo ago

I don't see how your position is compatible with the constant hype about the ever-growing capabilities of LLMs. Either they are improving rapidly, and your intuition keeps getting less and less valuable, or they aren't improving.

simonw 10mo ago
They're improving rapidly, which means your intuition needs to be constantly updated. Things that they couldn't do six months go might now be things that they can do - and knowing they couldn't do X six months ago is useful because it helps systematize your explorations. A key skill here is to know what they can do, what they can't do and what the current incantations are that unlock interesting capabilities. A couple I've learned in the past week: 1. Don't give Claude Code a URL to some code and tell it to use that, because by default it will use its WebFetch tool but that runs an extra summarization layer (as a prompt injection defense) which loses details. Telling it to use curl sometimes works but a guaranteed trick is to have it git clone the relevant repo to /tmp and look at the code there instead. 2. Telling Claude Code "use red/green TDD" is a quick to type shortcut that will cause it to write tests first, run them and watch them fail, then implement the feature and run the test again. This is a wildly effective technique for getting code that works properly while avoiding untested junk code that isn't needed. Now multiply those learnings by three years. Sure, the stuff I figure out in 2023 mostly doesn't apply today - but the skills I developed in learning how to test and iterate on my intuitions from then still count and still keep compounding. The idea that you don't need to learn these things because they'll get better to the point that they can just perfectly figure out what you need is AGI science fiction. I think it's safe to ignore.
- crakhamster01 10mo ago
  I feel like both of these examples are insights that won't be relevant in a year. I agree that CC becoming omniscient is science fiction, but the goal of these interfaces is to make LLM-based coding more accessible. Any strategies we adopt to mitigate bad outcomes are destined to become part of the platform, no? I've been coding with LLMs for maybe 3 years now. Obviously a dev who's experienced with the tools will be more adept than one who's not, but if someone started using CC today, I don't think it would take them anywhere near that time to get to a similar level of competency.

mmcnl 10mo ago

Why can't both be true at the same time? Maybe their problems are more complex than yours. Why do you assume it's a skill issue and ignore the contextual variables?

simonw 10mo ago
On the rare occasions that I can convince them to share the details of the problems they are tackling and the exact prompts they are using it becomes very clear that they haven't learned how to use the tools yet.
- UncleEntity 10mo ago
  I'm kind of curious about the things you're seeing since I find the best way is to have them come up with a plan for the work they're about to do and then make sure they actually finish it because they like to skip stuff if it requires too much effort. I mean, I just think of them like a dog that'll get distracted and go off doing some other random thing if you don't supervise them enough and you certainly don't want to trust them to guard your sandwich.

rubslopes 10mo ago

I don't disagree, knowing how to use the tools is important. But I wanted to add that great prompting skill nowadays are far far less necessary for top-tier models that it was years ago. If I'm clear about what I want and how I want it to behave, Claude Opus 4.5 almost always nails it first time. The "extra" that I do often, that maybe newcomers don't, is to setup a system where the LLM can easily check the results of its changes (verbose logs in terminal and, in web, verbose logs in console and playwright).

coffeemug 10mo ago

Strongly disagree. Claude Code is the most intuitive technology I've ever used-- way easier than learning to use even VS Code for example. It doesn't even take weeks. Maybe a day or two to get the hang of it and you're off to the races.

simonw 10mo ago
Don't underestimate the number of developers who aren't comfortable with tools that live in the terminal.
- HDThoreaun 10mo ago
  Well these people are left behind either way. Competent devs can easily learn to use coding assistants in a day or two
- coffeemug 10mo ago
  I actually don't use it in the terminal, I use the vs code extension. It's a better experience (bringing up the file being edited, nicer diffs, etc.) But both are trivial to pick up.
johnsmith1840 10mo ago
The difference is AI tooling lies to you. Day 0 you think it's perfect but the more you use ai tools you realize using them wrong can give you gnarly bugs. It's intuitive to use but hard to master
- coffeemug 10mo ago
  It took me a couple of days to find the right level of detail to prompt it. Too high level, and the codebase gets away from me/the tooling goes off the rails. Too low level, and I may as well do it myself. Maybe also learn the sorts of things Claude Code isn't good at yet. But once I got in the groove it was very easy from there. I think the whole process took 2-3 days.
  - 10mo ago

furyofantares 10mo ago

I think I'm also very good at getting great results out of coding agents and LLMs, and I disagree pretty heavily with you. It is just way easier for someone to get up to speed today than it was a year ago. Partly because capabilities have gotten better and much of what was learned 6+ months ago no longer needs to be learned. But also partly because there is just much more information out there about how to get good results, you might have coworkers or friends you can talk to who have gotten good results, you can read comments on HN or blog posts from people who have gotten good results, etc. I mean, ok, I don't think someone can fully catch up in a few weeks. I'll grant that for sure. But I think they can get up to speed much faster than they could have a year ago. Of course, they will have to put in the effort at that time. And people who have been putting it off may be less likely to ever do that. So I think people will get left behind. But I think the alarm to raise is more, "hey, it's a deep topic and you're going to have to put in the effort" rather than "you better start now or else it's gonna be too late".

biophysboy 10mo ago

What are your tips? Any resources you would recommend? I use Claude code and all the chat bots, but my background isn't programming, so I sometimes feel like I'm just swimming around.

Humorist2290 10mo ago

It needs to be said that your opinion on this is well understood by the community, respected, but also far from impartial. You have a clear vested interest in the success of _these_ tools. There's a learning curve to any toolset, and it may be that using coding agents effectively is more than a few weeks of upskilling. It may be, and likely will be, that people make their whole careers about being experts on this topic. But it's still a statistical text prediction model, wrapped in fancy gimmicks, sold at a loss by mostly bad faith actors, and very far from its final form. People waiting to get on the bandwagon could well be waiting to pick up the pieces once it collapses.

mattmanser 10mo ago
I have a lot of respect from Simon and read a lot of his articles. But I'm still seeing clear evidence it IS a statistical text prediction model. You ask it the right niche thing and it can only pump out a few variations of the same code, that's clearly someone else's code stolen almost verbatim. And I just use it 2 or 3 times a day. How are SimonW and AntiRez not seeing the same thing? How are they not seeing the propensity for both Claude + ChatGPT to spit out tons of completely pointless error handling code, making what should be a 5 line function a 50 line one? How are they not seeing that you constantly have to nag it to use modern syntax. Typescript, C#, Python, doesn't matter what you're writing in, it will regularly spit out code patterns that are 10 years out of date. And woe betide you using a library that got updated in the last 2 years. It will constantly revert back to old syntax over and over and over again. I've also had to deal with a few of my colleagues using AI code on codebases they don't really understand. Wrong sort, id instead of timestamp. Wrong limit. Wrong json encoding, missing key converters. Wrong timezone on dates. A ton of subtle, not obvious, bugs unless you intimately know the code, but would be things you'd look up if you were writing the code. And that's not even including the bit where the AI obviously decided to edit the wrong search function in a totally different part of the codebase that had nothing to do with what my colleague was doing. But didn't break anything or trigger any tests because it was wrapped in an impossible to hit if clause. And it created a bunch of extra classes to support this phantom code, so hundreds of new lines of code just lurking there, not doing anything but if I hadn't caught it, everyone thinks it does do something.

csmpltn 10mo ago

So where’s all of this cutting edge amazing and flawless stuff you’ve built in a weekend that everybody else couldn’t because they were too dumb or slow or clueless?

wild_egg 10mo ago
This is such a tired response at this point. People are under zero obligation to release their work to the public. Simon actually publishes and writes about a remarkable amount of the side projects he builds with AI. The rest of us just build tons of cool stuff for personal use or for $JOB. Releasing stuff to the public is, in general, a massive amount of extra work for very little benefit. There are loads of FOSS maintainers trapped spending as much time managing their communities as they do their actual projects and many of us just don't have time for that.
- Anamon 10mo ago
  The response may be tired when asked in this personal way, but in general, it's a fair question. Nobody is forced to share their work. But with all the high praises, we'd expect to see at least some uptick in the software world. But there is no surge in open source projects. No surge in app store entries. And for the bigger companies claiming high GenAI use, they're not iterating faster or building more. They are continually removing features and their software is getting worse, slower, less robust, and less secure. Software quality has been on a step downwards curve as far as quality and capabilities are concerned, for years before LLM coding had its breakthrough. For all the promises I'd have expected to, three years later, at least notice the downward trajectory easing off. But it hasn't been happening.
- rgoulter 10mo ago
  > The rest of us just build tons of cool stuff for personal use or for $JOB. Releasing stuff to the public is, in general, a massive amount of extra work for very little benefit. There are loads of FOSS maintainers trapped spending as much time managing their communities as they do their actual projects and many of us just don't have time for that. I wouldn't worry about this. There are many examples of people sharing a project they've used LLMs to help write, and the result was not a huge amount of attention & expectation of burden. Perhaps "I don't share it because I'm worried people will love it too much" even suggests the opposite: you can concretely demonstrate the kinds of things you've been able to build using LLMs. > This is such a tired response at this point. Lack of specificity & concrete examples frequently mean all that's left for discussion is emotion for hype and anti-hype, though. In this thread, the discussion was: pro: use LLMs or get left behind conserve: okay, I'll start using LLMs when they're good pro: no no they won't be that good, it takes effort to get to use them conserve: do you have any examples? pro: why should we have to share examples? I like LLMs. But making big claims while being reticent about concrete claims and demonstrations is irksome.

hollowturtle 10mo ago

I can't buy it because for many people like you it's always the other that uses the tools wrong, proving the contrary for skeptics that keep getting bad results from llms it simply is impossible with this narrative as the base of the discourse, eg "you're not using it well". I don't even get why you need to praise yourself so much being really good at using these tools, if not for building some tech influencer status around here... same thing I believe antirez is trying to do(who knows why)

kevin42 10mo ago
Have you considered that maybe you aren't using it well? It's something that can and should be learned. It's a tool, and you can't expect to get the most out of a tool without really learning how to use it. I've had this conversation with a few people so far, and I've offered to personally walk through a project of their choosing with them. Everyone who has done this has changed their perspective. You may not be convinced it will change the world, but if you approach it with an open mind and take the time to learn how to best use it, I'm 100% sure you will see that it has so much potential. There are tons of youtube videos and online tutorials if you really want to learn.
- hollowturtle 10mo ago
  > Have you considered that maybe you aren't using it well? Here we go, as I said, and again and again and again it's always out fault we're not using well. It is impossible to counter argument. Btw to reply to your question, yes many times and proved to be useful in very small specialized tasks and a couple of migrations. I really like how LLMs are helping me in my day to day, but still so far away from all this astroturfing

camel-cdr 10mo ago

How many thing you learned working with LLMs in 2022 are relevant today? How many things you learned now are relevant in the future?

y1n0 10mo ago
This question misses the point. Everything you learn today informs how you learn in the future.

tehnub 10mo ago

I guess this applies to the type of developer who needs years, not weeks, to become proficient in say Python?

deleted 10mo ago

[deleted]

noosphr 10mo ago

I've been building Ai apps since gpt 3 so 5 years now. The pro AI people don't understand what quadratic attention means and the anti-ai people don't understand how much information can be contained in a tb of weights. At the end of the day both will be hugely disappointed. >The best asset you can develop is an intuition for what works and what doesn't, and getting that intuition requires months if not years of personal experimentation. Intuition does not translate between models. Whatever you think dense llms were good at deepseek completely upended it in an afternoon. The difference between major revisions of model families is substantial enough that intuition is a drawback not an asset.

simonw 10mo ago
What does quadratic attention mean? I've so far found that intuition travels between models of a similar generation remarkably well. The conformance suite trick (find a 9,200 test existing conformance suite and tell an agent to build a fresh implementation that passes all those tests) I first found with GPT-5.2 turned out to work exactly as well against Claude Opus 4.5, for example.
- noosphr 10mo ago
  https://arxiv.org/abs/2209.04881 https://arxiv.org/abs/2209.04881
  - simonw 10mo ago
    To save anyone else the click, this is the paper "On The Computational Complexity of Self-Attention" from September 2022, with authors from NYU and Microsoft. It argues that the self-attention mechanism in transformers works by having every token "attend to" every other token in a sequence, which is quadratic - n^2 against input - which should limit the total context length available to models. This would explain why the top models have been stuck at 1 million tokens since Gemini 1.5 in February 2024 (there has been a 2 million token Gemini but it's not in wide use, and Meta claimed their Llama 4 Scout could do 10 million but I don't know of anyone who's seen that actually work.) My counter-argument here is that Claude Opus 4.5 has a comparatively tiny 200,000 token window which turns out to work incredibly well for the kinds of coding problems we're throwing at it, when accompanied by a cleverly designed harness such as Claude Code. So this limit from 2022 has been less "disappointing" than people may have expected.

tymscar 10mo ago

I think you are right in saying that there is some deep intuition that takes months, if not years, to hone about current models, however, the intuition some who did nothing but talk and use LLMs nonstop two years ago would be just as good today as someone who started from scratch, if not worse because of antipatterns that don’t apply anymore, such as always starting a new chat and never using a CLI because of context drift. Also, Simon, with all due respect, and I mean it, I genuinely look in awe at the amount of posts you have on your blog and your dedication, but it’s clear to anyone that the projects you created and launched before 2022 far exceed anything you’ve done since. And I will be the first to say that I don’t think that’s because of LLMs not being able to help you. But I do think it’s because what makes you really, really good at engineering you kept replacing slowly but surely with LLMs more and more by the month. If I look at Django, I can clearly see your intelligence, passion, and expertise there. Do you feel that any of the projects you’ve written since LLMs are the main thing you focus on are similar? Think about it this way: 100% of you wins against 100% of me any day. 100% of Claude running on your computer is the same as 100% of Claude running on mine. 95% of Claude and 5% of you, while still better than me (and your average Joe), is nowhere near the same jump from 95% Claude and 5% me. I do worry when I see great programmers like you diluting their work.

simonw 10mo ago
My great regret from the past few years is that experimenting with LLMs has been such a huge distraction from my other work! My https://llm.datasette.io/ https://llm.datasette.io/ tool is from that era though, and it's pretty cool.
- tymscar 10mo ago
  I do think your datasettes work is fantastic and I genuinely hope you take my previous message the right way. I’m not saying you do something bad, quite the opposite, I feel like we need more of you and I’m afraid because of LLMs we get less of you.
  - beaker52 10mo ago

grayhatter 10mo ago

Show me what you've made with AI? What's the impressive thing that can convince me it's equivalent, or better than anything created before, or without it? I understand you've produced a lot of things, and that your clout (which depends on the AI ferver) is based largely because of how refined a workflow you've invented. But I want to see the product, rather than the hype. Make me say; I wish I was good enough to create this! Without that, all I can see is the cost, or the negative impact. edit: I've read some of your other posts, and for my question, I'd like to encourage you to pick only one. Don't use the scatter shot approach that LLMs love, giving plenty of examples, hoping I'll ignore the noise for the single that sounds interesting. Pick only one. What project have you created that you're truly proud of? I'll go first, (even though it's unfinished): Verse

tsimionescu 10mo ago
There are so many projects named Verse (or similar) that you really need to be more specific.
- grayhatter 10mo ago
  The point wasn't to level set, or for it to feel like I'm promoting it. Only so that I couldn't back out later, or not have any skin in the game. But if you really couldn't figure out which I meant. It's the one on my github, and also hosting on srctree, which I link to from the site in my profile, and from my github.

yeasku 10mo ago

[dead]

lunar_mycroft 10mo ago

The core of your argument is that using LLMs is a skill that takes a significant amount of time to master. I'm not going to argue against that (although I have some doubts) because I think it's ultimately irrelevant. The question isn't "is prompting a skill that you'll need to be an effective software developer in the future" but "what other skills will you need to do so", and regardless of the answer you don't need to start adopting LLMs right away. Maybe AI gets good enough at writing code that it's users' knowledge of computer science and software development becomes irrelevant. In that case, approximately everyone on this site is just screwed. We're all in the business of selling that specialized knowledge, and if it's no longer required then companies aren't going to pay us to operate the AI, they're going to pay PMs, middle managers, executives, etc. But even that won't be particularly workable long term, because all their customers will realize they no longer need to pay the companies for software either. In this world, the price of software goes to zero (and hosting likely gets significantly more commoditized than it is now). Any time you put into learning to use LLMs for software development doesn't help you keep making money selling software, and actually stops you from picking up a new career. If, on the other hand, CS and software engineering knowledge is still needed, companies will have to keep/restart hiring or training new developers. In terms of experience using AI, it is impossible for anyone to have less experience than these new developers. We will, however, have much more experience and knowledge of the aforementioned non-LLM skills that we're assuming (in this scenario) are still necessary for the job. In this scenario you might be better off if you'd started learning to prompt a bit earlier, but you'll still be fine if you didn't.

elvis10ten 10mo ago

While intuition takes a while, I think it can be learned in less than a month or two. This has been my experience. When something gets good enough, someone will create some really good resource on it. Allowing the dust to settle, to me is a more efficient strategy than constantly trying to “keep up”. Maybe also not waiting too long to do so. This wouldn’t work of course if a person was trying to be some AI thought leader.

helloplanets 10mo ago
I'd say that it's a different type of learning process, where even a good resource doesn't help as much as it would with a traditional programming language. Sort of like you can't get very good at writing by just reading a ton of instructional books about it.
- elvis10ten 10mo ago
  Even CRUD programming: you can’t get very good at it with just reading.

stefanlindbohm 10mo ago

Maybe it’s just two different ways to reach the same result. You need to spend time to be great at prompting to get high-quality code from LLM’s, which might just be equivalent to the fact you need to spend time to write high-quality code without LLM’s too. From where I’m standing, I don’t see any massive difference on overall productivity between anyone all in on vibe coding than those who aren’t. There’s not more features, higher quality, etc from teams/companies out there than before on any high-level metrics/observations. Maybe it will come, but there’s also no evidence it will. I do, however, see great gains within certain specific tasks using LLM’s. Smaller scope code gen, rubber ducking, etc. But this seems much less difficult to get good at using (and I hope for tooling that help facilitate the specific types of use cases) and on the whole amounts to marginal gains. It seems fine to be a few years late to catch up, worst case.

noodletheworld 10mo ago

> I don't think you can just catch up in a few weeks, and I do think that the risk of falling behind isn't being taken seriously enough by much of the developer population. This is nonsense. This field moves so fast the things you did more than a year ago aren't relevant anymore. Claude code came out last year. Anyone using random shit from before that is not using it any more. It is completely obsolete in all but a handful of cases. To make matters worse “intuition” about models is wasted learning, because they change, significantly, often. Stop spreading FUD. You can be significantly less harmful to people who are trying to learn by sharing what you actually do instead of nebulously hand waving about magical BS. Dear readers: ignore this irritating post. Go and watch Armin Ronacher on youtube if you want to see what a real developer doing this looks like, and why its hard.

simonw 10mo ago
You're accusing me of spreading harmful advice here, when you're the one telling people that they don't need to worry about not investing in their skills because "This field moves so fast the things you did more than a year ago aren't relevant anymore." One of us is right here. I hope for your sake and the people that listen to you that it's you. I don't think it is.
- noodletheworld 10mo ago
  Simon, you're literally fear mongering. You're making wild claims, and absolutely failing to back them up with evidence. That is FUD. People should invest, they should try things. …but its far faaaaaar less clear cut that dropping everything and focusing on AI right now is so absolutely important. The difference between what you get prompting and a totally naïve user of claude code gets is marginal. People are not being left behind if they try it a bit and find its ok, not great, and come back later. It is not a deep topic. writing AI tools is a deep topic, but most people arent doing that. Youre in the wrong here. Stop making people scared. I quote antirez here, since you clearly arent interested in listening to me: > I have a single suggestion for you, my friend. Whatever you believe about what the Right Thing should be, you can't control it by refusing what is happening right now. > Skipping AI is not going to help you or your career. Think about it. Test these new tools, with care, with weeks of work, not in a five minutes test where you can just reinforce your own beliefs. > Find a way to multiply yourself, and if it does not work for you, try again every few months.

febusravenga 10mo ago

> Using this stuff well is a deep topic. These things can be applied in so many different ways, and to so many different projects. The best asset you can develop is an intuition You're basically saying that using LLMs is like using magic. Telling people to use intuition is basically telling that i don't know how it works and why, but works for me sometimes. That's why we programmers hate it - we have safe space where there's no intuition - namely programming languages & runtimes with deterministic behavior. And we're shoehorned back into mess of magic/intuition and wishfullthinking. (yes, i try llm, i have some results, i'm frustrated mostly by people AI-slopping _everything_ around me)

hahahahhaah 10mo ago
Intuition is the wrong word IMO. Tacit knowledge is the thing. Knowledge that is hard to communicate and needs experience. Problem with AI is it isn't woodwork. The material keeps changing!
theshrike79 10mo ago
It's like any other power tool. It requires skill to use it safely and efficiently. Anyone can use a band saw to cut things. Then go look what Jimmy DiResta makes with one and you see the difference. The chance of an inexperienced person cutting off their finger with a bandsaw is also way over zero, there are things you should not and must not do with it. As with any power tool.
simonw 10mo ago
Oddly enough I wrote about the magic analogy and why I stopped using is a few years ago (pre-ChatGPT, even): https://simonwillison.net/2022/Oct/5/spell-casting/ https://simonwillison.net/2022/Oct/5/spell-casting/ I am eternally frustrated that "intuition" is the key skill people need to work effectively with LLMs, because it's something I can't teach people! If I could figure out how to download my intuition into other people's heads I would do that. Instead I have to convince people that intuition is key, and the only way to get it is to invest in experimenting.

matsemann 10mo ago

I learned Django 15 years after its inception. After 5 years of experience I'm probably not too far behind someone doing the exact same work as me but for 15 years. Or would you say people shouldn't learn Django now? As it's useless as they're already far behind? They shouldn't study computer science, as it will be too late? Every profession have new people continuously entering the workforce, that quickly get up to speed on whatever is in vogue. Honestly, what you've spent years learning and experimenting with, someone else will be able to learn in months. People will figure out the best ways of using these tools after lots of attempts, and that distilled knowledge will be transferred quickly to others. This is surely painful to hear for those having spent years in the trenches, and is perhaps why you refuse to acknowledge it, but I think it's true.

simonw 10mo ago
I would not say that about a framework like Django - though I would encourage people not to under-invest in understanding web fundamentals since once you have those Django, Rails, Next.js etc are all quick to pick up. I would say that about LLMs. That's why I'm ringing the alarm bells here. LLM skills are not the same as framework or library usage skills. They aren't clearly documented or predictable - they're really weird! If you assume learning to use coding agents is the same category of challenge as learning to use something like Django you'll get burned by that assumption.
- matsemann 10mo ago
  But what makes it impossible to catch up? Does it matter if I wait a year and then start? It's not a linear thing, at some point I will catch up with those that started before me, as the logarithmic curve of learning flattens out. And then, why does it matter that I started a year later?

water-drummer 10mo ago

> Using this stuff well is a deep topic. These things can be applied in so many different ways, and to so many different projects. The best asset you can develop is an intuition for what works and what doesn't, and getting that intuition requires months if not years of personal experimentation. You feel that way because it took you years or months to reach that point. But after reaching that point, do you really think that it's equally—if not more—difficult to put what you learned into words compared to, let's say, programming or engineering? See, the thing about these tools is that they're designed to be operated via natural language, which is something most people (with a certain level of education) are quite comparable to each other at; consequently, the skill ceiling is considerably lower compared to something like programming. I am not saying there's no variance in people's ability to articulate, but that the variance is considerably less than what we get when comparing people's ability to write code or solve engineering problems. So, whatever you learned by trial and error was just different ways or methods to get around the imperfections of the existing LLMs—not ways to use them skillfully according to their design goals. Their design goal is to achieve whatever task is given to them, as long as the intent is clear. These workarounds and tricks that you learned aren't something you build an intuition for. What you build an intuition for is finding new workarounds, but once you've found them, they're quite concrete and easy to describe to someone else who can simply use them to achieve the same results as you. Tools that are designed to be operable via natural language aren't designed to be more thorough—it's actually the opposite. If you want more control, you have programming languages and search engines; thoroughness is where you get that high skill ceiling. The skill ceiling for using these tools is going to get narrower and narrower. The workarounds that you figure out may take skill to discover, but they don't take much skill to replicate. If you share your "tips and tricks" with someone, then yeah, it will take them a week to start getting the same results as you because the skill ceiling is low and the workarounds are concrete/require less thinking.

simonw 10mo ago
The more I see of how different people use LLMs the more convinced I am that communication skills differ wildly between different people. Clear, unambiguous communication is a key skill to unlock LLMs. I suspect it's a lot less common than you think!

nitwit005 10mo ago

> Using this stuff well is a deep topic. It might be now, but the intent of these tools is clearly not to have to learn a bunch of work arounds to get the tool to do what you want. If these tools do improve, that inefficiency would presumably reduce, or go away entirely, which means you wouldn't see an advantage to your head start.

ej88 10mo ago
I would rather learn how to use these tools effectively now and ship more and of higher quality now. If these tools improve to the point where anyone can pick it up - that's great! I enjoyed my head start while it lasted. If these tools continue to require experience and a skillset to use, that's great too - I'll continue to learn and pull ahead.