3 ms·
These are good examples of things that I wouldn't expect an LLM to get right, based purely on my own intuition. I don't believe they have much training materia
by simonw 2y ago
These are good examples of things that I wouldn't expect an LLM to get right, based purely on my own intuition.
I don't believe they have much training material on the UI for tools at the moment - it may well come in the future as these new "computer use" models get fed vast amounts of screen capture videos, but to date my hunch is that there hasn't been much focus on that, especially for tools like Photoshop and Blender (training them to use a web browser is a whole lot more useful for the moment).
I'd encourage you not to assume they suck at code just because they suck at answering questions about Photoshop and Blender. I wrote about that a while back: "Don’t fall into the trap of anthropomorphizing LLMs and assuming that failures which would discredit a human should discredit the machine in the same way." - https://simonwillison.net/2025/Mar/11/using-llms-for-code/#set-reasonable-expectations https://simonwillison.net/2025/Mar/11/using-llms-for-code/#s...
- esperent 2y agoHave you actually extensively tried using any LLMs for help with Blender/Photoshop/other apps, or are you just speaking based on your intuition? I use Claude for help with Blender all the time and it's amazing, for the most part. It does have an in depth knowledge of the UI and also of many specific technical ways of doing things. The main thing that it gets stuck on is UI changes between versions, and to be fair, I gets stuck on this too. I will try this same query on Claude tomorrow when I'm in my office. I suspect it'll get it wrong as well, but it's not so much the getting wrong that I had an issue with, but how persistent Gemini was in refusing to admit error and making it seem like I was the one at fault.
- simonw 2y agoI've tried it for a few different GUI things (and "how do I do X on website Y" things) with very mixed results. I've not used it for Blender. Really interesting to hear that Claude does well at this kind of problem! Maybe that's thanks to training they did relative to their Claude Computer Use research last year. https://simonwillison.net/2024/Oct/22/computer-use/ https://simonwillison.net/2024/Oct/22/computer-use/