5 ms·

I don’t think you’re a dimwit but I read your post[1] with an example and I am curious to whether you feel you’re losing something by telling the LLM it’s wrong

I don’t think you’re a dimwit but I read your post[1] with an example and I am curious to whether you feel you’re losing something by telling the LLM it’s wrong and to try again, rather than going through the exploratory/iterative learning process yourself. For example, would you have known to ask about GeoJSON if you had not come across and learned about it pre-LLM? More succinctly: do you feel you’re learning more or less or an equal amount when using LLMs? [1] https://simonwillison.net/2024/Mar/22/claude-and-chatgpt-case-study/ https://simonwillison.net/2024/Mar/22/claude-and-chatgpt-cas...

noduerme 2y ago
You seem to be writing from a lost world when people enjoyed coding because they enjoyed learning new things with every project. I think that's sadly something only enjoyed at work by a very small minority these days.
- simonw 2y ago
  I very much aim to learn something new with every project. The thing I like most about working with LLMs is that they let me take on MORE projects!
- eru 2y ago
  I don't think those 'good old days' ever existed. Or rather, the situation hasn't really gotten worse.
  - noduerme 2y ago
    I probably wasn't clear I still live in that world, both professionally and in personal projects. But my perception is that the vast majority of devs and engineers do not.
simonw 2y ago
Absolutely I could have learned more from that particular project if I'd spent more time with it rather than getting the LLM to do the work... but that's why I like it as an example: since it was effectively a distraction (a "side quest") the alternative wasn't learning more, it was not doing it at all (and learning nothing). I'm able to get really great results out of LLMs because I have 20+ years of experience helping me know what questions to ask of them. I do feel like my rate of learning has increased though, because I'm much more likely to try out a completely new technology when I know an LLM can flatten the learning curve for me a bit.
- gtirloni 2y ago
  > I do feel like my rate of learning has increased It's the same for me. Lots of experience knowing what to ask for. It does a better job in summarizing knowledge and getting me a relatively coherent explanation. Much faster than using Google Search to find and connect the dots from dozens of pages. I just don't see much benefit in its reasoning and code assistance features besides basic stuff.
- yawaramin 2y ago
  > I have 20+ years of experience helping me know what questions to ask of them. That seems rather unlikely given that LLMs didn't exist more than seven years ago, much less 20+.
  - Loocid 2y ago
    They mean they have 20 years experience in programming, which helps them ask good questions.
    - simonw 2y ago
      Yeah, that's what I meant. With LLMs having deep domain knowledge in a field helps enormously in terms of prompting them as productively as possible.
      - yawaramin 2y ago
        So, to maximize the usefulness of LLMs, requires expert level domain knowledge in a field? Sounds like a useful tool for highly trained experts. For the average Joe Developer, perhaps not.
- skydhash 2y ago
  > I'm able to get really great results out of LLMs because I have 20+ years of experience helping me know what questions to ask of them. While a direct answer is nice, I like an iterative/explorative process because of all the things I pick up alongside it. An example is when I was working on an epub reader for macOS (side project). I wanted to a native layout engine instead of a webview and I decided to go with muPDF. This has lead me to know more about text layout and rendering, and embedding C inside Swift than I would if I just have direct answers for every problem (if I'd know the correct questions in the first place). I accumulate side quests like that until I can do a nice experiment to learn as much as I can for a particular domain space.
erhaetherth 2y ago
I was trying to fetch key-value pairs out of a database using PHP+PDO the other day and I knew there was a nice easy to do it but I couldn't remember how. Something about fetchAll, maybe PDO::FETCH_GROUP|PDO::FETCH_COLUMN.... what was it? So I asked a couple LLMs. They wrote out loops for me to format the data how I wanted. I could have copy-and-pasted that in and it would probably have worked. But I felt there was something better yet, so back to Google I go. It's `PDO::FETCH_KEY_PAIR`. It's built-in. But oddly kind of hard to find unless you know the right thing to search for, and "key pair" was not springing to my mind. Point is, if you just let the LLMs do your work you won't even find these better ways of doing things. And I'm quite afraid of what the LLMs are going to do to Google, Stackoverflow and documentation in general. In a couple users it'll be ungoogleable too.
- fzeroracer 2y ago
  As noted LLMs can only give you what you ask for, but for a lot of problems what you ask for isn't what you need; it's two or three steps removed. And LLMs can't tell you that you're doing something wrong; unlike curmudgeonly users on SO or in various forums/channels. My gut feeling is that we're going to enter into a 'dark age' of coding where a lot of previously available resources are going to be ransacked and made hard to find in favor of big corporation owned LLMs. It's already having an extremely bad effect on search in general; we're potentially only a few fights away from sites like SO having users leave en masse. That's why I think having a strong network of engineers to talk with will become more important than ever, almost a return to the IRC days.
- DrSiemer 2y ago
  Strong disagree. I have learned so many new things since I started using LLM's. Some of them I'm actually embarrassed to admit, because I should have known about them a decade ago. If you work in a small company and you are the most experienced developer, you don't often get feedback on how you can improve things. The trick is, quite simply: just ask. I regularly dump some code I wrote in a language model and then ask what can be done better. I would never do that in any online space, because first, I don't wait an answer maybe some day, I need an answer NOW. And second, I prefer to avoid being called a fool.

noduerme 2y ago

You seem to be writing from a lost world when people enjoyed coding because they enjoyed learning new things with every project. I think that's sadly something only enjoyed at work by a very small minority these days.

simonw 2y ago
I very much aim to learn something new with every project. The thing I like most about working with LLMs is that they let me take on MORE projects!
eru 2y ago
I don't think those 'good old days' ever existed. Or rather, the situation hasn't really gotten worse.
- noduerme 2y ago
  I probably wasn't clear I still live in that world, both professionally and in personal projects. But my perception is that the vast majority of devs and engineers do not.

simonw 2y ago

Absolutely I could have learned more from that particular project if I'd spent more time with it rather than getting the LLM to do the work... but that's why I like it as an example: since it was effectively a distraction (a "side quest") the alternative wasn't learning more, it was not doing it at all (and learning nothing). I'm able to get really great results out of LLMs because I have 20+ years of experience helping me know what questions to ask of them. I do feel like my rate of learning has increased though, because I'm much more likely to try out a completely new technology when I know an LLM can flatten the learning curve for me a bit.

gtirloni 2y ago
> I do feel like my rate of learning has increased It's the same for me. Lots of experience knowing what to ask for. It does a better job in summarizing knowledge and getting me a relatively coherent explanation. Much faster than using Google Search to find and connect the dots from dozens of pages. I just don't see much benefit in its reasoning and code assistance features besides basic stuff.
yawaramin 2y ago
> I have 20+ years of experience helping me know what questions to ask of them. That seems rather unlikely given that LLMs didn't exist more than seven years ago, much less 20+.
- Loocid 2y ago
  They mean they have 20 years experience in programming, which helps them ask good questions.
  - simonw 2y ago
    Yeah, that's what I meant. With LLMs having deep domain knowledge in a field helps enormously in terms of prompting them as productively as possible.
    - yawaramin 2y ago
      So, to maximize the usefulness of LLMs, requires expert level domain knowledge in a field? Sounds like a useful tool for highly trained experts. For the average Joe Developer, perhaps not.
skydhash 2y ago
> I'm able to get really great results out of LLMs because I have 20+ years of experience helping me know what questions to ask of them. While a direct answer is nice, I like an iterative/explorative process because of all the things I pick up alongside it. An example is when I was working on an epub reader for macOS (side project). I wanted to a native layout engine instead of a webview and I decided to go with muPDF. This has lead me to know more about text layout and rendering, and embedding C inside Swift than I would if I just have direct answers for every problem (if I'd know the correct questions in the first place). I accumulate side quests like that until I can do a nice experiment to learn as much as I can for a particular domain space.

erhaetherth 2y ago

I was trying to fetch key-value pairs out of a database using PHP+PDO the other day and I knew there was a nice easy to do it but I couldn't remember how. Something about fetchAll, maybe PDO::FETCH_GROUP|PDO::FETCH_COLUMN.... what was it? So I asked a couple LLMs. They wrote out loops for me to format the data how I wanted. I could have copy-and-pasted that in and it would probably have worked. But I felt there was something better yet, so back to Google I go. It's `PDO::FETCH_KEY_PAIR`. It's built-in. But oddly kind of hard to find unless you know the right thing to search for, and "key pair" was not springing to my mind. Point is, if you just let the LLMs do your work you won't even find these better ways of doing things. And I'm quite afraid of what the LLMs are going to do to Google, Stackoverflow and documentation in general. In a couple users it'll be ungoogleable too.

fzeroracer 2y ago
As noted LLMs can only give you what you ask for, but for a lot of problems what you ask for isn't what you need; it's two or three steps removed. And LLMs can't tell you that you're doing something wrong; unlike curmudgeonly users on SO or in various forums/channels. My gut feeling is that we're going to enter into a 'dark age' of coding where a lot of previously available resources are going to be ransacked and made hard to find in favor of big corporation owned LLMs. It's already having an extremely bad effect on search in general; we're potentially only a few fights away from sites like SO having users leave en masse. That's why I think having a strong network of engineers to talk with will become more important than ever, almost a return to the IRC days.
DrSiemer 2y ago
Strong disagree. I have learned so many new things since I started using LLM's. Some of them I'm actually embarrassed to admit, because I should have known about them a decade ago. If you work in a small company and you are the most experienced developer, you don't often get feedback on how you can improve things. The trick is, quite simply: just ask. I regularly dump some code I wrote in a language model and then ask what can be done better. I would never do that in any online space, because first, I don't wait an answer maybe some day, I need an answer NOW. And second, I prefer to avoid being called a fool.