3 ms·

These metaphors all suck. Well, ok, yours is funny. But anyway, LLMs are just very different from any human. They are extremely shallow, even compared to a jun

These metaphors all suck. Well, ok, yours is funny. But anyway, LLMs are just very different from any human. They are extremely shallow, even compared to a junior developer. But extremely broad, even compared to the most experienced developer. They type real fuckin fast compared to anyone on earth, but they need to be told what to do much more carefully than anyone on earth.

Guthur 1y ago
I just spent a good 2 hours trying to debug a SM6 Vulkan issue with unreal engine using an LLM, it had got me to good state but UE kept falling to load a project, it transpired that the specific error message would provide a fix as the top Google result, which I found when I eventually decided to look for myself. LLM did help a lot to get some busy work out of the way, but it's difficult to know when you need to jump out of the LLM loop and go old skool.
- jack_pp 1y ago
  Fwiw I think the ratio of times I needed to go to google for a solution instead of an LLM is like 20:1 for me so your mileage may vary. Depends a lot on the specific niche you're working in. Unrelated to software but recently I wanted to revive an old dumbphone I haven't used since 2014 and apparently I had it password protected and forgot the password and wanted to factory reset it. I found the exact model of the phone and google had only content farm articles that didn't help me at all but Gemini gave me the perfect solution first try. I went to google first because I had no faith in Gemini since to me it seemed like a pretty obscure question but guess I was wrong.
  - Guthur 1y ago
    In the interest of full disclosure my setup is quite esoteric for unreal dev. Linux and nixos no less. To be honest I'd probably have given up on nixos long ago without LLM support. It's actually really quite handy to be able to share a declarative specification of my environment.
  - TexanFeller 1y ago
    Google search has been enshittified. Kagi is where you get real search results now.
    - platevoltage 1y ago
      Unsure why this is downvoted. "Google search has been enshittified" should be a common sentiment here.
CuriouslyC 1y ago
I've gotten Claude Code to make CUDA kernels and all kinds of advanced stuff that there's zero percent chance a junior would pull off. AI is like a super advanced senior wearing a blindfold. It knows almost everything, it's super fast, and it gets confused pretty quickly about things after you tell it.
- skeeter2020 1y ago
  it's not a senior though, because of the amount of oversight and guidance required. I trust senior-plus human developers to do the right thing and understand why it's the right thing. For mission critical things I get another human senior to verify. There's no way I'd autonomously trust 2, 10 or any number of LLMs to do the same thing.
- QuadmasterXLII 1y ago
  have you ever asked a junior developer to write a cuda kernel?
  - CuriouslyC 1y ago
    I've asked juniors to write easier things without success, and I applied the transitive property.
    - deleted 1y ago
      [deleted]
- achierius 1y ago
  You'd be surprised at what juniors can pull off. I have seen fresh-out-of-college new grads write performant GPU kernels that are used in real world library implementations for particular architectures.
  - computerex 1y ago
    What's your sample size?
tjwebbnorfolk 1y ago
> but they need to be told what to do much more carefully than anyone on earth. have you ever managed an offshore team. holy cow
- PessimalDecimal 1y ago
  "Offshore team." "Holy cow." I see what you did there.
  - tjwebbnorfolk 1y ago
    that was accidental
root_axis 1y ago
I think we had the analogy right with "fancy autocomplete". Sometimes, the completions are excellent and do exactly what we want, other times the completions fail to match our expectations and need human attention. It's a tool.
cortesoft 1y ago
And they are incessantly cheerful. I asked Claude to design me a UI, and it made a lovely one... but I wanted a web ui. It very happily through away all its work and made a brand new web UI. I can't imagine any employee being that quick to just move on after something like that.

Guthur 1y ago

I just spent a good 2 hours trying to debug a SM6 Vulkan issue with unreal engine using an LLM, it had got me to good state but UE kept falling to load a project, it transpired that the specific error message would provide a fix as the top Google result, which I found when I eventually decided to look for myself. LLM did help a lot to get some busy work out of the way, but it's difficult to know when you need to jump out of the LLM loop and go old skool.

jack_pp 1y ago
Fwiw I think the ratio of times I needed to go to google for a solution instead of an LLM is like 20:1 for me so your mileage may vary. Depends a lot on the specific niche you're working in. Unrelated to software but recently I wanted to revive an old dumbphone I haven't used since 2014 and apparently I had it password protected and forgot the password and wanted to factory reset it. I found the exact model of the phone and google had only content farm articles that didn't help me at all but Gemini gave me the perfect solution first try. I went to google first because I had no faith in Gemini since to me it seemed like a pretty obscure question but guess I was wrong.
- Guthur 1y ago
  In the interest of full disclosure my setup is quite esoteric for unreal dev. Linux and nixos no less. To be honest I'd probably have given up on nixos long ago without LLM support. It's actually really quite handy to be able to share a declarative specification of my environment.
- TexanFeller 1y ago
  Google search has been enshittified. Kagi is where you get real search results now.
  - platevoltage 1y ago
    Unsure why this is downvoted. "Google search has been enshittified" should be a common sentiment here.

CuriouslyC 1y ago

I've gotten Claude Code to make CUDA kernels and all kinds of advanced stuff that there's zero percent chance a junior would pull off. AI is like a super advanced senior wearing a blindfold. It knows almost everything, it's super fast, and it gets confused pretty quickly about things after you tell it.

skeeter2020 1y ago
it's not a senior though, because of the amount of oversight and guidance required. I trust senior-plus human developers to do the right thing and understand why it's the right thing. For mission critical things I get another human senior to verify. There's no way I'd autonomously trust 2, 10 or any number of LLMs to do the same thing.
QuadmasterXLII 1y ago
have you ever asked a junior developer to write a cuda kernel?
- CuriouslyC 1y ago
  I've asked juniors to write easier things without success, and I applied the transitive property.
  - deleted 1y ago
    [deleted]
achierius 1y ago
You'd be surprised at what juniors can pull off. I have seen fresh-out-of-college new grads write performant GPU kernels that are used in real world library implementations for particular architectures.
- computerex 1y ago
  What's your sample size?

tjwebbnorfolk 1y ago

> but they need to be told what to do much more carefully than anyone on earth. have you ever managed an offshore team. holy cow

PessimalDecimal 1y ago
"Offshore team." "Holy cow." I see what you did there.
- tjwebbnorfolk 1y ago
  that was accidental

root_axis 1y ago

I think we had the analogy right with "fancy autocomplete". Sometimes, the completions are excellent and do exactly what we want, other times the completions fail to match our expectations and need human attention. It's a tool.

cortesoft 1y ago

And they are incessantly cheerful. I asked Claude to design me a UI, and it made a lovely one... but I wanted a web ui. It very happily through away all its work and made a brand new web UI. I can't imagine any employee being that quick to just move on after something like that.