7 ms·

I don't think the current round of AI is capable of replacing anything but Junior developers and doing that would just result in a shock in a few decades when t

I don't think the current round of AI is capable of replacing anything but Junior developers and doing that would just result in a shock in a few decades when there is a lack of Senior developers since on the job training is important for that role. And it is difficult to listen to NVidia talk about what AI can do when it's $2 Trillion evaluation is rooted in AI everything. AI being used to make programming more efficient is fine. At least once it is capable of meaningfully doing that. What a lot of developers don't realize is if your code is long lived the time to write the first time isn't really a primary consideration when you are talking about say 40 hours vs 80 hours. Maintainability and extensibility are core. And if AI doesn't help with those (certainly this batch doesn't) it is only increasing the tech debt problem in the industry.

thechao 3y ago
I've found the AI tooling very helpful for 1:1 pair programming with junior developers. The AI tends to provide bad code at "100%" but it prevents me from constantly having to take the keyboard away from the junior — I can just prompt the junior to type a bit, the AI fleshes out the rest, and we talk about how/why the AI isn't quite right, and what should be correct. The junior is typing the whole time. It really helps, a lot. Probably cuts 1:1 time in half, or better (re: it doubles the amount of stuff we can cover).
- deleted 3y ago
  [deleted]
- kijin 3y ago
  When I'm working with a junior developer I have a rule: never, ever, touch their keyboard or mouse while they are driving. If I absolutely have to help them make some specific input, I'll spell it out, symbols and all, so that the junior can type it themselves. It takes time, but it's so much easier to remember something when you're the one who typed it out. Besides, even before AI, autocomplete was usually good enough that I rarely had to spell out more than a few letters. :)
  - thechao 3y ago
    Yeah — that's what I was doing before AI: clearly spelling out each character, character-by-character. The AI does about 75% of that, so we can skip the 0.5-baud C-over-bongos thing.
andyjohnson0 3y ago
> and doing that would just result in a shock in a few decades when there is a lack of Senior developers since on the job training is important for that role By which time senior management will have retired to the golf course / yacht / beach house. Reward systems etc.
Workaccount2 3y ago
At this rate I would be shocked if junior developers can't function at a higher level than senior ones with the help of AI in a few decades. I'd be shocked if there are developers at all in a few decades.
- nimblegorilla 3y ago
  If AI makes junior developers "better" than senior developers then what stops senior developers from adopting the tools and becoming once again better than the juniors?
  - Abroszka 3y ago
    They are older and learn slower, they also learned a lot of biases over the years. They are often unwilling to accept major changes to workflow.
    - cvhashim04 3y ago
      Maybe the second sentence is true but > They are older and learn slower, they also learned a lot of biases over the years Hasn’t this been disproven time and time again. And research has shown continuous learning helps to prevent or delay cognitive decline.
      - Abroszka 3y ago
        I don't see why that would contradict my statement. Yes learning is good, but it becomes harder as we age.
    - ar_lan 3y ago
      > They are older and learn slower, they also learned a lot of biases over the years. Wow, I rarely see such unwarranted confidence in such a blatantly wrong statement.
      - Abroszka 3y ago
voiceblue 3y ago
AI “feels” more efficient for programming since it spits out code in a jiffy, but once I am done with a session of coding I really have to wonder whether it wouldn’t have just been faster to look up APIs and online discussions rather than repeatedly trying to fix what the LLM was doing or attempting to force it down a given path if you know roughly what you want. (Plus half the time it’s not clear why a method it wants to use isn’t there - not imported, or doesn’t exist and never existed, or defined elsewhere in whatever context used to be around this in training?) It’s almost like using StackOverflow but a toddler is in charge of inputting your queries and only tells you a fraction of what he finds.
- red-iron-pine 3y ago
  this was kind of my experience as well, esp. when dealing with stuff I don't know super well. for example not super up on node.js but know enough to get done what I need to get done. got GPT to throw out some ideas re: linked lists with an approach I'd never seen before. so then the question was... is it smarter than me, and knows the answer better than I do? or is it dumber? I spent more time figuring that out -- it was dumber -- than if I'd just slapped in the node and mongo code myself. though I'll say it was a good learning experience, but not something really sustainable for full-time coding.
BearOso 3y ago
LLMs can only reproduce what has been done before. For many tasks like web programming it can do a lot of what you need because it was trained on that data many times over. When you need to use a concept or knowledge that's fairly rare or oblique it will fail. That's pretty much what Carmack is saying--problem solving will continue to be a needed skill.
- cvhashim04 3y ago
  Unless you’re at the cutting edge of your field, the rest of us are doing work that has been proven and solved by someone else.
  - SkyPuncher 3y ago
    Even when I’m working cutting edge, 99% of my work is still mundane. You still need to get all of the basic stuff in place.
- Guvante 3y ago
  I tell the computer exactly what I want done and it produces another program I can use to do that task. Am I describing an LLM or a programming language? Templates have been a thing for a long time and beyond being better than Google at finding them LLMs fall into the exact same problems those did: If they don't exactly fit your goals you end up wasting a ton of timing fixing the edge cases.
- kragen 3y ago
  this is really far from my experience coding with llms i've often been able to get them to write working code to harness libraries i just wrote (so they definitely aren't in their training set) or programming languages that i haven't even implemented yet. i've fed them obfuscated code in one programming language and asked them to translate it into another—getting sometimes very buggy output, sometimes not. i've asked them for problem-solving approaches for novel problems and gotten reasonable answers—sometimes
smusamashah 3y ago
> anything but Junior developers Not even junior developers. Only thing it can, and somewhat already has is the students with their assignments. And it is going to spoil a lot of future developers forever. Will all those students with LLM skills be able to enter the market and put generated code everywhere or will that serve as a filter. I believe first one because before GPTs the entry skill was googling and copy pasting stackoverflow code.
- Guvante 3y ago
  To be clear I don't mean ChatGPT today. I meant the technology we are using. As in while AI in general could do more the highest possible echelon of this kind of AI is junior developer.
raxxorraxor 3y ago
Nvidia says coding will decline because they want to sell the solution that does that for you. I really hope that nvidia gets competition, although it will be difficult because the AI environment is locked in to a significant degree in many tools. Not completely though, so maybe we will see competitors arise. I am still pretty disappointed with LLMs on their ability to solve logical problems. Just pick an easier logic quiz from the internet and plug it into copilot an co. Copilot seems to now have some templates to answer some of them and I think they fix the model pretty actively. Had some quizzes that were solved wrongly just a few weeks ago, that now get a correct answer. But the overall ability of LLM is quite low still. Always funny when your model says it knows this "classic puzzle" and still gets it wrong. Or the typical "find the cat in the box"-puzzle where the AI just wants to suffocate the cat so it cannot move anymore. Despite that I wouldn't want to miss AI for coding anymore. It is just added value. I have a decent rig that just runs codellama:7b with VSCode&Twinny locally. Not perfect, but a big help already. AI critics often called it autocomplete on steroids. Perhaps it is just that, but maybe that is already quite helpful. I don't see non-coders being able to leverage AI for coding at all. The concept of the citizen-developer was proven wrong so many times already... I think there will be some modified models for certain languages to improve overall tooling around a language.

thechao 3y ago

I've found the AI tooling very helpful for 1:1 pair programming with junior developers. The AI tends to provide bad code at "100%" but it prevents me from constantly having to take the keyboard away from the junior — I can just prompt the junior to type a bit, the AI fleshes out the rest, and we talk about how/why the AI isn't quite right, and what should be correct. The junior is typing the whole time. It really helps, a lot. Probably cuts 1:1 time in half, or better (re: it doubles the amount of stuff we can cover).

deleted 3y ago
[deleted]
kijin 3y ago
When I'm working with a junior developer I have a rule: never, ever, touch their keyboard or mouse while they are driving. If I absolutely have to help them make some specific input, I'll spell it out, symbols and all, so that the junior can type it themselves. It takes time, but it's so much easier to remember something when you're the one who typed it out. Besides, even before AI, autocomplete was usually good enough that I rarely had to spell out more than a few letters. :)
- thechao 3y ago
  Yeah — that's what I was doing before AI: clearly spelling out each character, character-by-character. The AI does about 75% of that, so we can skip the 0.5-baud C-over-bongos thing.

andyjohnson0 3y ago

> and doing that would just result in a shock in a few decades when there is a lack of Senior developers since on the job training is important for that role By which time senior management will have retired to the golf course / yacht / beach house. Reward systems etc.

Workaccount2 3y ago

At this rate I would be shocked if junior developers can't function at a higher level than senior ones with the help of AI in a few decades. I'd be shocked if there are developers at all in a few decades.

nimblegorilla 3y ago
If AI makes junior developers "better" than senior developers then what stops senior developers from adopting the tools and becoming once again better than the juniors?
- Abroszka 3y ago
  They are older and learn slower, they also learned a lot of biases over the years. They are often unwilling to accept major changes to workflow.
  - cvhashim04 3y ago
    Maybe the second sentence is true but > They are older and learn slower, they also learned a lot of biases over the years Hasn’t this been disproven time and time again. And research has shown continuous learning helps to prevent or delay cognitive decline.
    - Abroszka 3y ago
      I don't see why that would contradict my statement. Yes learning is good, but it becomes harder as we age.
  - ar_lan 3y ago
    > They are older and learn slower, they also learned a lot of biases over the years. Wow, I rarely see such unwarranted confidence in such a blatantly wrong statement.
    - Abroszka 3y ago

voiceblue 3y ago

AI “feels” more efficient for programming since it spits out code in a jiffy, but once I am done with a session of coding I really have to wonder whether it wouldn’t have just been faster to look up APIs and online discussions rather than repeatedly trying to fix what the LLM was doing or attempting to force it down a given path if you know roughly what you want. (Plus half the time it’s not clear why a method it wants to use isn’t there - not imported, or doesn’t exist and never existed, or defined elsewhere in whatever context used to be around this in training?) It’s almost like using StackOverflow but a toddler is in charge of inputting your queries and only tells you a fraction of what he finds.

red-iron-pine 3y ago
this was kind of my experience as well, esp. when dealing with stuff I don't know super well. for example not super up on node.js but know enough to get done what I need to get done. got GPT to throw out some ideas re: linked lists with an approach I'd never seen before. so then the question was... is it smarter than me, and knows the answer better than I do? or is it dumber? I spent more time figuring that out -- it was dumber -- than if I'd just slapped in the node and mongo code myself. though I'll say it was a good learning experience, but not something really sustainable for full-time coding.

BearOso 3y ago

LLMs can only reproduce what has been done before. For many tasks like web programming it can do a lot of what you need because it was trained on that data many times over. When you need to use a concept or knowledge that's fairly rare or oblique it will fail. That's pretty much what Carmack is saying--problem solving will continue to be a needed skill.

cvhashim04 3y ago
Unless you’re at the cutting edge of your field, the rest of us are doing work that has been proven and solved by someone else.
- SkyPuncher 3y ago
  Even when I’m working cutting edge, 99% of my work is still mundane. You still need to get all of the basic stuff in place.
Guvante 3y ago
I tell the computer exactly what I want done and it produces another program I can use to do that task. Am I describing an LLM or a programming language? Templates have been a thing for a long time and beyond being better than Google at finding them LLMs fall into the exact same problems those did: If they don't exactly fit your goals you end up wasting a ton of timing fixing the edge cases.
kragen 3y ago
this is really far from my experience coding with llms i've often been able to get them to write working code to harness libraries i just wrote (so they definitely aren't in their training set) or programming languages that i haven't even implemented yet. i've fed them obfuscated code in one programming language and asked them to translate it into another—getting sometimes very buggy output, sometimes not. i've asked them for problem-solving approaches for novel problems and gotten reasonable answers—sometimes

smusamashah 3y ago

> anything but Junior developers Not even junior developers. Only thing it can, and somewhat already has is the students with their assignments. And it is going to spoil a lot of future developers forever. Will all those students with LLM skills be able to enter the market and put generated code everywhere or will that serve as a filter. I believe first one because before GPTs the entry skill was googling and copy pasting stackoverflow code.

Guvante 3y ago
To be clear I don't mean ChatGPT today. I meant the technology we are using. As in while AI in general could do more the highest possible echelon of this kind of AI is junior developer.

raxxorraxor 3y ago

Nvidia says coding will decline because they want to sell the solution that does that for you. I really hope that nvidia gets competition, although it will be difficult because the AI environment is locked in to a significant degree in many tools. Not completely though, so maybe we will see competitors arise. I am still pretty disappointed with LLMs on their ability to solve logical problems. Just pick an easier logic quiz from the internet and plug it into copilot an co. Copilot seems to now have some templates to answer some of them and I think they fix the model pretty actively. Had some quizzes that were solved wrongly just a few weeks ago, that now get a correct answer. But the overall ability of LLM is quite low still. Always funny when your model says it knows this "classic puzzle" and still gets it wrong. Or the typical "find the cat in the box"-puzzle where the AI just wants to suffocate the cat so it cannot move anymore. Despite that I wouldn't want to miss AI for coding anymore. It is just added value. I have a decent rig that just runs codellama:7b with VSCode&Twinny locally. Not perfect, but a big help already. AI critics often called it autocomplete on steroids. Perhaps it is just that, but maybe that is already quite helpful. I don't see non-coders being able to leverage AI for coding at all. The concept of the citizen-developer was proven wrong so many times already... I think there will be some modified models for certain languages to improve overall tooling around a language.