3 ms·

> For those who are becoming far more productive than they previously were, learning new skills, and building the next generation of companies.. that same time

> For those who are becoming far more productive than they previously were, learning new skills, and building the next generation of companies.. that same time has been life changing. As someone who is very much in this camp, I feel that life changing is a bit of an exaggeration for my own experience. GitHub Copilot is a really good autocomplete. Stable Diffusion and company were a fun toy for a while. Perplexity.ai is pretty good at doing what Google could do before the enormous wave of AI junk made search completely intractible. But otherwise... life goes on. I got a few better tools out of it, but the chat models have been reliably useless for my professional work and I therefore don't trust them much for learning something new (see Gell-Mann amnesia [0]). And I'm not someone with my head in the sand—I fine tuned GPT-2 back before anything LLM-based had hit the tech mainstream, much less the general public. I was one of the first to sign up for the Copilot Beta, and I run Ollama locally with a bunch of different models that I've tried out on several side projects. I have access to both Anthropic and OpenAI models at work and regularly break them out to try again... it just never actually helps. People have all these amazing stories of all the productivity they gain by using these models, but at their best I always come away feeling like I spent a few minutes coaching a junior developer through a problem without actually having taught anyone anything. I'm left wondering if I'm somehow doing it wrong or if all the stories of amazing results are coming from juniors who don't realize the model isn't very much better at this than they are. [0] https://en.m.wikipedia.org/wiki/Michael_Crichton#GellMannAmnesiaEffect https://en.m.wikipedia.org/wiki/Michael_Crichton#GellMannAmn...

roenxi 2y ago
Life changing seems fair, I'm making changes to my life assuming that programming jobs are about to get cutthroat competitive and then disappear. We've got something that is about as good as a junior developer in ChatGPT, much cheaper to employ for a month and it's competence is improving. It may not learn anything in a single session but at a higher level the rate of improvement of these models seems to be much faster than a human can manage. A human couldn't possibly learn as much as ChatGPT has in the time it has taken from founding OpenAI to now, and hardware improvement doesn't look like it'll cap out any time soon. We've got all these people in a state of delusion that somehow the job of a developer isn't glorified autocomplete of what someone else prompts them with. We can't be sure what the speed is that it will happen, but the writing is on the wall that AI will obsolete software development roles as they currently exist.
- JTyQZSnP3cQGa8B 2y ago
  If ChatGPT becomes as good as a junior, it may replace this position and we will be stuck with a decreasing number of seniors. But right now it's bad, makes a lot of mistakes and hallucinates APIs which is unacceptable. > the job of a developer [is] glorified autocomplete of what someone else prompts them with Clients don't know what they want, need, or what machines are capable of. Managers can't properly translate that either to the developers. My job is to understand all this and propose solutions, or say no if it's impossible. Will AI do that in the future? Do you remember when Cobol was supposed to replace programmers? Same thing for me. I have yet to see a fully generated solution from the specs to the deployment that takes responsibilities for its mistakes. But not yet another ReactJS frontend that anyone can do in a few days, something real that is useful to me and that you can sell as a real product. We currently have a large number of business guys (look it up on LinkedIn) who learn "prompting" because they believe it will make them gods in the future even if they don't know what a programming language is. And a lot of scammers who sell their prompts to those people. It feels like cryptocurrencies all over again. Last but not least, who will create the new APIs, technologies, and languages that we use? Will we be stuck with C++2x forever? ChatGPT will need a few additional revolutions to do this and to be independent on the open-source that already exists.
  - namaria 2y ago
    Yeah I feel like people have been focusing on getting more, faster compute done. Both software and hardware side. But the hard question of 'what compute should be done' remains the important one.
- dns_snek 2y ago
  > We've got all these people in a state of delusion that somehow the job of a developer isn't glorified autocomplete of what someone else prompts them with. You could make the same oversimplification about any other knowledge or managerial job, up to and including C-suite executives. If this technology is coming and taking everyone's jobs, we're going to need incredibly strong social support structures like Universal basic income and radical redistribution of wealth from AI-owners to fund it. Do you have any ideas for tackling that?
r2_pilot 2y ago
> I'm left wondering if I'm somehow doing it wrong or if all the stories of amazing results are coming from juniors who don't realize the model isn't very much better at this than they are. And here I am, being impressed that SOTA AI passes the Turing programming test for a human junior programmer(using your own interpretation of the skill levels involved). Let's at least appreciate that 10 years ago this level of computer intellect was science fiction. And while I'm very much a "junior developer" (not my day job and I do have a BSCS), I can still use AI productively for my personal backlog of things I'd like to accomplish, and it's been pretty helpful for me notwithstanding its limitations.
- lolinder 2y ago
  > Let's at least appreciate that 10 years ago this level of computer intellect was science fiction. I definitely appreciate that the tech has come a long way! All I'm saying is it hasn't changed my life, and OP's characterization of the AI skeptics as essentially uninterested in learning or having sci-fi expectations is unfair. Some of us may simply be too experienced to get much out of it yet (if ever).
- p1esk 2y ago
  10 years ago this level of computer intellect was science fiction It was still science fiction just 5 years ago.
olvy0 2y ago
My experience is very somilar. In my own narrow domain chatgpt, Claude, copilot et al has been a toy, as in, mostly useless answers to my very specific questions.
rockemsockem 2y ago
The big way I think about how much value AI is providing me is by thinking about how much of my Google search traffic has shifted to chat GPT. It's not just that it's a bit better, it's that it can answer queries I had that Google never could answer, especially really vague stuff.
- lolinder 2y ago
  I've heard that too and it's strictly not true for me. I regularly try it but it's wrong more than half the time. Again, some of this may just come down to experience: by the time I'm turning to ChatGPT or Google it's because I'm in deep water and the docs have failed me. A search engine can turn up the one obscure github issue where someone had the same problem and someone else hints in a direction that ends up having the solution, but that won't have been encoded in the weights with enough strength to be spat out by ChatGPT.
  - rockemsockem 2y ago
    The funny thing is that's the opposite of my flow these days. I start with chatgpt and then fallback to Google if I can't get a good answer. Also, are you using the free version or paid? The difference between gpt3.5 and gpt4/4o is huge.
    - lolinder 2y ago
      At work for real problems, paid. And when I do try a GPT it usually is before turning to a search engine, but after I exhaust my own resources and direct searches in the docs.
JTyQZSnP3cQGa8B 2y ago
I had to scold and mentor juniors who tried to push their crappy ChatGPT code in our codebase. They believe it gives them amazing powers because they don't have the experience to know whether a solution is good, bad, or completely buggy. Most of the time it's inefficient and full of bugs. I do believe you learn from your own mistakes, but that requires thinking for yourself, not generating random answers.
orwin 2y ago
I think it's strictly inferior to a junior dev because it has no domain-specific knowledge, and even training it on your org repo does not really help it seems (ironically, a junior is doing it at my org as upper management want us to use GenAI, so maybe he fudged something)
osigurdson 2y ago
>> coaching a junior developer I have honestly never gotten GPT to work like a junior developer as it can't seem to get details right so the actual coding usually ends up being done by me. What it is great for is being a general teacher (sometimes wrong) and something to bounce ideas off of. It is enormously useful but not quite in the same way that people think it should. I am not the "manager" with the LLM being a direct report. At least I have never gotten it to work like that.

roenxi 2y ago

Life changing seems fair, I'm making changes to my life assuming that programming jobs are about to get cutthroat competitive and then disappear. We've got something that is about as good as a junior developer in ChatGPT, much cheaper to employ for a month and it's competence is improving. It may not learn anything in a single session but at a higher level the rate of improvement of these models seems to be much faster than a human can manage. A human couldn't possibly learn as much as ChatGPT has in the time it has taken from founding OpenAI to now, and hardware improvement doesn't look like it'll cap out any time soon. We've got all these people in a state of delusion that somehow the job of a developer isn't glorified autocomplete of what someone else prompts them with. We can't be sure what the speed is that it will happen, but the writing is on the wall that AI will obsolete software development roles as they currently exist.

JTyQZSnP3cQGa8B 2y ago
If ChatGPT becomes as good as a junior, it may replace this position and we will be stuck with a decreasing number of seniors. But right now it's bad, makes a lot of mistakes and hallucinates APIs which is unacceptable. > the job of a developer [is] glorified autocomplete of what someone else prompts them with Clients don't know what they want, need, or what machines are capable of. Managers can't properly translate that either to the developers. My job is to understand all this and propose solutions, or say no if it's impossible. Will AI do that in the future? Do you remember when Cobol was supposed to replace programmers? Same thing for me. I have yet to see a fully generated solution from the specs to the deployment that takes responsibilities for its mistakes. But not yet another ReactJS frontend that anyone can do in a few days, something real that is useful to me and that you can sell as a real product. We currently have a large number of business guys (look it up on LinkedIn) who learn "prompting" because they believe it will make them gods in the future even if they don't know what a programming language is. And a lot of scammers who sell their prompts to those people. It feels like cryptocurrencies all over again. Last but not least, who will create the new APIs, technologies, and languages that we use? Will we be stuck with C++2x forever? ChatGPT will need a few additional revolutions to do this and to be independent on the open-source that already exists.
- namaria 2y ago
  Yeah I feel like people have been focusing on getting more, faster compute done. Both software and hardware side. But the hard question of 'what compute should be done' remains the important one.
dns_snek 2y ago
> We've got all these people in a state of delusion that somehow the job of a developer isn't glorified autocomplete of what someone else prompts them with. You could make the same oversimplification about any other knowledge or managerial job, up to and including C-suite executives. If this technology is coming and taking everyone's jobs, we're going to need incredibly strong social support structures like Universal basic income and radical redistribution of wealth from AI-owners to fund it. Do you have any ideas for tackling that?

r2_pilot 2y ago

> I'm left wondering if I'm somehow doing it wrong or if all the stories of amazing results are coming from juniors who don't realize the model isn't very much better at this than they are. And here I am, being impressed that SOTA AI passes the Turing programming test for a human junior programmer(using your own interpretation of the skill levels involved). Let's at least appreciate that 10 years ago this level of computer intellect was science fiction. And while I'm very much a "junior developer" (not my day job and I do have a BSCS), I can still use AI productively for my personal backlog of things I'd like to accomplish, and it's been pretty helpful for me notwithstanding its limitations.

lolinder 2y ago
> Let's at least appreciate that 10 years ago this level of computer intellect was science fiction. I definitely appreciate that the tech has come a long way! All I'm saying is it hasn't changed my life, and OP's characterization of the AI skeptics as essentially uninterested in learning or having sci-fi expectations is unfair. Some of us may simply be too experienced to get much out of it yet (if ever).
p1esk 2y ago
10 years ago this level of computer intellect was science fiction It was still science fiction just 5 years ago.

olvy0 2y ago

My experience is very somilar. In my own narrow domain chatgpt, Claude, copilot et al has been a toy, as in, mostly useless answers to my very specific questions.

rockemsockem 2y ago

The big way I think about how much value AI is providing me is by thinking about how much of my Google search traffic has shifted to chat GPT. It's not just that it's a bit better, it's that it can answer queries I had that Google never could answer, especially really vague stuff.

lolinder 2y ago
I've heard that too and it's strictly not true for me. I regularly try it but it's wrong more than half the time. Again, some of this may just come down to experience: by the time I'm turning to ChatGPT or Google it's because I'm in deep water and the docs have failed me. A search engine can turn up the one obscure github issue where someone had the same problem and someone else hints in a direction that ends up having the solution, but that won't have been encoded in the weights with enough strength to be spat out by ChatGPT.
- rockemsockem 2y ago
  The funny thing is that's the opposite of my flow these days. I start with chatgpt and then fallback to Google if I can't get a good answer. Also, are you using the free version or paid? The difference between gpt3.5 and gpt4/4o is huge.
  - lolinder 2y ago
    At work for real problems, paid. And when I do try a GPT it usually is before turning to a search engine, but after I exhaust my own resources and direct searches in the docs.

JTyQZSnP3cQGa8B 2y ago

I had to scold and mentor juniors who tried to push their crappy ChatGPT code in our codebase. They believe it gives them amazing powers because they don't have the experience to know whether a solution is good, bad, or completely buggy. Most of the time it's inefficient and full of bugs. I do believe you learn from your own mistakes, but that requires thinking for yourself, not generating random answers.

orwin 2y ago

I think it's strictly inferior to a junior dev because it has no domain-specific knowledge, and even training it on your org repo does not really help it seems (ironically, a junior is doing it at my org as upper management want us to use GenAI, so maybe he fudged something)

osigurdson 2y ago

>> coaching a junior developer I have honestly never gotten GPT to work like a junior developer as it can't seem to get details right so the actual coding usually ends up being done by me. What it is great for is being a general teacher (sometimes wrong) and something to bounce ideas off of. It is enormously useful but not quite in the same way that people think it should. I am not the "manager" with the LLM being a direct report. At least I have never gotten it to work like that.