14 ms·

In before all the comments about how “most code is trivial” or “most programming is stuff that already exists” or “you’re missing the point look how it’s gettin

In before all the comments about how “most code is trivial” or “most programming is stuff that already exists” or “you’re missing the point look how it’s getting better”. I really am in awe of how much work people seem willing to do to justify this as revolutionary and programmers as infantile, and also why they do that. It’s fascinating. Thinking back to my first job out of college as a solid entry level programmer. ChatGPT couldn’t have done what I was doing on day 2. Not because it’s so hard or I’m so special. Just because programming is never just a snippet of code. Programming is an iterative process that involves a CLI, shell, many runtimes, many files, a REPL, a debugger, a lot of time figuring out a big codebase and how it all links together, and a ton of time going back and forth between designers, managers, and other programmers on your team, iterating in problems that aren’t fully clear, getting feedback, testing it across devices, realizing it feels off for reasons, and then often doing it and redoing it after testing for performance, feel, and feedback. Often it’s “spend a whole day just reading code and trying to replicate something very tricky to find” and you only produce a single tiny change deep in the code somewhere. GPT is absolutely terrible at stuff like this. And yes, often it is finding new solutions that aren’t anywhere on the internet. That’s the most valuable programming work, and a significant % of it. Feel like there’s 10 more points I could make here but I’m on my phone and don’t like wasting too much time on HN. But man, what a disappointment of critical thinking I’ve seen in this specific topic.

derefr 4y ago
Have you seen this video? https://www.youtube.com/watch?v=9QZlQMpNk-M https://www.youtube.com/watch?v=9QZlQMpNk-M I think the author is onto something – while AI might not be able to program per se, it can certainly be handed a code snippet and then use its huge corpus of Internet Learning™ to tell you things about it, code that looks like it, and ways (people on the Internet think) it might be solved better. In that sense, it isn't replacing the programmer; it's replacing IDE autocomplete.
- red75prime 4y ago
  > ways (people on the Internet think) It reminds me of arguments that it's not the computer that plays chess, but its programmers. You can describe a GPT's response as a statistical average of responses on the internet (for quite a contrived definition of average), but at some point it will be easier to describe it as analyzing a snippet and forming an opinion (based on what people on the Internet think). Are we past that point? I'm not sure yet, but we are close.
  - z3c0 4y ago
    > quite a contrived definition of average Sure, maybe an argmax mixed with an RNG isn't really an average, but some would say it's quite a contrived definition of forming an opinion.
    - red75prime 4y ago
      argmax is doing all the heavy lifting there. Consider argmax is_proof_of_P_eq_NP(string) You can describe solution of any well defined problem with argmax, it doesn't make it trivial.
- matwood 4y ago
  > replacing IDE autocomplete Co-pilot has been very useful the times I've used it. It's not perfect, but does cover a lot of boiler plate. It also makes it much easier to jump between languages.
  - brookst 4y ago
    I’ve been working with copilot a few months and the biggest surprise is how it has led to much better commented code. I used to comment tricky code to explain it. Now I comment trivial code instead of writing it. Often two lines of comment will get me 10 lines of copiloted code, faster than I could have typed it and with good comments to boot.
- aix1 4y ago
  I really enjoyed the video, thanks for sharing. I think the author is operating in what I consider to be the sweet spot of current LLMs - where I can ask a question I don't know the answer to but can reliably spot bullshit (either because I know enough or through other means). I think there's a lot of value to be had when those conditions are met, and not just for coding.
- sumtechguy 4y ago
  That is a good take on it. I have been saying this thing is like really good at summary and terrible on detail. So watch what it spits out. Last night I sat down and tried using it to write an 8086 emulator. It got an simple emulation outline fairly quickly. But when it came to getting each of the instructions and interrupts correct. It fell very flat very quickly. What was interesting is that it made the exact same mistakes many early emulator writers make. You could then correct it and it would give it a shot at 'doing better'. I at one point got a bit bored with it and kept feeding it 'can you make that more compact/better'. It did an adequate job at that, eventually using templates and jump lists. It did not get very far using duffs device or dynrec but I am sure I could have guided it into doing that. But probably for a majority of things like in emulation 'close enough is good enough'. That is an interesting finding of coding things up I think. This thing is also going to make seriously crazy amount of bugs we will be chasing for decades.
- jackcviers3 4y ago
  This is how I use it. It is the best rubber duck money can buy.
- tootie 4y ago
  I spent this afternoon asking ChatGPT (3.5 not 4) to help me query AWS resources into a csv. It gave me a 90% correct answer but made up a native csv output option. When I told it that option didn't exist it got almost sassy insisting it was correct. Eventually it gave me a closer answer using json and jq after I prodded it. I had a similar experience asking it to write an API client. It wrote something very plausible but just concocted an endpoint that looked real but didn't exist.
jacobedawson 4y ago
I think most people see where the puck is going, and even where it is right now is very impressive. It's not hard to see that it will be likely less than 5 years before it will be able to do what you did on day 2, and much more, at a tiny fraction of the cost, with no downtime, no attitude problems, sick days, etc. The things you mentioned (taking on board feedback, testing across devices, iterating on solutions) doesn't seem very far away at all. The rate of increase in capabilities is also unpredictable, which is what is amazing & terrifying.
- mlycv 4y ago
  That's what they said about Watson and Wolfram Alpha. You only hear this nonsense on HN.
  - refulgentis 4y ago
    Demonstrably untrue, it’s taken up semi-residence in everything from oped pages to boardrooms since the New Year. Welcome to the mid singularity, my friend.
    - dr_dshiv 4y ago
      You can’t say “singularity” on HN without a ton of downvotes. It’s not because people disagree (or they’d say so), but because they literally shit their pants when they think about it. Hard to blame them.
      - vbezhenar 4y ago
        I'm saving my money, so I can live enough off savings to learn a new profession. I'm not sure if that will happen, but I consider this option as likely enough to affect my life decisions. The world is changing at alarming rate. Though I think that it's more likely that I'll become shaman who talks to the spirits. Managers hate computers and they'd prefer to pay someone to deal with them.
      - suction 4y ago
JimmyRuska 4y ago
This post feels like the people that go into linux forums and say linux sucks because I can't get it to do X but microsoft can, but then get 400 replies and several that show <how to do thing> GPT has limited reasoning but given enough knowledge of the problem you can coerce it to do surprising things so long as you can relate it to something in else in the knowledge base. Given how big that knowledge base is, you can get lucky surprises where things just work if you fish around enough
- matwood 4y ago
  > you can coerce it to do surprising things Yeah, I've gotten it to write some pretty decent contracts, but only because I have written said contracts and can ask it all the right questions/prod it into adding what I need.
  - hnfong 4y ago
    And the next iteration of ChatGPT will likely generate a better contract because your interaction with it will be used as training data
    - jamiek88 4y ago
      Huh. There’s something scary about that in my lizard brain. Not sure what.
      - pixl97 4y ago
        Its like feeding a baby polar bear. When it grows up it is going to eat you.
        ashwagary 4y ago
        Extending this analogy into a question: Could the polar bear community eat enough expert food providers that their quality of food and eventually overall health declines?
kevinwang 4y ago
Eh, it's still impressive that these systems can write such good code despite pretty much just predicting the next word. I guess it's a matter of perspective. You can either be astounded by how much it can do relative to your expectations from 2018 or you can be skeptical relative to the volume of excitement. I think some excitement is also people extrapolating to the future: if predicting the next word gets you this far, what happens when you actually try to make it good?
- copperx 4y ago
  > despite pretty much just predicting the next word Is GPT really like the old school NLP trigram stuff?
  - hnfong 4y ago
    "predicting the next word" sounds trivial until you realize the value and complexity of "predicting the next word[s] by a world leading expert in a particular domain".
    - Jensson 4y ago
      Quoting famous people sounds smart until you realize they just memorized a ton of trivia. These models have demonstrated that they don't learn logical models, instead they learn to generate text that looks logical at first glance but is nonsense.
      - sgarland 4y ago
        I asked GPT-4 something fairly niche that I happen to know a fair amount about: to explain the concept of Xenon poisoning in a nuclear reactor. Other than skipping Te-135 being the initial fission product that starts the decay chain (and tbf, operationally it can be skipped since the half-life is 19 seconds), it got everything correct. I'm sure if I kept probing on smaller and smaller details it would eventually fail, but I'd argue for _most_ people, on _most_ subjects, it performs incredibly well.
pewpew2020 4y ago
I feel it's the average programmer who is enamoured with GPT. The programmers working on hard stuff know that AI writing programs is still far away.
- MrDresden 4y ago
  It is not just those who are working on hard problems who are of that opinion. Good luck having a ML model understand a 20 year old undocumented dataformat developed inhouse at a specific research lab to be used in their proprietary systems which are also undocumented and are a part of a spiderweb of interconnected systems at that lab (I have worked at this particular lab). It will be a long time (if ever) until a ML model will be able to handle these situations (and I hazard a guess that most of the worlds active code is something akin to this). As a supporting tool for the software engineers working there, sure. Just like a linter.
  - sgarland 4y ago
    Have you tried giving it a skeleton example of said undocumented data format? I've created some arbitrary ones and asked it to parse them, and it succeeded.
  - hackinthebochs 4y ago
    >It will be a long time (if ever) until a ML model will be able to handle these situations (and I hazard a guess that most of the worlds active code is something akin to this). ChatGPT can do that right now. Just provide it example data in the input and it can extrapolate the format and interact with this abstraction (i.e. describe the format, write code to it, etc). LLMs don't just remix data it has seen before, they perform in-context learning. This means abstracting out patterns in the input and then leveraging it in generating output.
- anonzzzies 4y ago
  It’s also programmers who work with average and bad programmers and finally have a way to not waste time with them anymore. I work on the hard stuff, gpt generate the rest (I don’t have to waste time and energy to try to pull dead horses) and my colleagues go do something else. It’s a reality today and will improve further. Sure it cannot do what I do, but that might improve; gpt4 is noticeably better than 3.5 at slightly harder things, especially if you work properly iterative with it and having it fix it’s own errors. But no, it cannot do hard things it never saw; however it can give me all the boilerplate around the hard things so I only have to do the hard things. My days are simply 3-5 in 1 now, as I also don’t have the overhead of explaining jira tasks to others who are never going to really grasp what they are doing anyway. So I am not average and I am enamoured with gpt, simply because it presents high value to me now, more than some actual real humans. For me that’s enough revolutionary.
maxdoop 4y ago
I don’t think you’ve tried GPT-4. If you have, I don’t think you are like majority of devs (maybe not on HN, but in real life). You sound lucky to have true, novel problems to solve each day. I’m with many here commenting that this is quite powerful stuff, especially when my day-to-day is writing simple CRUD apps, or transforming data from one format to another within an API, or configuring some new bit of infra or CI/CD. I’d love to be challenged in some new way, and have access to truly fascinating problems that require novel solutions. But most enterprises aren’t really like that nor do that need that from majority of engineers.
- whateveracct 4y ago
  There's not such thing as actually boring CRUD. I've worked at many companies and founded my own. Even when it felt like CRUD, a year+ in it was clear that tasteful decisions pay off and iteration and cost gradients matter. GPT doesn't sniff that. I agree with GP - day 2 dev me outclasses it, which means it isn't replacing anyone.
  - anonzzzies 4y ago
    HN echo chamber. Generally the better/best programmers hang around here; your day 2 was probably better than many coders hope to achieve in their whole lives. It is replacing people already; I personally know about 10 people who have been fired or assigned to a completely different position because gpt did a faster and better or equal job. So ‘not anyone’ is simply nonsense; I am only one person, there will be many many more who see the same thing. Of course they are not told they are replaced by ai when fired, but their managers or more skilled colleagues know. I do agree that there is no boring crud; that’s why gpt or no code tools are not good for full solutions (yet), but it’ll get there I am sure.
    - DaiPlusPlus 4y ago
      > I personally know about 10 people who have been fired or assigned to a completely different position because gpt did a faster and better or equal job Please elaborate. And, if true, this would be a major news story that Vox or any number of major newspapers would love to write about - so have you approached the media about this? If not, why not?
spaceman_2020 4y ago
I started learning how to code about 6 months ago, mostly to build prototypes of a couple of app ideas. I have no intention of getting a coding job - if the prototype is successful, I'll seek a technical co-founder. Last couple of months, I've been using chatGPT to write a lot of features and functions. I don't think it has made me a better coder, but it has made me massively more productive. Things like scraping data from a URL - something I would have had to sit through an hour long tutorial to learn - is accessible with a single query. I also think that the code quality has improved over the last few iterations. It makes fewer mistakes now.
- davewritescode 4y ago
  This is exactly what GPT is great for, accelerating learning.
  - spaceman_2020 4y ago
    I predict there will be an explosion of productivity and small scale entrepreneurship. These tools are going to give so many smart but technically unskilled people a path towards realizing their ideas and vision. Exciting times
qikInNdOutReply 4y ago
The reason is easy to imagine. Most non-programmers, are living like analphabets in a world were reading is valuable super power. They grudgingly accept this power assemtry, but ocassionally rebel - with "easier" visual programming languages made and excel. This is another one of those rebellions, non-programers hoping to avoid reading the book and closing it for good, while keeping the awesome around. The code-bases we will see, were the commits are basically chatgpt tags and tasks for each document.
Nevermark 4y ago
> I really am in awe of how much work people seem willing to do to justify this as revolutionary and programmers as infantile [...] Well it is revolutionary. And it isn't just where it is today, but how fast these models are improving - with no saturation in ability evident at this time. On the other hand, I am not sure anyone is saying programmers are infantile - although poorly written software is as at least as prevalent as poorly compensated or managed software development positions.
yucky 4y ago
> I really am in awe of how much work people seem willing to do to justify this as revolutionary and programmers as infantile, and also why they do that. It’s fascinating. Equally fascinating is all of the "this is fine" posts from programmers suddenly realizing they are not the gods they once thought. But fret not, programming is not the first industry that has been automated into a shell of itself. Yes, the industry is going to shrink massively, but this is what new skills are for. Just as farmers had to learn industrial jobs and then miners and autoworkers had to "learn to code", most programmers will have to learn to do something else. Humans are resilient and will adapt. And there will still be jobs in development for the most talented and in niche areas, but when the largest tech companies can layoff hundreds of thousands of employees without skipping a beat that should tell you all you need to know about the value of most "programming" jobs.
- dmm 4y ago
  > Just as farmers had to learn industrial jobs and then miners and autoworkers had to "learn to code" The transition from agrarian to industrial societies was extremely painful and arguably it was centuries before the people affected were better off. > Humans are resilient and will adapt. Based on recent events I think it's more likely people will elect extremist politicians who promise quick fixes while blaming a convenient out-group for everything.
- lamp987 4y ago
  "Yes, the industry is going to shrink massively" You meant "expand massively" i think. Did all the programmers manually making machine code get fired and the job of programmer disappear when compilers were invented and totally replaced these jobs? No, it just changed to use the new tool. There wont be any unprecedented mass layoffs, despite what the jaded STEM-hating twitter crowd wants. Companies will simply make their programmers use these tools to increase the amount of produce per employee, and make software that would have been otherwise financially impossible to be made. Because the competition will do so too to get ahead.
  - yucky 4y ago
    > You meant "expand massively" i think. Nope, I meant what I said. > There wont be any unprecedented mass layoffs Baghdad Bob strikes again.
SuoDuanDao 4y ago
I think we're seeing the early phases a prediction I made in my first book come true: That computers will be more suitable than humans for most of the functions humans currently use their left brain half for. Best case, that will have a whole lot more humans using their right brain halves on things like defining the problem. I like the thought of that, it's more pleasant work. But a lot of intelligent people define their intelligence by how well their left brain half works and uncomfortable with how good Chatgpt is at those tasks. I think you're pointing out there's more to programming than left-brain activities, and I think you're right that silicon will never eclipse carbon at those challenges, but I think a lot of people are also feeling threatened by the fact that chatgpt is getting better and better at the thing they used to be better than all humans at.
- davemp 4y ago
  > But a lot of intelligent people define their intelligence by how well their left brain half works and uncomfortable with how good Chatgpt is at those tasks. I think 'left brain' work also has a lot more predictability (knowing you can churn out ~X widgets/hr) so having only uncertain 'right brain' work can be uncomfortable for people to build their livelihoods upon. That being said. 'right brain' work is certainly more fulfilling for me.
  - danuker 4y ago
    > can be uncomfortable for people to build their livelihoods upon Absolutely. Not everyone can be consistently creative, and without a large enough financial buffer, it's a recipe for disaster.
  - SuoDuanDao 4y ago
    Great point. That's one reason I'm increasingly a single-issue voter focused on UBI.
- Quarrelsome 4y ago
  I feel like there's a slight inaccuracy here that this article covers. GPT-like models excel at logical problems that have already been solved. As the article suggests the models (at least at the moment) are extremely keen to utilise existing solutions as opposed to inventing new ones. The existing solutions end up somewhat being far too distracting when problem complexity compounds beyond the common. This implies that those truly talented at solving logical problems will still be spearheading the development of novel solutions. We might be able to state that GPT will easily trim away all the average workloads for both the left and right. It can perform "creative writing" or draw pictures to an average or even beyond average extent, but it continues to currently struggle to hit the exceptional examples that humanity are capable of.
munchbunny 4y ago
These statements can definitely be simultaneously true: * ChatGPT is revolutionary - honestly, it's genuinely impressive how much of a leap ChatGPT is compared to the attempts that came before it. * Programmers write a lot of simple code that has been written before - there are genuinely tons of cases of "write a web endpoint that takes an ID, looks it up in a database table, pulls an object through an ORM, and returns a JSON serialization of it." Most programmers? Doubt it. But tons of programmers write CRUD stuff and tons of IT admins do light scripting, and a lot of it is repeated code. Could ChatGPT do my job? Not even close. But it's still really impressive to me.
- Jeff_Brown 4y ago
  I feel safe too, and I'm amused at the dynamic. GPT could do a lot of the things I do, but it would take someone who knows what I know in order to explain the task in sufficient detail for it to do that.
staminade 4y ago
While I think there's truth to what you say, I'd also point our that workers in many pre-automated industries with an "artisan" approach also considered themselves irreplaceable because they figured, correctly, that nobody could build a machine with the capability of reproducing their workflow, with all its inherent uncertainty, flexibility and diverse physical and mental skills. What they failed to predict was that some people wouldn't try to automate them like-for-like. Instead they would reconfigure their entire approach to fit with the specific advantages and limitations of the machinery. And this new approach might even be qualitatively worse in various ways, but not so much as to overwhelm the economic advantages that provided by the things machines were good at. AI likely isn't going to slot into a developer-shaped hole in a software team. But it's possible we'll see new organisation approaches, companies, and development paradigms that say: How far can you get if you put prompt-generated code at the heart of the workflow and make everything else subservient to it. I'm not sure, right now, that that approach is feasible, but I'm not sure it won't be in a year or two.
- Quarrelsome 4y ago
  That's an extremely interesting thought. Perhaps we will see organisations in the future structure themselves more like a suite of unit tests. Instead of getting a developer or software house to plug a specific need in their org and being entirely outside of the development process: they will reflect the development process organisationally to ensure they catch any problems with the current output and just feed the box new prompts to increase their efficiency or efficacy. Their competitive advantage in their field then becomes the range of their tests (borne through experience), efficiency in running their pipeline of testing and ability to generate effective prompts.
  - yc-kraln 4y ago
    This is already how automotive companies work. They are huge organizations which do four things: marketing, financing, design, and requirements. The supplier management and project management all fall under requirements management and enforcement.
  - jackcviers3 4y ago
lionkor 4y ago
For me, its mostly that I have used GPT-3.5 a little for programming C++, and I wasnt impressed. For one, it made horrible, glaring mistakes (like defining extern functions which dont exist, using functions which are specific to a platform im not using, etc.), stuff beginners would do. It also decided to sneak in little issues, such as off-by-one errors (calling write() with a buffer and a size that is off by one in a place where its very hard to tell), missing edge cases (such as writing a C++ concept which worked, but actually did everything in slightly the wrong way to actually ensure the concept was requiring exactly what I asked). Even when asked to correct these mistakes, it often struggled, made me read paragraph after paragraph of "im sorry, ive been such a bad little machine" garbage, and didnt even correct the issue (or, in some cases, introduced new bugs). Im utterly unimpressed by this. GPT is great for a lot of things, but not writing code better than I would, in the same time. The time it took me to massage it to solve a nontrivial problem (write hello world with just syscalls) was way longer than reading the manual and writing it myself (and has less bugs). Not everyone unfazed by these articles is simply in denial. I feel sorry for people who write copy paste code and find that ChatGPT or Clippy from 2000 can relace them, but not everyone writes trivial code.
- woeirua 4y ago
  Two points: GPT4 is significantly better in this regard, and you should be concerned about the rate of progress more than it’s actual capabilities today.
  - Jeff_Brown 4y ago
    > you should be concerned about the rate of progress Kind of agree? On the one hand we don't even have a roadmap toward reliable AI. On the other, if we ever plug an LLM into something that has memory, acquires experiences, does experiments, observes the outcome and adjusts its worldview in response, consciousness might fall out of that. And writing good code might not even require consciousness.
    - flatline 4y ago
      Epistemologically speaking, I think we can roughly break down the potential nature of consciousness into three categories: - as a function of an independent human soul - as the fundamental substrate on which the rest of the universe is built - as a byproduct/secondary phenomenon of physical processes In the latter two cases I believe that the question of whether GPT is conscious is immaterial. In either case it is functioning in the same medium we all are when we talk, think, write. In the first case it is not, and the question is thornier.
yodsanklai 4y ago
The problem is that many people here have an extreme point of view. It's either "this is going to make all develops jobless" or "it's useless, my job isn't that". I think It'll help with some tasks, which is always good to take. After all, people tweak their vim settings because they feel it makes them more productive.
- Jeff_Brown 4y ago
  Software development seems safe for the time being, but as someone who has both paid professional translators and used ChatGPT for translation, I'm certain GPT is obliterating some jobs.
- travisjungroth 4y ago
  This tech is so powerful, cutting so close to the last of human output that’s uncopied, advancing so fast, I have an incredibly hard time imagining it ending in a space that deserves a muted a reaction. It seems like it will either dramatically change society or fail. And I have quite a hard time imagining failure. It seems on par with the Industrial Revolution, at least. Which, easy to forget, was a huge deal that totally changed society.
siftrics 4y ago
cope
janeway 4y ago
I think this clip section says a lot - if videos are permitted as replies. https://youtube.com/watch?v=MNtjSMvcjzs&t=211s https://youtube.com/watch?v=MNtjSMvcjzs&t=211s
DuckFeathers 4y ago
It's a similar waste of my time to respond to comments like this... but I will at least try to give some pointers so that those who encounter this comment won't immediately fall into that line of thinking: - a lot of programmers, including experienced ones, are absolutely infantile and they only have a job because there is a big shortage of programmers; not all of them get better with experience... hence a significant part of software development is dealing with problematic programmers and problems created by them. - GPT is not that great a programmer but a great thing about it is that it is not a human... and one can get thousands of instances of them for the price of one human. You only need one of those instances to produce usable code. - there have been many changes throughout the years which have definitely replaced a lot of programmers: library distribution services (pypi, npmjs), better software development tools and practices, SaaS delivery model, better programming languages etc.; so far, because the market need for programmers has continued to increase, most programmers continue to have jobs; this won't last forever.

derefr 4y ago

Have you seen this video? https://www.youtube.com/watch?v=9QZlQMpNk-M https://www.youtube.com/watch?v=9QZlQMpNk-M I think the author is onto something – while AI might not be able to program per se, it can certainly be handed a code snippet and then use its huge corpus of Internet Learning™ to tell you things about it, code that looks like it, and ways (people on the Internet think) it might be solved better. In that sense, it isn't replacing the programmer; it's replacing IDE autocomplete.

red75prime 4y ago
> ways (people on the Internet think) It reminds me of arguments that it's not the computer that plays chess, but its programmers. You can describe a GPT's response as a statistical average of responses on the internet (for quite a contrived definition of average), but at some point it will be easier to describe it as analyzing a snippet and forming an opinion (based on what people on the Internet think). Are we past that point? I'm not sure yet, but we are close.
- z3c0 4y ago
  > quite a contrived definition of average Sure, maybe an argmax mixed with an RNG isn't really an average, but some would say it's quite a contrived definition of forming an opinion.
  - red75prime 4y ago
    argmax is doing all the heavy lifting there. Consider argmax is_proof_of_P_eq_NP(string) You can describe solution of any well defined problem with argmax, it doesn't make it trivial.
matwood 4y ago
> replacing IDE autocomplete Co-pilot has been very useful the times I've used it. It's not perfect, but does cover a lot of boiler plate. It also makes it much easier to jump between languages.
- brookst 4y ago
  I’ve been working with copilot a few months and the biggest surprise is how it has led to much better commented code. I used to comment tricky code to explain it. Now I comment trivial code instead of writing it. Often two lines of comment will get me 10 lines of copiloted code, faster than I could have typed it and with good comments to boot.
aix1 4y ago
I really enjoyed the video, thanks for sharing. I think the author is operating in what I consider to be the sweet spot of current LLMs - where I can ask a question I don't know the answer to but can reliably spot bullshit (either because I know enough or through other means). I think there's a lot of value to be had when those conditions are met, and not just for coding.
sumtechguy 4y ago
That is a good take on it. I have been saying this thing is like really good at summary and terrible on detail. So watch what it spits out. Last night I sat down and tried using it to write an 8086 emulator. It got an simple emulation outline fairly quickly. But when it came to getting each of the instructions and interrupts correct. It fell very flat very quickly. What was interesting is that it made the exact same mistakes many early emulator writers make. You could then correct it and it would give it a shot at 'doing better'. I at one point got a bit bored with it and kept feeding it 'can you make that more compact/better'. It did an adequate job at that, eventually using templates and jump lists. It did not get very far using duffs device or dynrec but I am sure I could have guided it into doing that. But probably for a majority of things like in emulation 'close enough is good enough'. That is an interesting finding of coding things up I think. This thing is also going to make seriously crazy amount of bugs we will be chasing for decades.
jackcviers3 4y ago
This is how I use it. It is the best rubber duck money can buy.
tootie 4y ago
I spent this afternoon asking ChatGPT (3.5 not 4) to help me query AWS resources into a csv. It gave me a 90% correct answer but made up a native csv output option. When I told it that option didn't exist it got almost sassy insisting it was correct. Eventually it gave me a closer answer using json and jq after I prodded it. I had a similar experience asking it to write an API client. It wrote something very plausible but just concocted an endpoint that looked real but didn't exist.

jacobedawson 4y ago

I think most people see where the puck is going, and even where it is right now is very impressive. It's not hard to see that it will be likely less than 5 years before it will be able to do what you did on day 2, and much more, at a tiny fraction of the cost, with no downtime, no attitude problems, sick days, etc. The things you mentioned (taking on board feedback, testing across devices, iterating on solutions) doesn't seem very far away at all. The rate of increase in capabilities is also unpredictable, which is what is amazing & terrifying.

mlycv 4y ago
That's what they said about Watson and Wolfram Alpha. You only hear this nonsense on HN.
- refulgentis 4y ago
  Demonstrably untrue, it’s taken up semi-residence in everything from oped pages to boardrooms since the New Year. Welcome to the mid singularity, my friend.
  - dr_dshiv 4y ago
    You can’t say “singularity” on HN without a ton of downvotes. It’s not because people disagree (or they’d say so), but because they literally shit their pants when they think about it. Hard to blame them.
    - vbezhenar 4y ago
      I'm saving my money, so I can live enough off savings to learn a new profession. I'm not sure if that will happen, but I consider this option as likely enough to affect my life decisions. The world is changing at alarming rate. Though I think that it's more likely that I'll become shaman who talks to the spirits. Managers hate computers and they'd prefer to pay someone to deal with them.
    - suction 4y ago

JimmyRuska 4y ago

This post feels like the people that go into linux forums and say linux sucks because I can't get it to do X but microsoft can, but then get 400 replies and several that show <how to do thing> GPT has limited reasoning but given enough knowledge of the problem you can coerce it to do surprising things so long as you can relate it to something in else in the knowledge base. Given how big that knowledge base is, you can get lucky surprises where things just work if you fish around enough

matwood 4y ago
> you can coerce it to do surprising things Yeah, I've gotten it to write some pretty decent contracts, but only because I have written said contracts and can ask it all the right questions/prod it into adding what I need.
- hnfong 4y ago
  And the next iteration of ChatGPT will likely generate a better contract because your interaction with it will be used as training data
  - jamiek88 4y ago
    Huh. There’s something scary about that in my lizard brain. Not sure what.
    - pixl97 4y ago
      Its like feeding a baby polar bear. When it grows up it is going to eat you.
      - ashwagary 4y ago
        Extending this analogy into a question: Could the polar bear community eat enough expert food providers that their quality of food and eventually overall health declines?

kevinwang 4y ago

Eh, it's still impressive that these systems can write such good code despite pretty much just predicting the next word. I guess it's a matter of perspective. You can either be astounded by how much it can do relative to your expectations from 2018 or you can be skeptical relative to the volume of excitement. I think some excitement is also people extrapolating to the future: if predicting the next word gets you this far, what happens when you actually try to make it good?

copperx 4y ago
> despite pretty much just predicting the next word Is GPT really like the old school NLP trigram stuff?
- hnfong 4y ago
  "predicting the next word" sounds trivial until you realize the value and complexity of "predicting the next word[s] by a world leading expert in a particular domain".
  - Jensson 4y ago
    Quoting famous people sounds smart until you realize they just memorized a ton of trivia. These models have demonstrated that they don't learn logical models, instead they learn to generate text that looks logical at first glance but is nonsense.
    - sgarland 4y ago
      I asked GPT-4 something fairly niche that I happen to know a fair amount about: to explain the concept of Xenon poisoning in a nuclear reactor. Other than skipping Te-135 being the initial fission product that starts the decay chain (and tbf, operationally it can be skipped since the half-life is 19 seconds), it got everything correct. I'm sure if I kept probing on smaller and smaller details it would eventually fail, but I'd argue for _most_ people, on _most_ subjects, it performs incredibly well.

pewpew2020 4y ago

I feel it's the average programmer who is enamoured with GPT. The programmers working on hard stuff know that AI writing programs is still far away.

MrDresden 4y ago
It is not just those who are working on hard problems who are of that opinion. Good luck having a ML model understand a 20 year old undocumented dataformat developed inhouse at a specific research lab to be used in their proprietary systems which are also undocumented and are a part of a spiderweb of interconnected systems at that lab (I have worked at this particular lab). It will be a long time (if ever) until a ML model will be able to handle these situations (and I hazard a guess that most of the worlds active code is something akin to this). As a supporting tool for the software engineers working there, sure. Just like a linter.
- sgarland 4y ago
  Have you tried giving it a skeleton example of said undocumented data format? I've created some arbitrary ones and asked it to parse them, and it succeeded.
- hackinthebochs 4y ago
  >It will be a long time (if ever) until a ML model will be able to handle these situations (and I hazard a guess that most of the worlds active code is something akin to this). ChatGPT can do that right now. Just provide it example data in the input and it can extrapolate the format and interact with this abstraction (i.e. describe the format, write code to it, etc). LLMs don't just remix data it has seen before, they perform in-context learning. This means abstracting out patterns in the input and then leveraging it in generating output.
anonzzzies 4y ago
It’s also programmers who work with average and bad programmers and finally have a way to not waste time with them anymore. I work on the hard stuff, gpt generate the rest (I don’t have to waste time and energy to try to pull dead horses) and my colleagues go do something else. It’s a reality today and will improve further. Sure it cannot do what I do, but that might improve; gpt4 is noticeably better than 3.5 at slightly harder things, especially if you work properly iterative with it and having it fix it’s own errors. But no, it cannot do hard things it never saw; however it can give me all the boilerplate around the hard things so I only have to do the hard things. My days are simply 3-5 in 1 now, as I also don’t have the overhead of explaining jira tasks to others who are never going to really grasp what they are doing anyway. So I am not average and I am enamoured with gpt, simply because it presents high value to me now, more than some actual real humans. For me that’s enough revolutionary.

maxdoop 4y ago

I don’t think you’ve tried GPT-4. If you have, I don’t think you are like majority of devs (maybe not on HN, but in real life). You sound lucky to have true, novel problems to solve each day. I’m with many here commenting that this is quite powerful stuff, especially when my day-to-day is writing simple CRUD apps, or transforming data from one format to another within an API, or configuring some new bit of infra or CI/CD. I’d love to be challenged in some new way, and have access to truly fascinating problems that require novel solutions. But most enterprises aren’t really like that nor do that need that from majority of engineers.

whateveracct 4y ago
There's not such thing as actually boring CRUD. I've worked at many companies and founded my own. Even when it felt like CRUD, a year+ in it was clear that tasteful decisions pay off and iteration and cost gradients matter. GPT doesn't sniff that. I agree with GP - day 2 dev me outclasses it, which means it isn't replacing anyone.
- anonzzzies 4y ago
  HN echo chamber. Generally the better/best programmers hang around here; your day 2 was probably better than many coders hope to achieve in their whole lives. It is replacing people already; I personally know about 10 people who have been fired or assigned to a completely different position because gpt did a faster and better or equal job. So ‘not anyone’ is simply nonsense; I am only one person, there will be many many more who see the same thing. Of course they are not told they are replaced by ai when fired, but their managers or more skilled colleagues know. I do agree that there is no boring crud; that’s why gpt or no code tools are not good for full solutions (yet), but it’ll get there I am sure.
  - DaiPlusPlus 4y ago
    > I personally know about 10 people who have been fired or assigned to a completely different position because gpt did a faster and better or equal job Please elaborate. And, if true, this would be a major news story that Vox or any number of major newspapers would love to write about - so have you approached the media about this? If not, why not?

spaceman_2020 4y ago

I started learning how to code about 6 months ago, mostly to build prototypes of a couple of app ideas. I have no intention of getting a coding job - if the prototype is successful, I'll seek a technical co-founder. Last couple of months, I've been using chatGPT to write a lot of features and functions. I don't think it has made me a better coder, but it has made me massively more productive. Things like scraping data from a URL - something I would have had to sit through an hour long tutorial to learn - is accessible with a single query. I also think that the code quality has improved over the last few iterations. It makes fewer mistakes now.

davewritescode 4y ago
This is exactly what GPT is great for, accelerating learning.
- spaceman_2020 4y ago
  I predict there will be an explosion of productivity and small scale entrepreneurship. These tools are going to give so many smart but technically unskilled people a path towards realizing their ideas and vision. Exciting times

qikInNdOutReply 4y ago

The reason is easy to imagine. Most non-programmers, are living like analphabets in a world were reading is valuable super power. They grudgingly accept this power assemtry, but ocassionally rebel - with "easier" visual programming languages made and excel. This is another one of those rebellions, non-programers hoping to avoid reading the book and closing it for good, while keeping the awesome around. The code-bases we will see, were the commits are basically chatgpt tags and tasks for each document.

Nevermark 4y ago

> I really am in awe of how much work people seem willing to do to justify this as revolutionary and programmers as infantile [...] Well it is revolutionary. And it isn't just where it is today, but how fast these models are improving - with no saturation in ability evident at this time. On the other hand, I am not sure anyone is saying programmers are infantile - although poorly written software is as at least as prevalent as poorly compensated or managed software development positions.

yucky 4y ago

> I really am in awe of how much work people seem willing to do to justify this as revolutionary and programmers as infantile, and also why they do that. It’s fascinating. Equally fascinating is all of the "this is fine" posts from programmers suddenly realizing they are not the gods they once thought. But fret not, programming is not the first industry that has been automated into a shell of itself. Yes, the industry is going to shrink massively, but this is what new skills are for. Just as farmers had to learn industrial jobs and then miners and autoworkers had to "learn to code", most programmers will have to learn to do something else. Humans are resilient and will adapt. And there will still be jobs in development for the most talented and in niche areas, but when the largest tech companies can layoff hundreds of thousands of employees without skipping a beat that should tell you all you need to know about the value of most "programming" jobs.

dmm 4y ago
> Just as farmers had to learn industrial jobs and then miners and autoworkers had to "learn to code" The transition from agrarian to industrial societies was extremely painful and arguably it was centuries before the people affected were better off. > Humans are resilient and will adapt. Based on recent events I think it's more likely people will elect extremist politicians who promise quick fixes while blaming a convenient out-group for everything.
lamp987 4y ago
"Yes, the industry is going to shrink massively" You meant "expand massively" i think. Did all the programmers manually making machine code get fired and the job of programmer disappear when compilers were invented and totally replaced these jobs? No, it just changed to use the new tool. There wont be any unprecedented mass layoffs, despite what the jaded STEM-hating twitter crowd wants. Companies will simply make their programmers use these tools to increase the amount of produce per employee, and make software that would have been otherwise financially impossible to be made. Because the competition will do so too to get ahead.
- yucky 4y ago
  > You meant "expand massively" i think. Nope, I meant what I said. > There wont be any unprecedented mass layoffs Baghdad Bob strikes again.

SuoDuanDao 4y ago

I think we're seeing the early phases a prediction I made in my first book come true: That computers will be more suitable than humans for most of the functions humans currently use their left brain half for. Best case, that will have a whole lot more humans using their right brain halves on things like defining the problem. I like the thought of that, it's more pleasant work. But a lot of intelligent people define their intelligence by how well their left brain half works and uncomfortable with how good Chatgpt is at those tasks. I think you're pointing out there's more to programming than left-brain activities, and I think you're right that silicon will never eclipse carbon at those challenges, but I think a lot of people are also feeling threatened by the fact that chatgpt is getting better and better at the thing they used to be better than all humans at.

davemp 4y ago
> But a lot of intelligent people define their intelligence by how well their left brain half works and uncomfortable with how good Chatgpt is at those tasks. I think 'left brain' work also has a lot more predictability (knowing you can churn out ~X widgets/hr) so having only uncertain 'right brain' work can be uncomfortable for people to build their livelihoods upon. That being said. 'right brain' work is certainly more fulfilling for me.
- danuker 4y ago
  > can be uncomfortable for people to build their livelihoods upon Absolutely. Not everyone can be consistently creative, and without a large enough financial buffer, it's a recipe for disaster.
- SuoDuanDao 4y ago
  Great point. That's one reason I'm increasingly a single-issue voter focused on UBI.
Quarrelsome 4y ago
I feel like there's a slight inaccuracy here that this article covers. GPT-like models excel at logical problems that have already been solved. As the article suggests the models (at least at the moment) are extremely keen to utilise existing solutions as opposed to inventing new ones. The existing solutions end up somewhat being far too distracting when problem complexity compounds beyond the common. This implies that those truly talented at solving logical problems will still be spearheading the development of novel solutions. We might be able to state that GPT will easily trim away all the average workloads for both the left and right. It can perform "creative writing" or draw pictures to an average or even beyond average extent, but it continues to currently struggle to hit the exceptional examples that humanity are capable of.

munchbunny 4y ago

These statements can definitely be simultaneously true: * ChatGPT is revolutionary - honestly, it's genuinely impressive how much of a leap ChatGPT is compared to the attempts that came before it. * Programmers write a lot of simple code that has been written before - there are genuinely tons of cases of "write a web endpoint that takes an ID, looks it up in a database table, pulls an object through an ORM, and returns a JSON serialization of it." Most programmers? Doubt it. But tons of programmers write CRUD stuff and tons of IT admins do light scripting, and a lot of it is repeated code. Could ChatGPT do my job? Not even close. But it's still really impressive to me.

Jeff_Brown 4y ago
I feel safe too, and I'm amused at the dynamic. GPT could do a lot of the things I do, but it would take someone who knows what I know in order to explain the task in sufficient detail for it to do that.

staminade 4y ago

While I think there's truth to what you say, I'd also point our that workers in many pre-automated industries with an "artisan" approach also considered themselves irreplaceable because they figured, correctly, that nobody could build a machine with the capability of reproducing their workflow, with all its inherent uncertainty, flexibility and diverse physical and mental skills. What they failed to predict was that some people wouldn't try to automate them like-for-like. Instead they would reconfigure their entire approach to fit with the specific advantages and limitations of the machinery. And this new approach might even be qualitatively worse in various ways, but not so much as to overwhelm the economic advantages that provided by the things machines were good at. AI likely isn't going to slot into a developer-shaped hole in a software team. But it's possible we'll see new organisation approaches, companies, and development paradigms that say: How far can you get if you put prompt-generated code at the heart of the workflow and make everything else subservient to it. I'm not sure, right now, that that approach is feasible, but I'm not sure it won't be in a year or two.

Quarrelsome 4y ago
That's an extremely interesting thought. Perhaps we will see organisations in the future structure themselves more like a suite of unit tests. Instead of getting a developer or software house to plug a specific need in their org and being entirely outside of the development process: they will reflect the development process organisationally to ensure they catch any problems with the current output and just feed the box new prompts to increase their efficiency or efficacy. Their competitive advantage in their field then becomes the range of their tests (borne through experience), efficiency in running their pipeline of testing and ability to generate effective prompts.
- yc-kraln 4y ago
  This is already how automotive companies work. They are huge organizations which do four things: marketing, financing, design, and requirements. The supplier management and project management all fall under requirements management and enforcement.
- jackcviers3 4y ago

lionkor 4y ago

For me, its mostly that I have used GPT-3.5 a little for programming C++, and I wasnt impressed. For one, it made horrible, glaring mistakes (like defining extern functions which dont exist, using functions which are specific to a platform im not using, etc.), stuff beginners would do. It also decided to sneak in little issues, such as off-by-one errors (calling write() with a buffer and a size that is off by one in a place where its very hard to tell), missing edge cases (such as writing a C++ concept which worked, but actually did everything in slightly the wrong way to actually ensure the concept was requiring exactly what I asked). Even when asked to correct these mistakes, it often struggled, made me read paragraph after paragraph of "im sorry, ive been such a bad little machine" garbage, and didnt even correct the issue (or, in some cases, introduced new bugs). Im utterly unimpressed by this. GPT is great for a lot of things, but not writing code better than I would, in the same time. The time it took me to massage it to solve a nontrivial problem (write hello world with just syscalls) was way longer than reading the manual and writing it myself (and has less bugs). Not everyone unfazed by these articles is simply in denial. I feel sorry for people who write copy paste code and find that ChatGPT or Clippy from 2000 can relace them, but not everyone writes trivial code.

woeirua 4y ago
Two points: GPT4 is significantly better in this regard, and you should be concerned about the rate of progress more than it’s actual capabilities today.
- Jeff_Brown 4y ago
  > you should be concerned about the rate of progress Kind of agree? On the one hand we don't even have a roadmap toward reliable AI. On the other, if we ever plug an LLM into something that has memory, acquires experiences, does experiments, observes the outcome and adjusts its worldview in response, consciousness might fall out of that. And writing good code might not even require consciousness.
  - flatline 4y ago
    Epistemologically speaking, I think we can roughly break down the potential nature of consciousness into three categories: - as a function of an independent human soul - as the fundamental substrate on which the rest of the universe is built - as a byproduct/secondary phenomenon of physical processes In the latter two cases I believe that the question of whether GPT is conscious is immaterial. In either case it is functioning in the same medium we all are when we talk, think, write. In the first case it is not, and the question is thornier.

yodsanklai 4y ago

The problem is that many people here have an extreme point of view. It's either "this is going to make all develops jobless" or "it's useless, my job isn't that". I think It'll help with some tasks, which is always good to take. After all, people tweak their vim settings because they feel it makes them more productive.

Jeff_Brown 4y ago
Software development seems safe for the time being, but as someone who has both paid professional translators and used ChatGPT for translation, I'm certain GPT is obliterating some jobs.
travisjungroth 4y ago
This tech is so powerful, cutting so close to the last of human output that’s uncopied, advancing so fast, I have an incredibly hard time imagining it ending in a space that deserves a muted a reaction. It seems like it will either dramatically change society or fail. And I have quite a hard time imagining failure. It seems on par with the Industrial Revolution, at least. Which, easy to forget, was a huge deal that totally changed society.

siftrics 4y ago

cope

janeway 4y ago

I think this clip section says a lot - if videos are permitted as replies. https://youtube.com/watch?v=MNtjSMvcjzs&t=211s https://youtube.com/watch?v=MNtjSMvcjzs&t=211s

DuckFeathers 4y ago

It's a similar waste of my time to respond to comments like this... but I will at least try to give some pointers so that those who encounter this comment won't immediately fall into that line of thinking: - a lot of programmers, including experienced ones, are absolutely infantile and they only have a job because there is a big shortage of programmers; not all of them get better with experience... hence a significant part of software development is dealing with problematic programmers and problems created by them. - GPT is not that great a programmer but a great thing about it is that it is not a human... and one can get thousands of instances of them for the price of one human. You only need one of those instances to produce usable code. - there have been many changes throughout the years which have definitely replaced a lot of programmers: library distribution services (pypi, npmjs), better software development tools and practices, SaaS delivery model, better programming languages etc.; so far, because the market need for programmers has continued to increase, most programmers continue to have jobs; this won't last forever.