7 ms·
This article is entirely built on 2 big and wrong assumptions: 1. AI code ability will be the same as is today 2. Companies will replace people for AI en mass
by nerder92 2y ago
This article is entirely built on 2 big and wrong assumptions:
1. AI code ability will be the same as is today
2. Companies will replace people for AI en masse at a given moment in time
Of course both these assumptions are wrong, the quality of code produced by AI will improve dramatically as model evolves. And is not even just the model itself. The tooling, the Agentic capabilities and workflow will entirely change to adapt to this. (Already doing)
The second assumption is also wrong, intelligent companies will not layoff en masse to use AI only, they will most likely slow hiring devs because their existing enhanced devs using AI will suffice enough to their coding related needs. At the end of the day product is just one area of company development, build the complete e2e ultimate solution with 0 distribution or marketing will not help.
This article, in my opinion, is just doomerism storytelling for nostalgic programmers, that see programming only as some kind of magical artistic craft and AI as the villain arrived to remove all the fun from it. You can still switch off Cursor and write donut.c if you enjoy doing it.
- Madmallard 2y agoWhat makes you think (1) will be true? It is only generating based on training data. In mature code bases there is a massive amount of interconnected state that is not already present in any github repository. The new logic you'd want to add is likely something never done before. As other programmers have stated, it seems to be improving at generating useful boilerplate and making simple websites and such related to what's out there en masse on Github. But it can't make any meaningful changes in an extensively matured codebase. Even Claude Sonnet is absolutely hopeless at this. And the requirement before the codebase is "matured" is not very high.
- nerder92 2y ago> It is only generating based on training data This is not the case anymore, current SOTA CoT models are not just parroting stuff from the training data. And as of today they are not even trained exclusively on publicly (and not so publicly) available stuff, but they massively use synthetic data which the model itself generated or distilled data from other smarter models. I'm using and I know plenty of people using AI in current "mature" codebases with great results, this doesn't mean it does the work while you sip a coffee (yet) *NOTE: my evidence for this is that o3 could not break ARC AGI by parroting, because it's a banchmark made exactly for this reason. Not a coding banchmark per se, but still transposable imo.
- fragmede 2y agoTry Devin or OpenHands. OpenHands isn't quite ready for production, but it's informative on where things are going and to watch the LLM go off and "do stuff", kinda on its own, from my prompt (while I drink coffee).
- ryandrake 2y ago> The new logic you'd want to add is likely something never done before. 99% of software development jobs are not as groundbreaking as this. It’s mostly companies doing exactly what their competitors are doing. Very few places are actually doing things that an LLM model has truly never seen crawling through GutHub. Even new innovative products generally boil down to the same database fetches and CRUD glue and JSON parsing and front end form filling code.
- SpicyLemonZest 2y agoGroundbreakingness is different from the type of novelty that's relevant to an LLM. The script I was trying to write yesterday wasn't groundbreaking at all: it just needed to pull some code from a remote repository, edit a specific file to add a hash, then run a command. But it had to do that _within our custom build system_, and there's few examples of that, so our coding assistant couldn't figure out how to do it.
- skydhash 2y ago> Even new innovative products generally boil down to the same database fetches and CRUD glue and JSON parsing and front end form filling code. The simplest version of that is some CGI code a PHP script. Which everyone should be writing according to your description. But why so many books have been written to be able to do this seemingly simple task? So many frameworks, so many patterns, so many methodologies....
- Madmallard 2y agoI don't know man It can't do anything in these random Phaser games I'm making and even translating my 10,000 line XNA game to Phaser. It is totally hopeless. Phaser has been out forever now, and XNA used to be too.
- causal 2y agoNot to mention I haven't really seen AI replace anyone, except perhaps as a scapegoat for execs who were planning on layoffs anyway. That said, I do think there is real risk of letting AI hinder the growth of Junior dev talent.
- sanderjd 2y agoI think I've seen us be able to do more with fewer people than in the past. But that isn't the limiting factor for our hiring. All else equal, we'd like to just do more, when we can afford to hire the people. There isn't a fixed amount of work to be done. We have lots of ideas for products and services to make if we have the capacity.
- causal 2y agoAgreed, I often see AI discussed as if most companies wouldn't take 10x more developers if they could have them for free
- badgersnake 2y ago> the quality of code produced by AI will improve dramatically as model evolves This is the incorrect assumption, or at least there’s no evidence to support it.
- nerder92 2y agoIf benchmark means anything in evaluating how model capability progress, the evidence is that all the existing benchmark have been pretty much solved, except FrontierMath (https://epoch.ai/frontiermath https://epoch.ai/frontiermath)
- shihab 2y agoI recently saw Sam Altman bragging about OpenAI's performance on Codeforces (leetcode-like website), which I consider just about the worst benchmark possible. 1. All problems are small- the prompt and solution (<100 LOC, often <60LOC) 2. Solving those problems is more about recollecting patterns and less about good new insights. Now, top level human competitors do need original thinking, but that's only because our memory is too small to store all previously seen patterns. 3. Unusually good dataset- you have tens of thousands of problems, each with thousands of submissions, along with clear signals to train on (right/wrong, time taken etc), a very rich discussion sections etc. I think becoming 100th best Codeforces programmer is still an incredible achievement for a LLM. But for Sam Altman to specifically note the performance on this- I consider that a sign of weakness, not strength.
- badgersnake 2y agoAltman spouts even more bullshit than his models, if that’s even possible.
- layer8 2y agoGiven that I’ve seen excellent mathematicians produce poor-quality code in real-world software projects, I’m not sure how relevant these benchmarks are.
- mrguyorama 2y ago
- high_na_euv 2y ago>Of course both these assumptions are wrong, the quality of code produced by AI will improve dramatically as model evolves. How are you so sure?
- dev1ycan 2y agohe's not, just another delusional venture capitalist that hasn't bothered to look up the counter arguments to his point of view, done by mathematicians
- randmeerkat 2y ago> he's not, just another delusional venture capitalist that hasn't bothered to look up the counter arguments to his point of view, done by mathematicians Don’t hate on it, just spin up some startup with “ai” and LLM hype. Juice that lemon.
- iamleppert 2y agoWhen life gives you delusional VC's that don't understand the basics of what they are investing in, make lemonade! Very, very expensive lemonade!
- switchbak 2y agoQueue the Juicero reference!
- johnnyanmac 2y agoSadly my goals are more more ephemeral than opening up a lemonade stand.
- twelve40 2y agoRaising on hype != success, most of these lemons will rot before they can even do a small secondary for themselves. Plenty of startups died when the last hype wave ended (which was easy zirp money, before ai). I don't trust anything without a growing revenue anymore, and even then...
- croes 2y agoThat the second assumnoption is wrong is based on > intelligent companies will not layoff en masse to use AI How many companies are intelligent given how many dumb decisions we see? If we assume enough not so intelligent companies then better AI code we lead to mass firing.
- croes 2y agoBTW your reasoning for 1 sound like previous reasoning for FSD. Assuming the same kind of growth in capabilities isn't backed by reality. The last release of OpenAI's model wasn't dramatically better. At the moment it's more about getting cheaper.
- blah2244 2y agoTo be fair, his argument is valid for FSD! We have fully deployed FSD in multiple US cities now!
- croes 2y agoJust because it’s deployed doesn’t mean it’s working. https://www.autoevolution.com/news/viral-tesla-cybertruck-crash-on-fsd-shows-uphill-battle-to-advance-autonomous-driving-246836.html https://www.autoevolution.com/news/viral-tesla-cybertruck-cr...
- wincy 2y agoReally? Because I’m using ChatGPT Deep Research and it feels revolutionary in the quality of the data it’s able to output.
- croes 2y agoDeep Research is only available for Pro subscribers, I know only the benefits of the Plus subscription models and it’s pretty underwhelming.
- deleted 2y ago[deleted]
- ssimpson 2y agoI tend to agree with you. The general pattern behind "x tool came along that made work easier" isn't to fire a bunch of folks, its to make the people that are there work whatever increment of ease of work more. ie, if the tool cuts work in half, you'd be expected to do 2x more work. Automation and tools almost never "makes our lives easier", it just removes some of the lower value added work. It would be nice to live better and work less, but our overlords won't let that happen. Same output with less work by the individual isn't as good as same or more output with the same or less people.
- fragmede 2y ago> but our overlords won't let that happen If you have a job, working for a boss, you're trading your time for money. If you're a contractor and negotiate being paid by the project, you're being paid for results. Trading your time for money is the underlying contract. That's the fundamental nature of a job working for somebody else. You can escape that rat race if you want to. Someone I know builds websites for clients on a contract basis, and did so without LLMs. Within his market, he knows what a $X,000 website build entails. His clients were paying that rate for a website build out prior to AI-augmented programming, and it would take a week to do that job. With help from LLMs, that same job now takes half as much time. So now he can choose to take on more clients and take home more pay, or not, and be able to take it easy. So that option is out there, if you can make that leap. (I haven't)
- johnnyanmac 2y ago>You can escape that rat race if you want to. I'm working on it. But it takes money and the overlords definitely are trying to squeeze as of late. And yes, while I don't think I'm being replaced in months or years, I can a possibility in a decade or two of the ladder being pulled up on most programming jobs. We'll either be treated as well as artists (assuming we still don't unionize) or we'll have to rely on our own abilities to generate value without corporate overlords.
- wincy 2y agoInteresting. So it sounds like I need to get into the market and charge half as much and steal all of his customers. These things eventually always end up as a Red Queen’s race, where you have to run as fast as you can to stay in the same place.
- jayd16 2y agoThese two predictions seem contradictory. If the AI massively improves why would they slow roll adoption?
- throwaway290 2y agoPeople want their AI stocks to go up. So they say things like sky is the limit and jobs are not going away (aka please don't regulate) in one sentence. I think only one of this is true.
- dkjaudyeqooe 2y ago> Of course both these assumptions are wrong, the quality of code produced by AI will improve dramatically as model evolves. This is the fundamental delusion that is driving AI hype. Although scale has made LLMs look like magic, actual magic (AGI) is not on the scaling path. This is a conjecture (as is the converse), but I'm betting the farm on it personally and see LLMs as useful chat bots that augment other, better technologies for automation. If you want to pursue AGI, move on quickly to something structurally and fundamentally better. People don't understand that AGI is pure speculation. There is no rigorous, non-circular definition of human intelligence, let alone proof that AGI is possible or achievable in any reasonable time frame (like 100 years).
- aiono 2y ago> the quality of code produced by AI will improve dramatically as model evolves. That's a very bold claim. We are already seeing plateu in LLM capabilities in general. And there is little improvement in places where they fall short (like making holistic changes in a large codebase) since their birth. They only improve where they are already good at such as writing small glue programs. Expecting significant breakthroughs with just scaling without any fundamentally changes to the architecture seems like too optimistic to me.
- y-c-o-m-b 2y ago> The second assumption is also wrong, intelligent companies will not layoff en masse to use AI only, they will most likely slow hiring devs because their existing enhanced devs using AI will suffice enough to their coding related needs After 20 years in tech, I can't think of a single company I've worked for/with that would fit the profile of an "intelligent" company. All of them make poor and irrational decisions regularly. I think you over-estimate the intelligence of leadership whilst simultaneously under-estimating their greed and eventual ability to self-destruct. EDIT: you also over-estimate the desire for developers to increase their productivity with AI. I use AI to reduce complexity and give me more breathing room, not to increase my output.
- mulmboy 2y ago> After 20 years in tech, I can't think of a single company I've worked for/with that would fit the profile of an "intelligent" company. All of them make poor and irrational decisions regularly. I think you over-estimate the intelligence of leadership whilst simultaneously under-estimating their greed and eventual ability to self-destruct. Says nothing about companies and everything about you > you also over-estimate the desire for developers to increase their productivity with AI. I use AI to reduce complexity and give me more breathing room, not to increase my output. I'm the same. But I expect that once many begin to do this, there will be some who do use it for productivity and they will set the bar. Then people like you and I will either use it for productivity or fall behind.
- johnnyanmac 2y agoI'm happy you've only worked for altruistic, not-for-profit minded companies that care about employee growth and takes pride in their tach stack above all else. I have not had as fortunate an experience. >I expect that once many begin to do this, there will be some who do use it for productivity and they will set the bar. Yeah, probably. I've had companies so pinpointed on "velocoity" instead of quality. I imagine they will definitely try to expect triple the velocity just because one person "gets so much done". Not realizing how much of that illusion is correcting the submissions.
- leptons 2y ago> their existing enhanced devs using AI will suffice enough to their coding related needs. Not my experience. I spend as much time reading through and replacing wrong AI generated code as I do writing my own code, so it's really wasting my time more often than helping. It's really hit or miss, and about the only thing the AI gets right most often is writing console.log statements based on the variable I've just assigned, and that isn't really "coding". And even then it gets it right only about 75% of the time. Sure, that saves me some time, but I'm not seeing the supposed acceleration AI is hyped as giving.
- bodegajed 2y ago> Quality of code produced by AI will improve dramatically as model evolves. Where are your facts? What is your basis on these future prediction? Sure small code snippets have already improved since gpt2. What about larger applications where layers and layers of abstractions coming from private company sensitive data? > At the end of the day product is just one area of company development, build the complete e2e ultimate solution with 0 distribution or marketing will not help. this sounds like PR garbage to me why use "e2e ultimate solution" provide technical details so we can verify what you're saying is true