28 ms·

There's such a huge disconnect between people reading headlines and developers who are actually trying to use AI day to day in good faith. We know what it is g

by dham 2y ago

There's such a huge disconnect between people reading headlines and developers who are actually trying to use AI day to day in good faith. We know what it is good at and what it's not. It's incredibly far away from doing any significant change in a mature codebase. In fact I've become so bearish on the technology trying to use it for this, I'm thinking there's going to have to be some other breakthrough or something other than LLM's. It just doesn't feel right around the corner. Now completing small chunks of mundane code, explaining code, doing very small mundane changes. Very good at.

hammock 2y ago
> It's incredibly far away from doing any significant change in a mature codebase The COBOL crisis at Y2K comes to mind.
- Cascais 2y ago
  Is this the same Cobol crisis we have now? https://www.computerweekly.com/news/366588232/Cobol-knowledge-crisis-threatens-Dutch-financial-systems https://www.computerweekly.com/news/366588232/Cobol-knowledg...
lawlessone 2y ago
Same, LLMs are interesting but on their own are a dead end. I think something needs to actually experience the world in 3d in real time to understand what it is actually coding things or doing tasks for.
- falcor84 2y ago
  > LLMs are interesting but on their own are a dead end. I don't think that anyone is advocating for LLMs to be used "on their own". Isn't it like saying that airplanes are useless "on their own" in 1910, before people had a chance to figure out proper runways and ATC towers?
  - dingnuts 2y ago
    there was that post about "vibe coding" here the other day if you want to see what the OP is talking about
    - falcor84 2y ago
      You mean Karpathy's post discussed on https://twitter.com/karpathy/status/1886192184808149383 https://twitter.com/karpathy/status/1886192184808149383 ? If so, I quite enjoyed that as a way of considering how LLM-driven exploratory coding has now become feasible. It's not quite there yet, but we're getting closer to a non-technical user being able to create a POC on their own, which would then be a much better point for them in engaging an engineer. And it will only get better from here.
      - sarchertech 2y ago
        Technology to allow business people to create POCs has been around for a long time.
        falcor84 2y ago
        All previous examples have been of the "no code" variety, where you press buttons and it controls presets that the creators of the authoring tool have prepared for you. This is the first time where you can talk to it and it writes arbitrary code for you. You can argue that it's not a good idea, but it is a novel development.
        sarchertech 2y ago
        A no code solution at its most basic level is nothing more or less than a compiler. You wouldn’t argue that writing in a high level language doesn’t let you produce arbitrary code because the compiler is just spitting out presets its author prepared for you. There are 2 main differences between using an LLM to build an app for you and using a no code solution with a visual language. 1. The source code is English (which is definitely more expressive). 2. The output isn’t deterministic (even with temperature set to 0 which is probably not what you want anyway) Both 1 and 2 are terrible ideas. I’m not sure which is worse.
- cratermoon 2y ago
  > actually experience the world in 3d in real time AKA embodiment. Hubert L. Dreyfus discussed this extensively in "Why Heideggerian AI Failed and How Fixing it Would Require Making it More Heideggerian": http://dx.doi.org/10.1080/09515080701239510 http://dx.doi.org/10.1080/09515080701239510
- data-ottawa 2y ago
  I don’t know that it needs to experience the world in real-time, but when the brain thinks about things it’s updating its own weights. I don’t think attention is a sufficient replacement for that mechanism. Reasoning LLMs feel like an attempt to stuff the context window with additional thoughts, which does influence the output, but is still a proxy for plasticity and aha-moments that can generate.
  - lawlessone 2y ago
    >I think this is true only if there is a novel solution that is in a drastically different direction than similar efforts that came before. That's good point, we don't do that right now. it's all very crystalized.
pjmlp 2y ago
You missed the companies selling AI consulting projects, with the disconnect between sales team, customer, folks on the customer side, consultants doing the delivery, and what actually gets done.
lolinder 2y ago
Part of the problem is that many working developers are still in companies that don't allow experimentation with the bleeding edge of AI on their code base, so their experiences come from headlines and from playing around on personal projects. And on the first 10,000 lines of code, the best in class tools are actually pretty good. Since they can help define the structure of the code, it ends up shaped in a way that works well for the models, and it still basically all fits in the useful context window. What developers who can't use it on large warty codebases don't see is how poorly even the best tools do on the kinds of projects that software engineers typically work on for pay. So they're faced with headlines that oversell AI capabilities and positive experiences with their own small projects and they buy the hype.
- throwaway0123_5 2y ago
  Some codebases grown with AI assistance must be getting pretty large now, I think an interesting metric to track would be percent of code that is AI generated over time. Still isn't a perfect proxy for how much work the AI is replacing though, because of course it isn't the case that all lines of code would take the same amount of time to write by hand.
  - lolinder 2y ago
    Yeah, that would be very helpful to track. Anecdotally, I have found in my own projects that the larger they get the less I can lean on agent/chat models to generate new code that works (without needing enough tweaks that I may as well have just written it myself). Having been written with models does seem to help, but it doesn't get over the problem that eventually you run out of useful context window. What I have seen is that autocomplete scales fine (and Cursor's autocomplete is amazing), but autocomplete supplements a software engineer, it doesn't replace them. So right now I can see a world where one engineer can do a lot more than before, but it's not clear that that will actually reduce engineering jobs in the long term as opposed to just creating a teller effect.
    - ryandrake 2y ago
      It might not just be helpful but required one day. Depending on how the legality around AI-generated code plays out, it's not out of the question that companies using it will have to keep track of and check the provenance and history of their code, like many companies already do for any open source code that may leak into their project. My company has an "open source review" process to help ensure that developers aren't copy-pasting GPL'ed code or including copyleft libraries into our non-GPL licensed products. Perhaps one day it will be common to do an "AI audit" to ensure all code written complied with whatever the future regulatory landscape shapes up to be.
pgm8705 2y ago
Yes. I think part of the problem is how good it is at starting from a blank slate and putting together an MVP type app. As a developer, I have been thoroughly impressed by this. Then non-devs see this and must think software engineers are doomed. What they don't see is how terrible LLMs are at working with complex, mature codebases and the hallucinations and endless feedback loops that go with that.
- idle_zealot 2y ago
  The tech to quickly spin up MVP apps has been around for a while now. It gets you from a troubling blank slate to something with structure, something you can shape and build on. I am of course talking about npx create-{template name} Or your language of choice's equivalent (or git clone template-repo).
  - tiborsaas 2y ago
    Yes, but the LLM driven MVP-s are not only builerplates but actual functioning apps. The "create-" is somewhat good, but it's usually throwaway code and do it properly later. While my LLM made boilerplate is the actual first few steps to get the boring parts done. It also needs refactoring and polishing, but it's an order of magnitude better than the "MVP helper tooling" before.
hnthrow90348765 2y ago
> Now completing small chunks of mundane code, explaining code, doing very small mundane changes. Very good at. This is the only current threat. The time you save as a developer using AI on mundane stuff will get filled by something else, possibly more mundane stuff. A small company with only 2-5 Seniors may not be able to drop anyone. A company with 100 seniors might be able to drop 5-10 of them total, spread across each team. The first cuts will come at scaled companies. However, it's difficult to detect if companies are cutting people just to save money or if they are actually realizing any productivity gains from AI at this point.
- renegade-otter 2y ago
  Especially since the zero-interest bonanza led to over-hiring of resume-driven developers. Half of AWS is torching energy by runnning some bloat that should not even be there.
- sumoboy 2y ago
  I don't think companies realize AI is not free. A 100+ devs, openai, anthropic, gemini API costs, the hidden overhead of costs not spoken about. Too much speculation that productivity will increase substantially, especially when a majority of companies IT is just so broken and archaic.
csmpltn 2y ago
I think that LLMs are only going to make people with real tech/programming skills much more in demand, as younger programmers skip straight into prompt engineering and never develop themselves technically beyond the bare minimum needed to glue things together. The gap between people with deep, hands-on experience that understand how a computer works and prompt engineers will become so insanely deep. Somebody needs to write that operating system the LLM runs on. Or your bank's backend system that securely stores your money. Or the mission critical systems powering this airplane you're flying next week... to pretend like this will all be handled by LLMs is so insanely out of touch with reality.
- whynotminot 2y ago
  Isn’t this kind of thing the story of tech though? Languages like Python and Java come around, and old-school C engineers grouse that the kids these days don’t really understand how things work, because they’re not managing memory. Modern web-dev comes around and now the old Java hands are annoyed that these new kids are just slamming NPM packages together and polyfills everywhere and no one understands Real Software Design. I actually sort of agree with the old C hands to some extent. I think people don’t understand how a lot of things actually work. And it also doesn’t really seem to matter 95% of the time.
  - deleted 2y ago
    [deleted]
  - chucky_z 2y ago
    $1 for the pencil, $1000 for the line. That’s the 5% when it does matter.
    - whynotminot 2y ago
      Yes this is what people like to think. It’s not really true in practice.
  - shafyy 2y ago
    Just because there are these abstractions layers that happened in the past does not mean that it will continue to happen that way. For example, many no-code tools promised just that, but they never caught on. I believe that there's a "optimal" level of abstraction, which, for the web, seems to be something like the modern web stack of HTML, JavaScript and some server-side language like Python, Ruby, Java, JavaScript. Now, there might be tools that make a developer's life easier, like a nice IDE, debugging tools, linters, autocomplete and also LLMs to a certain degree (which, for me, still is a fancy autocomplete), but they are not abstraction layers in that sense.
hhh1111 2y ago
[dead]
renegade-otter 2y ago
AI will create more jobs, if anything, as the "engineers" out of their depth create massive unmaintainable legacy.
- OccamsMirror 2y ago
  It's Access databases all over again.
DanielHB 2y ago
The only thing I use it for is for small self-contained snippets of code on problems that require use of APIs I don't quite remember out of the top of my head. The LLM spits out the calls I need to make or attributes/config I need to set and I go check the docs to confirm. Like "How to truncate text with CSS alone" or "How to set an AWS EC2 instance RAM to 2GB using terraform"
bwfan123 2y ago
A "causal model" is needed to fix bugs ie, to "root-cause" a bug. LLMs yet dont have the idea of a causal-model of how something works built-in. What they do have is pattern matching from a large index and generation of plausible answers from that index. (aside: the plausible snippets are of questionable licensing lineage as the indexes could contain public code with restrictive licensing) Causal models require machinery which is symbolic, which is able to generate hypotheses and test and prove statements about a world. LLMs are not yet capable of this and the fundamental architecture of the llm machine is not built for it. Hence, while they are a great productivity boost as a semantic search engine, and a plausible snippet generator, they are not capable of building (or fixing bugs in) a machine which requires causal modeling.
- fiso64 2y ago
  >Causal models require machinery which is symbolic, which is able to generate hypotheses and test and prove statements about a world. LLMs are not yet capable of this and the fundamental architecture of the llm machine is not built for it. Prove that the human brain does symbolic computation.
  - bwfan123 2y ago
    We dont know what the human brain does, but we know it can produce symbolic theories or models of abstract worlds (in the case of math) or real worlds (in the case of science). It can also produce the "symbolic" turing machine which serves as an abstraction for all computation we use (cpu/gpu/etc)
tarkin2 2y ago
Sorry for inadvertently advising but I met a guy who used v0.dev to make impressive websites (although admittedly he did use react before so he was experienced) with professional success. It's more than arguable that his company will fire/hire fewer devs. Of course in a decade or so they'll be a skill gap unless LLMs can fill that gap too.
anavat 2y ago
My take is that AI's ability to generate new code will prove so valuable, it will not matter if it is bad at changing existing code. And that the engineers of the distant future (like, two years from now) will not bother to read the generated code, as long as it runs and passes the tests (which will also be AI-generated). I try to use AI daily, and every month I see how it is able to generate larger and more complex chunks of code from the first shot. It is almost there. We just need to adopt the new paradigm, build the tooling, and embrace the new weird future of software development.
- bdhcuidbebe 2y ago
  > I try to use AI daily You should reflect on the consequences of relying too much on it. See https://www.404media.co/microsoft-study-finds-ai-makes-human-cognition-atrophied-and-unprepared-3/ https://www.404media.co/microsoft-study-finds-ai-makes-human...
  - anavat 2y ago
    I don't buy it makes me dumber. It just makes me worse at some things I used to do before, while making better at some other things. Often times it doesn't feel like coding anymore, more like if I were training to be a lawyer or something. But that's my bet.
weatherlite 2y ago
I agree but too many serious people are hinting we are very close I can't ignore it anymore. Sure, when Sam Altman / Zuckerberg say we're close I don't know if I can believe him because obviously the dudes will say anything to sell/pump the stock price. But how about Demis Hassabis ? He doesn't strike me like that at all. Same for Geoff Hinton, Bengio and a couple of others.
- bdhcuidbebe 2y ago
  Market hype is all it is.
- layer8 2y ago
  People investing their lives in the field are inherently biased. This is not to diminish them, it’s just a fact of the matter. Nobody knows how general intelligence really works, nor even how to reliably test for it, so it’s all speculation.
RivieraKid 2y ago
I'm surprised to see a huge disconnect between how I perceive things and the vast majority of comments here. AI is obviously not good enough to replace programmers today. But I'm worried that it will get much better at real-world programming tasks within years or months. If you follow AI closely, how can you be dismissive of this threat? OpenAI will probably release a reasoning-based software engineering agent this year. We have a system that is similar to top humans at competitive programming. This wasn't true 1 year ago. Who knows what will happen in 1 year.
- layer8 2y ago
  When I see stuff like https://news.ycombinator.com/item?id=42994610 https://news.ycombinator.com/item?id=42994610 (continued in https://news.ycombinator.com/item?id=42996895 https://news.ycombinator.com/item?id=42996895), I think the field still has fundamental hurdles to overcome.
  - lordswork 2y ago
    This kind of error doesn't really matter in programming where the output can be verified with a feedback loop.
    - layer8 2y ago
      This is not about the numerical result, but about the way it reasons. Testing is a sanity check, not a substitute for reasoning about program correctness.
  - tmnvdb 2y ago
    Why do you think this is a fundamental hurdle, rather than just one more problem that can be solved? I dont have strong evidence either way, but I've seen a lot of 'fundamental unsurmountable problems' fall by the wayside over the past few years. So I'm not sure we can be that confident that a problem like this, for which we have very good classic algorithms, is a fundamental issue.
- Imanari 2y ago
ge96 2y ago
A friend of mine reached out with some code ChatGPT wrote for him to trade crypto. It had so much random crap in it and lines would say "AI enhanced trading algo" and it was just an np.randomint line. It was pulling in random deps not even used. I get it though like I'm terrible working with IMUs and I want to just get something going but I can't there's that wall I need to overcome/learn eg. the math behind it. Same with programming helps to have the background knowing how to read code and how it works.
- HDThoreaun 2y ago
  I used claude to help write a crypto trading bot. It helped me push out thousands of lines a day. What wouldve taken months took a couple weeks. Obviously you still need experienced pilots but unless we find an absolute fuckload of new work to do(not unlikely looking at history) its hard for me to see anything other than way less developers being needed.
jillesvangurp 2y ago
I think of it as an enabler that reduces my dependency on junior developers. Instead of delegating simple stuff to them, I now do it myself with about the same amount of overhead (have to explain what I want, have to triple check the results) on my side but less time wasted on their end. A lot of micro managing is involved either way. And most LLMs suffer from a severe case of ground hog day. You can't assume them to remember anything over time. Every conversation starts from scratch. If it's not in your recent context, specify it again. Etc. Quite tedious but it still beats me doing it manually. For some things. For at least the next few years, it's going to be an expectation from customers that you will not waste their time with stuff they could have just asked an LLM to do for them. I've had two instances of non technical CPO and CEO types recently figuring out how to get a few simple projects done with LLMs. One actually is tackling rust programs now. The point here is not that that's good code but that neither of them would have dreamed about doing anything themselves a few years ago. The scope of the stuff you can get done quickly is increasing. LLMs are worse at modifying existing code than they are at creating new code. Every conversation is a new conversation. Ground hog day, every day. Modifying something with a lot of history and context requires larger context windows and tools to fill those. The tools are increasingly becoming the bottleneck. Because without context the whole thing derails and micromanaging a lot of context is a chore. And a big factor here is that huge context windows are costly so there's an incentive for service providers to cut some corners there. Most value for me these days come from LLM tool improvements that result in me having to type less. "fix this" now means "fix the thing under my cursor in my open editor, with the full context of that file". I do this a lot since a few weeks.
belter 2y ago
> Now completing small chunks of mundane code, explaining code, doing very small mundane changes. Very good at. I would not trust them until they can do the news properly. Just read the source Luke. "AI chatbots unable to accurately summarise news, BBC finds" - https://www.bbc.com/news/articles/c0m17d8827ko https://www.bbc.com/news/articles/c0m17d8827ko
strangescript 2y ago
As context sizes get larger (and remain accurate within the size) and speeds increase, especially inference, it will start solving these large complex code bases. I think people lose sight of how much better it has gotten in just a few years.
hintymad 2y ago
> It's incredibly far away from doing any significant change in a mature codebase A lot of the use cases are on building something that has already been built before, like a web app, a popular algorithm, and etc. I think the real threat to us programmers is stagnation. If we don't have new use cases to develop but only introduce marginal changes, then we can surely use AI to generate our code from the vast amount of previous work.
giancarlostoro 2y ago
They all sounds like crypto bros talking about AI. It's really frustrating to talk to them, just like crypto bros.
- moogly 2y ago
  They're the same people in my experience.
  - giancarlostoro 2y ago
    Its the same energy for sure.
deeviant 2y ago
The huge disconnect is that the skill set to use LLMs for code effectively is not the same skill set of standard software engineering. There is a very heavy intersection and I would say you cannot be effective at LLM software development without being an effective software engineer, but being an effective software engineer does not by any means make somebody good at LLM development. Very talented engineers, coworkers, that I would place above myself in skill, seemed stumped by it, while I have realized at least a 10x productively gain. The claim that LLMs are not being applied in mature, complex code-bases is pure fantasy, example: https://arxiv.org/abs/2501.06972 https://arxiv.org/abs/2501.06972. Here Google is using LLMs to accelerate the migration of mature, complex production systems.
agentultra 2y ago
I’m more keen on formal methods to do this than LLMs. I take the view that we need more precise languages that require us to write less code that obviously has no errors in it. LLMs are primed to generate more code using less precise specifications; resulting in code that has no obvious errors in it.
micromacrofoot 2y ago
I think you're discounting efficiency gains — through a series of individually minor breakthroughs in LLM tech I think we could end up with things like 100M+ token context windows We've already seen this sort of incrementalism over the past couple of years, the initial buzz started without much more than a 2048 context window and we're seeing models with 1M out there now that are significantly more capable.
__MatrixMan__ 2y ago
I think it's more likely that we'll see a rise in workflows that AI is good at, rather than AI rising to meet the challenges of our more complex workflows. Let the user pair with an AI to edit and hot-reload some subset of the code which needs to be very adapted to the problem domain, and have the AI fine-tuned for the task at hand. If that doesn't cut it, have the user submit issues if they need an engineer to alter the interface that they and the AI are using. I guess this would resemble how myspace used to do it, where you'd get a text box where you could provide custom edits, but you couldn't change the interface.
rs186 2y ago
I use AI coding assistants daily, and whenever there is a task that those tools cannot do correctly/quickly enough so that I need to fallback to editing things by myself, I spend a bit of time thinking what is so special about the tasks. My observation is that LLMs do repetitive, boring tasks really well, like boilerplate code and common logic/basic UI that thousands of people have already done. Well, in some sense, jobs where developers who spend a lot of time writing generic code is already at risk of being outsourced. The tasks that need a ton of tweaking or not worth asking AI at all are those that are very specific to a specific product and need to meet specific requirements that often come from discussions or meetings. Well, I guess in theory if we had transcripts for everything, AI could write code like the way you want, but I doubt that's happening any time soon. I have since become less worried about the pace AI will replace human programmers -- there is still a lot that these tools cannot do. But for sure people need to watch out and be aware of what's happening.
xp84 2y ago
> It's incredibly far away from doing any significant change in a mature codebase. I agree with this completely. However the problem that I think the article gets at is still real because junior engineers also can't do significant changes on a mature codebase when they first start out. They used to do the 'easy stuff' which freed the rest of us up to do bigger stuff. But: 1. Companies like mine don't hire juniors anymore 2. With Copilot I can be so much more productive that I don't need juniors to do "the easy stuff" because Copilot can easily do that in 1/1000th the time a junior would. 3. So now who is going to train those juniors to get to the level where we need them to be to make those "significant changes"?
- hypothesis 2y ago
  > So now who is going to train those juniors to get to the level where we need them to be to make those "significant changes"? Founders will cash out long before that becomes an issue. Alternatively, the hype is true and they will obsolete programmers, also solving the issue above… This is quite devious if you think about it, withering pipeline of new devs and only them having an immediate fix in all cases.
outworlder 2y ago
> I'm thinking there's going to have to be some other breakthrough or something other than LLM's. We actually _need_ a breakthrough for the promises to materialize, otherwise we will have yet another AI Winter. Even though there seems to be some emergent behavior (some evidence that LLMs can, for example, create an internal chess representation by themselves when asked to play), that's not enough. We'll end up with diminishing returns. Investors will get bored of waiting and this whole thing comes crashing down. We'll get an useful too in our toolbox, as we do at every AI cycle.
Aiguru31415666 2y ago
[dead]
darepublic 2y ago
I dunno if it's always good at explaining code. It tends to take everything at face value and is unable to opinionatedly reject bs when it's presented with it. Which in the majority of cases is bad.
- bodegajed 2y ago
  this is also my problem. When I ask someone a technical question, and I did not provide context on some abstractions. Usually this is common because abstractions can be very deep. "Hmm, not sure.. can you check what's this supposed to do?" LLMs don't do this, it confidently hallucinate the abstraction out of thin air or uses their outdated knowledge store. Sending wrong use or wrong input parameters.
scotty79 2y ago
> We know what it is good at and what it's not. We know what it's good at today. And pretty sure it won't be any worse at it in the future. And 5 years ago state of the art was basically output of Markov Chain. In 5 years we might be at another place entirely.
necovek 2y ago
Agreed, and I haven't yet seen any single instance of a company firing software engineers because AI is replacing them (even if by increasing productivity of another set of software engineers): I've asked this a number of times, and while it's a common refrain, I haven't really seen any concrete news report saying it in so many words. And to be honest, if any company is firing software engineers hoping AI replaces their production, that is good news since that company will soon stop existing and treating engineers like shit which it probably did :)
dartos 2y ago
Marketing is really good at their job. That coupled with new money and retail investors being thinking they’re in a gold rush and you get the environment we’re in.
ericmcer 2y ago
A breakthrough is exactly what everyone is banking on. OpenAI was surprised by GPT3, they were just dumping data into an LLM and ended up with something that was way better than they expected. Everyone is hoping (probably delusional) that bigger and more impressive breakthroughs will keep leaping up if we just keep tweaking the models and increasing the size of the data sets.
jmspring 2y ago
My favorite has been everyday people claiming Elmo will find all sorts of corruption on Fed databases using AI. Trained on what datasets? What biases? Etc.
ein0p 2y ago
+1. I've tried really hard to replace even some parts of my job with AI ever since GPT3 era, unsuccessfully. All it does for me is it allows me to enter unfamiliar domains (such as e.g. SwiftUI) but then I'm all on my own. In domains where I already have expertise it just doesn't work well. So it is a productivity booster, sure, but I don't see it replacing anyone doing non-bullshit work. I don't even see a trend line pointing in that direction.
z3n0n 2y ago
Been heavily testing Cursor, Windsurf and VSCode w/CP lately. Most low level stuff works surprisingly well. But for anything slightly more complex I just end up wasting 90% credits watching the AI chase its own tail.
chefandy 2y ago
Counterpoint: https://medium.com/@sweaty.phd/yet-another-ai-will-take-your-job-story-programmers-edition-146a97211734 https://medium.com/@sweaty.phd/yet-another-ai-will-take-your...

by dham 2y ago

hammock 2y ago

> It's incredibly far away from doing any significant change in a mature codebase The COBOL crisis at Y2K comes to mind.

Cascais 2y ago
Is this the same Cobol crisis we have now? https://www.computerweekly.com/news/366588232/Cobol-knowledge-crisis-threatens-Dutch-financial-systems https://www.computerweekly.com/news/366588232/Cobol-knowledg...

lawlessone 2y ago

Same, LLMs are interesting but on their own are a dead end. I think something needs to actually experience the world in 3d in real time to understand what it is actually coding things or doing tasks for.

falcor84 2y ago
> LLMs are interesting but on their own are a dead end. I don't think that anyone is advocating for LLMs to be used "on their own". Isn't it like saying that airplanes are useless "on their own" in 1910, before people had a chance to figure out proper runways and ATC towers?
- dingnuts 2y ago
  there was that post about "vibe coding" here the other day if you want to see what the OP is talking about
  - falcor84 2y ago
    You mean Karpathy's post discussed on https://twitter.com/karpathy/status/1886192184808149383 https://twitter.com/karpathy/status/1886192184808149383 ? If so, I quite enjoyed that as a way of considering how LLM-driven exploratory coding has now become feasible. It's not quite there yet, but we're getting closer to a non-technical user being able to create a POC on their own, which would then be a much better point for them in engaging an engineer. And it will only get better from here.
    - sarchertech 2y ago
      Technology to allow business people to create POCs has been around for a long time.
      - falcor84 2y ago
        All previous examples have been of the "no code" variety, where you press buttons and it controls presets that the creators of the authoring tool have prepared for you. This is the first time where you can talk to it and it writes arbitrary code for you. You can argue that it's not a good idea, but it is a novel development.
        sarchertech 2y ago
        A no code solution at its most basic level is nothing more or less than a compiler. You wouldn’t argue that writing in a high level language doesn’t let you produce arbitrary code because the compiler is just spitting out presets its author prepared for you. There are 2 main differences between using an LLM to build an app for you and using a no code solution with a visual language. 1. The source code is English (which is definitely more expressive). 2. The output isn’t deterministic (even with temperature set to 0 which is probably not what you want anyway) Both 1 and 2 are terrible ideas. I’m not sure which is worse.
cratermoon 2y ago
> actually experience the world in 3d in real time AKA embodiment. Hubert L. Dreyfus discussed this extensively in "Why Heideggerian AI Failed and How Fixing it Would Require Making it More Heideggerian": http://dx.doi.org/10.1080/09515080701239510 http://dx.doi.org/10.1080/09515080701239510
data-ottawa 2y ago
I don’t know that it needs to experience the world in real-time, but when the brain thinks about things it’s updating its own weights. I don’t think attention is a sufficient replacement for that mechanism. Reasoning LLMs feel like an attempt to stuff the context window with additional thoughts, which does influence the output, but is still a proxy for plasticity and aha-moments that can generate.
- lawlessone 2y ago
  >I think this is true only if there is a novel solution that is in a drastically different direction than similar efforts that came before. That's good point, we don't do that right now. it's all very crystalized.

pjmlp 2y ago

You missed the companies selling AI consulting projects, with the disconnect between sales team, customer, folks on the customer side, consultants doing the delivery, and what actually gets done.

lolinder 2y ago

Part of the problem is that many working developers are still in companies that don't allow experimentation with the bleeding edge of AI on their code base, so their experiences come from headlines and from playing around on personal projects. And on the first 10,000 lines of code, the best in class tools are actually pretty good. Since they can help define the structure of the code, it ends up shaped in a way that works well for the models, and it still basically all fits in the useful context window. What developers who can't use it on large warty codebases don't see is how poorly even the best tools do on the kinds of projects that software engineers typically work on for pay. So they're faced with headlines that oversell AI capabilities and positive experiences with their own small projects and they buy the hype.

throwaway0123_5 2y ago
Some codebases grown with AI assistance must be getting pretty large now, I think an interesting metric to track would be percent of code that is AI generated over time. Still isn't a perfect proxy for how much work the AI is replacing though, because of course it isn't the case that all lines of code would take the same amount of time to write by hand.
- lolinder 2y ago
  Yeah, that would be very helpful to track. Anecdotally, I have found in my own projects that the larger they get the less I can lean on agent/chat models to generate new code that works (without needing enough tweaks that I may as well have just written it myself). Having been written with models does seem to help, but it doesn't get over the problem that eventually you run out of useful context window. What I have seen is that autocomplete scales fine (and Cursor's autocomplete is amazing), but autocomplete supplements a software engineer, it doesn't replace them. So right now I can see a world where one engineer can do a lot more than before, but it's not clear that that will actually reduce engineering jobs in the long term as opposed to just creating a teller effect.
  - ryandrake 2y ago
    It might not just be helpful but required one day. Depending on how the legality around AI-generated code plays out, it's not out of the question that companies using it will have to keep track of and check the provenance and history of their code, like many companies already do for any open source code that may leak into their project. My company has an "open source review" process to help ensure that developers aren't copy-pasting GPL'ed code or including copyleft libraries into our non-GPL licensed products. Perhaps one day it will be common to do an "AI audit" to ensure all code written complied with whatever the future regulatory landscape shapes up to be.

pgm8705 2y ago

Yes. I think part of the problem is how good it is at starting from a blank slate and putting together an MVP type app. As a developer, I have been thoroughly impressed by this. Then non-devs see this and must think software engineers are doomed. What they don't see is how terrible LLMs are at working with complex, mature codebases and the hallucinations and endless feedback loops that go with that.

idle_zealot 2y ago
The tech to quickly spin up MVP apps has been around for a while now. It gets you from a troubling blank slate to something with structure, something you can shape and build on. I am of course talking about npx create-{template name} Or your language of choice's equivalent (or git clone template-repo).
- tiborsaas 2y ago
  Yes, but the LLM driven MVP-s are not only builerplates but actual functioning apps. The "create-" is somewhat good, but it's usually throwaway code and do it properly later. While my LLM made boilerplate is the actual first few steps to get the boring parts done. It also needs refactoring and polishing, but it's an order of magnitude better than the "MVP helper tooling" before.

hnthrow90348765 2y ago

> Now completing small chunks of mundane code, explaining code, doing very small mundane changes. Very good at. This is the only current threat. The time you save as a developer using AI on mundane stuff will get filled by something else, possibly more mundane stuff. A small company with only 2-5 Seniors may not be able to drop anyone. A company with 100 seniors might be able to drop 5-10 of them total, spread across each team. The first cuts will come at scaled companies. However, it's difficult to detect if companies are cutting people just to save money or if they are actually realizing any productivity gains from AI at this point.

renegade-otter 2y ago
Especially since the zero-interest bonanza led to over-hiring of resume-driven developers. Half of AWS is torching energy by runnning some bloat that should not even be there.
sumoboy 2y ago
I don't think companies realize AI is not free. A 100+ devs, openai, anthropic, gemini API costs, the hidden overhead of costs not spoken about. Too much speculation that productivity will increase substantially, especially when a majority of companies IT is just so broken and archaic.

csmpltn 2y ago

I think that LLMs are only going to make people with real tech/programming skills much more in demand, as younger programmers skip straight into prompt engineering and never develop themselves technically beyond the bare minimum needed to glue things together. The gap between people with deep, hands-on experience that understand how a computer works and prompt engineers will become so insanely deep. Somebody needs to write that operating system the LLM runs on. Or your bank's backend system that securely stores your money. Or the mission critical systems powering this airplane you're flying next week... to pretend like this will all be handled by LLMs is so insanely out of touch with reality.

whynotminot 2y ago
Isn’t this kind of thing the story of tech though? Languages like Python and Java come around, and old-school C engineers grouse that the kids these days don’t really understand how things work, because they’re not managing memory. Modern web-dev comes around and now the old Java hands are annoyed that these new kids are just slamming NPM packages together and polyfills everywhere and no one understands Real Software Design. I actually sort of agree with the old C hands to some extent. I think people don’t understand how a lot of things actually work. And it also doesn’t really seem to matter 95% of the time.
- deleted 2y ago
  [deleted]
- chucky_z 2y ago
  $1 for the pencil, $1000 for the line. That’s the 5% when it does matter.
  - whynotminot 2y ago
    Yes this is what people like to think. It’s not really true in practice.
- shafyy 2y ago
  Just because there are these abstractions layers that happened in the past does not mean that it will continue to happen that way. For example, many no-code tools promised just that, but they never caught on. I believe that there's a "optimal" level of abstraction, which, for the web, seems to be something like the modern web stack of HTML, JavaScript and some server-side language like Python, Ruby, Java, JavaScript. Now, there might be tools that make a developer's life easier, like a nice IDE, debugging tools, linters, autocomplete and also LLMs to a certain degree (which, for me, still is a fancy autocomplete), but they are not abstraction layers in that sense.

hhh1111 2y ago

[dead]

renegade-otter 2y ago

AI will create more jobs, if anything, as the "engineers" out of their depth create massive unmaintainable legacy.

OccamsMirror 2y ago
It's Access databases all over again.

DanielHB 2y ago

The only thing I use it for is for small self-contained snippets of code on problems that require use of APIs I don't quite remember out of the top of my head. The LLM spits out the calls I need to make or attributes/config I need to set and I go check the docs to confirm. Like "How to truncate text with CSS alone" or "How to set an AWS EC2 instance RAM to 2GB using terraform"

bwfan123 2y ago

A "causal model" is needed to fix bugs ie, to "root-cause" a bug. LLMs yet dont have the idea of a causal-model of how something works built-in. What they do have is pattern matching from a large index and generation of plausible answers from that index. (aside: the plausible snippets are of questionable licensing lineage as the indexes could contain public code with restrictive licensing) Causal models require machinery which is symbolic, which is able to generate hypotheses and test and prove statements about a world. LLMs are not yet capable of this and the fundamental architecture of the llm machine is not built for it. Hence, while they are a great productivity boost as a semantic search engine, and a plausible snippet generator, they are not capable of building (or fixing bugs in) a machine which requires causal modeling.

fiso64 2y ago
>Causal models require machinery which is symbolic, which is able to generate hypotheses and test and prove statements about a world. LLMs are not yet capable of this and the fundamental architecture of the llm machine is not built for it. Prove that the human brain does symbolic computation.
- bwfan123 2y ago
  We dont know what the human brain does, but we know it can produce symbolic theories or models of abstract worlds (in the case of math) or real worlds (in the case of science). It can also produce the "symbolic" turing machine which serves as an abstraction for all computation we use (cpu/gpu/etc)

tarkin2 2y ago

Sorry for inadvertently advising but I met a guy who used v0.dev to make impressive websites (although admittedly he did use react before so he was experienced) with professional success. It's more than arguable that his company will fire/hire fewer devs. Of course in a decade or so they'll be a skill gap unless LLMs can fill that gap too.

anavat 2y ago

My take is that AI's ability to generate new code will prove so valuable, it will not matter if it is bad at changing existing code. And that the engineers of the distant future (like, two years from now) will not bother to read the generated code, as long as it runs and passes the tests (which will also be AI-generated). I try to use AI daily, and every month I see how it is able to generate larger and more complex chunks of code from the first shot. It is almost there. We just need to adopt the new paradigm, build the tooling, and embrace the new weird future of software development.

bdhcuidbebe 2y ago
> I try to use AI daily You should reflect on the consequences of relying too much on it. See https://www.404media.co/microsoft-study-finds-ai-makes-human-cognition-atrophied-and-unprepared-3/ https://www.404media.co/microsoft-study-finds-ai-makes-human...
- anavat 2y ago
  I don't buy it makes me dumber. It just makes me worse at some things I used to do before, while making better at some other things. Often times it doesn't feel like coding anymore, more like if I were training to be a lawyer or something. But that's my bet.

weatherlite 2y ago

I agree but too many serious people are hinting we are very close I can't ignore it anymore. Sure, when Sam Altman / Zuckerberg say we're close I don't know if I can believe him because obviously the dudes will say anything to sell/pump the stock price. But how about Demis Hassabis ? He doesn't strike me like that at all. Same for Geoff Hinton, Bengio and a couple of others.

bdhcuidbebe 2y ago
Market hype is all it is.
layer8 2y ago
People investing their lives in the field are inherently biased. This is not to diminish them, it’s just a fact of the matter. Nobody knows how general intelligence really works, nor even how to reliably test for it, so it’s all speculation.

RivieraKid 2y ago

I'm surprised to see a huge disconnect between how I perceive things and the vast majority of comments here. AI is obviously not good enough to replace programmers today. But I'm worried that it will get much better at real-world programming tasks within years or months. If you follow AI closely, how can you be dismissive of this threat? OpenAI will probably release a reasoning-based software engineering agent this year. We have a system that is similar to top humans at competitive programming. This wasn't true 1 year ago. Who knows what will happen in 1 year.

layer8 2y ago
When I see stuff like https://news.ycombinator.com/item?id=42994610 https://news.ycombinator.com/item?id=42994610 (continued in https://news.ycombinator.com/item?id=42996895 https://news.ycombinator.com/item?id=42996895), I think the field still has fundamental hurdles to overcome.
- lordswork 2y ago
  This kind of error doesn't really matter in programming where the output can be verified with a feedback loop.
  - layer8 2y ago
    This is not about the numerical result, but about the way it reasons. Testing is a sanity check, not a substitute for reasoning about program correctness.
- tmnvdb 2y ago
  Why do you think this is a fundamental hurdle, rather than just one more problem that can be solved? I dont have strong evidence either way, but I've seen a lot of 'fundamental unsurmountable problems' fall by the wayside over the past few years. So I'm not sure we can be that confident that a problem like this, for which we have very good classic algorithms, is a fundamental issue.
Imanari 2y ago

ge96 2y ago

A friend of mine reached out with some code ChatGPT wrote for him to trade crypto. It had so much random crap in it and lines would say "AI enhanced trading algo" and it was just an np.randomint line. It was pulling in random deps not even used. I get it though like I'm terrible working with IMUs and I want to just get something going but I can't there's that wall I need to overcome/learn eg. the math behind it. Same with programming helps to have the background knowing how to read code and how it works.

HDThoreaun 2y ago
I used claude to help write a crypto trading bot. It helped me push out thousands of lines a day. What wouldve taken months took a couple weeks. Obviously you still need experienced pilots but unless we find an absolute fuckload of new work to do(not unlikely looking at history) its hard for me to see anything other than way less developers being needed.

jillesvangurp 2y ago

I think of it as an enabler that reduces my dependency on junior developers. Instead of delegating simple stuff to them, I now do it myself with about the same amount of overhead (have to explain what I want, have to triple check the results) on my side but less time wasted on their end. A lot of micro managing is involved either way. And most LLMs suffer from a severe case of ground hog day. You can't assume them to remember anything over time. Every conversation starts from scratch. If it's not in your recent context, specify it again. Etc. Quite tedious but it still beats me doing it manually. For some things. For at least the next few years, it's going to be an expectation from customers that you will not waste their time with stuff they could have just asked an LLM to do for them. I've had two instances of non technical CPO and CEO types recently figuring out how to get a few simple projects done with LLMs. One actually is tackling rust programs now. The point here is not that that's good code but that neither of them would have dreamed about doing anything themselves a few years ago. The scope of the stuff you can get done quickly is increasing. LLMs are worse at modifying existing code than they are at creating new code. Every conversation is a new conversation. Ground hog day, every day. Modifying something with a lot of history and context requires larger context windows and tools to fill those. The tools are increasingly becoming the bottleneck. Because without context the whole thing derails and micromanaging a lot of context is a chore. And a big factor here is that huge context windows are costly so there's an incentive for service providers to cut some corners there. Most value for me these days come from LLM tool improvements that result in me having to type less. "fix this" now means "fix the thing under my cursor in my open editor, with the full context of that file". I do this a lot since a few weeks.

belter 2y ago

> Now completing small chunks of mundane code, explaining code, doing very small mundane changes. Very good at. I would not trust them until they can do the news properly. Just read the source Luke. "AI chatbots unable to accurately summarise news, BBC finds" - https://www.bbc.com/news/articles/c0m17d8827ko https://www.bbc.com/news/articles/c0m17d8827ko

strangescript 2y ago

As context sizes get larger (and remain accurate within the size) and speeds increase, especially inference, it will start solving these large complex code bases. I think people lose sight of how much better it has gotten in just a few years.

hintymad 2y ago

> It's incredibly far away from doing any significant change in a mature codebase A lot of the use cases are on building something that has already been built before, like a web app, a popular algorithm, and etc. I think the real threat to us programmers is stagnation. If we don't have new use cases to develop but only introduce marginal changes, then we can surely use AI to generate our code from the vast amount of previous work.

giancarlostoro 2y ago

They all sounds like crypto bros talking about AI. It's really frustrating to talk to them, just like crypto bros.

moogly 2y ago
They're the same people in my experience.
- giancarlostoro 2y ago
  Its the same energy for sure.

deeviant 2y ago

The huge disconnect is that the skill set to use LLMs for code effectively is not the same skill set of standard software engineering. There is a very heavy intersection and I would say you cannot be effective at LLM software development without being an effective software engineer, but being an effective software engineer does not by any means make somebody good at LLM development. Very talented engineers, coworkers, that I would place above myself in skill, seemed stumped by it, while I have realized at least a 10x productively gain. The claim that LLMs are not being applied in mature, complex code-bases is pure fantasy, example: https://arxiv.org/abs/2501.06972 https://arxiv.org/abs/2501.06972. Here Google is using LLMs to accelerate the migration of mature, complex production systems.

agentultra 2y ago

I’m more keen on formal methods to do this than LLMs. I take the view that we need more precise languages that require us to write less code that obviously has no errors in it. LLMs are primed to generate more code using less precise specifications; resulting in code that has no obvious errors in it.

micromacrofoot 2y ago

I think you're discounting efficiency gains — through a series of individually minor breakthroughs in LLM tech I think we could end up with things like 100M+ token context windows We've already seen this sort of incrementalism over the past couple of years, the initial buzz started without much more than a 2048 context window and we're seeing models with 1M out there now that are significantly more capable.

__MatrixMan__ 2y ago

I think it's more likely that we'll see a rise in workflows that AI is good at, rather than AI rising to meet the challenges of our more complex workflows. Let the user pair with an AI to edit and hot-reload some subset of the code which needs to be very adapted to the problem domain, and have the AI fine-tuned for the task at hand. If that doesn't cut it, have the user submit issues if they need an engineer to alter the interface that they and the AI are using. I guess this would resemble how myspace used to do it, where you'd get a text box where you could provide custom edits, but you couldn't change the interface.

rs186 2y ago

I use AI coding assistants daily, and whenever there is a task that those tools cannot do correctly/quickly enough so that I need to fallback to editing things by myself, I spend a bit of time thinking what is so special about the tasks. My observation is that LLMs do repetitive, boring tasks really well, like boilerplate code and common logic/basic UI that thousands of people have already done. Well, in some sense, jobs where developers who spend a lot of time writing generic code is already at risk of being outsourced. The tasks that need a ton of tweaking or not worth asking AI at all are those that are very specific to a specific product and need to meet specific requirements that often come from discussions or meetings. Well, I guess in theory if we had transcripts for everything, AI could write code like the way you want, but I doubt that's happening any time soon. I have since become less worried about the pace AI will replace human programmers -- there is still a lot that these tools cannot do. But for sure people need to watch out and be aware of what's happening.

xp84 2y ago

> It's incredibly far away from doing any significant change in a mature codebase. I agree with this completely. However the problem that I think the article gets at is still real because junior engineers also can't do significant changes on a mature codebase when they first start out. They used to do the 'easy stuff' which freed the rest of us up to do bigger stuff. But: 1. Companies like mine don't hire juniors anymore 2. With Copilot I can be so much more productive that I don't need juniors to do "the easy stuff" because Copilot can easily do that in 1/1000th the time a junior would. 3. So now who is going to train those juniors to get to the level where we need them to be to make those "significant changes"?

hypothesis 2y ago
> So now who is going to train those juniors to get to the level where we need them to be to make those "significant changes"? Founders will cash out long before that becomes an issue. Alternatively, the hype is true and they will obsolete programmers, also solving the issue above… This is quite devious if you think about it, withering pipeline of new devs and only them having an immediate fix in all cases.

outworlder 2y ago

> I'm thinking there's going to have to be some other breakthrough or something other than LLM's. We actually _need_ a breakthrough for the promises to materialize, otherwise we will have yet another AI Winter. Even though there seems to be some emergent behavior (some evidence that LLMs can, for example, create an internal chess representation by themselves when asked to play), that's not enough. We'll end up with diminishing returns. Investors will get bored of waiting and this whole thing comes crashing down. We'll get an useful too in our toolbox, as we do at every AI cycle.

Aiguru31415666 2y ago

[dead]

darepublic 2y ago

I dunno if it's always good at explaining code. It tends to take everything at face value and is unable to opinionatedly reject bs when it's presented with it. Which in the majority of cases is bad.

bodegajed 2y ago
this is also my problem. When I ask someone a technical question, and I did not provide context on some abstractions. Usually this is common because abstractions can be very deep. "Hmm, not sure.. can you check what's this supposed to do?" LLMs don't do this, it confidently hallucinate the abstraction out of thin air or uses their outdated knowledge store. Sending wrong use or wrong input parameters.

scotty79 2y ago

> We know what it is good at and what it's not. We know what it's good at today. And pretty sure it won't be any worse at it in the future. And 5 years ago state of the art was basically output of Markov Chain. In 5 years we might be at another place entirely.

necovek 2y ago

Agreed, and I haven't yet seen any single instance of a company firing software engineers because AI is replacing them (even if by increasing productivity of another set of software engineers): I've asked this a number of times, and while it's a common refrain, I haven't really seen any concrete news report saying it in so many words. And to be honest, if any company is firing software engineers hoping AI replaces their production, that is good news since that company will soon stop existing and treating engineers like shit which it probably did :)

dartos 2y ago

Marketing is really good at their job. That coupled with new money and retail investors being thinking they’re in a gold rush and you get the environment we’re in.

ericmcer 2y ago

A breakthrough is exactly what everyone is banking on. OpenAI was surprised by GPT3, they were just dumping data into an LLM and ended up with something that was way better than they expected. Everyone is hoping (probably delusional) that bigger and more impressive breakthroughs will keep leaping up if we just keep tweaking the models and increasing the size of the data sets.

jmspring 2y ago

My favorite has been everyday people claiming Elmo will find all sorts of corruption on Fed databases using AI. Trained on what datasets? What biases? Etc.

ein0p 2y ago

+1. I've tried really hard to replace even some parts of my job with AI ever since GPT3 era, unsuccessfully. All it does for me is it allows me to enter unfamiliar domains (such as e.g. SwiftUI) but then I'm all on my own. In domains where I already have expertise it just doesn't work well. So it is a productivity booster, sure, but I don't see it replacing anyone doing non-bullshit work. I don't even see a trend line pointing in that direction.

z3n0n 2y ago

Been heavily testing Cursor, Windsurf and VSCode w/CP lately. Most low level stuff works surprisingly well. But for anything slightly more complex I just end up wasting 90% credits watching the AI chase its own tail.

chefandy 2y ago

Counterpoint: https://medium.com/@sweaty.phd/yet-another-ai-will-take-your-job-story-programmers-edition-146a97211734 https://medium.com/@sweaty.phd/yet-another-ai-will-take-your...