14 ms·

Again one of the few advantages of having been round the sun a few more times than most is this isn’t the first time this has happened. Packages were supposed

Again one of the few advantages of having been round the sun a few more times than most is this isn’t the first time this has happened. Packages were supposed to replace programming. They got you 70% of the way there as well. Same with 4GLs, Visual Coding, CASE tools, even Rails and the rest of the opinionated web tools. Every generation has to learn “There is no silver bullet”. Even though Fred Brooks explained why in 1986. There are essential tasks and there are accidental tasks. The tools really only help with the accidental tasks. AI is a fabulous tool that is way more flexible than previous attempts because I can just talk to it in English and it covers every accidental issue you can imagine. But it can’t do the essential work of complexity management for the same reason it can’t prove an unproven maths problem. As it stands we still need human brains to do those things.

bambax 2y ago
Where AI really really shine is to help an engineer get proficient in a language they don't know well. Simon Willison says this somewhere and in my experience it's very true. If you can code, and you understand the problem (or are well on your way to understanding it), but you're not familiar with the exact syntax of Go or whatever, then working with AI will save you hundreds of hours. If you can't code, or do not (yet) understand the problem, AI won't save you. It will probably hurt.
- fatnoah 2y ago
  > Where AI really really shine is to help an engineer get proficient in a language they don't know well. I used GitHub Copilot in a project I started mostly to learn Go. It was amazing. I spent not so much time fiddling around with syntax, and much more time thinking about design.
- heelix 2y ago
  So very much this. As I was learning Rust, I'd ask what the equivalent was for a snippet I could create in Java. It is funny. I look at the Java code provided by prompts and go meh. The Rust code looks great. I realize this is probably due to 1) me being that junior level in Rust or 2) less legacy crap in the training model. I'm sure it is both, with more of the former as I work from working to beautiful code.
- dartos 2y ago
  I used to agree, but as an experienced engineer asking about rust and y-crdt, it sent me down so many wrong rabbit holes with half valid information. I used Claude recently to refresh my knowledge on the browser history API and it said that it gets cleared when the user navigates to a new page because the “JavaScript context has changed” I have the experience and know how to verify this stuff, but a new engineer may not, and that would be awful. Things like these made me cancel all my AI subscriptions and just wait for whatever comes after transformers.
  - cvwright 2y ago
    I think of AI like a very eager and bright 20 yr old intern who’s really into whatever tech I’m trying to learn. Good knowledge of facts, willing to give anything a try, but not much wisdom.
    - dartos 2y ago
      > Good knowledge of facts I haven’t seen this demonstrated in gpt-4 or Claude sonnet when asking anything beyond the most extreme basics. I consistently get subtly wrong answers and whenever I ask “oh okay, so it works like this” I always get “Yes! Exactly. You show a deep understanding of…” even though I was wrong based on the wrong info from the LLM. Useless for knowledge work beyond RAG, it seems. Search engines that I need to double check are worse than documentation. It’s why so many of us moved beyond stack overflow. Documentation has gotten so good.
    - tmpz22 2y ago
      ChatGPT and Claude outputs are equivalent to a 20 yr old intern who goes to MIT?
  - pizza 2y ago
    But actually whenever this happens you get rich signal about the hidden didactic assumptions of your method of prompting it about things you yourself are unsure you can verify for yourself, and also of how you thought the tool worked. This is a good meta skill to hone.
- osigurdson 2y ago
  It allows you to write a little code in languages that you are not familiar with. However, it is nothing like actually internalizing a language.
  - cruffle_duffle 2y ago
    That’s true. But if you are already an experienced developer who’s been around the block enough to call bullshit when you see it, these LLM thingies can be pretty useful for unfamiliar languages. But you need to be constantly vigilant, ask it the right questions (eg: “is this thing you wrote really best practice for this language? Cause it doesn’t seem that way”), and call bullshit on obvious bullshit… …which sometimes feels like it is more work than just fucking doing it yourself. So yeah. I dunno!
- adamc 2y ago
  Not sure I get how that would work. It seems to me that to do my job I will have to validate the semantics of the program, and that means I will have to become familiar with the syntax of Go or whatever, at a fairly sophisticated level. If I am glossing over the syntax, I am inevitably glossing over the fine points of how the program works.
- cruffle_duffle 2y ago
  It depends on the language and the libraries you will use. Python with a well known library? Sure no problem. Almost any model will crank out fairly error free boilerplate to get you started. Terraform? Hah. 4o and even o1 both absolutely sucked at it. You could copy & paste the documentation for a resource provider, examples and all, and it would still produce almost unusable code. Which was not at all helpful given I didn’t know the language or its design patterns and best practices at all. Sonnet 3.5 did significantly better but still required a little hand holding. And while I got my cloud architecture up and running now I question if I followed “best practices” at all. (Note: I don’t really care if I did though… I have other more important parts of my project to work on, like the actual product itself). To me one of the big issues with these LLM’s is they have zero ability to do reflection and explain their “thought process”. And even if they could you cannot trust what it says because it could be spouting off whatever random training data it hovered up or it could be “aligned” to agree with whatever you tell it. And that is the thing about LLM’s. They are remarkably good bullshitters. They’ll say exactly what you want them to and be right just enough that they fool you into thinking they are something more than an incredibly sophisticated next token generator. They are both incredibly overrated and underrated at the same time. And it will take us humans a little while to fully map out what they are actually good at and what they only pretend to be good at.
- DanielHB 2y ago
  LLMs are a great help with terraform and devops configuration, they often invent things but at least the point at the documentation I need to look up on. Of course everything needs double-checking but just asking the LLM: "how do I do X" will usually at least output all the names of terrraform resources and most configuration attributes I need to look up. They are great for any kind of work that requires "magical incantations" as I like to call them.
- vunderba 2y ago
  I guess it depends on how you define "proficiency". For me, proficiency implies a fundamental understanding of something. You're not proficient in Spanish if you have to constantly make use of Google Translate. Could code assistants be used to help actually learn a programming language? - Absolutely. Will the majority of people that use an LLM to write a Swift app actually do this? - Probably not, they'll hammer the LLM until it produces code that hobbles along and call it a day. Also, LEARNING is aided by being more active, but relying on an LLM inherently encourages you to adopt a significantly more passive behavior (reading rather than writing).
- mgens 2y ago
  Yes! Reading some basic documentation on the language or framework, then starting to build in Cursor with AI suggestions works so well. The AI suggests using functions you didn't even know about yet, then you can go read documentation on them to flesh out your knowledge. Learned basic web dev with Django and Tailwind this way and it accelerated the process greatly. Related to the article, this relies on being curious and taking the time to learn any concepts the AI is using, since you can't trust it completely. But it's a wonderfully organic way to learn by doing.
  - dogleash 2y ago
    Tell me you're a Cursor advertisement chatbot without telling me you're a Cursor advertisement chatbot.
    - mgens 2y ago
      I'll have you know that I successfully clicked on several traffic lights and motorcycles this week to get into a web site. Didn't even need Cursor!
- croes 2y ago
  If you don’t know the language well enough AI could lead you down the path of bad practices and you won’t recognize it.
- xboxnolifes 2y ago
  I agree with this take. AI has essentially served me well as a more useful search engine when working in new languages, libraries, or frameworks.
arikrak 2y ago
AI is a potential silver bullet since it can address the "essential complexity" that Fred Books said regular programming improvements couldn't address. It may not yet have caused an "order of magnitude" improvement in overall software development but it has caused that improvement in certain areas, and that will spread over time. https://en.wikipedia.org/wiki/No_Silver_Bullet https://en.wikipedia.org/wiki/No_Silver_Bullet
snarf21 2y ago
The most valuable thing I want AI to do with regards to coding is to have it write all the unit tests and get me to 100% code coverage. The data variance and combinatorics needed to construct all the meaningful tests is sometimes laborious which means it doesn't get done (us coders are lazy...). That is what AI to do, all the mind numbing draining work so I can focus more on the system.
- jyounker 2y ago
  Which is also a fantasy, since if you did achieve it, you'd just have tests that verified that all your bugs were in place.
  - Ferret7446 2y ago
    Not necessarily. I have used LLMs to write unit tests based on the intent of the code and have it catch bugs. This is for relatively simple cases of course, but there's no reason why this can't scale up in the future. LLMs absolutely can "detect intent" and correct buggy code. e.g., "this code appears to be trying to foo a bar, but it has a bug..."
- js8 2y ago
  How do you expect AI to write unit tests if it doesn't know the precise desired semantics (specification)? What I personally would like AI to do would be to refactor the program so it would be shorter/clearer, without changing its semantics. Then, I (human) could easily review what it does, whether it conforms to the specification. (For example, rewrite the C program to give exactly the same output, but as a Python code.) In cases where there is a peculiar difference between the desired semantics and real semantics, this would become apparent as additional complexity in the refactored program. For example, there might be a subtle semantic differences between C and Python library functions. If the refactored program would use a custom reimplementation of C function instead of the Python function, it would indicate that the difference matters for the program semantics, and needs to be somehow further specified, or it can be a bug in one of the implementations.
- 9rx 2y ago
whiplash451 2y ago
At this point, I don't think that (truly) AI-informed people believe that AI will replace engineers. But AI tools will likely bring a deep transformation to the workflow of engineers (in a positive and collaborative way). It may not be tab-tab-tab all the way, but a whole lot more tabs will sneak in.
- Workaccount2 2y ago
  I don't think it will replace engineers, but I do think it will turn them from $200k/yr golden geese into regular old $90k/yr ducks.
  - CuriouslyC 2y ago
    I think you have that backwards (sort of). The high tier programmers who can write things AI can't will be worth more since they'll be more productive, while the programmers below the AI skill floor will see their value drop since they've been commoditized. We already have a bimodal distribution of salaries for programmers between FAANG/not, this will just exacerbate that.
  - cruffle_duffle 2y ago
    As somebody who makes extensive use of LLM’s, I very much disagree. Large language models are completely incapable of replacing the kind of stuff you pay a developer $200k for. If anything they make that $200k developer even more of a golden goose.
  - nicholassmith 2y ago
    I suspect you're right, but I think it'll follow the COBOL engineer salary cycle, engineers that have a deeper understanding of the whole widget will be in demand when companies remember they need them.
  - whiplash451 2y ago
    I think the opposite will happen. Engineers will still be worth $200k/yr. Humankind will just develop a lot more software, faster and further into industries. Availability will create new demand.
orbisvicis 2y ago
If I could sketch out the architecture I wanted as a flow chart annotated with types and structures, implementable by an AI, that would be a revolutionary leap. I design top-down, component by component, and sometimes the parts don't fit together as I envisioned, so I have to write adapters or - worst case - return to the drawing board. If the AI could predict these mismatches, that would also be helpful. Unfortunately, I don't think AI is great with only the accidental tasks either.
- adamc 2y ago
  Maybe we need better terminology. But AI right now is more like pattern-matching than anything I would label as "understanding", even when it works well.
- CuriouslyC 2y ago
  AI is really good at goldfish programming. It's incredibly smart within its myopic window, but falls apart as it is asked to look farther. The key is to ask for bite sized things where that myopia doesn't really become a factor. Additionally, you as the user have to consider whether the model has seen seen similar things in the past, as it's really good at regurgitating variations but struggles with novelty.
EGreg 2y ago
AI totally is a silver bullet. If you don't think so, you're just using it wrong and it's your fault. If you think that it takes you just as long or longer to constantly double-check everything it does, then you don't understand the P vs NP problem. </sarcasm> AI will only get better. And AI has proven a lot of unproven maths problems as far back as 2019: https://mathscholar.org/2019/04/google-ai-system-proves-over-1200-mathematical-theorems/ https://mathscholar.org/2019/04/google-ai-system-proves-over...
DanielHB 2y ago
The better tech often lowers the barrier for people to do things but raises the bar of users (and stakeholders for contract projects) expectations. It is plainly visible with web development where the amount of tooling has grown dramatically (both frontend and backend) to do things. Like, for example, all the big-data stuff we do today was unthinkable 10 years ago, today every mid-sized company has a data team. 15 years ago all data in a single monolithic relational database was the norm, all you needed to know was SQL and some Java/C#/PHP and some HTML to get some data wired up into queries.
bloomingkales 2y ago
Same with 4GLs, Visual Coding, CASE tools, even Rails and the rest of the opinionated web tools. How many of those things were envisioned by futurists or great authors? This AI stuff is the stuff of dreams, and I think it’s unwise to consider it another go around the sun.
- no_wizard 2y ago
  Until it’s actually AI and not Machine Learning masquerading as AI because AI is the sectors marketing pitch, I would strongly hesitate considering it as anything other than a tool. Yes, a powerful tool, and as powerful tools go, they can re-shape how things get done, but a tool none the less and therefore we must consider what its limits are, which is all OP is getting at and the current and known near future state suggests we aren’t evolving passed the tool state
- latexr 2y ago
  > This AI stuff is the stuff of dreams This AI stuff? No, not really. The stuff of dreams is an AI that you can talk to and interact infinitely and trust that it doesn’t make mistakes. LLMs ain’t it.
  - deleted 2y ago
    [deleted]
latexr 2y ago
For reference, to those who want to learn more: https://en.wikipedia.org/wiki/No_Silver_Bullet https://en.wikipedia.org/wiki/No_Silver_Bullet
HarHarVeryFunny 2y ago
> The tools really only help with the accidental tasks I don't think that's really the problem with using LLMs for coding, although it depends on how you define "accidental". I suppose if we take the opposite of "essential" (the core architecture, planned to solve the problem) to be boilerplate (stuff that needs to be done as part of a solution, but doesn't itself really define the solution), then it does apply. It's interesting/amusing that on the surface a coding assistant is one of the things that LLMs appear better suited for, and they are suited for, as far as boilerplate generation goes (essentially automated stack overflow, and similar-project, cut and pasting)... But, in reality, it is one of the things LLMs are LEAST suited for, given that once you move beyond boilerplate/accidental code, the key skills needed for software design/development are reasoning/planning, as well as experienced-based ("inference time") learning to progress at the craft, which are two of the most fundamental shortcomings of LLMs that no amount of scale can fix. So, yeah, maybe they can sometimes generate 70% of the code, but it's the easy/boilerplate 70% of the code, not the 30% that defines the architecture of the solution. Of course it's trendy to call LLMs "AI" at the moment, just as previous GOFAI attempts at AI (e.g. symbolic problem solvers like SOAR, expert systems like CYC) were called "AI" until their limitations became more apparent. You'll know we're one step closer to AI/AGI when LLMs are in the rear view mirror and back to just being called LLMs again!
blueprint 2y ago
What? AI can't prove any unproven math problems...?
hinkley 2y ago
The first lead I worked with inoculated me to this. He taught me about hype trains long before the idea was formalized. He’d been around for the previous AI hype cycle and told me to expect this one to go the same. Which it did, and rather spectacularly. That was three cycles ago now and while I have promised myself I will check out the next cycle, because I actually do feel like maybe next time they’ll build systems that can answer why not just how, this one is a snooze fest I don’t need to get myself involved in.
- danielbln 2y ago
  Just be careful you don'tet your pendulum swing too much in the other direction, where your turn into an old curmudgeon that doesn't get excited by anything and that thinks nothing is novel or groundbreaking anymore.
banku_brougham 2y ago
The devs get it from what i can tell, management are the slow learners here
jamil7 2y ago
It’s an old one but I think Joel Spolsky‘s take on leaky abstractions is relevant again in this discussion as we add another abstraction layer with LLM assisted coding. https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-abstractions/ https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-a...
- louthy 2y ago
  Aligned with: “The purpose of abstraction is not to be vague, but to create a new semantic level in which one can be absolutely precise.” — Edsger Dijkstra The abstraction has never been this leaky or vague — Me
  - olvy0 2y ago
    That is a great quote, thank you.
  - truefossil 2y ago
    Agree. So far "the progress" implied understanding (discovering) previously unknown things. AI is exactly the opposite: "I don't understand how, but it sorta works!"
  - rob74 2y ago
    That's the purpose that all abstractions strive to achieve, but most of them fall short - and become leaky...
- rossmurphy 2y ago
  Joel's blog posts should be hung in the Louvre.
smokel 2y ago
Too bad most of society is accidental as well. With which I mean to say that there are a lot of nonsensical projects being done out there, that still make a living for many people. Modern AI may well change things, similar to how computers changed things previously. I get your sentiment, I've been through a few hype cycles as well, but besides learning that history repeats itself, there is no saying how it will repeat itself.
- notRobot 2y ago
  > With which I mean to say that there is a lot of nonsensical projects being done out there, that still make a living for many people. I don't know why this is a bad thing. I don't think projects that you believe are nonsensical shouldn't exist just because of your opinion, especially if they're helping people survive in this world. I'm sure the people working on them don't think they're nonsensical.
  - jajko 2y ago
    Yeah when you take 2 steps back, ignore IT for a second and look on whole mankind, there are hundreds of millions of jobs that could be called nonsensical from certain points of view. We are not above this in any meaningful ways, maybe its just a bit more obvious to keen eye. Yet society and economy keeps going and nobody apart from some academic discussions really cares. I mean companies have 100% incentive to trim fat to raise income yet they only do the least minimum.
    - chuankl 2y ago
      There is a book about it: "Bullshit Jobs" by the late anthropologist David Graeber.
  - lukan 2y ago
    "I'm sure the people working on them don't think they're nonsensical." Rise of antidepressants imply otherwise. The more bullshit we stop doing, the more energy there is for the awesome things we could do instead.
andrewstuart 2y ago
None of those even came within 10% of delivering to the extent that LLMs have.
- ulfw 2y ago
  Exactly what you said Andrew. The comparisons are lacking and are almost at whataboutism level. The amount of actual 'work' that AI does versus the tools of yesterday are an order of magnitude away
  - kruxigt 2y ago
    [dead]
  - smokel 2y ago
    Going from punched cards to interactive terminals surely must have been a big productivity boost. And going from text based CAD to what is possible on modern workstations has probably also helped a bit in that field. In that view I'd say the productivity boost by LLMs is somewhat disappointing, especially with respect to how amazing they are.
  - player1234 2y ago
    Quantify it, show us the numbers.
    - NitpickLawyer 2y ago
      I think the field is too new and the successful stories too private atm. However I think the best apples to apples example in this context is Amz's codebase update project that they've blogged about. From memory, they took some old java projects, and had some LLM driven "agents" update the codebase to recent java. I don't know java enough to know how "hard" this task is, but asking around I've heard that "analog" tools for this exist, but aren't that good, bork often, are hardcoded and so on. Amz reported ~70% of code that came out passed code review, presumably the rest had to be tweaked by humans. I don't know if there are any "classical" tools that can do that ootb. So yeah, that's already imrpessive and "available today" so to speak.
Dalewyn 2y ago
>AI is a fabulous tool that is way more flexible than previous attempts because I can just talk to it in English In an era when UIs become ever more Hieroglyphic(tm), Aesthetical(tm), and Nouveau(tm), "AI" revolutionizing and redefining the whole concept of interacting with computers as "Just speak Human." is a wild breath of fresh air.
- skydhash 2y ago
  Programming and interacting with computers in general is just translation to a more restricted and precise language. And that what's make them more efficient. Speaking human is just going the other way and losing productivity. It's akin to how everyone can build a shelter, but building a house requires a more specialized knowledge. The cost of the later is training time to understand stuff. The cost of programming is also training time to understand how stuff works and how to manipulate them.
  - Dalewyn 2y ago
    An inefficient computer you can use is more productive than an efficient computer you can't use. Most people can't use mice or keyboards with speed, touchscreens are marginally better except all the "gestures" are unnatural as hell, and programming is pig latin. Mice and keyboards and programming languages and all the esoteric ways of communicating with computers came about simply because we couldn't just talk Human to them. Democratizing access to computers is a very good and very productive thing.
    - skydhash 2y ago
      That's the thing. You don't communicate with computers. You use them. You have a task to do that the computer have been programmed for and what you want is to get the parameters of that tasks to the computer. And you learn how to use the computer because the tasks is worth it, just like you learn how to play a game because you enjoy the time doing it. The task supersedes the tool. Generative AI can be thought as an interface to the tool, but it's been proven that they are unreliable. And as the article outlines, if it can get to 70% of the task, but you don't have the knowledge requires to complete it, that's pretty much the same as 0%. And if you have the knowledge, more often than not you realize that it just go faster on a zigzag instead of the straight route you would have taken with more conventional tools.
koliber 2y ago
As the adage goes: If you aim for the stars and fail, you'll end up on the moon.
- steveBK123 2y ago
  Kind of the opposite lesson of Icarus?
- rsynnott 2y ago
  Other options are available, for instance ploughing into a village because your second stage didn't light, or, well, this: https://youtu.be/mTmb3Cqb2qw?t=16 https://youtu.be/mTmb3Cqb2qw?t=16 Most of the "you'll never need programmers again!" things have ended up more "cars-showered-with-chunks-of-flaming-HTPB" than "accidentally-land-on-moon", tbh. 4GLs had an anomaly, and now we don't talk about them anymore. (It's a terrible adage, really. "Oops, the obviously impossible thing didn't happen, but an unrelated good thing did" just doesn't happen that often, and when it does there's rarely a causal relation between A and B.)
- cwillu 2y ago
  If you aim for the stars and fail, you end up going in circles for all eternity.
  - CapeTheory 2y ago
    If you aim for the stars and fail, others can remember your folly by the smoking crater you leave behind.
    - christophilus 2y ago
      Not if you fail by running out of fuel and smashing into Europa.
    - mrjin 2y ago
      Hmmm, for 3 minutes?
Izkata 2y ago
> Even though Fred Brooks explained why in 1986. There are essential tasks and there are accidental tasks. The tools really only help with the accidental tasks. I don't know this reference, so I have to ask: Was "accidental" supposed to be "incidental"? Because I don't see how "accidental" makes any sense.
- dkdbejwi383 2y ago
  "Accident" in the same sense as "accident of birth" not "traffic accident".
- bluGill 2y ago
  Buy and read the book. There is a reason the 25th aniversery eddition has been still in print for more than 30 years. It is a timeless combuter book that everyone should read and keep an their bookshelf.
- racingmars 2y ago
  The Mythical Man-Month, by Fred Brooks. Chapter 16 is named "No Silver Bullet—Essence and Accident in Software Engineering." I'll type out the beginning of the abstract at the beginning of the chapter here: "All software construction involves essential tasks, the fashioning of the complex conceptual structures that compose the abstract software entity, and accidental tasks, the representation of these abstract entities in programming languages and the mapping of these onto machine languages within space and speed constraints. Most of the big past gains in software productivity have come from removing artificial barriers that have made the accidental tasks inordinately hard, such as severe hardware constraints, awkward programming languages, lack of machine time. How much of what software engineers now do is still devoted to the accidental, as opposed to the essential? Unless it is more than 9/10 of all effort, shrinking all the accidental activities to zero time will not give an order of magnitude improvement."
  - Izkata 2y ago
    From the abstract that definitely sounds like he meant "incidental": Something that's a necessary consequence of previous work and / or the necessary but simpler part of the work.
lxdlam 2y ago
Totally agree. The software development is absolutely a fractal. In 1960s we were solving the complexity by using high level language that compile to machine code to enable more people write simple code. This has happened again and again and again. But different generations face different problems, which requires another level of thinking, abstraction, and push both boundaries until we reach the next generation. All of this is not solved by a single solution, but the combination based on basic principles that never changes, and these things, at least for now, only human can do.
- uludag 2y ago
  > The software development is absolutely a fractal. I think this analogy is more apt than you may realize. Just like a fractal, the iterated patterns get repeated on a much smaller scale. The jump to higher-level languages was probably a greater leap then the the rest of software innovation will provide. And with each iterative gain we approach some asymptote, but never get there. And this frustration of never reaching our desired outcome results in ever louder hype cycles.
- dkrich 2y ago
  Interestingly it seems like we are investing many more magnitudes of capital for smaller and smaller gains. For example, the jump in productivity from adding an operating system to a computer is orders of magnitude larger than adding an LLM to a web development process despite the LLM requiring infrastructure that cost tens of billions to create. It seems that while tools are getting more and more sophisticated, they aren’t really resulting in much greater productivity. It all still seems to be resulting in software that solves the same problems as before. Whereas when html came around it opened up use cases that has never been seen before despite being a very simple abstraction layer by today’s standards. Perhaps the opportunities are greatest when you are abstracting the layer that the fewest understand when LLMs seem to assume the opposite.
  - mattgreenrocks 2y ago
    The real gains in software are still to be had by aggressively destroying incidental complexity. Most of the gunk in a web app doesn't absolutely need to exist, but we write it anyway. (Look at fasthtml for an alternate vision of building web apps.) The issue with LLMs is they enshrine the status quo. I don't want ossified crappy software that's hard to work with. Frameworks and libraries should have to fight to justify their existence in the marketplace of ideas. Subverting this mechanism is how you ruin software construction.
GistNoesis 2y ago
>But it can’t do the essential work of complexity management I think this is the "closing the loop" ( https://en.wikipedia.org/wiki/Control_loop#Open-loop_and_closed-loop https://en.wikipedia.org/wiki/Control_loop#Open-loop_and_clo... ) moment for coding AI. All pieces are there, we just need to decide to do it. Today's AI are able to produce an increasing tangled mess of code. But it's also able to reorganize the code. It's also capable of writing test code, and assess the quality of the code. It's also capable to make architectural decision. Today's AI code, is more like a Frankenstein's composition. But with the right prompt OODA loop and quality assessment rigor, it boils down to just having to sort and clean the junk pile faster than you produce it. Once you have a coherent unified codebase, things get fast quickly, capabilities grows exponentially with the number of lines of code. Think of things like Julia Language or Wolfram Language. Once you have a well written library or package, you are more than 95% there and you almost don't need AI to do the things you want to do.
- qazpot 2y ago
  > I think this is the "closing the loop" ( https://en.wikipedia.org/wiki/Control_loop#Open-loop_and_clo https://en.wikipedia.org/wiki/Control_loop#Open-loop_and_clo... ) moment for coding AI. > All pieces are there, we just need to decide to do it. Another silver bullet.
  - GistNoesis 2y ago
    There is a huge gap in performance and reliability in control systems between open-loop and closed-loop. You've got to bite the bullet at one point and make the transition from open-loop to closed-loop. There is a compute cost associated to it, and there is also a tuning cost, so it's not all silver lining.
- rafaelmn 2y ago
  >Once you have a coherent unified codebase, things get fast quickly, capabilities grows exponentially with the number of lines of code. Think of things like Julia Language or Wolfram Language. >Once you have a well written library or package, you are more than 95% there and you almost don't need AI to do the things you want to do. That's an idealistic view. Packages are leaky abstractions that make assumptions for you. Even stuff like base language libraries - there are plenty of scenarios where people avoid them - they work for 9x% of cases but there are cases where they don't - and this is the most fundamental primitive in a language. Even languages are leaky abstractions with their own assumptions and implications. And these are the abstractions we had decades of experience writing, across the entire industry, and for fairly fundamental stuff. Expecting that level of quality in higher level layers is just not realistic. I mean just go look at ERP software (vomit warning) - and that industry is worth billions.
dkrich 2y ago
This seems to apply to all areas of AI in its current form and in my experience 70% may be a bit generous. AI is great at getting you started or setting up scaffolds that are common to all tasks of a similar kind. Essentially anything with an identifiable pattern. It’s yet another abstraction layer sitting on an abstraction layer. I suspect this is the reason we are really only seeing AI agents being used in call centers, essentially providing stand ins for chatbots- because chatbots are designed to automate highly repetitive, predictable tasks like changing an address or initiating a dispute. But for things like “I have a question about why I was charged $24.38 on my last statement” you will still be escalated to an agent because inquiries like that require a human to investigate and interpret an unpredictable pattern. But creative tasks are designed to model the real world which is inherently analog and ever changing and closing that gap of identifying what’s missing between what you have and the real world and coming up with creative solutions is what humans excel at. Self driving, writing emails, generating applications- AI gets you a decent starting point. It doesn’t solve problems fully, even with extensive training. Being able to fill that gap is true AI imo and probably still quite a ways off.
- jeremyjh 2y ago
  Yeah, its more like it can generate 70% of the code by volume, rather than get you 70% of the way to a complete solution. 12 week projects don't become 4 week projects, at best they are 9-10 week projects.
- adamc 2y ago
  Great analysis, and I agree it's Fred Brooks' point all over again. None of these tools hurt, but you still need to comprehend the problem domain and the tools -- not least because you have to validate proposed solutions -- and AI cannot (yet) do that for you. In my experience, generating code is a relatively small part of the process.
- marcosdumay 2y ago
  > because chatbots are designed to automate highly repetitive, predictable tasks like changing an address or initiating a dispute You know what is even cheaper, more scalable, more efficient, and more user-friendly than a chatbot for those use cases? A run of the mill form on a web page. Oh, and it's also more reliable.
singularity2001 2y ago
>> for the same reason it can’t prove an unproven maths problem that aged like LeCun

bambax 2y ago

Where AI really really shine is to help an engineer get proficient in a language they don't know well. Simon Willison says this somewhere and in my experience it's very true. If you can code, and you understand the problem (or are well on your way to understanding it), but you're not familiar with the exact syntax of Go or whatever, then working with AI will save you hundreds of hours. If you can't code, or do not (yet) understand the problem, AI won't save you. It will probably hurt.

fatnoah 2y ago
> Where AI really really shine is to help an engineer get proficient in a language they don't know well. I used GitHub Copilot in a project I started mostly to learn Go. It was amazing. I spent not so much time fiddling around with syntax, and much more time thinking about design.
heelix 2y ago
So very much this. As I was learning Rust, I'd ask what the equivalent was for a snippet I could create in Java. It is funny. I look at the Java code provided by prompts and go meh. The Rust code looks great. I realize this is probably due to 1) me being that junior level in Rust or 2) less legacy crap in the training model. I'm sure it is both, with more of the former as I work from working to beautiful code.
dartos 2y ago
I used to agree, but as an experienced engineer asking about rust and y-crdt, it sent me down so many wrong rabbit holes with half valid information. I used Claude recently to refresh my knowledge on the browser history API and it said that it gets cleared when the user navigates to a new page because the “JavaScript context has changed” I have the experience and know how to verify this stuff, but a new engineer may not, and that would be awful. Things like these made me cancel all my AI subscriptions and just wait for whatever comes after transformers.
- cvwright 2y ago
  I think of AI like a very eager and bright 20 yr old intern who’s really into whatever tech I’m trying to learn. Good knowledge of facts, willing to give anything a try, but not much wisdom.
  - dartos 2y ago
    > Good knowledge of facts I haven’t seen this demonstrated in gpt-4 or Claude sonnet when asking anything beyond the most extreme basics. I consistently get subtly wrong answers and whenever I ask “oh okay, so it works like this” I always get “Yes! Exactly. You show a deep understanding of…” even though I was wrong based on the wrong info from the LLM. Useless for knowledge work beyond RAG, it seems. Search engines that I need to double check are worse than documentation. It’s why so many of us moved beyond stack overflow. Documentation has gotten so good.
  - tmpz22 2y ago
    ChatGPT and Claude outputs are equivalent to a 20 yr old intern who goes to MIT?
- pizza 2y ago
  But actually whenever this happens you get rich signal about the hidden didactic assumptions of your method of prompting it about things you yourself are unsure you can verify for yourself, and also of how you thought the tool worked. This is a good meta skill to hone.
osigurdson 2y ago
It allows you to write a little code in languages that you are not familiar with. However, it is nothing like actually internalizing a language.
- cruffle_duffle 2y ago
  That’s true. But if you are already an experienced developer who’s been around the block enough to call bullshit when you see it, these LLM thingies can be pretty useful for unfamiliar languages. But you need to be constantly vigilant, ask it the right questions (eg: “is this thing you wrote really best practice for this language? Cause it doesn’t seem that way”), and call bullshit on obvious bullshit… …which sometimes feels like it is more work than just fucking doing it yourself. So yeah. I dunno!
adamc 2y ago
Not sure I get how that would work. It seems to me that to do my job I will have to validate the semantics of the program, and that means I will have to become familiar with the syntax of Go or whatever, at a fairly sophisticated level. If I am glossing over the syntax, I am inevitably glossing over the fine points of how the program works.
cruffle_duffle 2y ago
It depends on the language and the libraries you will use. Python with a well known library? Sure no problem. Almost any model will crank out fairly error free boilerplate to get you started. Terraform? Hah. 4o and even o1 both absolutely sucked at it. You could copy & paste the documentation for a resource provider, examples and all, and it would still produce almost unusable code. Which was not at all helpful given I didn’t know the language or its design patterns and best practices at all. Sonnet 3.5 did significantly better but still required a little hand holding. And while I got my cloud architecture up and running now I question if I followed “best practices” at all. (Note: I don’t really care if I did though… I have other more important parts of my project to work on, like the actual product itself). To me one of the big issues with these LLM’s is they have zero ability to do reflection and explain their “thought process”. And even if they could you cannot trust what it says because it could be spouting off whatever random training data it hovered up or it could be “aligned” to agree with whatever you tell it. And that is the thing about LLM’s. They are remarkably good bullshitters. They’ll say exactly what you want them to and be right just enough that they fool you into thinking they are something more than an incredibly sophisticated next token generator. They are both incredibly overrated and underrated at the same time. And it will take us humans a little while to fully map out what they are actually good at and what they only pretend to be good at.
DanielHB 2y ago
LLMs are a great help with terraform and devops configuration, they often invent things but at least the point at the documentation I need to look up on. Of course everything needs double-checking but just asking the LLM: "how do I do X" will usually at least output all the names of terrraform resources and most configuration attributes I need to look up. They are great for any kind of work that requires "magical incantations" as I like to call them.
vunderba 2y ago
I guess it depends on how you define "proficiency". For me, proficiency implies a fundamental understanding of something. You're not proficient in Spanish if you have to constantly make use of Google Translate. Could code assistants be used to help actually learn a programming language? - Absolutely. Will the majority of people that use an LLM to write a Swift app actually do this? - Probably not, they'll hammer the LLM until it produces code that hobbles along and call it a day. Also, LEARNING is aided by being more active, but relying on an LLM inherently encourages you to adopt a significantly more passive behavior (reading rather than writing).
mgens 2y ago
Yes! Reading some basic documentation on the language or framework, then starting to build in Cursor with AI suggestions works so well. The AI suggests using functions you didn't even know about yet, then you can go read documentation on them to flesh out your knowledge. Learned basic web dev with Django and Tailwind this way and it accelerated the process greatly. Related to the article, this relies on being curious and taking the time to learn any concepts the AI is using, since you can't trust it completely. But it's a wonderfully organic way to learn by doing.
- dogleash 2y ago
  Tell me you're a Cursor advertisement chatbot without telling me you're a Cursor advertisement chatbot.
  - mgens 2y ago
    I'll have you know that I successfully clicked on several traffic lights and motorcycles this week to get into a web site. Didn't even need Cursor!
croes 2y ago
If you don’t know the language well enough AI could lead you down the path of bad practices and you won’t recognize it.
xboxnolifes 2y ago
I agree with this take. AI has essentially served me well as a more useful search engine when working in new languages, libraries, or frameworks.

arikrak 2y ago

AI is a potential silver bullet since it can address the "essential complexity" that Fred Books said regular programming improvements couldn't address. It may not yet have caused an "order of magnitude" improvement in overall software development but it has caused that improvement in certain areas, and that will spread over time. https://en.wikipedia.org/wiki/No_Silver_Bullet https://en.wikipedia.org/wiki/No_Silver_Bullet

snarf21 2y ago

The most valuable thing I want AI to do with regards to coding is to have it write all the unit tests and get me to 100% code coverage. The data variance and combinatorics needed to construct all the meaningful tests is sometimes laborious which means it doesn't get done (us coders are lazy...). That is what AI to do, all the mind numbing draining work so I can focus more on the system.

jyounker 2y ago
Which is also a fantasy, since if you did achieve it, you'd just have tests that verified that all your bugs were in place.
- Ferret7446 2y ago
  Not necessarily. I have used LLMs to write unit tests based on the intent of the code and have it catch bugs. This is for relatively simple cases of course, but there's no reason why this can't scale up in the future. LLMs absolutely can "detect intent" and correct buggy code. e.g., "this code appears to be trying to foo a bar, but it has a bug..."
js8 2y ago
How do you expect AI to write unit tests if it doesn't know the precise desired semantics (specification)? What I personally would like AI to do would be to refactor the program so it would be shorter/clearer, without changing its semantics. Then, I (human) could easily review what it does, whether it conforms to the specification. (For example, rewrite the C program to give exactly the same output, but as a Python code.) In cases where there is a peculiar difference between the desired semantics and real semantics, this would become apparent as additional complexity in the refactored program. For example, there might be a subtle semantic differences between C and Python library functions. If the refactored program would use a custom reimplementation of C function instead of the Python function, it would indicate that the difference matters for the program semantics, and needs to be somehow further specified, or it can be a bug in one of the implementations.
9rx 2y ago

whiplash451 2y ago

At this point, I don't think that (truly) AI-informed people believe that AI will replace engineers. But AI tools will likely bring a deep transformation to the workflow of engineers (in a positive and collaborative way). It may not be tab-tab-tab all the way, but a whole lot more tabs will sneak in.

Workaccount2 2y ago
I don't think it will replace engineers, but I do think it will turn them from $200k/yr golden geese into regular old $90k/yr ducks.
- CuriouslyC 2y ago
  I think you have that backwards (sort of). The high tier programmers who can write things AI can't will be worth more since they'll be more productive, while the programmers below the AI skill floor will see their value drop since they've been commoditized. We already have a bimodal distribution of salaries for programmers between FAANG/not, this will just exacerbate that.
- cruffle_duffle 2y ago
  As somebody who makes extensive use of LLM’s, I very much disagree. Large language models are completely incapable of replacing the kind of stuff you pay a developer $200k for. If anything they make that $200k developer even more of a golden goose.
- nicholassmith 2y ago
  I suspect you're right, but I think it'll follow the COBOL engineer salary cycle, engineers that have a deeper understanding of the whole widget will be in demand when companies remember they need them.
- whiplash451 2y ago
  I think the opposite will happen. Engineers will still be worth $200k/yr. Humankind will just develop a lot more software, faster and further into industries. Availability will create new demand.

orbisvicis 2y ago

If I could sketch out the architecture I wanted as a flow chart annotated with types and structures, implementable by an AI, that would be a revolutionary leap. I design top-down, component by component, and sometimes the parts don't fit together as I envisioned, so I have to write adapters or - worst case - return to the drawing board. If the AI could predict these mismatches, that would also be helpful. Unfortunately, I don't think AI is great with only the accidental tasks either.

adamc 2y ago
Maybe we need better terminology. But AI right now is more like pattern-matching than anything I would label as "understanding", even when it works well.
CuriouslyC 2y ago
AI is really good at goldfish programming. It's incredibly smart within its myopic window, but falls apart as it is asked to look farther. The key is to ask for bite sized things where that myopia doesn't really become a factor. Additionally, you as the user have to consider whether the model has seen seen similar things in the past, as it's really good at regurgitating variations but struggles with novelty.

EGreg 2y ago

AI totally is a silver bullet. If you don't think so, you're just using it wrong and it's your fault. If you think that it takes you just as long or longer to constantly double-check everything it does, then you don't understand the P vs NP problem. </sarcasm> AI will only get better. And AI has proven a lot of unproven maths problems as far back as 2019: https://mathscholar.org/2019/04/google-ai-system-proves-over-1200-mathematical-theorems/ https://mathscholar.org/2019/04/google-ai-system-proves-over...

DanielHB 2y ago

The better tech often lowers the barrier for people to do things but raises the bar of users (and stakeholders for contract projects) expectations. It is plainly visible with web development where the amount of tooling has grown dramatically (both frontend and backend) to do things. Like, for example, all the big-data stuff we do today was unthinkable 10 years ago, today every mid-sized company has a data team. 15 years ago all data in a single monolithic relational database was the norm, all you needed to know was SQL and some Java/C#/PHP and some HTML to get some data wired up into queries.

bloomingkales 2y ago

Same with 4GLs, Visual Coding, CASE tools, even Rails and the rest of the opinionated web tools. How many of those things were envisioned by futurists or great authors? This AI stuff is the stuff of dreams, and I think it’s unwise to consider it another go around the sun.

no_wizard 2y ago
Until it’s actually AI and not Machine Learning masquerading as AI because AI is the sectors marketing pitch, I would strongly hesitate considering it as anything other than a tool. Yes, a powerful tool, and as powerful tools go, they can re-shape how things get done, but a tool none the less and therefore we must consider what its limits are, which is all OP is getting at and the current and known near future state suggests we aren’t evolving passed the tool state
latexr 2y ago
> This AI stuff is the stuff of dreams This AI stuff? No, not really. The stuff of dreams is an AI that you can talk to and interact infinitely and trust that it doesn’t make mistakes. LLMs ain’t it.
- deleted 2y ago
  [deleted]

latexr 2y ago

For reference, to those who want to learn more: https://en.wikipedia.org/wiki/No_Silver_Bullet https://en.wikipedia.org/wiki/No_Silver_Bullet

HarHarVeryFunny 2y ago

> The tools really only help with the accidental tasks I don't think that's really the problem with using LLMs for coding, although it depends on how you define "accidental". I suppose if we take the opposite of "essential" (the core architecture, planned to solve the problem) to be boilerplate (stuff that needs to be done as part of a solution, but doesn't itself really define the solution), then it does apply. It's interesting/amusing that on the surface a coding assistant is one of the things that LLMs appear better suited for, and they are suited for, as far as boilerplate generation goes (essentially automated stack overflow, and similar-project, cut and pasting)... But, in reality, it is one of the things LLMs are LEAST suited for, given that once you move beyond boilerplate/accidental code, the key skills needed for software design/development are reasoning/planning, as well as experienced-based ("inference time") learning to progress at the craft, which are two of the most fundamental shortcomings of LLMs that no amount of scale can fix. So, yeah, maybe they can sometimes generate 70% of the code, but it's the easy/boilerplate 70% of the code, not the 30% that defines the architecture of the solution. Of course it's trendy to call LLMs "AI" at the moment, just as previous GOFAI attempts at AI (e.g. symbolic problem solvers like SOAR, expert systems like CYC) were called "AI" until their limitations became more apparent. You'll know we're one step closer to AI/AGI when LLMs are in the rear view mirror and back to just being called LLMs again!

blueprint 2y ago

What? AI can't prove any unproven math problems...?

hinkley 2y ago

The first lead I worked with inoculated me to this. He taught me about hype trains long before the idea was formalized. He’d been around for the previous AI hype cycle and told me to expect this one to go the same. Which it did, and rather spectacularly. That was three cycles ago now and while I have promised myself I will check out the next cycle, because I actually do feel like maybe next time they’ll build systems that can answer why not just how, this one is a snooze fest I don’t need to get myself involved in.

danielbln 2y ago
Just be careful you don'tet your pendulum swing too much in the other direction, where your turn into an old curmudgeon that doesn't get excited by anything and that thinks nothing is novel or groundbreaking anymore.

banku_brougham 2y ago

The devs get it from what i can tell, management are the slow learners here

jamil7 2y ago

It’s an old one but I think Joel Spolsky‘s take on leaky abstractions is relevant again in this discussion as we add another abstraction layer with LLM assisted coding. https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-abstractions/ https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-a...

louthy 2y ago
Aligned with: “The purpose of abstraction is not to be vague, but to create a new semantic level in which one can be absolutely precise.” — Edsger Dijkstra The abstraction has never been this leaky or vague — Me
- olvy0 2y ago
  That is a great quote, thank you.
- truefossil 2y ago
  Agree. So far "the progress" implied understanding (discovering) previously unknown things. AI is exactly the opposite: "I don't understand how, but it sorta works!"
- rob74 2y ago
  That's the purpose that all abstractions strive to achieve, but most of them fall short - and become leaky...
rossmurphy 2y ago
Joel's blog posts should be hung in the Louvre.

smokel 2y ago

Too bad most of society is accidental as well. With which I mean to say that there are a lot of nonsensical projects being done out there, that still make a living for many people. Modern AI may well change things, similar to how computers changed things previously. I get your sentiment, I've been through a few hype cycles as well, but besides learning that history repeats itself, there is no saying how it will repeat itself.

notRobot 2y ago
> With which I mean to say that there is a lot of nonsensical projects being done out there, that still make a living for many people. I don't know why this is a bad thing. I don't think projects that you believe are nonsensical shouldn't exist just because of your opinion, especially if they're helping people survive in this world. I'm sure the people working on them don't think they're nonsensical.
- jajko 2y ago
  Yeah when you take 2 steps back, ignore IT for a second and look on whole mankind, there are hundreds of millions of jobs that could be called nonsensical from certain points of view. We are not above this in any meaningful ways, maybe its just a bit more obvious to keen eye. Yet society and economy keeps going and nobody apart from some academic discussions really cares. I mean companies have 100% incentive to trim fat to raise income yet they only do the least minimum.
  - chuankl 2y ago
    There is a book about it: "Bullshit Jobs" by the late anthropologist David Graeber.
- lukan 2y ago
  "I'm sure the people working on them don't think they're nonsensical." Rise of antidepressants imply otherwise. The more bullshit we stop doing, the more energy there is for the awesome things we could do instead.

andrewstuart 2y ago

None of those even came within 10% of delivering to the extent that LLMs have.

ulfw 2y ago
Exactly what you said Andrew. The comparisons are lacking and are almost at whataboutism level. The amount of actual 'work' that AI does versus the tools of yesterday are an order of magnitude away
- kruxigt 2y ago
  [dead]
- smokel 2y ago
  Going from punched cards to interactive terminals surely must have been a big productivity boost. And going from text based CAD to what is possible on modern workstations has probably also helped a bit in that field. In that view I'd say the productivity boost by LLMs is somewhat disappointing, especially with respect to how amazing they are.
- player1234 2y ago
  Quantify it, show us the numbers.
  - NitpickLawyer 2y ago
    I think the field is too new and the successful stories too private atm. However I think the best apples to apples example in this context is Amz's codebase update project that they've blogged about. From memory, they took some old java projects, and had some LLM driven "agents" update the codebase to recent java. I don't know java enough to know how "hard" this task is, but asking around I've heard that "analog" tools for this exist, but aren't that good, bork often, are hardcoded and so on. Amz reported ~70% of code that came out passed code review, presumably the rest had to be tweaked by humans. I don't know if there are any "classical" tools that can do that ootb. So yeah, that's already imrpessive and "available today" so to speak.

Dalewyn 2y ago

>AI is a fabulous tool that is way more flexible than previous attempts because I can just talk to it in English In an era when UIs become ever more Hieroglyphic(tm), Aesthetical(tm), and Nouveau(tm), "AI" revolutionizing and redefining the whole concept of interacting with computers as "Just speak Human." is a wild breath of fresh air.

skydhash 2y ago
Programming and interacting with computers in general is just translation to a more restricted and precise language. And that what's make them more efficient. Speaking human is just going the other way and losing productivity. It's akin to how everyone can build a shelter, but building a house requires a more specialized knowledge. The cost of the later is training time to understand stuff. The cost of programming is also training time to understand how stuff works and how to manipulate them.
- Dalewyn 2y ago
  An inefficient computer you can use is more productive than an efficient computer you can't use. Most people can't use mice or keyboards with speed, touchscreens are marginally better except all the "gestures" are unnatural as hell, and programming is pig latin. Mice and keyboards and programming languages and all the esoteric ways of communicating with computers came about simply because we couldn't just talk Human to them. Democratizing access to computers is a very good and very productive thing.
  - skydhash 2y ago
    That's the thing. You don't communicate with computers. You use them. You have a task to do that the computer have been programmed for and what you want is to get the parameters of that tasks to the computer. And you learn how to use the computer because the tasks is worth it, just like you learn how to play a game because you enjoy the time doing it. The task supersedes the tool. Generative AI can be thought as an interface to the tool, but it's been proven that they are unreliable. And as the article outlines, if it can get to 70% of the task, but you don't have the knowledge requires to complete it, that's pretty much the same as 0%. And if you have the knowledge, more often than not you realize that it just go faster on a zigzag instead of the straight route you would have taken with more conventional tools.

koliber 2y ago

As the adage goes: If you aim for the stars and fail, you'll end up on the moon.

steveBK123 2y ago
Kind of the opposite lesson of Icarus?
rsynnott 2y ago
Other options are available, for instance ploughing into a village because your second stage didn't light, or, well, this: https://youtu.be/mTmb3Cqb2qw?t=16 https://youtu.be/mTmb3Cqb2qw?t=16 Most of the "you'll never need programmers again!" things have ended up more "cars-showered-with-chunks-of-flaming-HTPB" than "accidentally-land-on-moon", tbh. 4GLs had an anomaly, and now we don't talk about them anymore. (It's a terrible adage, really. "Oops, the obviously impossible thing didn't happen, but an unrelated good thing did" just doesn't happen that often, and when it does there's rarely a causal relation between A and B.)
cwillu 2y ago
If you aim for the stars and fail, you end up going in circles for all eternity.
- CapeTheory 2y ago
  If you aim for the stars and fail, others can remember your folly by the smoking crater you leave behind.
  - christophilus 2y ago
    Not if you fail by running out of fuel and smashing into Europa.
  - mrjin 2y ago
    Hmmm, for 3 minutes?

Izkata 2y ago

> Even though Fred Brooks explained why in 1986. There are essential tasks and there are accidental tasks. The tools really only help with the accidental tasks. I don't know this reference, so I have to ask: Was "accidental" supposed to be "incidental"? Because I don't see how "accidental" makes any sense.

dkdbejwi383 2y ago
"Accident" in the same sense as "accident of birth" not "traffic accident".
bluGill 2y ago
Buy and read the book. There is a reason the 25th aniversery eddition has been still in print for more than 30 years. It is a timeless combuter book that everyone should read and keep an their bookshelf.
racingmars 2y ago
The Mythical Man-Month, by Fred Brooks. Chapter 16 is named "No Silver Bullet—Essence and Accident in Software Engineering." I'll type out the beginning of the abstract at the beginning of the chapter here: "All software construction involves essential tasks, the fashioning of the complex conceptual structures that compose the abstract software entity, and accidental tasks, the representation of these abstract entities in programming languages and the mapping of these onto machine languages within space and speed constraints. Most of the big past gains in software productivity have come from removing artificial barriers that have made the accidental tasks inordinately hard, such as severe hardware constraints, awkward programming languages, lack of machine time. How much of what software engineers now do is still devoted to the accidental, as opposed to the essential? Unless it is more than 9/10 of all effort, shrinking all the accidental activities to zero time will not give an order of magnitude improvement."
- Izkata 2y ago
  From the abstract that definitely sounds like he meant "incidental": Something that's a necessary consequence of previous work and / or the necessary but simpler part of the work.

lxdlam 2y ago

Totally agree. The software development is absolutely a fractal. In 1960s we were solving the complexity by using high level language that compile to machine code to enable more people write simple code. This has happened again and again and again. But different generations face different problems, which requires another level of thinking, abstraction, and push both boundaries until we reach the next generation. All of this is not solved by a single solution, but the combination based on basic principles that never changes, and these things, at least for now, only human can do.

uludag 2y ago
> The software development is absolutely a fractal. I think this analogy is more apt than you may realize. Just like a fractal, the iterated patterns get repeated on a much smaller scale. The jump to higher-level languages was probably a greater leap then the the rest of software innovation will provide. And with each iterative gain we approach some asymptote, but never get there. And this frustration of never reaching our desired outcome results in ever louder hype cycles.
dkrich 2y ago
Interestingly it seems like we are investing many more magnitudes of capital for smaller and smaller gains. For example, the jump in productivity from adding an operating system to a computer is orders of magnitude larger than adding an LLM to a web development process despite the LLM requiring infrastructure that cost tens of billions to create. It seems that while tools are getting more and more sophisticated, they aren’t really resulting in much greater productivity. It all still seems to be resulting in software that solves the same problems as before. Whereas when html came around it opened up use cases that has never been seen before despite being a very simple abstraction layer by today’s standards. Perhaps the opportunities are greatest when you are abstracting the layer that the fewest understand when LLMs seem to assume the opposite.
- mattgreenrocks 2y ago
  The real gains in software are still to be had by aggressively destroying incidental complexity. Most of the gunk in a web app doesn't absolutely need to exist, but we write it anyway. (Look at fasthtml for an alternate vision of building web apps.) The issue with LLMs is they enshrine the status quo. I don't want ossified crappy software that's hard to work with. Frameworks and libraries should have to fight to justify their existence in the marketplace of ideas. Subverting this mechanism is how you ruin software construction.

GistNoesis 2y ago

>But it can’t do the essential work of complexity management I think this is the "closing the loop" ( https://en.wikipedia.org/wiki/Control_loop#Open-loop_and_closed-loop https://en.wikipedia.org/wiki/Control_loop#Open-loop_and_clo... ) moment for coding AI. All pieces are there, we just need to decide to do it. Today's AI are able to produce an increasing tangled mess of code. But it's also able to reorganize the code. It's also capable of writing test code, and assess the quality of the code. It's also capable to make architectural decision. Today's AI code, is more like a Frankenstein's composition. But with the right prompt OODA loop and quality assessment rigor, it boils down to just having to sort and clean the junk pile faster than you produce it. Once you have a coherent unified codebase, things get fast quickly, capabilities grows exponentially with the number of lines of code. Think of things like Julia Language or Wolfram Language. Once you have a well written library or package, you are more than 95% there and you almost don't need AI to do the things you want to do.

qazpot 2y ago
> I think this is the "closing the loop" ( https://en.wikipedia.org/wiki/Control_loop#Open-loop_and_clo https://en.wikipedia.org/wiki/Control_loop#Open-loop_and_clo... ) moment for coding AI. > All pieces are there, we just need to decide to do it. Another silver bullet.
- GistNoesis 2y ago
  There is a huge gap in performance and reliability in control systems between open-loop and closed-loop. You've got to bite the bullet at one point and make the transition from open-loop to closed-loop. There is a compute cost associated to it, and there is also a tuning cost, so it's not all silver lining.
rafaelmn 2y ago
>Once you have a coherent unified codebase, things get fast quickly, capabilities grows exponentially with the number of lines of code. Think of things like Julia Language or Wolfram Language. >Once you have a well written library or package, you are more than 95% there and you almost don't need AI to do the things you want to do. That's an idealistic view. Packages are leaky abstractions that make assumptions for you. Even stuff like base language libraries - there are plenty of scenarios where people avoid them - they work for 9x% of cases but there are cases where they don't - and this is the most fundamental primitive in a language. Even languages are leaky abstractions with their own assumptions and implications. And these are the abstractions we had decades of experience writing, across the entire industry, and for fairly fundamental stuff. Expecting that level of quality in higher level layers is just not realistic. I mean just go look at ERP software (vomit warning) - and that industry is worth billions.

dkrich 2y ago

This seems to apply to all areas of AI in its current form and in my experience 70% may be a bit generous. AI is great at getting you started or setting up scaffolds that are common to all tasks of a similar kind. Essentially anything with an identifiable pattern. It’s yet another abstraction layer sitting on an abstraction layer. I suspect this is the reason we are really only seeing AI agents being used in call centers, essentially providing stand ins for chatbots- because chatbots are designed to automate highly repetitive, predictable tasks like changing an address or initiating a dispute. But for things like “I have a question about why I was charged $24.38 on my last statement” you will still be escalated to an agent because inquiries like that require a human to investigate and interpret an unpredictable pattern. But creative tasks are designed to model the real world which is inherently analog and ever changing and closing that gap of identifying what’s missing between what you have and the real world and coming up with creative solutions is what humans excel at. Self driving, writing emails, generating applications- AI gets you a decent starting point. It doesn’t solve problems fully, even with extensive training. Being able to fill that gap is true AI imo and probably still quite a ways off.

jeremyjh 2y ago
Yeah, its more like it can generate 70% of the code by volume, rather than get you 70% of the way to a complete solution. 12 week projects don't become 4 week projects, at best they are 9-10 week projects.
adamc 2y ago
Great analysis, and I agree it's Fred Brooks' point all over again. None of these tools hurt, but you still need to comprehend the problem domain and the tools -- not least because you have to validate proposed solutions -- and AI cannot (yet) do that for you. In my experience, generating code is a relatively small part of the process.
marcosdumay 2y ago
> because chatbots are designed to automate highly repetitive, predictable tasks like changing an address or initiating a dispute You know what is even cheaper, more scalable, more efficient, and more user-friendly than a chatbot for those use cases? A run of the mill form on a web page. Oh, and it's also more reliable.

singularity2001 2y ago

>> for the same reason it can’t prove an unproven maths problem that aged like LeCun