7 ms·

> it is sad to me just how much people are trying to automate away programming and delegating it to a black box I take it you're not using a compiler to genera

• jodhpurcapital 2y ago
  Nailed it.
• csallen 2y ago
  Thank you for saying this. It's always baffled me that people will decry ChangeX as unnatural and wrong when it happens in their lifetime, but happily build their lives upon NearlyIdenticalChangeY so long as it came before them.
  • pottspotts 2y ago
    We think of ourselves more intelligent than the generation before us, and more wise than the ones after.
    • danielbln 2y ago
      Makes you wonder if that sort of dynamic between generations is ever going to be something that can be overcome. Maybe if humanity cures aging.
  • JimDabell 2y ago
    “I've come up with a set of rules that describe our reactions to technologies: 1. Anything that is in the world when you’re born is normal and ordinary and is just a natural part of the way the world works. 2. Anything that's invented between when you’re fifteen and thirty-five is new and exciting and revolutionary and you can probably get a career in it. 3. Anything invented after you're thirty-five is against the natural order of things.” ― Douglas Adams, The Salmon of Doubt: Hitchhiking the Galaxy One Last Time
    • stavros 2y ago
      This is so accurate I'm offended.
• eternauta3k 2y ago
  There is a difference between those abstraction levels: with LLMs you are moving into the realm of management.
  • mypalmike 2y ago
    That was precisely the goal of many high level languages.
    • layer8 2y ago
      …which all failed for that reason.
• rstat1 2y ago
  Saying a compiler, and an "AI" hallucination model are even remotely the same is a pretty huge stretch.
  • trueismywork 2y ago
    Well, AI models don't have to hallucinate.
    • thaumasiotes 2y ago
      True, but LLMs do.
    • KeplerBoy 2y ago
      All current LLMs do though.
  • disconcision 2y ago
    how do you figure? people were making directly analogous arguments about compilers back in the day. (not trying to argue that they are 'the same', but their is definitely a spectrum of code generation methods, with widely varying genres of guarantees, suiting a widely varying range of use cases)
  • lovethevoid 2y ago
    They aren't saying they're the same, I'm not sure how you got that interpretation. It's very clear they're highlighting the hypocrisy that arises from claiming to be against automating away aspects of programming while relying on tools that do exactly that for you - only being Ok with it as long as they aren't called "AI".
    • advael 2y ago
      The crux of why this is a bad analogy is that everyone talking about "automating" things with LLMs is misusing the word "automation". A machine can automate a repetitive manual task. A computer can automate the operation of machinery. A machine instruction set is an abstraction on top of circuitry that can automate the labor of extrapolating the logic physically executed by that circuitry into human-comprehensible routines. In the same way, a programming language implementation (e.g. a compiler) can somewhat automate programming (in the sense that it uses higher levels of abstraction to describe the same thing, saving labor while keeping determinism). What do these things have in common? We can reliably make them approach deterministic behavior. In the case of compilers, completely and reliably and transparently so. Just because you haven't bothered to read what a compiler is doing doesn't mean someone can't verify what it's doing. Physical machines are less reliable, but we have reliable ways to test them, reliable error margins, reliable failure modes, reliable variance. When you are on a stack of abstractions like a programming language on top of a compiler on top of transistors on top of a machine, an error at the top of that stack can have a lot of implications. A tool that probabilistically generates code is not automation. We have no guarantees about how and when it will get things wrong, how much this will happen, and what kinds of things it will get wrong. We have no way to audit their results that will generalize to every problem upstream of them. We have no way to reliably measure improvement in consistency, let alone improve that margin of error reliably. The entire idea that this is an automation at all is nonsense.
• habosa 2y ago
  While the base of your argument is true, it’s also a bit dishonest. LLMs are significantly different than any of these other abstractions because they can’t be reasoned about or meaningfully examined/debugged. They’re also the first of these advances which anyone has claimed would eliminate the need for programmers at all. I don’t believe the C compiler was meant to do my whole job for me.
  • spencerchubb 2y ago
    I don't buy that you're actually examining compiled programs. Very few people do. Theoretically you could, but the whole point of the compiler is to find optimizations that you wouldn't think of yourself.
    • bena 2y ago
      No, the point of the compiler is to translate code into machine instruction. Yes, it can optimize things for you, but that is not its purpose.
      • spencerchubb 2y ago
        Okay I'll revise my statement A big feature of compilers is to find optimizations you wouldn't think of. I tried to make the point that compiled output is typically not read by humans
    • kibwen 2y ago
      The point of an optimizing compiler is to find optimizations which, crucially, are semantics-preserving. This is the contract that we have with compilers, is the reason that we trust them to transform our code, and is the reason why people get up in arms every time some C compiler starts leveraging undefined behavior in new and exciting ways. We have no such contract with LLMs. The comparison to compilers is highly mistaken, and feels like how the cryptocurrency folks used to compare cryptocurrency to gestures vaguely "the internet" in an attempt to appropriate legitimacy.
    • Jensson 2y ago
• minkles 2y ago
  Um those black boxes have determinism.
  • kqr 2y ago
    Not at all. See the cries for reproducible builds, exploits like Spectre etc.
    • fire_lake 2y ago
      You are precisely right but practically wrong here
      • deleted 2y ago
        [deleted]
• pajeets 2y ago
  We can argue the same thing for artists. Who wrote the algorithms for your favorite Photo/Image editor? Who created the image formats and standards, infrastructures for you to be able to push binary files to millions of people?
• Palomides 2y ago
  there's a strict contract between the programmer and the compiler (the language spec)
  • MarkusQ 2y ago
    Which would be relevant if either side respected it. In practice, compilers frequently have bugs and programmers even more frequently make use of "what the compiler actually does" rather than adhering to the language specification -- to the point where the de facto spec for many languages is "what the canonical implementation does". And the specifications change over time.
    • kibwen 2y ago
      The frequency with which compiler implementations functionally diverge from language specifications is dwarfed, by many orders of magnitude, by the frequency with which LLMs generate provably nonsensical code in response to a prompt. To wit, a compiler diverging from the specification is so relatively rare that people will get angry about it and demand that it be fixed, while an LLM spewing creative nonsense is so accepted and par for the course that complaining about that fact is met with a shrug and "well, what did you expect?"
    • layer8 2y ago
      LLMs don’t have a canonical output that could serve as a specification. And if they had, we wouldn’t consider that a satisfactory specification at all.
  • deleted 2y ago
    [deleted]
• milemi 2y ago
  You can learn how compilers work and understand how they do what they do. Nobody understands what’s in those billions of parameters, and no one ever will.
  • fuzztester 2y ago
    why are there so many parameters in the first place? and was it humans who generated so many? seems like a very big job for a human to do, or even a team of humans to do. disclaimer: I know next to nothing about llms. and I'm not that interested to learn about them. just asking casually.
    • jdiff 2y ago
      They're not manually generated or anything, it's just a setting. Too few and the model doesn't have enough flexibility to capture complex patterns. Too many and the model can just memorize the data you train it on rather than capturing the patterns driving it.
    • Jensson 2y ago
      > why are there so many parameters in the first place? Because parsing and writing human language in a natural way is extremely complex. > and was it humans who generated so many? No, it is generated using an algorithm that tries to predict the next word in human written text using the words that comes before it. It ingests basically all the text on the internet to do this, without that much text the LLM performs horribly.
• throwaway22032 2y ago
  I don't think that this is a fair comparison because at some point the nature of the craft actually does change. To give an analogy, a carpenter might be happy with hand tools, happy with machine tools, happy with plywood, and happy with MDF. For routine jobs they may be happy to buy pre-fabbed cabinets. But for them to employ an apprentice (AI in this example) and outsource work to them - suddenly they are no longer really acting as a carpenter, but a kind of project manager. edit: I agree that LLMs in their current state don't really fundamentally change the game - the point I am trying to make is that it's completely understandable that everyone has their own "stop" point. Otherwise, we'd all live in IKEA mansions.
  • kqr 2y ago
    Running state of the art LLMs for programming is nowhere near project management. At least in my experience, all LLMs are really good at is dumping plausible tokens quickly. They can't think, design, or handle tradeoffs intelligently. They help me with the keyboard work, not any of the actual programming.
  • jclulow 2y ago
    An apprentice is another person performing the same kind of work as the carpenter. That's fundamentally different from using an LLM, which is not a person and does not function like a person. Whether you think LLMs are spectacularly worthwhile or odious and destructive, it's crucial not to classify them as being a person instead of a software tool.
    • fuzztester 2y ago
      yep, I would call this the anthropomorphisation of llms. undesirable, just as any other kind of anthropomorphisation is.
• kif 2y ago
  This is a very poor analogy. It's not a matter of abstractions, it's a matter of getting someone or something else to do the work, while you mostly watch and fix any errors you're able to catch.
• discreteevent 2y ago
  HN commenter: Samuel, why don't you use an LLM to write this play you are working on? Beckett: What? HN commenter: Well it's just like when you decided to work in French instead of English. Your art was no less because of it. Now you can use an LLM instead of French. It will be so much quicker.
• YeGoblynQueenne 2y ago
  >> I take it you're not using a compiler to generate machine code, then? The dismissive glibness of your comment makes me wonder if it's worth it trying to point out the obvious error in the analogy you're making. Compilers translate, LLMs generate. They are two completely different things. When you write a program in a high-level language and pass it to a compiler, the compiler translates your program to machine code, yes. But when you prompt an LLM to generate code, what are you translating? You can pretend that you are "translating natural language to code" but LLMs are not translators, they're generators, and what you're really doing is providing a prefix for the generated string. You can generate strings form an LLM with an empty prefix; but try asking a compiler to compile an empty program. >> Even the Apollo spacecraft programmers at MIT had a black box: they offloaded the weaving of core rope memory to other people. You're referring to core rope memory: https://en.wikipedia.org/wiki/Core_rope_memory https://en.wikipedia.org/wiki/Core_rope_memory There is no "black box" here. Programmers created the program and handed it over to others to code it up. That's like hiring someone to type your code for you at a keyboard, following your instructions to do so. You have to stretch things very far to see this as anything like compilation. Also, really, compilers are not black boxes. Just because most people treat them as a scary unknowable thing doesn't mean that's what they are. LLms are "black boxes" because no matter how much we peer at their weights, arrays of numerical values, there's nothing we can ... er ... glean from them. They're incomprehensible to humans. Not so the code of a compiler. Even raw binary is comprehensible, with some experience.
  • CJefferson 2y ago
    I recently had used an LLM to convert a lot of Python to Rust. It got it 99% right, and it took me a short while to fix the compile-time errors, and carefully check the tests weren't broken (as I trusted the code worked when the tests passed). Is that "compiling" or "translating"? Lots of people use language to C "compilers".
    • shepherdjerred 2y ago
      Compilation is generally deterministic with strict semantics. LLMs are great, but they are the opposite of a compiler (in a good way)
• elric 2y ago
  I get what you're trying to say, but I don't entirely agree. Raising levels of abstraction is generally a good thing. But up until now, those have mostly been deterministic. We can be mostly confident that the compiler will generate correct machine code based on correct source code. We can be mostly confident that the magnetised needle does the right thing. I don't think this is true for LLMs. Their output is not deterministic (up for discussion). Their weights and the sources thereof are mostly unknown to us. We cannot really be confident that an LLM will produce correct output based on correct input.
  • computerex 2y ago
    So what if the output is stochastic? LLM's have self consistency, so you can repeat the inference several times and pick the most frequent output.
    • saulpw 2y ago
      And you can pay in time and/or $$ for the privilege of having to do do this extra unnecessary work.
    • dazilcher 2y ago
      Most frequent output does not imply correctness, LLMs often are confidently wrong. They can't even perform basic arithmetic (which is not surprising since they operate at the syntactic level, oblivious to any semantic rules), yet people seem to think offloading more complex tasks with strict correctness requirements is a good idea. Boggles the mind tbh.
  • quaintdev 2y ago
    What if the we can't do abstraction anymore. I mean we certainly can but we will loose ability to configure tiny details of system. So other path forward could very well be LLMs as they can save lot of time with writing boilerplate code
  • krageon 2y ago
    For the average programmer the infinite layers of abstraction, libraries and middleware isn't deterministic either. The LLMs actually honest to god being probabilistic estimators doesn't change anything about what they produce or how they see their own stuff.
• rodrigosetti 2y ago
  This is a qualitatively different kind of abstraction. All other abstractions still require the programmer to express the solution in a formal language, while LLMs are allowing the user to express the solution in natural language. It's no longer programming, but much more like talking to a programmer as a manager.
  • deleted 2y ago
    [deleted]
• freetanga 2y ago
  By that light, the artist that is painting using store-bought pigments instead of hand made, or brushing paints instead of reallocating molecules, or even better, atoms, is also using a black box. I think OP has a point, and is about guiding the design, then overall structure and dynamic of the code. Nobody expects to write in Assembler or not using libraries, but making a concious decision on the design. I have met few programmers, but many coders. For them coding is a job, and generally they don’t care about overall architecture, algorithm efficiency, and code elegance is limited to syntax-coloring-themes in their editor. I respect their existence, but generally they are building on top off somobody else’s effort.
• lagrange77 2y ago
  I guess we are currently in the special situation, that we as human programmers can understand the output of a coding LLM. That's because programming languages are designed to be human readable. And we had an incentive to learn those languages. I imagine that machine learning powered coding will evolve to an even blacker box, than it is today: It will transform requirements to CPU instructions (or GPU instructions, netlists, ...). Why bother to follow those indirections, that are just convenience layers for those weak carbon units (urgh)? Simultaneously, automation will likely lead to fewer skilled programmers in the future, because there will be fewer incentives to become one. Together those effects could lead to a situation where we are condemned to just watch.
  • kalind 2y ago
    So what do you do when the LLM creates a bug in a multi-million CPU instruction program it generated and you can't prompt it to fix the bug?
    • lagrange77 2y ago
      Yeah, good question. You could ask the same question for today's or near future coding LLMs used by non programmers.
• amelius 2y ago
  >> delegating it to a black box > I take it you're not using a compiler to generate machine code, then? An LLM is much, much closer to a "black box" than a compiler ... In fact, an LLM is pretty much a black box even to the people who created it.
• toprerules 2y ago
  Your argument is a false dichotomy and thus a logical fallacy. Not all steps forward in abstraction or code generation are necessary good steps and have to be considered on their own merits. If LLMs are indeed superior, then you should be able to articulate their merits without condescending to fallacious attacks.
• mgaunard 2y ago
  The transformations you're referring to are fully deterministic and guaranteed to be correct. LLMs provide statistically probable answers with no guarantee of correctness. It takes more time to review LLM code than it takes to write it correctly from scratch.
• tpoacher 2y ago
  I 100% agree with you. Whenever I see artists buy their brushes I cringe. Real artists don't draw anything until they've grown a tree and raised horses to obtain the raw materials (wood and horse hair) to make their first brush. Using a bought brush to paint and generating a painting via a prompt are basically identical. /s
• bayindirh 2y ago
  The thing is, all these stacks are built by people and verified against specifications. When they failed to perform the way they should, we fixed them. Plus, all the parts are deterministic in this stack. Their behavior is fixed for a given input, and all the parts are interpretable, readable, verifiable and observable. LLMs are none of that. They are stochastic probability machines, which are nondeterministic. We can't guarantee their output's correctness, and we can't fix them to guarantee correct output. They are built on tons of (unethically sourced) data, which has no correctness and quality guarantees. Some people will love LLMs, and/or see programming as a task/burden they have to complete. Some of us love programming for the sake of it, and earn money by doing it that way, too. So putting LLMs to the same bucket with a deterministic, task specific programming tool is both wrong, and disservice to both. I'm also strongly against LLMs, not because of the tech, but because of how they are trained and how their shortcomings are hid and they're put forward as the "oh the savior of the woeful masses, and the silver bullet of all thy problems", and it's neither of them. LLMs are just glorified tech demos which shows what stochastic parrots can pose as accomplishing when you feed the whole world to them.
  • red75prime 2y ago
    > We can't guarantee their output's correctness We can (for programming, at least): run the output thru theorem prover, ensure that proof is constructive, the Curry-Howard correspondence guarantees that you can turn the output into a correct program. It doesn't guarantee that formal properties of the program correspond to the informal problem statement. But even people occasionally make such errors (a provably correct program doesn't do what we wanted it to do). > and we can't fix them to guarantee correct output Same thing with other systems capable of programming. That is people. You just can't make a system that guarantees correct transformation from an informal problem statement into a formally correct implementation. "Informal" implies that there's wiggle room for interpretation. No, it doesn't mean that current LLMs are ready to replace programmers, it also doesn't mean that ML models of 2030s will not be able to.
    • bamboozled 2y ago
      This doesn’t mean the LLM is deterministic. It means it can spoof the right answer give enough time.
• carapace 2y ago
  This is completely wrong. Compilers are complex programs fraught with bugs. Modern microprocessors are hideously complex devices fraught with bugs. But at least we understand them in principle and practice. LLMs are nonsense generators, you need a second device that can recognize correct programs to use them effectively. Only humans can do that all-important second part. > Programming has always been about finding the next black box that is both powerful and flexible. That's the opposite of programming. Programming is the art and science of developing reliable algorithms. You can treat programs as black boxes only after you're sure that they work correctly. Otherwise you're just engaged in a kind of cargo cult. That the scary thing about the LLM fad: so many people seem so willing to abdicate their responsibility to actually think.
• w0m 2y ago
  Reminds me long ago of working with a guy that did everything in C when we were rewriting things in Perl . Yes, his stuff was faster. Yes, it was also buggier, harder to debug, and took 3x as long to write for similar levels of functionality (it wasn't speed dependent code by any stretch.)
• sifar 2y ago
  The question is about the abstraction being understandable and predictable. All the examples you have follow that, LLMs throw that out of the window. >> Scratch that, I guess you're not using a modern microprocessor to generate microcode from a higher-level instruction set either? Hell, I design gate level logic -> map it to instructions -> use them in C for the very LLMs and can fully understand[0] every aspect of it (if it doesn't behave as expected, that is a bug) but I cannot fathom or predict how the LLMs behave when i use them even though I know their architecture and implementation. [0] Admittedly I treat the tools I use during the process, like cad tools, compiler, as black boxes, however I know that if I want to or the need arises, I can debug/understand them.
• deleted 2y ago
  [deleted]
• archagon 2y ago
  I think it would be a mistake to classify an LLM as a level of abstraction. You cannot use it as a stable black-box foundation to build on top of.
• limit499karma 2y ago
  > black box Actually the opacity of the abstraction layer is the core of the issue. First we note that opacity is a measure in both the inner-workings of the 'box' and orthogonally (in context of LLMs) a measure of deterministic outcome. Programming, it is asserted by some of us, is exactly the act of instructing a deterministic 'black' box. Peer-coding with an LLM is the act of cajoling a mechanism to hopefully consistently produce the input to a sensible "blackbox". It is not programming, it is getting help. Now if the help was 100% reliable, we could discuss programming the helper. The other day I had a vision of the future AI whisperer in the corporate setting. They wear capes of varied colors and possibly sport a wand. "It's an art you see".