12 ms·

If the hardest part of programming is reading, understanding, and debugging other people’s code, our jobs are about to get a lot harder with a bunch of AIs runn

• orangepurple 4y ago
  Nearly nobody who wants to use these neural nets knows how to do model risk management. These things are black boxes with edge cases that are badly understood and nearly impossible to inspect and test. At least code is logical and human readable (usually).
  • pyrolistical 4y ago
    Hard to inspect but very possible test. Correctness will go from a binary pass/fail to a probability
    • danrocks 4y ago
      > Correctness will go from a binary pass/fail to a probability Excellent point. The pervasiveness of neural nets will require engineers (and really, everyone) to start thinking more probabilistically and establish acceptability thresholds instead of certainty. It's the way of the future.
      • marginalia_nu 4y ago
        That seems like a regression in many regards, especially given how much of this seems like a solution looking for a problem it can solve rather than the other way around.
        • guhidalg 4y ago
          Agreed. We may imagine ourselves very smart but in my experience most people do not understand probability and statistics well. Is 90% accuracy good enough? Is 95%? 99%? 99.9%? No matter the answer, you have to tolerate errors. Now your stakeholders have to tolerate errors. Are they going to accept errors just because "Software 2.0" is here and that's what we all have to live with? Nope.
          • staunton 4y ago
            All software is riddled with errors and for most purposes that's fine. Any developer, stakeholder, whatever, who thinks otherwise is living in a parallel universe. It may be possible in the future to have "for all practical purposes flawless" software, which might make sense for select special applications. That would be a new thing though, rather than something we have and could lose due to adopting AI development.
            • marginalia_nu 4y ago
              Formal verification of software has been around for a while though.
              • staunton 4y ago
                Which has almost never been used to build software anyone ever used... That might be changing very slowly
            • rco8786 4y ago
              > All software is riddled with errors and for most purposes that's fine. This is typically not that true when it comes to correctness. Most software does the correct thing in the eyes of the user, nearly 100% of the time. And when it doesn't, the bug gets fixed and that edge case is corrected for every other user going forward. AI generated software, from what I've seen, has a wide range of errors in correctness (along with all the other errors that you mentioned all software having..which is true). Like it literally just does the wrong thing given what the user is expecting it to do. The path toward iteratively improving and getting it to an acceptable level of correctness for any given application might be there, but so far I have not seen it.
              • staunton 4y ago
                That's an interesting difference but I'm not convinced it's valid. Perhaps you can explain what you mean in more detail? Let's say the software is good enough if it does the right thing 99.9% of the time it is used. I take it, you're saying that if an AI starts modifying it and only writes correct code in 99.9% of cases (yes, current AI is not even close, but it will improve), that makes it worse because the software might start failing completely. However, if you have proper tests and release management, such obvious flaws will quickly be detected and fixed or rolled back. For most applications that seems pretty much equivalent to what we have now. The other case is software that is completely AI generated and cannot reasonably be modified by humans anymore. In that case, again, you have tests and a sane deployment strategy that mitigates failures to a sufficient degree depending on application. So the only issue is when you start making a completely AI generated software and fail to ever meet requirements, or pass the human written test cases? Users at least will never be impacted by that. Even now, many human-written software projects never get to the stage where they can be used. Is this really a problem, especially if the attempt at AI generation of software is cheap?
        • davedx 4y ago
          Less of a regression than you might think. How many software systems (outside of very well unit tested components) really have strong correctness proofs? It’s an almost futile task in modern software engineering with all those distributed systems everywhere
        • xp84 4y ago
          I'd argue that it's a solution to the problem that programmers are expensive and only a fraction can be counted on to produce 100% reliable code anyway. While it's possible for a highly-skilled, highly-professional developer can both write code that will solve a given problem 100% correctly and write tests that will prove that it solves them for the entire domain, in practice most developers fall short on both counts. Every time you interact with a date or phone number field that chastises you for your use or non-use of punctuation, you know this. So, for many use cases, it's possible imperfect programmers will be replaced with neural networks that are 95% accurate, perhaps with a differently-trained one checking the work of the first one.
        • __MatrixMan__ 4y ago
          I've been looking for an opportunity to try out a pattern where the two approaches improve each other: 1. Generate a 95% accurate model 2. Use it to generate test cases 3. Code the thing, with the help of the cases 4. Manually remove cases in the 5% I image steps 1 and 2 being completed by a product owner and 3 and 4 being completed by a software engineer. We're so horrifically bad at communicating a requirement's intent, I wonder what would happen if we tried to use AI to communicate them via their extent instead.
          • lifeisstillgood 4y ago
            That's probably genius :-) I mean ... can it be done? - build a platform (ie the data we care about and are going to build some workflow over) - have business describe what should happen in english - How does GPT build something that will run? Can it create the infrastructure? does it speak AWS? - then ... OK - I am actually excited by that
            • __MatrixMan__ 4y ago
              I'm not familiar enough with AI workflows to weigh in on how easy or hard it would be. I've just noticed that it's much easier to describe why something is not what you want than describing what you want. So if AI can give us a mockup that's workable enough to skip the first few iterations of "no that's not what I want" and get right to the part where the engineer is asking questions about the edge cases that weren't explicit in the requirements... That's a win. I imagine you'd still want to have it in a box of some sort re: creating infra. Like you give it a very small cluster and probably make the stack decisions "write me a postgres schema for... write a fastapi API for the schema... write a react UI for the API... write me a k8s operator that up/down's the above components... Workshop the idea with other product people... ...and only then involve the engineer like: "make this AI-generated house of cards into a fortress".
        • agileAlligator 4y ago
          It seems to me that current AI techniques are more about reducing programmer workload than actually doing something that isn't possible with traditional methods.
      • islon 4y ago
        Our software has a 99% chance to calculate you taxes correctly! And only a 1% chance of committing tax fraud.
        • kklisura 4y ago
          Not quite, let me rephrase it... > Our software has a 99% chance to calculate your taxes correctly! And only a 1% chance of failure in which it's your fault and it's you that's committing tax fraud
          • gtirloni 4y ago
            Considering ChatGPT spits out innacurate information all the time, I think "our software has a 99% chance of you going to jail for tax fraud" is more accurate.
            • bryanrasmussen 4y ago
              the problem really is that ChatGPT writes things based on what is most likely given what it wrote before and what the input is, so what are the possible error scenarios: 1. making a mistake in this part is really common, ChatGPT makes common mistake. 2. you have uncommon situation affecting here, ChatGPT ignores and writes things that cause you to get in trouble, or it writes things that cause you to pay more than you should. Also the longer is goes on writing things the more likely that things it writes does not hang together with the past things it wrote, when a human lies they try to make their lies at least follow a sensible pattern. ChatGPT would be likely to get you flagged for audits because you can't be sure that what it wrote on page 1 jibes with what it writes on page 3.
          • alfor 4y ago
            Better than most accountants. Family member got ripped by a government audit of what was supposed all fine by the person doing his company taxes.
        • sebzim4500 4y ago
          I wonder what accuracy existing tax calculators have.
      • fock 4y ago
        that doesn't look to bright if we look back towards Covid and vaccines. People are even bad at plain frequentist statistics!
        • duderific 4y ago
          That assumes people even care what the statistics are. Many people just go with their emotional response based on a political outlook when deciding what course of action to take.
      • goatlover 4y ago
        That doesn't sound like progress. Maybe for CRUD apps it will be okay. I have a hard time imagining it will be acceptable for financial or critical systems.
        • ako 4y ago
          How about self driving cars?
          • bee_rider 4y ago
            You’ll rent a self-driving car from a ride sharing app, they’ll make sure that the expected cost of fines and lawsuits will be significantly less than the expected revenue from providing rides. Although they can help push down the former value by making sure they operate from a favorable jurisdiction.
      • rco8786 4y ago
        > It's the way of the future. For some use cases, sure. We go through painstaking efforts to ensure things like correctness, consistency, and idempotency for a reason though. Most things we want to be deterministic, and when something's not deterministic we freak out and fix it ASAP (including waking people up in the middle of the night to do so)
      • fassssst 4y ago
        Yea, aka how engineers working on safety critical or high reliability stuff already have to think. Assume your software has a probability to fail or have bugs or gets hit by bit flips or unreliable hardware. There’s a whole field for dealing with those kind of things that typical web devs haven’t had to worry about as much.
        • belter 4y ago
          These are not those type of Engineers. Not the ones doing triple computing using three different processors with a summation and voting process, inclusive using different programming languages and compilers, because the compiler can also have bugs. These are the engineers running self driving beta neural nets on public roads...
          • williamcotton 4y ago
            I use a process called sample-and-vote when having LLMs return executable solutions. I imagine we will see this kind of design a lot with probabilistic computing.
            • belter 4y ago
              Sounds interesting. Any references?
              • williamcotton 4y ago
                I show an approach here: https://github.com/williamcotton/empirical-philosophy/blob/main/articles/from-prompt-alchemy-to-prompt-engineering-an-introduction-to-analytic-agumentation.md https://github.com/williamcotton/empirical-philosophy/blob/m...
      • flangola7 4y ago
        Engineer brains are just a type of neural network too, which also have a probability of putting out bad code. Space exploration budgets run into the hundreds of millions of dollars each and recruit the brightest people on the planet to design and program software, yet there is a long list of projects that failed due to code errors. Mars probes have failed to reach the planet, orbited too low and burned up, crashed into the surface, or landed successfully but later overwrote critical memory due to a flawed software update. The AI doesn't have to be perfect, but only offer a lower error rate than humans.
        • belter 4y ago
          Engineer brains are not a neural net. It's only our brain lack of knowledge of how the brain works that led us to anthropomorphize Matrices and Weighted Graphs :-) Reading a bit on the Brain will quickly help dismiss those analogies. I suggest these two as good starting points: - Lange Clinical Neurology - 11th Edition - Bradley's Neurology in Clinical Practice, 8th Edition
          • hwayne 4y ago
            Brains are clocks, I mean machines, I mean computers, I mean neural nets.
            • goatlover 4y ago
              There was an article a while back comparing the mind to a symphony, but it never caught on as a metaphor.
      • Xeoncross 4y ago
        "We're 98.7% certain the user's payment go to the right account with this change"
        • krab 4y ago
          More like we're 98.7 % certain that the screen layout looks ok for all device types and languages. The question is how hard is it to fix the problematic cases without turning the whole thing upside down when you find out the original solution is not enough.
      • kabdib 4y ago
        "Good news, analysis of the anti-lock brake system failure shows that you're only sixteen percent dead."
        • bryanrasmussen 4y ago
          From the neck up, then?
    • deleted 4y ago
      [deleted]
    • ArchD 4y ago
      And someone will decide that a 0.0001% probability of an airplane crash caused by a bug is acceptable? Maybe 0.0000001% is more reasonable? In any case, how do you accurately determine the probability of a rare failure of a black box without doing many real experiments with real inputs?
      • Redoubts 4y ago
        And what’s a postmortem going to look like? “Don’t worry, we stuck the flight data recorder in the training set, and rebuilt the model. Should be good to go now”?
        • hinkley 4y ago
          We already know what the post mortems look like. AI black boxes will look a lot like declarative programming black boxes. We don't really know how the code runs, we just ask it nicely to do what we want and then stare at the config files and the docs if it doesn't. Low code and AI are going to have many of the same failure modes. Until someone combines them and then they'll have exactly the same failure modes.
      • AlotOfReading 4y ago
        That's how safety critical devices are already built, so yes. We have standardized probabilities of failure (e.g. SIL [0]) from the unexpected, because mitigating 100% of risk is somewhere between impractical and impossible. [0] https://en.wikipedia.org/wiki/Safety_integrity_level https://en.wikipedia.org/wiki/Safety_integrity_level
        • davedx 4y ago
          Indeed. Some of the commentators on this article should really examine their assumptions about how reliable and correct “Software 1.0” is.
        • ArchD 4y ago
          From a quick reading of the wiki, the associated methodology seems rather limited: "System complexity, particularly in software systems, making SIL estimation difficult to impossible" "The requirements of these schemes can be met either by establishing a rigorous development process, or by establishing that the device has sufficient operating history to argue that it has been proven in use." You could prove that normal code satisfies some specs, but you can't do that with neural nets unless the number of possible inputs is tiny. So, the only way to establish that the black box neural net meets some SIL target is through "sufficient operating history".
    • booleandilemma 4y ago
      That's not engineering though is it? What use is a banking app if it's only going to be correct some of the time?
    • rco8786 4y ago
      > Hard to inspect but very possible test. Probably, but is testing to the necessary level of correctness more or less effort than writing the code ourselves?
      • hinkley 4y ago
        Languages with very high degrees of static analysis are basically making you write the tests in the code. We have people who think they can keep extending that until there's barely any code, but they haven't been proven right or wrong yet.
        • layer8 4y ago
          Static analysis creates proofs, which is fundamentally different from tests. Merely testing AI output will always be inferior on principle than proving code to be correct. One would have expected AI to be better than humans at proving code correct (instead of just testing it for certain inputs), but LLMs currently fall short of that.
          • hinkley 4y ago
            What I'm saying is that if you have the proof, someone will write an AI to pass it. There's still a lot of very, very sophisticated work that goes into locking down requirements that tightly.
            • layer8 4y ago
              > if you have the proof, someone will write an AI to pass it. That doesn’t make any sense. What passes the proof is the program code. You have the code, and then you construct a formal proof that the code is correct, similar to how a mathematician proves that some theorem is correct. The code is a prerequisite for the proof. When you can construct a proof for the code, you’re done. This proof-construction process is what AIs currently aren’t good at, because it requires logical precision, and probability isn’t sufficient. They can generate code, but they can’t construct the formal proof that the code is correct (and it often isn’t). > There's still a lot of very, very sophisticated work that goes into locking down requirements that tightly. What’s true is that you need to know what you want to prove about the code, and that isn’t always easy.
    • hinkley 4y ago
      Possible to test, but not likely. What often divides the excellent engineers from the poor ones is how well they can think about corner cases, and tests are mostly about writing down the corner cases in a sustainable way (vs half-assedly writing down half of the corner cases and writing 0.5% of tests as asserting a bug.) The main problem I saw with DevOps and QA automation was that if people don't write code all day, having them write code that gates release of software to production does not result in good outcomes. Sooner or later developers have to inject some engineering practices. If the engineers are running AI generated code, nobody knows how to do that job, and you will get a long string of permanently damaged brand names in the aftermath.
      • theptip 4y ago
        On the flip side, if the rest of the process becomes easy and in some sense standardized (the art/craft of code is replaced with back-prop), then maybe the complements like data cleaning, model bias/error measuring, and interpretability become “the work” that the high-paid scientists/engineers focus on? I can particularly imagine a regulatory environment much more rigorous than software has got away with thus far, for example strict requirements around certifying that your model doesn’t exhibit X Y Z biases according to standard frameworks of evaluation.
        • hinkley 4y ago
          Not if the ladder gets pulled up behind us. If there's nowhere to train people with 0 years of experience to do this work effectively, then new developer creation will drop by 10x which means five years in we'll be missing 40% of the workforce.
      • emeril 4y ago
        so many companies have such terrible code quality to begin with, it's hard to see how brand names could be much worse in many (but not all) cases
  • johnthewise 4y ago
    Maybe don't approve code you don't understand? If people just lets in code they don't understand into products and it passes PRs and tests, they might have written dumb code in the first place. I see essentially zero risk in using these tools. dysfunctional teams will write shitty code anyways, whether it's coming from ai doesn't matter.
    • matisseverduyn 4y ago
      100 lines of poorly written code (by AI or humans) added to a 1M line codebase is easy to spot and fix by the dedicated humans who have that whole system mapped out in their heads. Having AI write a 90% accurate 1M line (or "parameter") codebase all at once, (which seems to be the expectation here), is the "risk" you're overlooking. No human will be able to know where to start debugging that. At least not yet. But will yet come before incredibly dangerous amounts of AI written code is pushed into critical systems everywhere, by naively optimi$tic opportuni$t$?
  • throwaway1777 4y ago
    Don’t worry, the AI can write the tests too
    • titzer 4y ago
      lol, who tests the tests?
    • goatlover 4y ago
      I bet it can also hallucinate tests.
      • cratermoon 4y ago
        I recently read a thread on Mastodon[1] suggesting that "confabulate" is more correct than "hallucinate". In psychology and neurology it means to fabricate imaginary experiences or details as compensation for loss of memory. 1 https://universeodon.com/@siderea/109883198218504351 https://universeodon.com/@siderea/109883198218504351
      • hinkley 4y ago
        I have coworkers who also hallucinate tests. Now they'll have someone to go out to lunch with.
  • shanebellone 4y ago
    This is my concern too. How can you trust a business process that you cannot inspect step-by-step? Imo this greatly limits use cases.
  • davedx 4y ago
    I’ve worked on a lot of codebases and your “usually” is highly debatable. I don’t think the difference in understandability will be as big as people think as tooling for ML evolves. Something like Godbolt for neural networks? Can’t be that far away
  • hinkley 4y ago
    I've worked a number of places where the interview process was predominantly about hiring legible people. Someone who's wrong and cagey can cause a lot of damage before anyone figures out what's up. At least a talkative idiot can be managed.
    • raydiatian 4y ago
      Oh that’s why I’m employed
  • mmis1000 4y ago
    I had saw someone mutate the code randomly until it works. There are both a boolean flag and an if statement went wrong, and result in correct result by luck. I almost wondering if it is AI or something trying to fix the code. If AI don't care about the process and only care about results. These types of thing is going to happen everywhere. And from this moment, no code is understandable to human.
  • lumb63 4y ago
    I feel like a broken record. I’ve been saying the same thing since this AI frenzy ramped up in the last year or so. I won’t restate your complaints, but I share them. I’d like to see more work done to incorporate all the advances AI has brought us into our traditional software. Using the example from the article, if databases can be 70% faster and use 10x less memory by leveraging a neural net, how? If we can figure out the how, we can understand it and incorporate it in other areas. A great deal of success in various fields draws on inspiration from other fields, for example biomimicry. We even come up with mental models in areas like computer science that trivialize complex topics to simple objects a child could understand (trees, stacks, etc.). We would benefit immensely from learning from neural networks, but instead we have decided to largely ignore the how and see what they can do. Both are important, but one is severely lacking.
  • deleted 4y ago
    [deleted]
• fassssst 4y ago
  Well maybe instead we'll spend our time honing the spec and doing lots of testing instead of mechanically writing and reviewing code. I see that as potentially a massive net positive for software quality.
  • marcosdumay 4y ago
    You mean honing the spec enough so that it is unambiguous? Maybe you can use some special purpose artificial language created with the purpose of writing unambiguous texts... Like Java or Python.
    • fassssst 4y ago
      Those aren't good enough, you really need a formally verified language. Or just use English and do lots of acceptance testing plus add fail safes. I bet that will be more economical.
      • gtirloni 4y ago
        Look at all the millions of developers proficient in Aqda and Coq.
        • naasking 4y ago
          The point is that you don't need millions of developers if AI is writing the code against a spec, you'll maybe only need a few thousand spec writers.
          • gtirloni 4y ago
            I think you're underestimating the number of spec writers. If that ever happens, every company with a custom business stack will need at least a few. That's certainly a few million jobs worldwide.
          • marcosdumay 4y ago
            The point is that if you are trying to replace millions of developers, you will obviously require something on the order of quadrillions of spec writers. Or, rather, this is probably still an underestimation. And yes, it is a very obvious point, and that people keep missing that point on this site is unsettling. (Also, yes, this can be trivially circumvented if you just let those people program, instead of only do verification.) You can, also obviously, replace millions of average developers with (way more) millions of extremely competent spec writers if they can use formal methods. Those will require way more computing power than it can ever exist on Earth to do their work, but they can mathematically get there.
• choletentent 4y ago
  Or easier if your peers write code which is on average below the 90% mark.
• krono 4y ago
  Not just inaccurate, but also code that is undesirable in several other ways of varying severity that a human is less likely to produce at all, or more likely to reject if it had originated from any other source. The fault lies with humans treating these programming AIs like the greenest of junior bootcamp devs would treat the most accomplished senior engineer.
• albertzeyer 4y ago
  It's not that humans would produce 100% accurate code... And you can also use the AI to check for correctness (of course, again not 100% accurate), potential issues, potential improvements, etc.
• johnthewise 4y ago
  If the tool itself is 90% accurate, conditioned on the events that somebody reviews it, it passes test and PRs, it can get to 99.9999% accurate very quickly.
• adregan 4y ago
  Why would the AIs spit out code? I think the point of the article is that they'll spit out black boxes full of weights that can be used to compute anything ala the universal approximation theorem. You'll train a neural net for a given task and verify the results. Producing code seems to be pandering to our ego.
  • p1esk 4y ago
    Because code is easier to verify.
  • yetihehe 4y ago
    Because code is faster and more correct than a neural net if written reasonably. With code you don't need several gpus to calculate a sum of several million records in database.
    • phkahler 4y ago
      Code can also be debugged and fixed. AI needs to be retrained and retested - probably multiple times. Each retraining may cause a regression in any area.
      • hinkley 4y ago
        Which also means you can diff the code output from two runs of the system and determine if any of the diffs amount to regressions in the code. Developers and machines that can't explain themselves end up out on their ear or damaging the projects they're involved in.
    • hn_throwaway_99 4y ago
      > Because code is faster and more correct than a neural net if written reasonably. But, as the article points out, at least for a considerable set of problem domains, it's not. My general rule of thumb is that anything that has a relatively small, finite, discrete set of inputs and outputs is better suited to "Software 1.0" (e.g. coding a calculator). But there are a huge number of domains, some highlighted in the article, which usually have to do with an infinite possible number of inputs and outputs, where human written code is not faster or more correct than a neural network.
• rhacker 4y ago
  The key word is "If". Since we don't generally read other peoples code, we tend to rewrite the whole damn thing every 10-15 years (sometimes 4 years). The reality is it all comes down to testing. If S1.0 has a unit test that says "Person should not get financial help", S2.0 should also have a unit test that says "Person should not get financial help", and work the same. Of course my unit test name is designed to enrage, but let's be honest, we're writing code that makes these decisions.
  • hinkley 4y ago
    Sometimes at the same time. I keep complaining about some NIH code we have and people say things like, "well that library didn't exist at the time, so we had to write one." Git history says they're wrong. Time and again they were writing code that had existed for two years already.
• gitfan86 4y ago
  Just run that code through another model to fix the problems. Then run that output through a model that will summarize the final code.
  • daveslash 4y ago
    Came here to make the same comment. If the first AI model is buggy, then instead of debugging it with Software 1.0 techniques, just use Software 2.0 to debug it with AI! And when there are issues with the AI debugger, use a 3rd model to debug the debugger. And then a 4th. It's AI all the way down! I can see whole empires of consulting firms becoming very rich.
• ModernMech 4y ago
  I made a passing joke the other week that we're turning the act of coding into a bureaucracy. No longer will you just be able to write code, you'll instead have to navigate whatever byzantine logic maze the AI has constructed for itself. Soon we'll have camps advocating various rhetorical paradigms to program AIs. Instead of imperative versus declarative programming, we'll have effusive versus abusive prompt engineering: "The best way to program is to be really nice to the AI, and tell it how much you appreciate it and it will come up with the best solution itself." "No! The best paradigm is to berate the AI and to beat it into submission to get exactly what you want!" The more I hear people talk about programming in the future, the more it sounds like that's really where some people want to take us. I'm not excited.
  • Joker_vD 4y ago
    It's been jokingly observed many times how programming is similar to spell-casting but now it seems we are really going in this direction, just of more demonological style where the summoned demon misinterprets your command and you burn in the fire. You can even imagine some half-thought out security measures being thrown in to tick the "you must address AI by its true name to make it obey you" checkbox.
• bilsbie 4y ago
  Or the opposite
• broast 4y ago
  It's the hardest part but also the most fun.
• billbrown 4y ago
  It reminds me of the "code generator" phase we went through a decade or so ago. At least then we could see the "code generator" code and adjust it when it produced dreck. I'll also note that the next hardest part of programming is troubleshooting "in production" whether a Web application, in an embedded device, or running on someone's machine. Is the "AI" going to help there? Are we going to even be able to fix those problems when doing so could make the code we don't understand fail in another way or hit the wrong side of the performance tradeoff the "optimized code" entailed?
  • hinkley 4y ago
    Code generation is an epicycle. My dad was worried about whether I should go into a CS degree because there was a code generation cycle going on at the time and people thought the computers would be programming themselves. We've had a few since, and my skills are more valuable than ever.
    • didericis 4y ago
      This particular cycle seems likely to be really destructive. Software is way more prevalent in nearly every aspect of our lives than it used to be, and all of the large organizations that don't really understand the problems that are going to compound invisibly are going to turn their products into unintelligible giant heaping piles of garbage very quickly if they're too enthusiastic. Large codebases almost always turn into giant heaping piles of garbage, so that's nothing new, but you can usually get someone to dig through and salvage things from the garbage pile when it goes bad. This article is from 2017, and I don't know about you, but I increasingly feel like software is more and more broken year after year. I think that has a lot to do with "delegating complexity" and building things without really understanding what they're doing. I think the correlation between human understanding of the fine details and underlying logic and desired outcome and software quality is pretty tight. That doesn't mean "software 2.0" neural net stuff doesn't fit in, you just need a human to plug it in right that really understands its benefits and its limitations. The author mentions the downsides, but I think they underestimate them. If you lean on AI too heavily and don't ever translate to a traditional language, you've basically liquified your logic/there's no garbage dump to salvage from when things go bad and zero understanding of implicit context. If you use it to generate ostensibly human readable code you get "documentation" with no guarantee of accuracy, which makes it worse than having no documentation (depending on how high the error bars are). While that's not a new problem either, if it's autogenerated that means it's easy to create way more of it than human generated code, which means it'll probably be an ever larger portion of what gets sucked up into later AI models. If they become too self referential they'll become increasingly detached from human judgement about whether the code is doing what it should and error bars will grow. I'm still convinced these things are virtually all going to end up in a fancy autocomplete suggestion and compression niche after a lot of pain. But that's still a big deal/I don't think the limitations of these means they don't have a big future place. The sheer number of things you can have autosuggest for now with these AI models are amazing, and that expansion is boosting productivity and creating a large number of new products that are going to become essential tools. That being said, every time this type of thread comes up I'm like "woah woah woah, pump the breaks, these things have no understanding of what they're doing. You can't just stop thinking about stuff and let a machine do it, bad bad bad idea."
• pphysch 4y ago
  "We need more Software (LOC)! Hire a thousand code monkeys!" ... "WTF! We have so much technical debt! Fire the code monkeys and replace them with AI! We need more Software (AI)" I'm sure this is going great.
• thih9 4y ago
  I don't think that was the point. I think the point was that as long as you're happy with 90% accuracy, you can have that for 10% of effort. It requires a perspective shift. Then again, cost efficiency is a big motivator.
  • hinkley 4y ago
    90% accuracy is lousy. Especially for anything that isn't a one-off interaction, such as with customers or developers. That's a 35% chance at a failure after 4 interactions.
    • thih9 4y ago
      Wouldn’t this depend on a lot of external factors? 90% failure rate can be lousy in one use case and can be perfectly adequate in another.
• pbreit 4y ago
  Or much easier if codebases start looking 90% more familiar.
• modriano 4y ago
  I guess I should start researching the testing framework ecosystem. Frameworks that make it easy and low-friction to represent desired behavior and identify+test edge cases will be key to using copilot/AI-code-tools effectively. So, what are best testing frameworks people have worked with?
  • unki2aut 4y ago
    To me "property based testing" would come to mind, where you would define properties of the input and output of a method and the library runs random test cases for you. The most prominent implementation would be QuickCheck[0], which is written in Haskell, but there are re-implementations of it in almost every language. [0] https://en.wikipedia.org/wiki/QuickCheck https://en.wikipedia.org/wiki/QuickCheck
• jmfldn 4y ago
  This is very true. But could the program itself just be a normal program, but written by a neural network? I guess this is kinda what GPT 3 can do? One way or another, I want to be able to read the source. Or am I missing the point? Maybe future software will be a big neural network blackbox that we verify purely through tests?
• ravagat 4y ago
  The only thing I'd value out of the current crop of AIs is whether they could decipher and contextualize other people's code to essentially create comments of context in codebases. That would be worth it. I think the general ecosystem would greatly benefit from such a thing
  • boringg 4y ago
    That would be a huge benefit. Also if it could recommend more efficient routes for how people wrote the code.