15 ms·
Software 2.0 (2017)
- andrewfromx 4y ago[flagged]
- l33t233372 4y ago> The process of training the neural network compiles the dataset into the binary, which is the final neural network. If I didn’t know better, I would think the author of this sentence didn’t know what any of those words mean.
- packetlost 4y agoThey definitely don't. There's no way GPT-3/4 replaces software development, but it will save us some typing with fancy autocomplete and prose-ey documentation!
- actionfromafar 4y agoI read it as, people will direct AI to search for solutions, then use these refined solutions to search for more solutions. A bit like how we use libraries and packages, and improved languages, to enhance traditional programming practices. Taking an extreme helicopter view I think I can see it, but on the other hand I'm not convinced it's a very interesting observation. Throughout history we used machines to make more complicated machines. Edit with this quote: Because you only have to provide Software 1.0 implementation for a small number of the core computational primitives (e.g. matrix multiply), it is much easier to make various correctness/performance guarantees. This is true, but it raises another question which is (to me) comically hand waved aside: how do you make correctness guarantees of the output of the neural net? It's not addressed, probably because it's very hard to do so. It's like the NAND gates inside CPUs and GPUs. Since they are so simple building blocks, they are very easy to verify. It does not follow that the business logic I write to run on these things, is easy to verify. The same goes for neural nets, but more so. I'm not saying these new AI tools aren't useful, they are. But it's easy to misunderstand what they can do.
- govg 4y agoThe "correctness" of neural networks is a vasy field of active research, with many prominent minds in deep learning as well as traditional CS theory working on it. We have many results for small scale networks, but I don't know of any results that can "prove" the correctness of an image classifier, for example. After some point, the correctness of such methods becomes very ill-defined, since unlike the normal programming world where everything is mostly in binary, here you will have to answer questions with some variation of "this is 98% likely" with no scope for 100% certainity.
- sebzim4500 4y agoThe analogy makes sense to me, might be missing something though.
- revskill 4y agoI need an AI to teach me which kind of neuron net architecture to use based on input and output.
- IncRnd 4y agoChatGPT will happily answer that for you.
- revskill 4y agoIt doesn't know it's happy or not ?
- IncRnd 4y agoI apologize for the confusion. In this context, the word happily is somewhat equivalent to "perform without reservation". The comment's sentence can be rewritten as, "ChatGPT will answer that for you." The difference is a slight loss of emphasis that had been meant to show chatgpt doesn't require many prompts in order to convince the model to answer the situation that you had posed. The word "happily" wasn't used in the sense of chatgpt experiencing emotions
- deleted 4y ago[deleted]
- ramesh31 4y agoReally needs a (2017). This stuff has moved at warp speed since then.
- dangoor 4y ago(2017) Interesting to see that this is from 2017, given what the AI explosion has been like recently.
- isoprophlex 4y agoYes, great foresight. One thing didn't stand the test of time: "You’ll notice that many of my links above involve work done at Google. This is because Google is currently at the forefront of re-writing large chunks of itself into Software 2.0 code." IMO google is dropping the ball pretty hard right now when it comes to AI.
- langitbiru 4y agoGoogle is the Xerox of AI.
- Tepix 4y agoI think it's too early to tell, neither Microsoft nor OpenAI have something to show that is trustworthy. If that is their goal they can still be first.
- et1337 4y agoIf the hardest part of programming is reading, understanding, and debugging other people’s code, our jobs are about to get a lot harder with a bunch of AIs running around spitting out 90% accurate code.
- orangepurple 4y agoNearly nobody who wants to use these neural nets knows how to do model risk management. These things are black boxes with edge cases that are badly understood and nearly impossible to inspect and test. At least code is logical and human readable (usually).
- pyrolistical 4y agoHard to inspect but very possible test. Correctness will go from a binary pass/fail to a probability
- danrocks 4y ago> Correctness will go from a binary pass/fail to a probability Excellent point. The pervasiveness of neural nets will require engineers (and really, everyone) to start thinking more probabilistically and establish acceptability thresholds instead of certainty. It's the way of the future.
- marginalia_nu 4y agoThat seems like a regression in many regards, especially given how much of this seems like a solution looking for a problem it can solve rather than the other way around.
- guhidalg 4y agoAgreed. We may imagine ourselves very smart but in my experience most people do not understand probability and statistics well. Is 90% accuracy good enough? Is 95%? 99%? 99.9%? No matter the answer, you have to tolerate errors. Now your stakeholders have to tolerate errors. Are they going to accept errors just because "Software 2.0" is here and that's what we all have to live with? Nope.
- 8f2ab37a-ed6c 4y agoWonder if / how Andrej's take has changed over the last 6 years.
- leetharris 4y agoHe talked a little about this on Lex Fridman's podcast about 6 months ago. I'd recommend listening to him discuss it as I will butcher a TL;DR.
- umeshunni 4y agoLink to Podcast: https://www.youtube.com/watch?v=cdiD-9MMpb0 https://www.youtube.com/watch?v=cdiD-9MMpb0 Neural Networks: https://www.youtube.com/watch?v=cdiD-9MMpb0&t=58s https://www.youtube.com/watch?v=cdiD-9MMpb0&t=58s Language Models: 41:50 https://www.youtube.com/watch?v=cdiD-9MMpb0&t=2510s https://www.youtube.com/watch?v=cdiD-9MMpb0&t=2510s Software 2.0: https://www.youtube.com/watch?v=cdiD-9MMpb0&t=3944s https://www.youtube.com/watch?v=cdiD-9MMpb0&t=3944s
- thundergolfer 4y agoHe gave the keynote talk about this at TreeHacks 2023 on Friday. Another commenter in this thread gives his updated take and a link to a relevant tweet.
- swyx 4y agohttps://news.ycombinator.com/item?id=34882831 https://news.ycombinator.com/item?id=34882831
- ArchD 4y agoThe "concrete examples" he gave in the "Ongoing transition" section are traditional AI tasks. What about other tasks like program compilation and theorem proving that require more hard-core logic? I will be more convinced if neural networks can reliably do those tasks. If not, at best a human will still need to manually break the high-level task into smaller subtasks to be solved by neural networks and then somehow glue the parts together.
- derefr 4y agoSo, why would you want to rely on business logic embodied directly as a chaotic ML model (in the Chaos Theory sense of “chaotic” — subtle changes in stateful hyperparameters making for discontinuous shifts in output space), when you could instead ask the model to precisely describe a rigid decision workflow (i.e. “Software 1.0” business logic) it would implement if needing to explain its own decision-making process as parsimoniously as possible?
- agentultra 4y agoI think we need to train people to write formal specifications before we're going to see machine learning techniques generating useful programs. Sure sometimes an LLM generates a correct program. And sometimes horoscopes predict the future. I will be impressed when we can write a precise and formally verifiable specification of a program and some other program can generate the code for us from that specification and prove the generated implementation is faithful to the specification. An active area of research here, code synthesis, is promising! It's still a long way off from generating whole programs from a specification. The search spaces are not small. And even using a language as precise as mathematics leaves a lot of search space and ambiguity. Where we're going today with LLM's trying to infer a program from an imprecise specification written in informal language is simply disappointing.
- canes123456 4y agoA precise and formally verified specification is WAY harder to write then the code for it
- deleted 4y ago[deleted]
- agentultra 4y agoExactly. It seems like we prefer the easy way out. If we can't write a precise specification then we don't know exactly what we want the computer to do. And so all we can get are guesses. The software industry has gotten this far with very little help from the formal methods community... that's changing in recent years in certain spaces where errors are magnified by scale like cloud computing, etc. But instead of getting better at writing precise specifications we're going to continue to be bad at it and hope that an LLM can manage to infer the correct program. It might be millions of lines of code but hopefully it does some of the things we want most of the time. Update: To be clear, I'm not saying AI/ML programs cannot help us to write programs at all, just that the inputs need to be better if we're going to have any confidence that the programs it generates are any better than a horoscope.
- henrik_w 4y agoThere is also this in a similar vein: "The End of Programming" https://cacm.acm.org/magazines/2023/1/267976-the-end-of-programming/fulltext https://cacm.acm.org/magazines/2023/1/267976-the-end-of-prog... Discussion: https://news.ycombinator.com/item?id=34087000 https://news.ycombinator.com/item?id=34087000
- Iv 4y agoI remember a pretty old interview with Linus Torvalds where they are talking about object oriented programming. The interviewer asked him if he expected a similar paradigm change in the coming years and I remember being surprised by his answer: (quoting from memory) No, I don't see anything big coming. Probably the next change will be caused by AI. Yes, differentiable code is already a new paradigm (write a function with millions of parameter, a loss function that requires more craft than people realize and train). That has a property that used to be the grail of IT project management: it is a field where, when you want to improve your code performance, you can just throw more compute at it. And I think that the clumsy but still impressive attempts at code generation hints at the possibility that yet another AI-caused paradigm change is on the horizon: coding through prompt, adding another huge step on the abstraction ladder we have been climbing. Forget ChatGPT coding mistakes, but down the road there is a team that will manage to propose a highly abstract yet predictable code generator fueled by language models. It will change our work totally.
- mattgreenrocks 4y agoThis feels like re-discovering DSLs, except the syntax is English, and the implementation is a blackbox.
- umeshunni 4y agoDSL being https://en.wikipedia.org/wiki/Domain-specific_language https://en.wikipedia.org/wiki/Domain-specific_language
- giobox 4y agoI don't feel a comparison to DSLs works here at all. If you are just using plain human language, is a comparison to DSLs apt? The point of DSLs are to provide a deliberately limited-scope language optimised for a specific problem or problem domain. LLMs that use general human language is like the furthest opposite of a DSL - its the broadest scope language for describing any problem, and they try to solve them all. Also, few popular DSLs are truly blackbox in the sense chatGPT is - many of them have exposed source or even line-by-line debuggers available. There are a ton of other reasons this doesn't make sense to compare.
- blowski 4y agoAren’t we on something like v15.0 by now? First there was software of the Enigma machine variety, then assembly, then massive IBM machines running COBOL, then C, then Ruby/PHP/Python. We could also talk about how networking and persistence fundamentally changes software. Each of those is a big iteration in itself, probably just as big as moving to ML generated code.
- pier25 4y agoIt's an interesting take which makes total sense with stuff like self driving. But I have a hard time picturing this approach for making UIs, data models, cruds, gameplay, audio, graphic engines, etc. Nothing is really a silver bullet so I guess the future of programming is really hybrid. Stuff like Github's Copilot.
- thuuuomas 4y ago“we should obviate things I find distasteful, but my livelihood must be preserved”
- pier25 4y agoNot really. I'd be extremely happy to be able to produce the same results with less work.
- ahussain 4y agoWhy do you think neural nets will be bad at making UIs, data models, cruds, gameplay, audio, graphic engines, etc? There are already some compelling examples of AIs making progress on those kinds of tasks.
- pier25 4y agoYes, AIs can certainly generate stuff. But can an AI generate the specific stuff I need for this specific use case? Probably not, although I'd be more than happy to be able to delegate work to our robot overlords.
- jimbokun 4y agoWhat's a good heuristic for determining if a problem better fits into Software 1.0 or Software 2.0?
- pyrolistical 4y agoThink of the hello world of machine learning. Handwritten digit recognition. https://youtu.be/aircAruvnKk https://youtu.be/aircAruvnKk
- jstx1 4y agoIt boils down to "do I need ML?".
- kklisura 4y agoIf assume the article is correct and there will be Software 2.0, what's the cost of running it vs cost of running Software 1.0? The reason I'm asking is given this quote "when the network fails in some hard or rare cases, we do not fix those predictions by writing code, but by including more labeled examples of those cases." - is it more costly to do "curating, growing, massaging and cleaning labeled datasets" and ultimately training neural networks than to just write code? Maybe not for small NNs, but for DNNs?
- gtirloni 4y agoIdeally, providing labeled examples can be done by relatively cheaper humans.
- deleted 4y ago[deleted]
- zelphirkalt 4y agoThe higher level the labeled examples become, the more effort is needed to make them in the first place. For example take a website. How are we going to provide enough examples of websites to make the code generated fit what we need and not have annoying properties we want to void? Lets say we have a website and we tell the code generator of choice, that we want that website to be accessible for blind people. How do we create the amount of labeled examples, that make the code generator understand what to create? Maybe that very creation of labeled examples will be a software developer's future work activity.
- gz5 4y agoAndrej's podcast with Lex Friedman touched on this too - worth listening to if this topic is of interest. The pod was in the past year, so many years after Andrej's software 2.0 post, and after many years of great AI experience at Tesla to add to or potentially change his views. Likewise, will be interesting to see how the HN community's experience with ML and AI over those years may have changed our views.
- Dave3of5 4y agoI'm sure that will be the case at some point but for the moment it's not true. It's also important to note that NN's have been around for a long long time with relation to computing in general. The problems caused by using them have been reduced by not completely removed. Imagine if the Therac-25 software was written by chatGPT.
- dividuum 4y ago> Imagine if the Therac-25 software was written by chatGPT. I guess in that case we wouldn't learn and teach from the design flaws made. Instead we would "It's just a glitch. No one is really to blame. Just feed it more data and maybe it won't kill anyone next time". Having been partially responsible for a (back then) SVM based machine learning system and seeing how it's difficult to explain to management why it fails and why fixing it isn't just a missing line of code somewhere was pretty frustrating. I'm not sure I like this future.
- Dave3of5 4y ago> I'm not sure I like this future Me too! But to allay our fears I think that the OP here is saying all programming will change to some NN powered large language model. That is not true. There will still be "manual" i.e. not NN powered programming and I suspect that it will be the case that type of thing is the majority for the rest of my career. Good luck to the poor sods in 100 years time arguing with a poorly trained LLM to output some unit tests whilst a virtual chatbot runs the standups.
- deleted 4y ago[deleted]
- jevgeni 4y agodon't show this to any consultants
- a-dub 4y agomachine learning is good at end to end closed loop analog signal analysis problems (decode/translate/synthesize) and electrical engineeringish tasks, as it always has been ... but i'm not convinced that it is yet good at the more needly real world problems that most software faces. so yes, sure, the best speech recognition algorithm will remain something ee-ish, and will probably make use of highly parallel numerical computing and data driven optimization based solution finding... but i think whether or not that will be the road to correct implementation of entire discrete information systems with all of their knotty discrete rough edges remains to be seen...
- marcus_holmes 4y agoI think it's about fault-tolerance. So "programming" in the Software 2.0 world is basically training a model. Which isn't a task with an end. You stop when you're bored, or when it passes a given level of accuracy, not when it's "completely trained" because it will never get to 100% accurate. In e.g. speech recognition, handwriting recognition, speech synthesis, drawing pictures, writing a response to a human's question, all those messy "organic" problems, this is fine. A 99.9% accuracy rating at e.g. speech recognition is better than humans do. But there are problems that absolutely need 100% accuracy, and you can only get that if you code it up the old-fashioned way (though probably not Agile - I love my iterative development cycles but they're equally prone to not quite getting it 100% right).
- a-dub 4y agommm... i disagree. fault tolerance is more of a reliability property that sits outside of the performance of a discrete logic or signal processing system, it describes how well a system can remain functioning in the face of adverse conditions like internal failures, hardware substrate failures, dependency failures, adverse inputs, etc. but... yeah, some problems have continuous performance variables (typically ee'ish) and others have discrete ones (typically cs'ish). the discrete account ledger either computes the correct value or it does not, where many signal processing problems have to contend with noise and are allowed to produce a wide range of noisy outputs and therefore their measures of correctness are based on statistical arguments.
- veselin 4y agoThe interesting part of that prediction is that depending on how you read it, you may say it failed embarrassingly, or you may say it predicted the current reality fairy well. The next software as a differentiable thing that is the program has certainly failed. But now, there are amazing opportunities to connect text-to-text models to other another, to search engines, that it is likely to become a new programming.
- claytongulick 4y agoAt some point the complexity of the prompts required to generate a program that meets a non-trivial specification become so complex as to be indistinguishable from a programming language.
- wvenable 4y agoThe only complete specification is one that can be compiled and executed. AI will take incomplete specifications and guess the rest -- just like humans do. Whether or not it makes those guesses better than a human remains to be seen.
- titzer 4y ago> Think about how amazing it could be if your web browser could automatically re-design the low-level system instructions 10 stacks down to achieve a higher efficiency in loading web pages. This sounds great in theory, but in practice, a system that has that much dynamic adaptation has brutally steep performance cliffs and is massively complex. I for one, will be opting out of that giant vertical slice of hell. This is one of the _good_ reasons for having layers: separate failure zones, separate levels of abstraction--true reuse and modularity. Bugs break all that. And no, given the hallucinations of large models just in the natural language space, I do not want to reason through the mad ravings of a tripping AI to debug a monster pile that happens to make web property X go 10% faster.
- samstave 4y agoELI5 what "10 stacks down" means? I havent heard this phrase before.
- titzer 4y agoI assume that the author means "Framework XYZ in TypeScript running on JS running on V8, which is written in C++, compiled by clang, running on Linux with glibc N, running on VMWare, running on x86, running on the Rapture Cove microarchitecture." I don't know if there are 10 I could name, but stacks are deep.
- deleted 4y ago[deleted]
- diydsp 4y agoYup, that graphics stacks could easily add a few layers for ya! When I was at a gfx card mfr, there were many compatibility and shim layers :)
- swyx 4y agoi’d honestly love a full accounting of those stacks from browser all the way down to transistor. almost feel like we need to record it for posterity in case anything happens to our civilization
- piokoch 4y agoSo we will have some abstract language that will allow business people to define what software should do and on that base AI will generate source code and working project. Wait, wait, I have heard that it was called BPMN and it generated underneath Enterprise Java Beans and it was working amazingly, all software is written today, right? Right? But, but nobody touches this crap besides generating pictures to show on Powerpoint slides. Because writing software is circa 10% of all effort, specification, legal stuff, maintenance, avoiding technical debt, proper test cases, anomaly testing, performance testing the right stuff. That is hard, that matters. Software 2.0 is barking the wrong tree.
- Anuiran 4y agoA step beyond that, Software 2.0 and beyond does not generate source code. Your app simply lives in the LLM.
- goatlover 4y agoI'm wondering how the LLM will be able to make use of hardware, or handle high demand. Can it be relied on to have data integrity?
- marcus_holmes 4y agoAnd COBOL before that. The goal of COBOL was to allow a non-technical business person to describe the problem in English and that description could be used as the basis of the program. Needless to say it didn't work. Accurately describing what a program should do is what a programmer does. We're not telling the computer what to do (move this value from memory into this register, etc). We're describing a program's behaviour. So I think we'll just get another language that is a good fit for describing what the program should do, and instead of compiling that to machine code, it will be used to train a model.
- hwayne 4y agoCOBOL actually worked pretty well for its time. The ultimate reason COBOL died was because it was a first generation language that nobody really iterated on. It's similar to how most of us don't use FORTRAN, ALGOL, or LISP 1.5. Unlike those, COBOL didn't have the backing of the "tech world" (at the time, IBM and universities), so it doesn't have descendant languages we do use.
- jvanderbot 4y agoOh boy. Hate to be that old guy, but I've heard this one before.
- mvkel 4y agoThis is the right take on the future. Unsurprising that it came from Karpathy all the way back in 2017. The quality of the code is irrelevant, as the point of Software 2.0 is that it's another layer of abstraction on top of traditional code. "Coding" becomes "I need something to do a thing," rather than "def doSomething: ..." As long as the output gives you what you need, the code quality ultimately is an efficiency play. But as AI coding improves, it can refactor itself, so it's a short-term problem. In my own experience coding with an "AI assistant," I've been able to mentally stay in "architecture mode," which makes me feel twice as creative, twice as productive. That alone is a net positive.
- goatlover 4y ago> The quality of the code is irrelevant, Until it needs to be maintained, or has weird bugs. > As long as the output gives you what you need, the code quality ultimately is an efficiency play. But as AI coding improves, it can refactor itself, so it's a short-term problem. Not sure how this is going to work on large codebases.
- mvkel 4y agoI think the "until it needs to be maintained" is an oft-quoted excuse that is either 1) unavoidable, no matter how elegant the code, or 2) way overstated.
- guhidalg 4y agoPerhaps he's right but it's a very depressing view of the future. > "Coding" becomes "I need something to do a thing," rather than "def doSomething: ..." More likely, corporate overlords will decide that you cannot just "do a thing" but rather that you are allowed to do X, Y and Z things for which they have pre-trained commercial models for. > As long as the output gives you what you need, the code quality ultimately is an efficiency play. But as AI coding improves, it can refactor itself, so it's a short-term problem. Have you ever debugged a problem with generated source code? Or even a compiler bug? Now imagine leaving your AI to go find the bug or iterate until the bug disappears hehehe... > In my own experience coding with an "AI assistant," I've been able to mentally stay in "architecture mode," which makes me feel twice as creative, twice as productive. That alone is a net positive. IMO, if your work benefits from an AI assistant then your work is to produce many lines of code and you would benefit equally from creating high-level abstractions than from using pre-trained black box models (or as some call them "new hires").
- terminal_d 4y agoThe adoption of the so-called "Software 2.0" depends on how influential OpenAI is as a company. The majority of the use-cases of GPT-3 are only about "remixing" information, and the "notoriety" of the AI is (IMO) mostly a publicity stunt; I'm assuming that there are prompts for personalities / jokes / poems / etc that are never shown to the end-user and are hailed as an "organic outcome" of the model. Obviously, no one can check for these right now. Seems to me that microsoft has taken a few leaves out of the OpenAI playbook and applied them (rather chaotically). So when you're looking at actually writing software that needs to be dependable / modifiable / bug free, you'd need a massive overhaul of whatever software stack is being used, so there's very little human-assisting "cruft", and instead you'd want a lot of supporting material for a model, which might look like something written in languages used for formal verification of programs. The promise of GOFAI was about having a human-understandable bottom-to-top framework, and the current "AI" paradigm is at odds with it. The "formal verification" assumption, then, skews towards GOFAI. But since there has to be some human support for the current not-there-yet AI to write software, we might see yet another abstraction layer based on NN / something newer in the years to come.
- willio58 4y ago> the "notoriety" of the AI is (IMO) mostly a publicity stunt Have you _used_ ChatGPT? I mean not just asking it random factoids but using it to genuinely help you with something. Are you aware that it's hit 100 million users faster than Facebook, Instagram, or TikTok did? It's not a perfect product but it's hard to argue with those numbers. I work at a startup and most people I work with use ChatGPT daily. I'm talking project managers, devs, personal assistants, etc. I guess that's all to say OpenAI is influential as heck _already_, now imagine 5 years down the line if they play their cards right.
- terminal_d 4y ago>Have you _used_ ChatGPT? I have, and I didn't find it to be useful for anything I did. I can do what it does with a search engine and trusty C-f. Also, TTS exists. It's 50% tech and 50% marketing (and I doubt it's 50% tech at that), it's not gonna upend anything. Except maybe increase the authenticity of online scams and make people get more degrees in machine learning. And yeah, make the people that rely on it bound as it degrades their skills. It's basically the "internet is educationally useful" argument. At some point everyone's gotta use it but you can live without it just fine. And even though people tout its usefulness for everything good, the majority of data transferred is porno.
- jstx1 4y agoWhat the article is saying is that when you develop and build ML systems, your workflow is different from the workflow you would have if you were solving the same problem without ML. The weird part is presenting it as 2.0 which implies that it replaces 1.0; it doesn't, apart from some edge cases like image recognition - we don't hand code rules to recognise images anymore but that's a tiny tiny part of all the software development work out there.
- jinay 4y agoI recently attended a hackathon (TreeHacks) where Andrej was a keynote speaker, and he's since updated his vision to include the significance of LLMs. The new software is that of prompt engineering, using LLMs as a simulator for the types of programs you want to build [1]. The definition of programming will keep changing as we get to higher and higher levels of abstraction. [1] https://twitter.com/karpathy/status/1618311660539904002?lang=en https://twitter.com/karpathy/status/1618311660539904002?lang...
- startupsfail 4y agoIt is more or less Software 3.0 now, DNNs, if compared to LLNs are like analog computers, if compared to VLSI CPUs.
- nice_byte 4y agoit's all very depressing. still, hoping that the future predicted here is 25-30 years out
- nice_byte 4y agoto be clear, i don't think that the current crop of llms can write better or even good enough software. it may well be that llms won't be the cornerstone of the future predicted in the article. still, there is a huge economic incentive to replacing "manual" programmers, and some attempt is bound to succeed eventually.
- barrenko 4y agoBased on what I can achieve with GPT in a non-Python language as a junior junior developer, I really see almost no limit to what an CS educated experienced mid developer could achieve with GPT and Python experience. Yes there is a lot of hand-holding and guardrailing but still, it's kind of insane.
- deleted 4y ago[deleted]
- nice_byte 4y agomy experience is that chatGPT in its current form is much more of a hindrance than help. it takes longer to achieve the same desired result than just writing it directly. it seems really good at refactoring type tasks (i.e. rename method, move this block to a different routine, etc.) but it needs real ide integration for that to be useful.
- adregan 4y agoI found it interesting that most of the list of benefits sound like they could be describing Forth.
- palotasb 4y agoI think a legitimate criticism of this article is using the clickbait "Software 2.0" label for something that's – as very well described by the same article – so very different from "Software 1.0." Considering ALL software, the intersection of uses cases where Software 1.0 and Software 2.0 truly compete are a very narrow niche. In the real world Software 2.0 will thrive in the next decades but Software 1.0 will be there just as much as it is today.
- binarymax 4y agoAn interesting treatise but I don’t fully agree with the statement that gathering data or stating a goal is easy. Getting quality data is arguably the hardest part of ML. Getting people to fully describe their intentions and all edge cases (“requirements”) in a clear manner is also the one of the hardest parts of software development - which is why things like agile exist. Maybe at the granular function or module level, but business requirements are not easy.
- jdoliner 4y agoOn an entirely silly tangent. It's a bit of a shame that Karpathy stopped working at Tesla. Just because having a guy named Karpathy working on car pathing was such a great example of nominative determinism.
- 0xdeadbeefbabe 4y agoNonfiction for the win.
- BulgarianIdiot 4y agoDo you believe in nominative determinism? It's an interesting concept with gigantic implications.
- nealabq 4y agoDo you? Does your username have deterministic implications? Or aspirational?
- BulgarianIdiot 4y agoHaha, both!
- xyzelement 4y agoNot the person you are replying to, but I do (but didn't know the term, thank you!) I find that teams and products with negative pun acronyms fail for example. I find that the trend of the name Lilith being popular correlates with abortion rates (in the bible, Lilith killed unborn babies) etc etc etc
- albert_e 4y agoIf you were from India you would read his name as car-pathy (Lord of Cars) cf. gana-pathy (Lord of the Ganas)
- ryloric 4y agoCompletely missed this lmao
- Lapsa 4y agoWeb5 is all the rage
- bjornlouser 4y ago"Unfortunately, this interpretation completely misses the random forest for the decision trees." Nice one Andrej!
- darksaints 4y agoTo contribute my personal lowbrow dismissal, this is the guy in charge of computer vision at Tesla, where cars have had a habit of doing absolutely bonkers behavior that doesn’t make any sense, when faced with completely normal phenomenon like lanes that fork into two. His single sentence caveat about how Neural Networks can fail in unintuitive and embarrassing ways is the understatement of the century. I’d like to add that Tesla still hasn’t solved that lane forking problem even eight years since it was first identified. I guess just throw more data at it, and eventually it will get better? At what point does the belief that things will get better with more data fed into the same algorithm become a religious creed? Neural Networks are significant advances in the state of not just machine learning, but the world as a whole. But the caveat that we don’t really understand what they’re doing is the whole fucking problem. Until Neural Networks can take advantage of, constrain to, and augment human models, they don’t have a snowballs chance in hell at replacing the types of software we rely on the most. Until then, you’ll just create a massively inefficient system where the neural network writes the software but you spend 10x on engineering your training datasets so that your brilliant neural network knows that it is better to commit to one of two lanes in a forked road than it is to crash into the concrete lane divider. Or to not be racist. Or to not go haywire because of a sticker on a stop sign.
- daveslash 4y agoRe >> "At what point does the belief that things will get better with more data fed into the same algorithm become a religious creed?" Oh, ye of little faith! It is heresy to criticize our new religion! ~Some AI consulting firm or AI "thought-leader" probably. Edit: I'm sure there are some useful use-cases, but I'm not an unquestioning devout adherent. That said, I should probably learn more about it just so I can intelligently defend the use-cases in which it doesn't make sense.
- hooande 4y ago"Neural networks are bad because some times they fail" is a low brow dismissal, indeed. What matters is the percentage accuracy. A black box with a 10% failure rate is better than a fully explainable system that fails 20% of the time. Explanations make us feel better and they can be very important. But for most cheap, repeated processes they aren't necessary. Not to mention that neural networks can be tested and interrogated in ways that other systems cannot.
- deleted 4y ago[deleted]
- deleted 4y ago[deleted]
- jejeyyy77 4y agoBut all of these models are learned from code written by humans. We've nowhere near enough "training data" to be able to generate all the software needed moving forward.
- chazeon 4y agoI remember there is a great analog for the era of the neuro network. The neuro network is a binary encoding of an algorithm/program. It could therefore be very efficient and simple. We will move on to get comfortable with it just like we are now comfortably familiar with the binary encoding of information (files).
- swyx 4y agoshould put (2017) in the title.
- Duwensatzaj 4y agoSid Meier's Alpha Centauri brought this up in 1999. "We are no longer particularly in the business of writing software to perform specific tasks. We now teach the software how to learn, and in the primary bonding process it molds itself around the task to be performed. The feedback loop never really ends, so a tenth year polysentience can be a priceless jewel or a psychotic wreck, but it is the primary bonding process—the childhood, if you will—that has the most far-reaching repercussions." – Bad'l Ron, Wakener, "Morgan Polysoft"
- wewtyflakes 4y agoThat game is worth playing for all of those neat "quotes"; they seem more on point by the day. https://civilization.fandom.com/wiki/Pre-Sentient_Algorithms_(SMAC) https://civilization.fandom.com/wiki/Pre-Sentient_Algorithms...
- Winsaucerer 4y agoThat game really deserves a sequel, with the same cast of characters. Beyond Earth tried to do generic leader personalities, and it suffered as a result. The unique personalities of the original cast were a large part of the game.
- swyx 4y agoit was a magical time in my youth. i tried buying and playing it again recently but the magic is gone, we are too used to the quick casual games and rich AAA shooters. for others as nostalgic as i am you should know about https://paeantosmac.wordpress.com/ https://paeantosmac.wordpress.com/ for a bit more philosophical exploration
- petilon 4y agoMachine Learning and Neural Networks are not applicable for solving all kinds of problems. In fact, in Tesla AI Day Elon Musk remarked that "I discourage the use of machine learning... unless you have to use machine learning, don't do it" [1]. Getting machine learning to work right is hard. If it can be avoided it should be. So to cast this as Software 1.0 vs 2.0 doesn't make sense. There is a class of problems where neural networks work better. Everywhere else we will continue to use traditional code. [1] https://youtu.be/j0z4FweCy4M?t=9319 https://youtu.be/j0z4FweCy4M?t=9319
- lysecret 4y agoI think a lot of people are interpreting this (implicitly) with a kind of Generative AI lense. However, this article was much more about "classical" ML type work, e.g. training your own Neural Net while I think the paradigm seems to be shifting towards "Zero or Few-Shot" learning e.g. there is no more fine tuning or even complete training step involved.
- cgearhart 4y agoThis is quite an interesting take… I’m not yet convinced that neural networks are “computing” in the classical sense, but maybe that’s moot. More interestingly, this makes me wonder if there are some Gödel-like proofs waiting out there that limit the capabilities of efficiently-optimizable programs. What new kinds of undecidable or uncomputable functions exist in the subspace of programs that an NN can learn? Would be exciting to find out.
- BulgarianIdiot 4y agoI hope he's smarter at machine learning than at blog posts. Yes, a lot of software will include NN models. Traditional software is going nowhere, because it's the only means of being 100% sure of what the outcome will be, non-probabilistically. Neural Networks are a tool for solving probabilistic, fuzzy logic problems.
- travisjungroth 4y agoYour upper bound for the probability of software written by people being correct is higher than mine.
- valenterry 4y agoYeah I agree. I can see those models helping us improve productivity quite a bit on the side of (still) writing code. Essentially it is a much better form of context aware c&p from stackoverflow. And then, as you say, there will be certain parts where those models are actually gonna be integrated in software in one way or the other. And I think this is powerful. It would be awesome if I can just toss certain problems to the business folks and empower them to figure out the solution AND implementation by themselves. But even that will probably take quite some time.
- thurn 4y agoThis just feels like taking two unrelated things and bunching them together under a label in order to be provocative and imply that software 2.0 will "replace" existing code. If you surveyed e.g. all of the code Google has in their piper repository, you would find significantly less than 1% of it could be replaced by even an extremely good neural network.
- juliangmp 4y agolol is this satire?
- throwaway744678 4y agoMy bullshitometer was ringing strongly reading this article.
- pulse7 4y agoEven without AI the systems have become VERY COMPLEX, hard to extend and debug. It is much harder to get started for newbies than it was say 25 years ago. If AI is software 2.0 it will be EVEN MORE COMPLEX. I guess that Software 2.0 must be something which will reduce complexity and not increase it...
- davedx 4y agoThere’s a huge amount of pessimism and downright snark in the comments. I see a call to action to improve the state of the art of the tooling universe around ML, to make it even more broadly applicable, more understandable, and probably unlock huge economic value. How about a more glass half full take on progress?
- pelagicAustral 4y agoI don't think product managers a very big on 'glass half full' nor bank institutions in 99.99% accuracy.
- azubinski 4y agoSo we have Web 2.0 thanks to which the Most Important Extension Distill is installed in my browser. Because without it, my head hurts a lot from Web 2.0. Ok. I feel like I'll have to install something that will make life easier with Software 2.0.
- hn_throwaway_99 4y agoHmm, a little frustrated because I feel like a lot of comments here are missing the forest for the trees. For example, as someone who works with financial software, I don't see Karpathy's "Software 2.0" replacing, say, account ledgering software anytime soon. "Yeah, we calculate our clients' balances correctly 99.9% of the time!" isn't going to cut it. But I don't think that's what Karpathy is arguing. There is a large set of problem domains where Karpathy's Software 2.0 is a much better solution than what he calls Software 1.0. For example, even in finance, stuff like fraudulent transaction detection, or financial security software for intrusion detection, is very well-suited to Software 2.0. So yes, I think Software 1.0 will always be around, but I don't think it makes sense to use it for domains where Software 2.0 is a better fit. What I feel like Karpathy is arguing for is really now a recognition that Software 2.0 really is a whole new paradigm shift, and we need better tooling (he uses "GitHub for Software 2.0" as an example) to support it.
- obviouslynotme 4y agoI agree that almost no Software 1.0 will be effected by this. Software 2.0 will start doing the jobs humans do now because Software 1.0 can't, e.g. security guard, customer support, as well as your examples. The real interesting things will be Software 1.0 and 2.0 working together. You use 1.0 to run and validate the work of 2.0 that is guided by prompts. An example of this would be using prompts to generate source code that is compiled and tested. The real TDD is only writing tests and letting Software 2.0 create the code for you. This extends to other work like engineering as well.
- underdeserver 4y agoThe problem is that the term "2.0" usually means a newer, better, more feature-rich version (literally version) of the same thing. The kinds of software that "Software 1.0" is suitable for are markedly different that the ones "Software 2.0" are. As Karpathy argues, it's a different tool, suitable for different tasks, and it should have a different name.
- ricw 4y agoThis thesis seems very credible, but it misses one downside of his "software 2.0" definition: you need significant amounts of data to train the neural networks. Most problems do not have this amount of data. Not even close. So yes, this will revolutionize and enable unseen performance in the few areas where there is significant data. For all the rest it'll be business as usual.
- hn_throwaway_99 4y ago> you need significant amounts of data to train the neural networks. Most problems do not have this amount of data. Not even close. Going to push back against this one. I think we have a lot more training data than most people realize. I wrote this comment yesterday, https://news.ycombinator.com/item?id=34862450 https://news.ycombinator.com/item?id=34862450, about how a large government contractor is using ChatGPT to generate first drafts of responses to government RFPs. Now, most of these RFPs are in very specific areas, technologically speaking (e.g. specific technologies around cloud network security, for example). These folks were actually blown away by how technically accurate ChatGPT was on many different areas, even very specific niche areas, and even considering ChatGPT's view of the world hasn't been updated since late 2021. Again, the first draft needed to be edited, but there are is a huge amount of data out there that ChatGPT is able to use coherently on even niche, esoteric topics.
- wufufufu 4y ago> Across many applications areas, we’ll be left with a choice of using a 90% accurate model we understand, or 99% accurate model we don’t. And how do you show that it is 99% accurate besides creating enough automated tests to the point that you could write the procedural version? I think what I was missing from this article is how to evaluate a domain where neural nets or LLMs can be applied. Image-from-text generation is a great one because accuracy isn't strictly defined. However, telling ChatGPT "code this pacemaker for me" would have a real accuracy attached to it that you could confirm with unit tests.
- ansgri 4y agoI was more excited by the concept of Software 2.0 when it meant differentiable programming -- i.e. ordinary programs with logic etc, but everything is differentiable, optimizable and can gradually be replaced by opaque NN-based components. That would be largely compatible with current mainstream software development and allow gradual transition to highly-efficient domain-specific architectures. Instead we got this LLM-based "paradigm-quake".
- mikewarot 4y agoI reject this premise. A neural network is not a program, they are essentially huge multi-sheet Excel workbooks that compute a score for a given set of inputs. Unlike the Excel workbooks made by domain experts, it's almost impossible to even find out what the function of any given cell is in the overall computation. See the effort it took to find the "neuron" responsible for a/an differences in GPT-2.[1] Neural networks have their roles as a black box, but they are not programs, constructed with intent by humans, to be read by other humans, and compilers. [1] https://www.lesswrong.com/posts/cgqh99SHsCv3jJYDS/we-found-an-neuron-in-gpt-2 https://www.lesswrong.com/posts/cgqh99SHsCv3jJYDS/we-found-a...
- dang 4y agoRelated: Software 2.0 (2017) - https://news.ycombinator.com/item?id=23766796 https://news.ycombinator.com/item?id=23766796 - July 2020 (22 comments) Software 2.0 - https://news.ycombinator.com/item?id=15678587 https://news.ycombinator.com/item?id=15678587 - Nov 2017 (36 comments)
- impalallama 4y agoFunny how little a splash this made then compared to now
- 0xdeadbeefbabe 4y agoMaybe I missed the quickstart, but is CUDA the next step to doing those matrix multiplications. Or do I start with PyTorch or TensorFlow? Or do I start by classifying lots of data to train the classifier?
- kyle-rb 4y agoI'm skeptical about ML being a "2.0" when we'll never stop using the 1.0 version altogether. But maybe he means it in the Python 2 -> 3 sense, where there's no real expectation that the older version will ever stop being used.
- impalallama 4y agoI can see this Software 2.0 being used for content moderation in the near future, if it isn’t already.
- kabdib 4y ago"Garbage In, Gospel Out"
- fhd2 4y agoI doubt the premise that everything will be NNs, but I can't predict the future any more than anybody else. What I find interesting in this imagined future is that problem definition usually happens, in my experience, while attempting to encode it, removing all ambiguity. If we skip that step n years from now, will we still even understand the problems we try to solve? Sounds scary to have systems where we can neither reason about solution nor problem.
- mkleczek 4y agoThis begs the question: what data is going to be used as a training set? How do we make sure our black boxes don't become just echo chambers giving us answers we already know? At the end of the day it is always: garbage in, garbage out.
- deleted 4y ago[deleted]
- koonsolo 4y agoCan I see GPT as some sort of lossy compression? You feed it loads and loads of data, where the neural net basically compresses it all into the structure and weights. Then you give it some (un)compressed part, and it gives you the other.
- thisoneworks 4y agoYou want guaranteed specific behavior from a software system (that's what SLAs and contracts are for) and easily reason about it so you can hire a college grad to tweak it. And that's not even talking about datasets, you can only train something accurate if you have enough good data on it? I can only see software 2.0 being a better autocorrect for internet scale usecases (in the short term atleast, until the next breakthrough) This also reminds me of a quote from the Book I'm currently reading (Practical Wisdom by Barry Schwarz) “Most of us think about empathy as a “feeling” or an “emotion.” It is. To be empathetic is to be able to feel what the other person is feeling. But empathy is more than just a feeling. In order to be able to feel what another person is feeling, you need to be able to see the world as that other person sees it. This ability to take the perspective of another demands perception and imagination. Empathy thus reflects the integration of thinking and feeling.” "Mind reading" is another way to put it (https://yosefk.com/blog/people-can-read-their-managers-mind.html https://yosefk.com/blog/people-can-read-their-managers-mind....) - this practical wisdom + mind reading is basically the salient human feature that NNs would never be able to replace so you would always have humans in the system.
- tkiolp4 4y agoI think AI like ChatGPT and similars will become what APIs are today. Just that. Today we wire together a crypto library with some JWT library with some Facebook API using some axios library. We write the code in between. Since we are able to do more, user requirements get more complex, and so software engineers are in more demand. Sure thing, what required 10 engineers in the past (1980) can be done by two today. In the same sense, in the future we will be wiring together AI APIs (probably because it will be cheaper to wire together manually N AIs than to write one that is the sum of the N AIs). Since we’ll be able to do more, user requirements will get more complex… and so the demand for software engineers will go up as well. In the future only a couple of engineers will be needed when today we need 10.
- ThouYS 4y agoyawn, it's all rubbish
- joaofiliperocha 4y agowhat about ...low code software ?
- unhammer 4y agoNeural nets are great at language and image processing, compression, search and probabilistic prediction problems. In many ways they open up new possibilities for what software can do at all. But replace existing solutions? How on earth do you create a single neural net architecture in say pytorch that turns into a CRUD app with oauth, email verification, css+html, support for parsing some ancient xml format uploaded by the user and stored in S3 along with various database connections? I can't even imagine what a labelled or unlabelled training set for that would look like. (Unless you're talking about making GPT write the code that does all of the above – I'm sure people are working on that – but unless I'm completely misreading Karpathy's article that's not at all what he intended; IIUC he's talking about programmers making datasets for specific problems + simple neural net architectures to train on that dataset. If ChatGPT wrote some Java code for you, you can't make it go faster by removing half the nodes, which nodes would those be? It definitely won't go faster if you send the same prompt to a lobotomised ChatGPT)
- zmmmmm 4y agoI've always thought this was the critical point where Google missed the boat. It was 2015. Google was clearly and unambiguously ahead in AI and ML. They also were rampaging to massive majority market share in the mobile OS space. They had the opportunity right at that point to turn Android into the very first AI native operating system. I remember seeing a glimpse of this when I found buried in the Android SDK a face detection feature that accurately located eyes and faces in photographs. I thought at the time: this is it - this is where Google will build AI level features into Android and expose them as platform OS features and Android will race ahead in the computing space and take over the world. The whole programming model will look completely different as devs start using first class AI as a basic feature of their toolkit. And then it never happened. They focused on cloning iPhone features and approach, dumbing down and simplifying the OS to the point where it's pretty hard to distinguish any more.
- tourist2d 4y agoBy far the worst take in the comments. Google missed the boat and focused on the completely wrong things and yet Android is at 70%+ phone OS market share. No phone consumer would care whether the developers of their apps had AI as part of their development toolkit.
- cranium 4y agoDon't forget it was written in Nov 2017, AlphaGo Zero was just released at this point. Seeing how the field evolved in a 5-year span, I can understand the desire to be at the forefront. Personally, I want to keep working on things that I can get to the bottom of. In the same way proprietary software is a nightmare to debug, having an AI blackbox in the middle of my stack could wreck all traceability. However, I can see myself using an AI blackbox when its output is consumed/checked by a human or when "best-effort" is good enough (but treat output as dirty). Example: sorting documents by relevance (human consumes), code assistant (human checks), transcription of my audio notes (wouldn't do it myself or pay anyone for that, so any output is good enough), Counter-examples (too dangerous!): AI personal assistant that accept/reject meetings, a ChatGPT text box as an interface for settings, auto-generated tests, Infrastructure-as-a-Desire: input your software and get a new K8S cluster provisioned for it.
- kilgnad 4y agoI have this crazy idea of using software 2.0 to write software 1.0.
- LLyaudet 4y agoI see AIs uses in two types of uses : - I never run this query : I have found a super Google, but maybe I will miss out some pearls available on the Internet; - I always run this query, but I'm too lazy to have text files, bookmarks in my web-browser to stock answers that I don't want/cannot memorize. It is somewhat similar to the arrival of Google, Unity UIs, Gnome 3, Windows 8, where they wanted to replace all menus with a single search. It is the way forward to idiocracy if we are no more able to tidy our ideas/data. It is like when we are child and we always ask our parents instead of look it up in the dictionary. It is regressive. It is already the case with Google. Since I can look it up on the Internet, I memorize less things. It is human to go a regressive/easy path without will. I'm looking forward to a blog post "Use AI without declining mentally" : - Interact with AI to tidy a corpus of tools/knowledges in personal files - Delegate to the IA the tidying and research of pearls you keep in your folders - BUT have a shared mental model of your data and their structures with your IA, like a manager may had a shared knowledge of the way the files were sorted in boxes and furniture with her secretary. Like this if your IA/secretary is in holidays, you can still work.