16 ms·

> LLMs aren’t just the biggest change since social, mobile, or cloud–they’re the biggest thing since the World Wide Web. And on the coding front, they’re the bi

> LLMs aren’t just the biggest change since social, mobile, or cloud–they’re the biggest thing since the World Wide Web. And on the coding front, they’re the biggest thing since IDEs and Stack Overflow, and may well eclipse them both. I personally feel the technology is over-hyped. Sure, the ability of LLMs to generate "decent" code from a prompt is pretty impressive, but I don't think they are biger than Stack Overflow or IDEs. So far my experience is that ChatGPT is great for generating code from languages I not proficient in or when I don't remember how to do something and I need a quick fix. So in a way it feels like a better "Google" but still I would rank it as inferior than Stack Overflow. I am also hesitant about the statement that it makes us 5 times as productive because we only need to "check the code is good" for two main reasons: 1. It is my belief that if you are proficient enough in the task at hand, it is actually a distraction to be checking "someone else code" over just writing it yourself. When I wrote the code, I know it by heart and I know what it does (or is supposed to do). At least for me, having to be creating prompts and then reviewing the code that generates is slower and takes me out of the flow. It is also more exhausting than just writing the thing myself. 2. I am only able to check the correctness of the code, if am am proficient enough as a programmer (and possibly in the language as well). To become proficient I need to write a lot of code, but the more I use LLMs, the less repetitions I get in. So in a way it feels like LLMs are going to make you a "worse" programmer by doing the work for you. Does anyone feel that way? Maybe I am wrong and the technology hasn't really clicked for me yet.

_qua 4y ago
Did you try GPT-4 yet? It's a huge increment over 3.5/ChatGPT
- aaomidi 4y ago
  Yep its actually able to create ideas that have never been done before.
  - NationalPark 4y ago
    Do share!
- roflyear 4y ago
  It's way better. But equally slower!
- ModernMech 4y ago
  I’ve tried 4 and I really can’t say the results are a qualitatively better than 3.5 for the tasks I’ve been trying (which have been trying to get it to generate documentation for my project). In fact, I find 3.5 turbo the best overall model as a tool, because to quality of responses really depends on the quality of prompts, and the quality of prompts is improved by reacting to responses, which come more quickly in 3.5-turbo. So while ChatGPT-4 is still writing the first not-good response, ChatGPT-3.5-Turbo will be on the 2nd or 3rd and it will be much more cogent.
LeftHandPath 4y ago
I agree - it’s hard to enter a flow state while reviewing someone else’s - or some AI’s - code. That’s a major reason why I haven’t started using LLMs for code, personally. I am glad someone else feels this way. Maybe it’s not going to be as big a paradigm shift as I originally expected.
- actionfromafar 4y ago
  It will probably raise the floor a lot. The least competent (not meaning in a bad way! I was one of those) coders will be a lot more competent all of a sudden.
  - uoaei 4y ago
    I'm skeptical. It's easy to make something sound correct at a first glance but that has subtle fundamental flaws that invalidate it. Knowing humans and the LGTM phenomenon, these kinds of issues will slip by quite readily.
counttheforks 4y ago
To me it's so funny when people say ChatGPT will make developers 5x more productive, because those people are basically just admitting they're not good at their jobs and assume the same holds true for everyone.
- micromacrofoot 4y ago
  not really, this is more like having a strangely knowledgable yet naive junior employee - I can tell gpt-4 to put something together that gets me 90% of what I need faster than I could possibly even type it, it’s reducing my known tasks
basch 4y ago
I think this is mistaking the current .01 iteration with what the technology will be able to achieve. All sorts of groundbreaking technology looks like a minor improvement over the previously refined version until it gets implemented in a way that takes advantage of its strengths, as opposed to just being plugged into old workflows. LLMs cannot be judged by their first few incarnations. What can be trained into them currently exceeds imagination. Imagination is our limiting factor. And I don’t say that from the context of “I jumped on the hype train at the end of last year”. I remember reading the 2017 Google transformer paper and thinking “whoa, this is really happening.” The fact it happened in only 5 years is pretty impressive. Im not sure many papers or innovations got my mind spinning quite like that one.
- cornholio 4y ago
  But there is an unanswered question of how far this technology can go based on its fundamentals. Coding is much like driving, you can't do 80% and let the human do the final 20%, because that final 20% requires reasoning about a well understood design that was implemented throughout the first 80%. If your fancy AI coder thingy can't really reason about the end task that the code is solving - and there is little to indicate that it does, or that, any moment now, technology will advance to the point that it will - then the 80% will be crap and there exists no human that can finish the last 20%, not even if they put up 200% of the effort required. We still don't have a working AI solution for driving, a well understood and very limited problem domain, never-mind the infinite domain of all problems that can be explained in natural language and solved with software. What you end up with is a fancier autocomplete, not an AI coder. Boilerplate and coder output might simply increase to take advantage of the new more productive way of generating source code, just like they did for the last decades whenever there was a "revolutionary" new tech, like high level languages, source control, IDEs and debuggers, component distribution etc. etc.
  - basch 4y ago
    You’re already limiting your imagination to “coding.” These are data transformers that can transform raw data without coding at all. At what point does a model itself replace code? It’s sort of like a CPU, right. You can have hardware that specialized, or general purpose hardware that can do anything once instructed. LLMs have the ability to be general purpose data manipulators without first having to be designed (or coded) to perform a task.
cabirum 4y ago
The keyword is "hype". It seems like any new "thing", no matter how useless, will get its hype cycle rolling. Crypto, NFT, Blockchain, AR, Metaverse, -- from the top of my head -- now AI. The point of hype is to attract investment. Big Corps must be driven by the fear of missing out on yet another world changing shiny new thing.
- ctoth 4y ago
  Is your assertion that because there have been other hyped things in the past, that nothing which is spoken of positively will ever actually be useful? Because you're gonna miss some pretty big stuff with those sort of glasses on. You know what else was hyped? Most everything you use today. Sometimes people use something and are absolutely blown away by it and are excited to talk about it. Not everything is 100% fake yet, I promise.
  - macNchz 4y ago
    I think there is a degree of fatigue from the stream of breathless "this is going to change the world, if you disagree you're wrong or don't understand, and if you don't participate you'll wind up poor" takes. We're barely one year out from nearly identical language around NFTs and "web3". IMO these AI technologies have obviously more tangible utility than some of the other hyped things on the list, however a lot remains to be seen about where they go.
908B64B197 4y ago
> only need to "check the code is good" ... because we all know proving correctness is the easy part of writing software! I can't wait to read about software engineers finding out some MBA had a huge codebase written by a language model and a few offshored contractors only to realize it's incredibly bugged and being hired to "just go and find the mistakes the error the ai made, should be easy all the code is written".
mtrycz2 4y ago
It can appear reasonably smart on the surface, but all it is is a stochastic parrot. It cannot reason with you about the code. To best illustrate what I mean, watch this chess match[0] it's quite riveting. Since it read millions of matches, it can predict a legal move most of the time, and even some good moves some of the time, but it cannot "understand" the rules of chess, and makes some hilariously illegal moves, especially if the match lasts longer. [0] https://www.reddit.com/r/AnarchyChess/comments/10ydnbb/i_placed_stockfish_white_against_chatgpt_black/ https://www.reddit.com/r/AnarchyChess/comments/10ydnbb/i_pla...
- Vanclief 4y ago
  Exactly and I personally think that will always be the largest limiter to how good can the technology get. No matter how good the stochastic parrot gets, its still a parrot.
- Verdex 4y ago
  On a similar vein, I tried to get chatgpt to play wordle. The result looked something like: Me: crane GPT: _ _ _ _ e Me: moist GPT: _ _ _ r _ Me: glyph GPT: you guessed it, the word was glyph Now, maybe GPT 4 or other future developments will give better results, but to me this highlights exactly what you're saying. LLMs do not have an internal structure in their 'minds' that they're pondering about. It's a very impressive engine for guessing the next character to produce into a stream. There's definitely usages for this, but not what a lot of people are saying.
  - mtrycz2 4y ago
    > you guessed it, the word was glyph My pet conspiracy theory is that is is wired to please the user, to get better coverage from the media and social media.
    - codetrotter 4y ago
      I don’t think so. In Wordle you have to guess the word in six attempts. It’s a fun game and often simple. So it could be that ChatGPT picked up on a pattern in the training data where after a couple of guesses, a lot of the time people pick the right word. So statistically it might go like. Guess a word. Probably not the right one. Guess a couple more and suddenly it’s statistically likely to be the correct word, and because of that the LLM ends up outputting the congrats and so on
twelfthnight 4y ago
I think there are ways in which LLMs will be very important, especially if we are able to get access to raw models /embeddings. That will let the models be extended to create new models and use cases. For example, personally I want to search Google and not ask a chatbot questions. LLMs could still be useful for identifying SEO spam and removing it from search results. Thus LLMs improve search but aren't giving me a watered down summary of everything I'm looking for.
yoyohello13 4y ago
> To become proficient I need to write a lot of code, but the more I use LLMs, the less repetitions I get in. So in a way it feels like LLMs are going to make you a "worse" programmer by doing the work for you. I've been experiencing this myself recently. I've been using co-pilot in some side projects. I've noticed myself getting more 'lazy' as I use it more. Recently I used it when doing some old (2015) advent of code puzzles I hadn't done before. I would read the puzzle prompt and have a pretty good idea of what I wanted to do. I wrote out some comments for functions and co-pilot was able to write what I needed with minimal changes. Even though I read through co-pilot's code and understood what it was doing I don't feel like I really retained anything from the time spent. If anything, I feel like co-pilot stunts my learning.
abhaynayar 4y ago
The technology hadn't clicked for me either. Today I had to write a script for which it would have taken me maybe 30 minutes or so on my own. I asked ChatGPT (GPT-4) to write it for me, and it got it right in the first try. I just spent a few minutes checking over the code. It truly is magical when the code just runs. Later I asked it to make several non-trivial changes to the code based on more requirements I thought of, and it aced those on the first go as well. Again, I checked the code for a negligible amount of time - compared to how much it would have taken me to write the code on my own. I do think humans will slowly get worse at lower-layers of the computer stack. But I don't think there's anything inherently bad with it. Compilers are also doing the work for you, and they are making you bad at writing assembly code - but would you rather live in a world where everyone has to hand-write tedious assembly-code? Maybe, in the future, writing Python would be like what writing assembly is today. We might go down the layer-cake once in a while to work with Python code. That does not mean we give up on the gains we get from whatever layers are going to be put on top of Python.
- rakejake 4y ago
  The compiler is a deterministic tool (even undefined behaviour is documented). So you can spend some time understanding the abstractions provided to you by your compiler and then you know exactly what it is going to do with your code. What is the equivalent of this for LLMs? Is there anyway generative models can give a guarantee that this prompt will 100% translate to this assembly? As far as I understand, no. And the way autoregressive models are built I don't think this is possible. I agree that they are useful for one-offs like you said, and their ability to tailor the solution for your problem (as opposed to reading multiple answers on stackoverflow and then piecing it yourself) is quite deadly, but for anything that is even slightly consequential, you are going to have to read everything it generates. I just can't figure out how it integrates into my workflow.
  - deleted 4y ago
    [deleted]
- bee_rider 4y ago
  The article compares to Stack Overflow, but this comment makes it look more like a comparison to compilers which is a much bigger deal than some website, and actually worth paying attention to. Anyway, people still write assembly kernels, so it is just that they only do it for cases that really matter. And there are a lot more coders than there were back when every program was assembly. So, it seems like great news.
qsort 4y ago
I don't quite know how to put it, what follows is a rough draft of an idea, maybe someone can help me to reword it, or perhaps it's trash. Since its inception, computer science has had two "camps": those who believe CS is engineering, and those who believe CS is mathematics. The reason why we are seeing all of this fuss around LLMs is that they are a new front of this feud. This "extends" the usual debate on emerging technologies between Thymoetes and Laocoon. Something that works 99 times out of 100 is 99% correct from the first perspective and 100% wrong from the second. LLMs are therefore a step forward if you take the first view, a step back if you take the second. If you accept this interpretation, an interesting consequence of it is that your outlook on LLMs is entirely dependent on what amounts to your aesthetic judgement. And it's very hard not to have rather strong aesthetic judgements on what we do 40 hours a week.
- inimino 4y ago
  There's never been a camp of computer science that said anything but the truth, which is that CS is applied mathematics. However, there is a pragmatic school of hacking, which says that results are all the matters. If you're in a startup, you should be pragmatic, and worse is better. Nobody truly believes that CS is engineering.
  - teaearlgraycold 4y ago
    Software Engineering is engineering
    - inimino 4y ago
      Software engineering is actually not engineering either.
      - tester756 4y ago
        TIL I need to tell my school to change my degree on diploma What reasoning are you using to come to conclusion that software is not engineering? >The creative application of scientific principles to design or develop structures, machines, apparatus, or manufacturing processes, or works utilizing them singly or in combination; or to construct or operate the same with full cognizance of their design; or to forecast their behavior under specific operating conditions; all as respects an intended function, economics of operation and safety to life and property It is purely software engineering.
redleggedfrog 4y ago
"Does anyone feel that way? Maybe I am wrong and the technology hasn't really clicked for me yet." No, I'm trying mightily to do what Yegge is talking about in the context of the programming work I do everyday. First v3 then v4. I've given up until maybe v7 or something. The problem is it doesn't have experience with my code-base. Sure, tell it to open a file and return a stream, it'll do that (after I fix the using statements), but for what I'm doing every day it doesn't even begin to know what to do. And because I'm careful about KISS and SOLID I don't really need a lot of simple code generation. I don't see 5x productivity. I actually don't see much advantage over the built in tools in VS. Maybe I'm doing it wrong, or maybe this make sense for people who write a lot of boilerplate, but that's not a lot of what I do.
- HanClinto 4y ago
  You might look at Github Copilot then. It actually looks at my project and helps me write within the context of all of the other code that I've written.
manmal 4y ago
> To become proficient I need to write a lot of code, but the more I use LLMs, the less repetitions I get in. So in a way it feels like LLMs are going to make you a "worse" programmer by doing the work for you. You will definitely learn from LLM suggestions. The mantra „Read other people’s code“ is accurate IMO - as long as the code is at least ok-ish. I‘ve learned a ton from code that ChatGPT generated for me already.
BWStearns 4y ago
I've found it to be extremely useful when _either_ you know the language really well but you're kind of exploring some new domain, or when you know the domain really well and you're new to the language. When you know both it's just really good autocomplete, which is great but not a huge game changer. If you know neither then you're not in a position to assess the output. But when you're still learning either the tool or the space I've found GPT to be a good tool for leveraging one expertise to create the other.
gsatic 4y ago
It is overhyped (thanks to every rando broadcasting how amazed they are). There is no causal learning happening. The randos will takes their own sweet time to work it out.
xbmcuser 4y ago
look at what the web was 20 years ago and then look at what it is now. I dont get why people in the tech field where there advances every year look at gpt and say oh it does not do this or that like wtf is the tech stagnant will it not improve. You guys should be the ones that say if it can do this today how will it improve what it will be able to do tomorrow. Most advances come when there is war/competition in the next decade 100s of billions will be spend on this do you really think their will be no improvement?
- lionkor 4y ago
  Im sure you have a good point, but its difficult to grasp with such hasty writing :/
ResearchCode 4y ago
I didn't get five times as productive yet. It's something closer to a few percent or less, which makes LLMs about as useful as syntax highlighting. It's nice to have, but not essential. We will see in a few years.
aglavine 4y ago
just imagine LLMs output as input to any other device.
chordalkeyboard 4y ago
Yes to 1. and 2.
uoaei 4y ago
There is a massive gulf between "correct code" and "correct implementation" in many real-world scenarios. Business logic and baking in domain expertise into your data model is most of the work. Making the code work efficiently doesn't matter if your code doesn't even do what it's supposed to. Normally this is an argument in favor of human-in-the-loop LLM-based development -- "the human just needs to curate and verify!" However it seems all too easy to me (especially having witnessed it more than a few times) that subtle discrepancies emerge between the stakeholders' desires for the function of the code and the developers' understanding of those requests. Hopefully we reach a best-case scenario where that's all developers need to focus on, but more likely we'll see some pretty egregious things slip through the cracks (the wave will likely start with security/privacy issues before the phenomenon is recognized) as this technology matures into the common workplaces.
senko 4y ago
I'm quite proficient in Python and Django (main tools I use daily). Yet I find myself asking ChatGPT every now and then "hey how do I do <foo>", where <foo> is something I last needed to do a year or more ago. I can recognize the correct answer but don't need to search docs/net for it. The reason this is faster (for me) than Googling or using Dash/Zeal is that the answer is already in the context of what I'm trying to do, whereas if I'm only looking at the docs, I will probably need to go through several pages to get a complete picture.
ravenstine 4y ago
> 1. It is my belief that if you are proficient enough in the task at hand, it is actually a distraction to be checking "someone else code" over just writing it yourself. When I wrote the code, I know it by heart and I know what it does (or is supposed to do). At least for me, having to be creating prompts and then reviewing the code that generates is slower and takes me out of the flow. It is also more exhausting than just writing the thing myself. I'm sure there were programmers who said the same thing in regards to high-level programming languages. > 2. I am only able to check the correctness of the code, if am am proficient enough as a programmer (and possibly in the language as well). To become proficient I need to write a lot of code, but the more I use LLMs, the less repetitions I get in. So in a way it feels like LLMs are going to make you a "worse" programmer by doing the work for you. Maybe that becomes irrelevant the more that the skill of the programmer shifts from handwriting "correct" code to supervising code generators while proofreading their work, and of course providing effective acceptance criteria. There's also a massive bias towards failed predictions of the past that serves to discredit predictions that may see a greater degree of manifestation. For every time someone says "but people predicted this before and it didn't pan out", I can point to technology that did fundamentally change how an industry works and even make jobs obsolete. Seems to me a lot of programmers on HN are refusing to believe that their ability to be proficient with code may be either outdated or supplanted by the efficiency of a system that writes code that is not necessarily "elegant" in human terms. > So in a way it feels like LLMs are going to make you a "worse" programmer by doing the work for you. Most programmers aren't great at what they do to start with, whereas LLMs can only get better from here on.
hdjjhhvvhga 4y ago
Nobody can deny the fact that ChatGPT can easily generate solutions for bazillions of relatively simple problems in various programming languages. What bothers me is how often it is completely wrong and how confident it is about its solution. A sample example. I asked it to generate Terraform code for registering an organizational unit in AWS Control Tower. This is impossible because the API of Control Tower is very limited. But ChatGPT was very happy to generate a solution pretending to use the official AWS module with a made up resource. Of course, the "solution" was not working at all. But if I ask it to do a trivial task, such as attaching an OU to an organization using AWS Organizations, it can do it perfectly well. And this, for me, is the difference between a human programmer and a machine that is good at certain tasks.
- yencabulator 4y ago
  Imagine a narcissistic human programmer who is a compulsive liar and won't admit to 1) being wrong 2) something being impossible or 3) not knowing something, and instead just making up plausible sounding business synergy bullshit to please the PHB. That's pretty much every ChatGPT-programming sample I've read so far. This one thinks character `i` in elisp regexps is matched with `\i`.
avereveard 4y ago
Code requires too much precision and is entangled with legal hurdles The value here is that the llm can act as a knowledge graph were common sense is preloaded on almost every topic, so that the user can add node and edges on the graph in natural language and perform extraction in natural language And you don't need fine tuning as long as you can fit the topic in their token space, and with gpt4 reaching 32k tokens you can load a huge amount of text and perform queries on it. That's what makes the tax return example so interesting. The model has already learned a lot of common and uncommon sense so it will not need the instruction on how to process the text or parse the query. Forget coding, but everything else is great for.
anigbrowl 4y ago
I've stopped using Stack Overflow almost completely (vs 10 times a day) and I don't miss it.
beyang 4y ago
I think you make fantastic points (Sourcegraph CTO, here). This is one of the reasons why we focused on code understanding rather than code generation for the initial version of Cody (in contrast to, say, GH Copilot). For code understanding tasks, the issues with standalone LLMs is that they have a certain amount of "memory" which is limited to their training data (SO and OSS)—and even that can be unreliable. A big "a-ha" moment for us was the realization that LLMs get much more helpful and reliable when coupled with a competent context fetching mechanism that can surface relevant code snippets from your own codebase. This makes Q&A much more factually accurate (and code generations that learns from the patterns in your codebase). We don't think LLMs will ever replace human coders, but we think they can be super helpful in eliminating a lot of the tedious, boring, duplicative writing and reading code that devs do every day. The entirety of Sourcegraph (not just the LLM part) is focused on eliminating these pain points.
pjungwir 4y ago
In my experience programmers hate to read each other's code. That's why rewrites are so popular. Do they really want to read an AI's? I bet the AI writes even worse comments your predecessor. One of the more toilsome bits of coding I do personally is rebasing. I have a patch to add application-time temporal tables to the Postgres project, and I've been rebasing it for several years now. It's a pretty big patch (actually a series of four patches), so there are almost always non-trival conflicts to deal with. If ChatGPT could do that for me it would be awesome. But it's probably the hardest thing for an LLM to do. It's not a routine program that has been written thousands of times across Github projects and StackOverflow posts. Every rebase is completely new. OTOH it would be awesome if git had just a bit more intelligence around merge conflicts. . . .
- Vanclief 4y ago
  At least the AI won't complain about the refactor haha
  - droopyEyelids 4y ago
    It was trained on human data about the same subject so we have every reason to expect it'd complain
- jemmyw 4y ago
  Your first point I agree with, I've already encountered chunks of AI generated code and I don't want to read them. Second point about the comments, actually I'm seeing the AI write much better comments (i.e. some) than most devs (none).
  - nocman 4y ago
    > actually I'm seeing the AI write much better comments (i.e. some) than most devs (none). Some comments are far worse than no comments at all. I would agree that even semi-decent comments are far better than nothing. However, "no-new-information" comments are just noise, and misleading comments have a huge negative effect. I would not be surprised if an AI produced a large number of the former, and perhaps some of the latter.
nine_k 4y ago
These are good points. I think though that LLM-based tools will eventually formalize to achieve a greater precision at what's required. I suspect that they could be a base for a new crop of different, much-higher-level programming languages. Programming languages went a long way; somebody from 1960 would have hard time putting things like Haskell or even SQL into the same conceptual bin as the original Fortran. We routinely see them as programming languages though. I don't see why this trend can't continue upwards, relegating even more legwork onto the machine while talking to it in reasonably precise, standardized, domain-specific terms.
joshka 4y ago
Hard disagree. I actually think the 80/20 mentioned in this article is low if done correctly. Let's think about the software engineering process for a minute and work out which bits to automate. Here's GPT3.5 on the process (PROMPT: List the steps going from a goal or dream to a software product in software engineering) 1. Idea Generation: The first step in creating a software product is to come up with an idea or goal that the software will achieve. 2. Research: Once you have an idea, it is important to conduct research to determine the feasibility of the idea and identify any potential challenges. 3. Planning: After research, planning is necessary to determine the scope of the project, the timeline, and the resources required. 4. Design: The design phase involves creating a detailed plan for the software, including the user interface, functionality, and architecture. 5. Development: In the development phase, the software is created by writing code, testing, and debugging. 6. Testing: After development, the software must undergo rigorous testing to identify and fix any issues. 7. Deployment: Once the software is tested and ready, it is deployed to the target audience. 8. Maintenance: Finally, the software must be maintained to ensure that it continues to function properly and meets the needs of the users. Each of those steps has a back and forth with a LLM that can enhance and speed up things. You're talking about 4 as being problematic, but right now there's a lot of "human in the loop" type issues that people are encountering. Imagine having the following loop: 1. LLM has generated a list of features to implement. AI: "Does this user story look good?" Human: "Y" 2. For each feature, generate an short English explanation of the feature and steps to implement it. Your job as a human is just to confirm that the features match what you want. "Should the shopping cart 3. For each step, LLM generates tests and code to implement the feature. AI: "Shall I implement the enter address feature by doing ..." Human "Y" 4. Automatically compile the code and run the tests until all tests implemented and feature is complete according to spec. 5. Automatically document the code / feature. Generate release notes / automated demo of feature. Confirm feature looks right. AI: "Here's what I implemented... Here's how this works... Does this look good?" 6. Lint / simplify / examine code coverage / examine security issues in the the code. Automatically fix the issues. I think you also miss that the LLM can be prompted to ask you for more details. e.g. PROMPT: "I'm building a shopping cart. Ask me some questions about the implementation." 1. What programming language are you using for the implementation of the shopping cart? 2. Are you using a specific framework for the shopping cart or are you building it from scratch? 3. How are you storing the products and their information in the shopping cart? 4. How are you handling the calculation of taxes, shipping costs, and discounts in the shopping cart? 5. What payment gateway(s) are you integrating with the shopping cart? Which can then be fed back to the LLM to make choices on the features or just plain enter the answer. PROMPT: "For each question give me 3 options and note the most popular choice.", and then your answers are fed back in too. At each point you're just a Y/N/Option 1,2,3 monkey. More succinctly, in each step of the software game, it's possible to codify practices that result in good working software. Effectively LLMs allow us to build out 5GL approaches[1] + processes. And in fact, I'd bet that there's a meta task that would end up with creating the product that does this using the same methodology manually. e.g. PROMPT: "Given what we've discussed so far, what is the next prompt that would drive the solution to the product that utilizes LLMs to automatically create software products towards completion" ;) [1]: https://en.wikipedia.org/wiki/Fifth-generation_programming_language https://en.wikipedia.org/wiki/Fifth-generation_programming_l...
Idiot_in_Vain 4y ago
The current ChatGPT is just a preview of what's possible. 2 years from now it will be able to create a DB, a set of microservices and web and mobile frontends, deploy these on a cloud platform and app stores and test them, all from a 30 min chat with a person, going over a business idea on very high level. Think about for example how Windows 1.0 looked. For an expirienced DOS user it was offering very little. Expirienced DOS users were saying GUIs are over hyped. Today there are probably a few dozen people worldwide who use a computer without a GUI (or a voice interface). ChatGPT&Co will obviously make 90% of the software developers out there obsolete in just a few years. An industrial revolution is happening in the software industry.
- kikimora 4y ago
  When you need to rename a button you’ll spend another 30 minutes talking about your app because ChatGPT does not “understand” code.
a2dam 4y ago
They aren't for you in this context then. The value of an LLM that can write passable code is not to take an experienced developer and make them better, it's to take someone who can't code at all and allow them to generate code. Whereas it might make you 10% better (or whatever your estimate is), it makes them infinity times better as it allows them to do it at all, even if it's not very good. Think of it like an accessibility aid. It doesn't help people who don't need them, but for those who do it's life changing.

_qua 4y ago

Did you try GPT-4 yet? It's a huge increment over 3.5/ChatGPT

aaomidi 4y ago
Yep its actually able to create ideas that have never been done before.
- NationalPark 4y ago
  Do share!
roflyear 4y ago
It's way better. But equally slower!
ModernMech 4y ago
I’ve tried 4 and I really can’t say the results are a qualitatively better than 3.5 for the tasks I’ve been trying (which have been trying to get it to generate documentation for my project). In fact, I find 3.5 turbo the best overall model as a tool, because to quality of responses really depends on the quality of prompts, and the quality of prompts is improved by reacting to responses, which come more quickly in 3.5-turbo. So while ChatGPT-4 is still writing the first not-good response, ChatGPT-3.5-Turbo will be on the 2nd or 3rd and it will be much more cogent.

LeftHandPath 4y ago

I agree - it’s hard to enter a flow state while reviewing someone else’s - or some AI’s - code. That’s a major reason why I haven’t started using LLMs for code, personally. I am glad someone else feels this way. Maybe it’s not going to be as big a paradigm shift as I originally expected.

actionfromafar 4y ago
It will probably raise the floor a lot. The least competent (not meaning in a bad way! I was one of those) coders will be a lot more competent all of a sudden.
- uoaei 4y ago
  I'm skeptical. It's easy to make something sound correct at a first glance but that has subtle fundamental flaws that invalidate it. Knowing humans and the LGTM phenomenon, these kinds of issues will slip by quite readily.

counttheforks 4y ago

To me it's so funny when people say ChatGPT will make developers 5x more productive, because those people are basically just admitting they're not good at their jobs and assume the same holds true for everyone.

micromacrofoot 4y ago
not really, this is more like having a strangely knowledgable yet naive junior employee - I can tell gpt-4 to put something together that gets me 90% of what I need faster than I could possibly even type it, it’s reducing my known tasks

basch 4y ago

I think this is mistaking the current .01 iteration with what the technology will be able to achieve. All sorts of groundbreaking technology looks like a minor improvement over the previously refined version until it gets implemented in a way that takes advantage of its strengths, as opposed to just being plugged into old workflows. LLMs cannot be judged by their first few incarnations. What can be trained into them currently exceeds imagination. Imagination is our limiting factor. And I don’t say that from the context of “I jumped on the hype train at the end of last year”. I remember reading the 2017 Google transformer paper and thinking “whoa, this is really happening.” The fact it happened in only 5 years is pretty impressive. Im not sure many papers or innovations got my mind spinning quite like that one.

cornholio 4y ago
But there is an unanswered question of how far this technology can go based on its fundamentals. Coding is much like driving, you can't do 80% and let the human do the final 20%, because that final 20% requires reasoning about a well understood design that was implemented throughout the first 80%. If your fancy AI coder thingy can't really reason about the end task that the code is solving - and there is little to indicate that it does, or that, any moment now, technology will advance to the point that it will - then the 80% will be crap and there exists no human that can finish the last 20%, not even if they put up 200% of the effort required. We still don't have a working AI solution for driving, a well understood and very limited problem domain, never-mind the infinite domain of all problems that can be explained in natural language and solved with software. What you end up with is a fancier autocomplete, not an AI coder. Boilerplate and coder output might simply increase to take advantage of the new more productive way of generating source code, just like they did for the last decades whenever there was a "revolutionary" new tech, like high level languages, source control, IDEs and debuggers, component distribution etc. etc.
- basch 4y ago
  You’re already limiting your imagination to “coding.” These are data transformers that can transform raw data without coding at all. At what point does a model itself replace code? It’s sort of like a CPU, right. You can have hardware that specialized, or general purpose hardware that can do anything once instructed. LLMs have the ability to be general purpose data manipulators without first having to be designed (or coded) to perform a task.

cabirum 4y ago

The keyword is "hype". It seems like any new "thing", no matter how useless, will get its hype cycle rolling. Crypto, NFT, Blockchain, AR, Metaverse, -- from the top of my head -- now AI. The point of hype is to attract investment. Big Corps must be driven by the fear of missing out on yet another world changing shiny new thing.

ctoth 4y ago
Is your assertion that because there have been other hyped things in the past, that nothing which is spoken of positively will ever actually be useful? Because you're gonna miss some pretty big stuff with those sort of glasses on. You know what else was hyped? Most everything you use today. Sometimes people use something and are absolutely blown away by it and are excited to talk about it. Not everything is 100% fake yet, I promise.
- macNchz 4y ago
  I think there is a degree of fatigue from the stream of breathless "this is going to change the world, if you disagree you're wrong or don't understand, and if you don't participate you'll wind up poor" takes. We're barely one year out from nearly identical language around NFTs and "web3". IMO these AI technologies have obviously more tangible utility than some of the other hyped things on the list, however a lot remains to be seen about where they go.

908B64B197 4y ago

> only need to "check the code is good" ... because we all know proving correctness is the easy part of writing software! I can't wait to read about software engineers finding out some MBA had a huge codebase written by a language model and a few offshored contractors only to realize it's incredibly bugged and being hired to "just go and find the mistakes the error the ai made, should be easy all the code is written".

mtrycz2 4y ago

It can appear reasonably smart on the surface, but all it is is a stochastic parrot. It cannot reason with you about the code. To best illustrate what I mean, watch this chess match[0] it's quite riveting. Since it read millions of matches, it can predict a legal move most of the time, and even some good moves some of the time, but it cannot "understand" the rules of chess, and makes some hilariously illegal moves, especially if the match lasts longer. [0] https://www.reddit.com/r/AnarchyChess/comments/10ydnbb/i_placed_stockfish_white_against_chatgpt_black/ https://www.reddit.com/r/AnarchyChess/comments/10ydnbb/i_pla...

Vanclief 4y ago
Exactly and I personally think that will always be the largest limiter to how good can the technology get. No matter how good the stochastic parrot gets, its still a parrot.
Verdex 4y ago
On a similar vein, I tried to get chatgpt to play wordle. The result looked something like: Me: crane GPT: _ _ _ _ e Me: moist GPT: _ _ _ r _ Me: glyph GPT: you guessed it, the word was glyph Now, maybe GPT 4 or other future developments will give better results, but to me this highlights exactly what you're saying. LLMs do not have an internal structure in their 'minds' that they're pondering about. It's a very impressive engine for guessing the next character to produce into a stream. There's definitely usages for this, but not what a lot of people are saying.
- mtrycz2 4y ago
  > you guessed it, the word was glyph My pet conspiracy theory is that is is wired to please the user, to get better coverage from the media and social media.
  - codetrotter 4y ago
    I don’t think so. In Wordle you have to guess the word in six attempts. It’s a fun game and often simple. So it could be that ChatGPT picked up on a pattern in the training data where after a couple of guesses, a lot of the time people pick the right word. So statistically it might go like. Guess a word. Probably not the right one. Guess a couple more and suddenly it’s statistically likely to be the correct word, and because of that the LLM ends up outputting the congrats and so on

twelfthnight 4y ago

I think there are ways in which LLMs will be very important, especially if we are able to get access to raw models /embeddings. That will let the models be extended to create new models and use cases. For example, personally I want to search Google and not ask a chatbot questions. LLMs could still be useful for identifying SEO spam and removing it from search results. Thus LLMs improve search but aren't giving me a watered down summary of everything I'm looking for.

yoyohello13 4y ago

> To become proficient I need to write a lot of code, but the more I use LLMs, the less repetitions I get in. So in a way it feels like LLMs are going to make you a "worse" programmer by doing the work for you. I've been experiencing this myself recently. I've been using co-pilot in some side projects. I've noticed myself getting more 'lazy' as I use it more. Recently I used it when doing some old (2015) advent of code puzzles I hadn't done before. I would read the puzzle prompt and have a pretty good idea of what I wanted to do. I wrote out some comments for functions and co-pilot was able to write what I needed with minimal changes. Even though I read through co-pilot's code and understood what it was doing I don't feel like I really retained anything from the time spent. If anything, I feel like co-pilot stunts my learning.

abhaynayar 4y ago

The technology hadn't clicked for me either. Today I had to write a script for which it would have taken me maybe 30 minutes or so on my own. I asked ChatGPT (GPT-4) to write it for me, and it got it right in the first try. I just spent a few minutes checking over the code. It truly is magical when the code just runs. Later I asked it to make several non-trivial changes to the code based on more requirements I thought of, and it aced those on the first go as well. Again, I checked the code for a negligible amount of time - compared to how much it would have taken me to write the code on my own. I do think humans will slowly get worse at lower-layers of the computer stack. But I don't think there's anything inherently bad with it. Compilers are also doing the work for you, and they are making you bad at writing assembly code - but would you rather live in a world where everyone has to hand-write tedious assembly-code? Maybe, in the future, writing Python would be like what writing assembly is today. We might go down the layer-cake once in a while to work with Python code. That does not mean we give up on the gains we get from whatever layers are going to be put on top of Python.

rakejake 4y ago
The compiler is a deterministic tool (even undefined behaviour is documented). So you can spend some time understanding the abstractions provided to you by your compiler and then you know exactly what it is going to do with your code. What is the equivalent of this for LLMs? Is there anyway generative models can give a guarantee that this prompt will 100% translate to this assembly? As far as I understand, no. And the way autoregressive models are built I don't think this is possible. I agree that they are useful for one-offs like you said, and their ability to tailor the solution for your problem (as opposed to reading multiple answers on stackoverflow and then piecing it yourself) is quite deadly, but for anything that is even slightly consequential, you are going to have to read everything it generates. I just can't figure out how it integrates into my workflow.
- deleted 4y ago
  [deleted]
bee_rider 4y ago
The article compares to Stack Overflow, but this comment makes it look more like a comparison to compilers which is a much bigger deal than some website, and actually worth paying attention to. Anyway, people still write assembly kernels, so it is just that they only do it for cases that really matter. And there are a lot more coders than there were back when every program was assembly. So, it seems like great news.

qsort 4y ago

I don't quite know how to put it, what follows is a rough draft of an idea, maybe someone can help me to reword it, or perhaps it's trash. Since its inception, computer science has had two "camps": those who believe CS is engineering, and those who believe CS is mathematics. The reason why we are seeing all of this fuss around LLMs is that they are a new front of this feud. This "extends" the usual debate on emerging technologies between Thymoetes and Laocoon. Something that works 99 times out of 100 is 99% correct from the first perspective and 100% wrong from the second. LLMs are therefore a step forward if you take the first view, a step back if you take the second. If you accept this interpretation, an interesting consequence of it is that your outlook on LLMs is entirely dependent on what amounts to your aesthetic judgement. And it's very hard not to have rather strong aesthetic judgements on what we do 40 hours a week.

inimino 4y ago
There's never been a camp of computer science that said anything but the truth, which is that CS is applied mathematics. However, there is a pragmatic school of hacking, which says that results are all the matters. If you're in a startup, you should be pragmatic, and worse is better. Nobody truly believes that CS is engineering.
- teaearlgraycold 4y ago
  Software Engineering is engineering
  - inimino 4y ago
    Software engineering is actually not engineering either.
    - tester756 4y ago
      TIL I need to tell my school to change my degree on diploma What reasoning are you using to come to conclusion that software is not engineering? >The creative application of scientific principles to design or develop structures, machines, apparatus, or manufacturing processes, or works utilizing them singly or in combination; or to construct or operate the same with full cognizance of their design; or to forecast their behavior under specific operating conditions; all as respects an intended function, economics of operation and safety to life and property It is purely software engineering.

redleggedfrog 4y ago

"Does anyone feel that way? Maybe I am wrong and the technology hasn't really clicked for me yet." No, I'm trying mightily to do what Yegge is talking about in the context of the programming work I do everyday. First v3 then v4. I've given up until maybe v7 or something. The problem is it doesn't have experience with my code-base. Sure, tell it to open a file and return a stream, it'll do that (after I fix the using statements), but for what I'm doing every day it doesn't even begin to know what to do. And because I'm careful about KISS and SOLID I don't really need a lot of simple code generation. I don't see 5x productivity. I actually don't see much advantage over the built in tools in VS. Maybe I'm doing it wrong, or maybe this make sense for people who write a lot of boilerplate, but that's not a lot of what I do.

HanClinto 4y ago
You might look at Github Copilot then. It actually looks at my project and helps me write within the context of all of the other code that I've written.

manmal 4y ago

> To become proficient I need to write a lot of code, but the more I use LLMs, the less repetitions I get in. So in a way it feels like LLMs are going to make you a "worse" programmer by doing the work for you. You will definitely learn from LLM suggestions. The mantra „Read other people’s code“ is accurate IMO - as long as the code is at least ok-ish. I‘ve learned a ton from code that ChatGPT generated for me already.

BWStearns 4y ago

I've found it to be extremely useful when _either_ you know the language really well but you're kind of exploring some new domain, or when you know the domain really well and you're new to the language. When you know both it's just really good autocomplete, which is great but not a huge game changer. If you know neither then you're not in a position to assess the output. But when you're still learning either the tool or the space I've found GPT to be a good tool for leveraging one expertise to create the other.

gsatic 4y ago

It is overhyped (thanks to every rando broadcasting how amazed they are). There is no causal learning happening. The randos will takes their own sweet time to work it out.

xbmcuser 4y ago

look at what the web was 20 years ago and then look at what it is now. I dont get why people in the tech field where there advances every year look at gpt and say oh it does not do this or that like wtf is the tech stagnant will it not improve. You guys should be the ones that say if it can do this today how will it improve what it will be able to do tomorrow. Most advances come when there is war/competition in the next decade 100s of billions will be spend on this do you really think their will be no improvement?

lionkor 4y ago
Im sure you have a good point, but its difficult to grasp with such hasty writing :/

ResearchCode 4y ago

I didn't get five times as productive yet. It's something closer to a few percent or less, which makes LLMs about as useful as syntax highlighting. It's nice to have, but not essential. We will see in a few years.

aglavine 4y ago

just imagine LLMs output as input to any other device.

chordalkeyboard 4y ago

Yes to 1. and 2.

uoaei 4y ago

There is a massive gulf between "correct code" and "correct implementation" in many real-world scenarios. Business logic and baking in domain expertise into your data model is most of the work. Making the code work efficiently doesn't matter if your code doesn't even do what it's supposed to. Normally this is an argument in favor of human-in-the-loop LLM-based development -- "the human just needs to curate and verify!" However it seems all too easy to me (especially having witnessed it more than a few times) that subtle discrepancies emerge between the stakeholders' desires for the function of the code and the developers' understanding of those requests. Hopefully we reach a best-case scenario where that's all developers need to focus on, but more likely we'll see some pretty egregious things slip through the cracks (the wave will likely start with security/privacy issues before the phenomenon is recognized) as this technology matures into the common workplaces.

senko 4y ago

I'm quite proficient in Python and Django (main tools I use daily). Yet I find myself asking ChatGPT every now and then "hey how do I do <foo>", where <foo> is something I last needed to do a year or more ago. I can recognize the correct answer but don't need to search docs/net for it. The reason this is faster (for me) than Googling or using Dash/Zeal is that the answer is already in the context of what I'm trying to do, whereas if I'm only looking at the docs, I will probably need to go through several pages to get a complete picture.

ravenstine 4y ago

> 1. It is my belief that if you are proficient enough in the task at hand, it is actually a distraction to be checking "someone else code" over just writing it yourself. When I wrote the code, I know it by heart and I know what it does (or is supposed to do). At least for me, having to be creating prompts and then reviewing the code that generates is slower and takes me out of the flow. It is also more exhausting than just writing the thing myself. I'm sure there were programmers who said the same thing in regards to high-level programming languages. > 2. I am only able to check the correctness of the code, if am am proficient enough as a programmer (and possibly in the language as well). To become proficient I need to write a lot of code, but the more I use LLMs, the less repetitions I get in. So in a way it feels like LLMs are going to make you a "worse" programmer by doing the work for you. Maybe that becomes irrelevant the more that the skill of the programmer shifts from handwriting "correct" code to supervising code generators while proofreading their work, and of course providing effective acceptance criteria. There's also a massive bias towards failed predictions of the past that serves to discredit predictions that may see a greater degree of manifestation. For every time someone says "but people predicted this before and it didn't pan out", I can point to technology that did fundamentally change how an industry works and even make jobs obsolete. Seems to me a lot of programmers on HN are refusing to believe that their ability to be proficient with code may be either outdated or supplanted by the efficiency of a system that writes code that is not necessarily "elegant" in human terms. > So in a way it feels like LLMs are going to make you a "worse" programmer by doing the work for you. Most programmers aren't great at what they do to start with, whereas LLMs can only get better from here on.

hdjjhhvvhga 4y ago

Nobody can deny the fact that ChatGPT can easily generate solutions for bazillions of relatively simple problems in various programming languages. What bothers me is how often it is completely wrong and how confident it is about its solution. A sample example. I asked it to generate Terraform code for registering an organizational unit in AWS Control Tower. This is impossible because the API of Control Tower is very limited. But ChatGPT was very happy to generate a solution pretending to use the official AWS module with a made up resource. Of course, the "solution" was not working at all. But if I ask it to do a trivial task, such as attaching an OU to an organization using AWS Organizations, it can do it perfectly well. And this, for me, is the difference between a human programmer and a machine that is good at certain tasks.

yencabulator 4y ago
Imagine a narcissistic human programmer who is a compulsive liar and won't admit to 1) being wrong 2) something being impossible or 3) not knowing something, and instead just making up plausible sounding business synergy bullshit to please the PHB. That's pretty much every ChatGPT-programming sample I've read so far. This one thinks character `i` in elisp regexps is matched with `\i`.

avereveard 4y ago

Code requires too much precision and is entangled with legal hurdles The value here is that the llm can act as a knowledge graph were common sense is preloaded on almost every topic, so that the user can add node and edges on the graph in natural language and perform extraction in natural language And you don't need fine tuning as long as you can fit the topic in their token space, and with gpt4 reaching 32k tokens you can load a huge amount of text and perform queries on it. That's what makes the tax return example so interesting. The model has already learned a lot of common and uncommon sense so it will not need the instruction on how to process the text or parse the query. Forget coding, but everything else is great for.

anigbrowl 4y ago

I've stopped using Stack Overflow almost completely (vs 10 times a day) and I don't miss it.

beyang 4y ago

I think you make fantastic points (Sourcegraph CTO, here). This is one of the reasons why we focused on code understanding rather than code generation for the initial version of Cody (in contrast to, say, GH Copilot). For code understanding tasks, the issues with standalone LLMs is that they have a certain amount of "memory" which is limited to their training data (SO and OSS)—and even that can be unreliable. A big "a-ha" moment for us was the realization that LLMs get much more helpful and reliable when coupled with a competent context fetching mechanism that can surface relevant code snippets from your own codebase. This makes Q&A much more factually accurate (and code generations that learns from the patterns in your codebase). We don't think LLMs will ever replace human coders, but we think they can be super helpful in eliminating a lot of the tedious, boring, duplicative writing and reading code that devs do every day. The entirety of Sourcegraph (not just the LLM part) is focused on eliminating these pain points.

pjungwir 4y ago

In my experience programmers hate to read each other's code. That's why rewrites are so popular. Do they really want to read an AI's? I bet the AI writes even worse comments your predecessor. One of the more toilsome bits of coding I do personally is rebasing. I have a patch to add application-time temporal tables to the Postgres project, and I've been rebasing it for several years now. It's a pretty big patch (actually a series of four patches), so there are almost always non-trival conflicts to deal with. If ChatGPT could do that for me it would be awesome. But it's probably the hardest thing for an LLM to do. It's not a routine program that has been written thousands of times across Github projects and StackOverflow posts. Every rebase is completely new. OTOH it would be awesome if git had just a bit more intelligence around merge conflicts. . . .

Vanclief 4y ago
At least the AI won't complain about the refactor haha
- droopyEyelids 4y ago
  It was trained on human data about the same subject so we have every reason to expect it'd complain
jemmyw 4y ago
Your first point I agree with, I've already encountered chunks of AI generated code and I don't want to read them. Second point about the comments, actually I'm seeing the AI write much better comments (i.e. some) than most devs (none).
- nocman 4y ago
  > actually I'm seeing the AI write much better comments (i.e. some) than most devs (none). Some comments are far worse than no comments at all. I would agree that even semi-decent comments are far better than nothing. However, "no-new-information" comments are just noise, and misleading comments have a huge negative effect. I would not be surprised if an AI produced a large number of the former, and perhaps some of the latter.

nine_k 4y ago

These are good points. I think though that LLM-based tools will eventually formalize to achieve a greater precision at what's required. I suspect that they could be a base for a new crop of different, much-higher-level programming languages. Programming languages went a long way; somebody from 1960 would have hard time putting things like Haskell or even SQL into the same conceptual bin as the original Fortran. We routinely see them as programming languages though. I don't see why this trend can't continue upwards, relegating even more legwork onto the machine while talking to it in reasonably precise, standardized, domain-specific terms.

joshka 4y ago

Hard disagree. I actually think the 80/20 mentioned in this article is low if done correctly. Let's think about the software engineering process for a minute and work out which bits to automate. Here's GPT3.5 on the process (PROMPT: List the steps going from a goal or dream to a software product in software engineering) 1. Idea Generation: The first step in creating a software product is to come up with an idea or goal that the software will achieve. 2. Research: Once you have an idea, it is important to conduct research to determine the feasibility of the idea and identify any potential challenges. 3. Planning: After research, planning is necessary to determine the scope of the project, the timeline, and the resources required. 4. Design: The design phase involves creating a detailed plan for the software, including the user interface, functionality, and architecture. 5. Development: In the development phase, the software is created by writing code, testing, and debugging. 6. Testing: After development, the software must undergo rigorous testing to identify and fix any issues. 7. Deployment: Once the software is tested and ready, it is deployed to the target audience. 8. Maintenance: Finally, the software must be maintained to ensure that it continues to function properly and meets the needs of the users. Each of those steps has a back and forth with a LLM that can enhance and speed up things. You're talking about 4 as being problematic, but right now there's a lot of "human in the loop" type issues that people are encountering. Imagine having the following loop: 1. LLM has generated a list of features to implement. AI: "Does this user story look good?" Human: "Y" 2. For each feature, generate an short English explanation of the feature and steps to implement it. Your job as a human is just to confirm that the features match what you want. "Should the shopping cart 3. For each step, LLM generates tests and code to implement the feature. AI: "Shall I implement the enter address feature by doing ..." Human "Y" 4. Automatically compile the code and run the tests until all tests implemented and feature is complete according to spec. 5. Automatically document the code / feature. Generate release notes / automated demo of feature. Confirm feature looks right. AI: "Here's what I implemented... Here's how this works... Does this look good?" 6. Lint / simplify / examine code coverage / examine security issues in the the code. Automatically fix the issues. I think you also miss that the LLM can be prompted to ask you for more details. e.g. PROMPT: "I'm building a shopping cart. Ask me some questions about the implementation." 1. What programming language are you using for the implementation of the shopping cart? 2. Are you using a specific framework for the shopping cart or are you building it from scratch? 3. How are you storing the products and their information in the shopping cart? 4. How are you handling the calculation of taxes, shipping costs, and discounts in the shopping cart? 5. What payment gateway(s) are you integrating with the shopping cart? Which can then be fed back to the LLM to make choices on the features or just plain enter the answer. PROMPT: "For each question give me 3 options and note the most popular choice.", and then your answers are fed back in too. At each point you're just a Y/N/Option 1,2,3 monkey. More succinctly, in each step of the software game, it's possible to codify practices that result in good working software. Effectively LLMs allow us to build out 5GL approaches[1] + processes. And in fact, I'd bet that there's a meta task that would end up with creating the product that does this using the same methodology manually. e.g. PROMPT: "Given what we've discussed so far, what is the next prompt that would drive the solution to the product that utilizes LLMs to automatically create software products towards completion" ;) [1]: https://en.wikipedia.org/wiki/Fifth-generation_programming_language https://en.wikipedia.org/wiki/Fifth-generation_programming_l...

Idiot_in_Vain 4y ago

The current ChatGPT is just a preview of what's possible. 2 years from now it will be able to create a DB, a set of microservices and web and mobile frontends, deploy these on a cloud platform and app stores and test them, all from a 30 min chat with a person, going over a business idea on very high level. Think about for example how Windows 1.0 looked. For an expirienced DOS user it was offering very little. Expirienced DOS users were saying GUIs are over hyped. Today there are probably a few dozen people worldwide who use a computer without a GUI (or a voice interface). ChatGPT&Co will obviously make 90% of the software developers out there obsolete in just a few years. An industrial revolution is happening in the software industry.

kikimora 4y ago
When you need to rename a button you’ll spend another 30 minutes talking about your app because ChatGPT does not “understand” code.

a2dam 4y ago

They aren't for you in this context then. The value of an LLM that can write passable code is not to take an experienced developer and make them better, it's to take someone who can't code at all and allow them to generate code. Whereas it might make you 10% better (or whatever your estimate is), it makes them infinity times better as it allows them to do it at all, even if it's not very good. Think of it like an accessibility aid. It doesn't help people who don't need them, but for those who do it's life changing.