5 ms·

Well... yes? I don't know about this 'flow based programming' per se, but what you describe is indeed how one uses ChatGPT for programming. I don't understand t

by roel_v 4y ago

Well... yes? I don't know about this 'flow based programming' per se, but what you describe is indeed how one uses ChatGPT for programming. I don't understand the pushback on every article like this. How aren't there obvious major productivity gains when a computer can generate 80% or more of your solution, and you just have to fine tune it? Let me give you an example of what I did just this morning. In fact, ChatGPT is generating code in another tab as I'm typing this - GPT4 is quite slow (well, at least it takes longer than my attention span which has been reduced to that of a sea urchin by years of social media and usage of internet in general), especially when you let it create 50 to 100 lines at a time. I'm working on a specialized ETL tool for geospatial data. In fact, I've had several iterations of this tool over 5 or more years, but for the last few months I've been working on a new version that incorporates everything I've learned about the domain in that time. This conceptualization, the learning of what the tool should do and how all components fit together, is not something a computer can do. I would have to be able to describe it to get a computer to generate it, of course, but I couldn't a few years ago because I was still finding out what it should do to provide the most value. The reason I had several versions is because I kept learning more about what I need it to do. But many of the boring machinations I absolutely can have a computer do. For example, a part of this tool is a tiny DSL which is basically a way of specifying combinations of parameters with some dress-up. I'd been dreading making a proper parser for weeks, since a) the 'design' of this DSL has fluctuated quite a bit, and b) I'd have to dreg up my knowledge of parsers from the last time I wrote one 10 years ago. So I've been getting by using a combination of regex, split() and some hand massaging of strings. But this morning, I had ChatGPT write me the biggest part of a parser using three questions: - "Write a parser in python for the following format: [optional identifier] [open square braces] [either a list of values, separated by commas; or a list of key-value pairs, separated by commas, the keys and values are separAted by equals signs] [close square braces]. The identifier is optionally followed by an 'at' sign, and a qualifier after that. Show me the EBNF and a parser in python that implements this grammar." - "The qualifier should come before the square brackets, it's part of the identifier." - "Write some unit tests for this grammar. Include corner cases for all varieties of the allowed input, so with or without identifier, with or without qualifier, with plain lists and with lists of key/value pairs. Include cases for 0, 1 and 3 values in all places where more than one element can appear." (try it, I used GPT4 but you'll probably get similar results if you use the free version) The result I got from this has 2 mistakes: in one case I had to swap two tokens in the EBNF. This one I found by reading the docs while waiting for the output of whatever ChatGPT suggested as a fix after me copy and pasting the error message I got. So this was fixed, by me, in a few minutes (ChatGPT's suggestion for a fix made no sense). The second one, ChatGPT suggested a correct fix after I copy and pasted the error message also. So, ChatGPT gave me a skeleton grammar, complete with the syntax of the library which I had never heard about 2 hours ago. The second of my first three questions was because I gave it ambiguous input, I only thought of that part while I was writing the question and didn't bother to go back and edit it in in the right place where it would have been more obvious. Then I copy/pasted the raw output from ChatGPT in a new file, ran it, I got the issues fixed (in collab with ChatGPT) in 20 minutes or so. This would have taken me all morning and probably all day if I include all unit tests and all other details just a few months ago. How is this not a massive game changer? Now, I would not have been able to do any of this hadn't I already known what an 'EBNF grammar' was, and hadn't known how to read and implement one. This tool will not replace programmers. It will however eliminate the need for the very low skill ones, and make the rest (much) more productive.

lucubratory 4y ago
What would you estimate as the industry-wide productivity increase for programmers from this tool? I've seen estimates anywhere from 1.5x to 10x from friends, which astounds me because in any other context whatsoever a 50% productivity increase is absolute gold, let alone the wild and frankly unbelievable higher numbers. Something I'm really interested in, after we've gotten the full 32k context and multi-modal capabilities of GPT-4 and done all the Langchain/ReAct shenaniganery we can, is GPT-5, or 6. These things are coming so fast and delivering step changes in productivity, it really does feel like something new.
- roel_v 4y ago
  Oh I don't know about that, I can only comment on my own experiences. I don't have sufficient breadth of experience to be able to say how much of this would be applicable to other developers in other subfields. I do however, like you, look forward to multi modal capabilities. I fully expect it to be able to take a Balsamiq mockup and generate the code to set up the UI for that mockup, with handlers for widget events and all. I've already had it generate UI's from text descriptions, which was cumbersome (relatively speaking - hedonistic adaptation strikes hard here). Wrt to productivity - I do think it's misleading to look at it like 'it used to take me 5 hours, now only 1, so that's 5x' or some other (imo naive) quantification. It's much more about removing drudgery and thus freeing up mental resources for interesting things. Say I do a task with a certain tool in the same time it took me without it, but in a much less mentally taxing way (because that's what drudgery does to me - activate bore-out mode). From the naive metric pov there is no productivity improvement. But if that leaves me capable of doing more in the rest of the day/week because I don't have to fight myself to move on with something, I'd argue it's still a productivity gain. Does typing with a nice keyboard and in a good chair make me more 'productive'? Well, I don't think anyone will argue that I can't type on a e5 Chinese crap one while sitting on a bucket before a door on two sawhorses. I can even type and think equally fast in those circumstances. But better tools still make one more 'productive' in the larger picture, I don't think anyone with experience will deny this either. This 'productivity gain' is in that sense somewhat ethereal, in a poetic sense of that word. To me that's where these tools are at now.
- scld 4y ago

by roel_v 4y ago

lucubratory 4y ago

What would you estimate as the industry-wide productivity increase for programmers from this tool? I've seen estimates anywhere from 1.5x to 10x from friends, which astounds me because in any other context whatsoever a 50% productivity increase is absolute gold, let alone the wild and frankly unbelievable higher numbers. Something I'm really interested in, after we've gotten the full 32k context and multi-modal capabilities of GPT-4 and done all the Langchain/ReAct shenaniganery we can, is GPT-5, or 6. These things are coming so fast and delivering step changes in productivity, it really does feel like something new.

roel_v 4y ago
Oh I don't know about that, I can only comment on my own experiences. I don't have sufficient breadth of experience to be able to say how much of this would be applicable to other developers in other subfields. I do however, like you, look forward to multi modal capabilities. I fully expect it to be able to take a Balsamiq mockup and generate the code to set up the UI for that mockup, with handlers for widget events and all. I've already had it generate UI's from text descriptions, which was cumbersome (relatively speaking - hedonistic adaptation strikes hard here). Wrt to productivity - I do think it's misleading to look at it like 'it used to take me 5 hours, now only 1, so that's 5x' or some other (imo naive) quantification. It's much more about removing drudgery and thus freeing up mental resources for interesting things. Say I do a task with a certain tool in the same time it took me without it, but in a much less mentally taxing way (because that's what drudgery does to me - activate bore-out mode). From the naive metric pov there is no productivity improvement. But if that leaves me capable of doing more in the rest of the day/week because I don't have to fight myself to move on with something, I'd argue it's still a productivity gain. Does typing with a nice keyboard and in a good chair make me more 'productive'? Well, I don't think anyone will argue that I can't type on a e5 Chinese crap one while sitting on a bucket before a door on two sawhorses. I can even type and think equally fast in those circumstances. But better tools still make one more 'productive' in the larger picture, I don't think anyone with experience will deny this either. This 'productivity gain' is in that sense somewhat ethereal, in a poetic sense of that word. To me that's where these tools are at now.
scld 4y ago