13 ms·

Using ChatGPT to generate a GPT project end-to-end

235 points by ixaxaar 3y ago

lionkor 3y ago
What value does a developer deliver when their entire process is done by an LLM? Is there no desire for a creative process?
- mirekrusin 3y ago
  What value does driver deliver if entire process is done by a car?
  - lionkor 3y ago
    Not much, hence I dont need a dedicated driver to drive my car.
    - mirekrusin 3y ago
      Until bird shits on the camera, your kid vomits in the car, tire is punctured, somebody breaks window, police hails to stop, you're choking with peanut, there is a crash nearby or crash with your car and all other kind of edge cases.
      - duckmysick 3y ago
        I think the OP meant "I dont need a dedicated driver to drive my car [because I can drive it on my own]". The process is simplified so you can do it yourself if you have the right tool, instead of relying on dedicated professionals. The process can be traveling or designing and writing an app. I don't understand the point you're trying to make with those edge cases, especially choking with a peanut, but driving your own car is extremely popular, despite those.
- deleted 3y ago
  [deleted]
- Avalaxy 3y ago
  It answers that in the post: > It was more like handholding a fresh grad who had absorbed all of human knowledge but needed someone to tie various parts of that knowledge to create something useful. Also ChatGPT is bad at dealing with abstractions beyond 2 layers. > ChatGPT is definitely a productivity multiplier. I think it is rather a differential productivity multiplier, as it would enhance more the capabilities of those who already know more. If I did not understand deep learning and FAISS, or how projects are structured, I don't think I would have been able to pull this off. On the other hand, it also has some sort of a leveling effect—I have not worked on PyTorch in a while, have no idea of FAISS's new APIs, etc., but these gaps were filled in by ChatGPT.
- Lerc 3y ago
  It seems to me the author maximised his creative process by getting a machine to do the less creative bits.
- haunter 3y ago
  People had the same thoughts about cake mixes in the 40s. Oh you can just buy a cake mix? That's not cooking anymore444 https://www.youtube.com/watch?v=r6wKaLQ66r8 https://www.youtube.com/watch?v=r6wKaLQ66r8
  - qup 3y ago
    I kind of agree about the cake mixes, but not the developer. It's pretty clear to me the value the developer provided, as he elaborated about it at length near the end and said the LLM isn't getting his job anytime soon. The cake mix really isn't "cooking," by my standards. Neither is microwaving popcorn. But it's an arbitrary line, I wouldn't defend it very hard.
  - nklrta 3y ago
    They were right. The quality of home cooking has gone down dramatically throughout the 20th century.
  - AnimalMuppet 3y ago
    Fun fact: People didn't like cake mixes when they first came out, precisely because it wasn't "really cooking". Then someone (Betty Crocker?) changed the mix (and the instructions) so that the person had to add an egg, not just water. Then the humans felt like they were actually cooking, and cake mixes were more accepted. A really smart AI would leave enough for the humans to do that they feel like they're still in charge.
- qumpis 3y ago
  The dev gave the initial idea to the LLM. That's the creative process. Everything after that, arguably, is just technical details in order to realize the idea. Sure, implementation requires plenty of creativity, but of different kind.
  - jvanderbot 3y ago
    Believe me, only a dev can get this working. Maybe in the future, LLM wizards will conjure all our technology, but at this point, having a working knowledge of all APIs from 2021 is an assistive technology, not a magical code-machine. I've used LLM to generate a lot of code recently on side projects. It's a 10x jump in productivity, but it can only reliably do 50-80% of the work, and the last tail needs editing, verification, setup with infrastructure, etc. It won't read your mind, you need to iterate, re-create, and guide. And each of those 3 relies on a working knowledge of software, libraries, tech, and user experience to get right. For now.
    - ixaxaar 3y ago
      Exactly this. I doubt a non-programmer would be able to produce similar output of similar quality and completeness. Like I said, I am not losing my job yet. Maybe next year...
    - baq 3y ago
      > [snip] This is mind blowing. ‘Only 80%’ is a bonkers thing to say. > For now. The really scary part, right here.
    - qumpis 3y ago
      I hope so, speaking from the perspective of someone who wants to keep their job. But at the same time I feel it's not trivial to bring good arguments against LLMs taking over without resorting to "they can't take into account all the context and might make tiny mistakes". But maybe people can be trained to be verifiers/testers as opposed to code writers.
  - deleted 3y ago
    [deleted]
- freehorse 3y ago
  By this argument we should be writing in machine code. Is there no desire for creative process if we use compilers?
  - nklrta 3y ago
    This is not even wrong. The level of analogies people use in the first few months of $hype_cycle is mind boggling.
    - dhoe 3y ago
      It's a spot on - moving up a level in the abstraction hierarchy.
      - nklrta 3y ago
        Human: ChatGPT, please devise an algorithm that solves the traveling salesman problem in polynomial time. A: Certainly, as an AI language model I'm happy to oblige. Here is the algorithm. Human: I used a higher abstraction level! I have solved P == NP!
        freehorse 3y ago
        This is a non-example, because it is not how interaction with LLMs to write code works right now. You can check the linked page to see that this took the author several hours spanned within 3 weekends to implement, where they had lengthy back and forth discussions with chatGPT building the code. And you still need to have some model of the code structure, you need to understand what is going on to go back and forth with it. It takes care of a part of the work that is quite standard so you work in a different level, the analogy is with a compiler taking care of certain optimisation part so that you do not have to invent it every time. If you think that you can build anything meaningful with chatGPT with one line prompts I would suggest you try to engage yourself in the context of a language/paradigm you are only a novice with to see how it works and the parts that can be a good learning experience, engaging and entertaining. Do not use some obscure language and libraries because it will start hallucinating a lot though.
- bravura 3y ago
  TBH, programming has lacked creativity since complilers got within 90% as good as hand-rolled assembly. Hand-rolled assembly wasn't really fun because you could type it and get a response instantly, rather than the creative good old days of mailing in punch cards and waiting weeks for a result. Punch cards weren't fun either, because using a computer wasn't creative. Doing math by hand was. Ad nauseum. If you think that really fucking excellent portal-opening tools don't enable creativity, you just have a dim view of what creativity is.
- bobsmooth 3y ago
  Someone has to type the prompt.
- flangola7 3y ago
  Artists are way ahead of you on this.
- ChatGTP 3y ago
  My manager is pretty confused about what ChatGPT is, just in the same way the same way they’re confused about Python. So not too worried yet. They’re not going to be coding tomorrow.
zerop 3y ago
It's good at writing new code, with sufficient prompting. But the big open question as of now for engineering orgs is - can it edit existing code like developers, just by instructions. Is there any hands on experience anyone has on copilot-x?
- jvanderbot 3y ago
  Copilot without -x does this relatively well. But it's a hands-on process. I can't just give it some source files and say "go". But it can easily make you 10x. I often spend more time tab-completing than writing.
  - deleted 3y ago
    [deleted]
  - eurekin 3y ago
    Any publically available demos? I've only played around with faux pilot and it seemed that it was mostly limited by my prompting skill. Also, is copilot aware of anything outside of currently edited file?
    - throwuwu 3y ago
      60 day free trial not enough for you?
  - davidktr 3y ago
    I'm still trying to figure out how to do this. Copilot is extremely helpful. But more often than not, it suggests completions which would steer the project in another direction. It is extremely good once the code has been organised and an overall structure exists. But until then it can be a distraction.
    - zackify 3y ago
      How are you guys enjoying copilot? For me, it wastes time so often. It blocked my editor auto importing functionality and times where I really have some easy suggestion that I need, maybe 50% of the time it doesn’t suggest the right thing. I’m hopeful for -x version though.
tobr 3y ago
You don’t mention it explicitly, but I assume you’ve manually copied and pasted all the code, as well as the various patches with updates? In my experience, that quickly makes new suggestions from ChatGPT go out of sync with the actual state of the code. Did you occasionally start the conversation over and pasted in all the code you currently had, or did this not turn out to be an issue for you?
- andai 3y ago
  I asked GPT to write a program which displays the skeleton of a project, i.e. folders, files, functions, classes and methods. I put that at the top of the prompt. I had a spooky experience with a project that was written almost entirely by GPT. I gave it the skeleton and one method and asked it for a modification. It gave it and also said "don't forget to update this other method", and showed me the updated code for that too. The spooky part is, I never told it the code for that method, but it was able to tell from context what it should be. (I told it that it itself had written it, but I don't know if that made any difference: does GPT know how it "would" have done things, i.e. can predict any code it knows that it wrote?)
  - jvanderbot 3y ago
    It's very good at guessing from function names and context. This is very impressive the first few times, and very frustrating when working with existing codebases, because it assumes the existence of functions based on naming schemes. When those don't exist, you can go ask it to write them, but this is a rabbit hole that leads to more distractions than necessary (oh now we need that function, which needs this one). It starts to feel like too much work. Often, it will posit the existence of a function that is named slightly differently. This is great for helping you find corners of an API with functionality you didn't know existed, but insanely frustrating when you're just trying to get code working the first time. You end up manually verifying the API calls.
    - freehorse 3y ago
      It only works well with codebases that are probably quite well represented in its training data ime. For more obscure ones one is better off just doing it by themselves. Finetuning may be a way to overcome this, though.
amelius 3y ago
Is anyone letting an LLM code and run its code by itself, then iteratively fix any bugs in it without human intervention until it e.g. passes some black box tests? Would it be possible to significantly improve an LLM using such unsupervised sessions?
- cube2222 3y ago
  I did, works fine for short pieces of code, but the context window size quickly becomes prohibitive in a naive approach. Also, gpt-4 does this much better than gpt-3.5, but gpt-4 is really slow, so the iteration process can take tens of minutes.
- ixaxaar 3y ago
  Exactly! During the process, it seemed like if there were like two GPTs self-playing to both generate the proper prompts iteratively and the other generates like the output, all triggered by one concise command from a human - say write tests and dont stop iterating till the tests pass - basically automating the human out of the loop - could get rid of the loops fixing tests, but also take away control.
- jiggywiggy 3y ago
  Yeah at some point gpt4 loses track and just consistently is wrong. Lately I can't feed it too much info, the longer the context the more issues. With your suggestion, it doesn't know which part of the iteration is correct at the moment. For us iteration is logically but for chatgpt I think it's just more variables that make the chance of being wrong larger. So you need to build that in somehow that it can iteratively filter and prompt
  - shmoogy 3y ago
    Isn't the issue that things fall out of context and then it starts hallucinating a lot more ? Sometimes it helps to just start a new prompt
    - jiggywiggy 3y ago
braindead_in 3y ago
Combine TDD and Self debugging into a workflow and you almost have a new paradigm of software development where entire applications can be developed with a series of prompts. Software programmers have finally programmed themselves out of jobs! It's kind of poetic justice that LLMs trained on open source code is replacing us. We should have never listened to Richard Stallman. /s
- hklwrand 3y ago
  I'm not writing new open source. First, web developers have profited from the work of others while locking down their own work on servers. They became rich and started looking down on those who wrote real complex code. Now "Open"-AI launders the output while keeping their IP safe. I wonder though if the whole thing is a scheme by Microsoft to kill open source.
  - chaxor 3y ago
    In what world have web devs ever been able to look down on real software engineers?
- gwoolhurme 3y ago
  If you really do feel that way. Honest non troll question. Why not quit? There still seems plenty of software work even if there was a feedback loop in the workflow. However if you feel we’ve wrote our selves out why not do something more physical. Not to sound snarky.
- enraged_camel 3y ago
  Writing code is such a small part of software development though. It’s an even smaller portion of software engineering (which involves tons of other skills such as translating user requirements/complaints into features/bug reports, recognizing trade offs and knowing when to make which one, having historical knowledge about the project such as which approaches to a particular problem have been tried and discarded and why, and so on).
snorkel 3y ago
Good preview of the near future of software dev, but I’m also wondering of the trend of companies forbids use of AI generated code do to copyright ambiguity. I suppose pressure to reduce cost as and move faster will overcome that concern.
- w1zard0f0z 3y ago
  Besides copyright ambiguity, I think a big problem will be security: someone that knows the tech stack and the business model in detail of a bank app, for example, that was generated with ChatGpt would be able to use ChatGpt to generate for him the same (or similar) code. This would turn any ChatGtp app almost in an open source app, no? And once you know the details of the implementation it's easier to find the security holes in it.
pwillia7 3y ago
Cool -- I did something similar with the goal: Imagine and simulate an instrument that doesn't exist and ended up with this -- it even created the assets or prompts for other AIs to make assets where it couldn't, including the model https://pwillia7.github.io/echosculpt3/ https://pwillia7.github.io/echosculpt3/
- bobsmooth 3y ago
  A truly 3D instrument, interesting!
- noman-land 3y ago
  How is this an instrument? It looks like a rock.
  - stevenhuang 3y ago
    It beeps with a certain tone depending where on the rock you poke it
    - ShamelessC 3y ago
      Not on safari it doesn’t.
    - noman-land 3y ago
      Oh I didn't notice that!
  - olddustytrail 3y ago
    The second request produced a roll.
- cristiancavalli 3y ago
  Seems like this type of spherical instrument exists and this seems like a mash up of this https://images.app.goo.gl/oW38YzoYMQUndH7Z9 https://images.app.goo.gl/oW38YzoYMQUndH7Z9 And a hang https://images.app.goo.gl/TTJ9uLiiRYoNZNtX8 https://images.app.goo.gl/TTJ9uLiiRYoNZNtX8
bobsmooth 3y ago
>It was more like handholding a fresh grad who had absorbed all of human knowledge but needed someone to tie various parts of that knowledge to create something useful. 10 INT but 0 WIS, that's a good mental model for LLMs.
ineedasername 3y ago
This is not really the same, but may be interesting to some: I subscribe to ChatGPT plus for a month to check out GPT-4. The rate limits were cumbersome though and it can be easy to waste a prompt, so I started to bootstrap: I would explain my problem to 3.5 and ask it to suggest comprehensive prompts to use with 4 to maximize my limited quota. It worked very well. In the long years to come the most advance AIs may become so far removed from us that the best intermediaries will be their less advanced brethren.
- muffles 3y ago
  Thats a great idea. I'm going to start doing this. For me it also seems GPT-4 just prints out slower. I find I can get most done with 3.5 and its faster to achieve what Im looking for. Then when I'm not satisfied with the 3.5 response I can clean it up and feed over into 4.
  - visarga 3y ago
    I kick off with the 4 as there's no time to waste, and solely utilize 3.5 in API mode for my apps. It's way speedier, and if you're certain the task is doable, it's a no-brainer to employ it. Scripted uses are often like that.
- hnlmorg 3y ago
  I see this as an example of the reverse: AI is still stupid enough that it takes humans a degree of skill to craft a request which generates the desired output.
  - srcreigh 3y ago
    Kolmogorov complexity would like a word! It seems intractable for AI to read minds, there should always be some degree of skill involved in prompt writing.
    - ineedasername 3y ago
      Agreed. But I think we may reach a point where it is difficult to prompt the most advance AI (which may are may note be an LLM, though maybe contain an LLM mode) in a productive way, especially due to computational demands on resources. And so AI's with lesser, cheaper capabilities may be reasonably competent in "understanding"-- a term I user very loosely-- the problem of dealing with more advanced but resource constrained systems and assist in the best practices in prompting them.
sentrysapper 3y ago
Why the random forgetfulness?
- ixaxaar 3y ago
  I thought of that as a kind of dropout for memory, so that the network also learns to build resilience around what it stores. I was also unsure of how to manage the memory - should I reset it every iteration? resulting in it behaving more like working memory maybe - should I reset it every epoch? would that pollute the memory from previous iterations and what would happen if it got full? - finally, why not maybe delete say 0.01% of the memory or maybe 1 cell per iteration randomly, this would imply the memory would not be that reliable as biological memory behaves and the neural net has to build resilience to use it effectively (by storing multiple copies of what is really useful (hyppocampal novelty detection type?)).
c7b 3y ago
The technological singularity is approaching fast. It's pretty clear that, given terminal access and an appropriate outer loop, GPT models can iteratively create new GPT models (either by writing and executing Pyhton code, or later versions trained on LLM weights may even be able to output new weights directly). If the inner workings of the loop are sufficiently obfuscated (in the code-based version), it wouldn't necessarily be clear to us what had changed, and the model weights/architecture on their own are not interpretable. That's very close to a singularity definition (machines that self-improve faster than what our understanding can keep up with), the only open question is whether the new versions would actually be improvements. But that sounds solvable too (except that it wouldn't be easy for us to tell for sure).
- gwoolhurme 3y ago
  Could you explain in detail how that causes the singularity? I’m lost. I saw a cool tool used to make another cool tool. You saw the singularity. Where is this logic coming from? Why would terminal access which I’m fairly certain I’ve seen in autogpt, change much.
  - jimbokun 3y ago
    The cool tool makes another cool tool, which in turn makes another cool tool, faster and faster, until we really don't understand at all what the latest cool tool is doing. But it just keeps getting smarter/faster/more effective/whatever it's optimizing for. That's the basic definition of "the singularity".
    - gwoolhurme 3y ago
      I don’t see that in this specific case for one thing or how we are closer from before. Code generation and auto correction getting closer is very cool, the singularity needs to be spelled out for me how a tool making a tool will get us there. How does that solve all of the hurdles that go into AGI or the singularity. How does it change a large language model.
    - monkpit 3y ago
piqufoh 3y ago
Great article! It reminds me a lot of becoming a manager, especially this line > Finally, it was also tiring. Imagine being reduced to giving only instructions and doing code review. Reading and understanding code is tiring!
RedTwoOne 3y ago
I really appreciate the well documented, well commented code. It's usually hard to find energy to do this on personal projects, if nothing else that aspect makes GBT coding worth it to me Especially in this domain, most of the pytorch projects are impossible to read with the lack of comments and obscure variable names
durdn 3y ago
I liked this metaphor: >It was more like handholding a fresh grad who had absorbed all of human knowledge but needed someone to tie various parts of that knowledge to create something useful.
- leetbulb 3y ago
  This has definitely been my experience. I experiment every so often with ChatGPT, usually having it create a simple multiplayer browser-based app with a server-side backend. The most recent being a collaborative pixel art app similar to /r/place. Usually by the time ChatGPT generates something that actually works, after some guidance and generally minimal code edits (usually due to its context loss), I could've written a far more optimized version myself. Its capabilities are still extremely impressive nonetheless and I look forward to future iterations of this technology. Super nice tool to have for mundane code generation and it'll only get better. Really wish I could use anything like this at work to generate tests... it's really good at that.
hoangbao0201 3y ago
[flagged]
anotherpaulg 3y ago
I had chat gpt 3.5 build a small web app for me too. I have since been building some tooling for this sort of GPT-assisted programming. https://github.com/paul-gauthier/easy-chat https://github.com/paul-gauthier/easy-chat
- samstave 3y ago
  This deserves its own post, but that aside, please expand. And take it from the perspective of ELI5 where one needs to basically set this up on a fresh install, what does it take?
  - anotherpaulg 3y ago
    The README in that repo tells the story of building that particular app. I recently got access to gpt-4, and the tooling I've built has become much more reliable. I will likely polish it up and put it onto GitHub sometime soon.
- drdaeman 3y ago
  How good was the process? I've had GPT3.5 teach me how to make a watchOS application to control a BLE device, and after a couple evenings I have one working on my watch right now. On the bright side, it gave some concrete hands-on guidance that I just wasn't been able to get from Apple documentation. It quickly gave me enough pointers to make things click in my head. While I've never wrote a single line for any Apple ecosystem before, I have a fair amount of experiences in various other niches, so I just needed a crash-course introduction showing me the ropes until I start to see the similarities with things I already know about - and it gave me exactly what I wanted. Unlike beginner "write your first app" blog articles, this crash course was tailored specifically to my requirements, which is incredibly helpful. However, on the downside, the code it wrote was awful, lacking any design. It gave me enough to get familiar with Swift syntax and how to setup CoreBluetooth, but the overall architecture was non-existent - just some spaghetti that kind-of worked (save for some bugs that were easier to fix on my own than explain to GPT) in my described happy scenario. It was like a junior developer would've grabbed some pieces from StackOverflow. In my attempts to bring things to order, I've hit a knowledge cutoff barrier (it has no clue about latest Swift with new its async syntax or Xcode 14 specifics - as a few things were moved around) and heavy hallucinations (ChatGPT actively trying to refer to various classes and methods that never existed). Still, I'm impressed. Not a replacement for a developer, but definitely a huge boon. Wonder what others' experiences and opinions are.
rapiz 3y ago
Every post that claimed using ChatGPT to achieve non-trivial tasks turned out to have non-trivial human intervention. > (from the original article) In fact, I found it better to let ChatGPT generate a toy-ish version of the code first, then let it add things to it step-by-step. This resulted in much better output than, say, asking ChatGPT to generate production-quality code with all features in the first go. This also gave me a way to break down my requirements and feed them one at a time - as I was also acting as a code-reviewer for the generated output, and so this method was also easier for me to work with. It takes a human who really knows the area to instruct ChatGPT and review the output, point out silly mistakes in the generated non-sense, and start next iteration. This kind of curated posts always cut off the most part of the conversations and the failed attempts, and then concatenate successful attempts with outputs of quality. Sure, it will be helpful as a super-IntelliSense. But not as helpful as the post suggested. I've tried to do something like in the post, but I was quickly bored with waiting output, reviewing, all the iterations. One important aspect about programming is that reading code may not be easier than writing code. And in my case, it's more painful.
- MattRix 3y ago
  Ok but this is extremely new tech, all of that stuff will get better over time, and the AI will require less and less intervention.
  - rapiz 3y ago
    I don't think so. Ultimately there's not enough information in prompts to produce "correct" code. And any attempt to deliver more information will result in a worse programming language, or as it is now, more iterations.
    - janeway 3y ago
      Seems like if it can eventually test that the output meets the criteria then it will excel.
      - 3y ago
franze 3y ago
I directed chatGPT to code https://chrome.google.com/webstore/detail/franz-ai-text-rewriter/lclcncbdhfpolppomniofbkiaaebaefg?hl=en https://chrome.google.com/webstore/detail/franz-ai-text-rewr... a chrome extension which rewrites any content in place on any website. not a single line of code was written by me by hand. but damn many prompts.
- rapiz 3y ago
  Sometimes it's really hard to say "damn many prompts" is better than writing on your own.
  - overbytecode 3y ago
    One of the most frustrating about this approach is that it feels asymptotic, you’re always approaching but never arriving at a solution. You start seeing diminishing returns on further prompting. It’s great for scaffolding, but not that great for non-trivial end-to-end projects.
akiselev 3y ago
I've found the best way to pair program with ChatGPT is with GPT4 API through a VSCode extension by @jakear [1] that uses the Notebook interface. Instead of setting a language for each cell, you set roles like "system", "user", or "assistant" and when you run a cell it sends the cells as chat messages. A huge benefit of this format is that you can delete cells, edit the responses from GPT4 to incorporate changes from future queries, and even rearrange or add mock assistant messages to prime the conversation. As ChatGPT suggests changes, I incorporate them into the main code cells and replace the old queries/feedback with new queries feedback. Since the old changes are incorporated into the parent cells, it loses track a lot less and I can also touch it up to use the right file paths, APIs, etc when it messes up. You can go a step further and monitor the llm file with inotify and extract assistant messages, infer the file path from the responses, and automatically write them to file as you update the notebook. That eliminates the back and forth copy pasting. It'd be nice to extend that interface to include Jupyter notebook cells so we can use ChatGPT to generate notebook cells that can be parsed and executed in the interface directly. Edit to add another tip: I use a variation of the below system prompt for working on larger sessions. Each user message begins with a file path and contains a code block with the contents of the file. After each user message containing a file, I manually add an assistant message that just says "continue", which allows adding several files at different paths. The last user message, the one I actually execute, contains the <request> tokens and the description of the modifications I want in the code. I incorporate the suggested changes into the messages then rinse and repeat. Prompt (sadly I forgot to record where I found it): You are a Rust AI programming assistant. The user will send you the relevant code over several requests. Please reply "continue" until you receive a message from the user starting with the tokens "<request>". Upon receiving a message from the user starting with the tokens "<request>" please carry out the request with reference to the code that the user previously sent. Assume the user is a senior software engineer who needs minimal instruction. Limit your commentary as much as possible. Under ideal circumstances, your response should just be code with no commentary. In some cases, commentary may be necessary: for example, to correct a faulty assumption of the user or to indicate into which file the code should be placed. [1] https://news.ycombinator.com/item?id=35605159 https://news.ycombinator.com/item?id=35605159
savbell 3y ago
I've also made six small apps completely coded by ChatGPT (with GitHub Copilot contributing a bit as well). Here are the two largest: PlaylistGPT (https://github.com/savbell/playlist-gpt https://github.com/savbell/playlist-gpt): A fun little web app that allows you to ask questions about your Spotify playlists and receive answers from Python code generated by OpenAI's models. I even added a feature where if the code written by GPT runs into errors, it can send the code and the error back to the model and ask it to fix it. It actually can debug itself quite often! One of the most impressive things for me was how it was able to model the UI after the Spotify app with little more than me asking it to do exactly that. WhisperWriter (https://github.com/savbell/whisper-writer https://github.com/savbell/whisper-writer): A small speech-to-text app that uses OpenAI's Whisper API to auto-transcribe recordings from a user's microphone. It waits for a keyboard shortcut to be pressed, then records from the user's microphone until it detects a pause in their speech, and then types out the Whisper transcription to the active window. It only took me two hours to get a working prototype up and running, with additions such as graphic indicators taking a few more hours to implement. I created the first for fun and the second to help me overcome a disability that impacts my ability to use a keyboard. I now use WhisperWriter literally every day (I'm even typing part of this comment with it), and I used it to prompt ChatGPT to write the code for a few additional personal projects that improve my quality-of-life in small ways. If people are interested, I may write up more about the prompting and pair programming process, since I definitely learned a lot as I worked through these, including some similar lessons to the article! Personally, I am super excited about the possibilities these AI technologies open up for people like me, who may be facing small challenges that could be easily solved with a tiny app written in a few hours tailored specifically to their problem. I had been struggling to use my desktop computer because the Windows Dictation tool was very broken for me, but now I feel like I can use it to my full capacity again because I can type with WhisperWriter. Coding now takes a minimal amount of keyboard use thanks to these AI coding assistants -- and I am super grateful for that!
- daqhris 3y ago
  Spot on, right! Glad you achieved all of the above. By design, tech advances to enhance human's ability to create. In your case, the AI tech (LLMs) truly augment your own capabilities. Therefore reaching the comfort that others enjoy freely. Hope to see more use cases, like yours, brought forward to inspire some anxious humans who are terrified by the rapid advancement of AI tech.
vrglvrglvrgl 3y ago
[dead]
m3kw9 3y ago
When you actually work with a high level engineer they can do a lot automouusly and can cut through ambiguous instructions based on experience, but they also require interactions that clarify important decision points and there are many. Gpt-x is miles away from this outcome
stuckkeys 3y ago
As someone who always wanted to work on multiple projects but was lacking the time and manpower. GPT has truly made that possible. Looking forward to GPT5.
1024core 3y ago
I just asked ChatGPT (v4) to write an app (a trivial one), and it refused: As an AI language model, I'm unable to write an entire app for you. I guess they've shut down that capability...
- SamPatt 3y ago
  It would be trivially easy to get it to do this, just not in one prompt. Ask it for help in making your app instead of asking it for the whole thing.
MuffinFlavored 3y ago
> Also ChatGPT is bad at dealing with abstractions beyond 2 layers. I wonder if this will be true in 5 years.

235 points by ixaxaar 3y ago

What value does a developer deliver when their entire process is done by an LLM? Is there no desire for a creative process?

mirekrusin 3y ago
What value does driver deliver if entire process is done by a car?
- lionkor 3y ago
  Not much, hence I dont need a dedicated driver to drive my car.
  - mirekrusin 3y ago
    Until bird shits on the camera, your kid vomits in the car, tire is punctured, somebody breaks window, police hails to stop, you're choking with peanut, there is a crash nearby or crash with your car and all other kind of edge cases.
    - duckmysick 3y ago
      I think the OP meant "I dont need a dedicated driver to drive my car [because I can drive it on my own]". The process is simplified so you can do it yourself if you have the right tool, instead of relying on dedicated professionals. The process can be traveling or designing and writing an app. I don't understand the point you're trying to make with those edge cases, especially choking with a peanut, but driving your own car is extremely popular, despite those.
deleted 3y ago
[deleted]
Avalaxy 3y ago
It answers that in the post: > It was more like handholding a fresh grad who had absorbed all of human knowledge but needed someone to tie various parts of that knowledge to create something useful. Also ChatGPT is bad at dealing with abstractions beyond 2 layers. > ChatGPT is definitely a productivity multiplier. I think it is rather a differential productivity multiplier, as it would enhance more the capabilities of those who already know more. If I did not understand deep learning and FAISS, or how projects are structured, I don't think I would have been able to pull this off. On the other hand, it also has some sort of a leveling effect—I have not worked on PyTorch in a while, have no idea of FAISS's new APIs, etc., but these gaps were filled in by ChatGPT.
Lerc 3y ago
It seems to me the author maximised his creative process by getting a machine to do the less creative bits.
haunter 3y ago
People had the same thoughts about cake mixes in the 40s. Oh you can just buy a cake mix? That's not cooking anymore444 https://www.youtube.com/watch?v=r6wKaLQ66r8 https://www.youtube.com/watch?v=r6wKaLQ66r8
- qup 3y ago
  I kind of agree about the cake mixes, but not the developer. It's pretty clear to me the value the developer provided, as he elaborated about it at length near the end and said the LLM isn't getting his job anytime soon. The cake mix really isn't "cooking," by my standards. Neither is microwaving popcorn. But it's an arbitrary line, I wouldn't defend it very hard.
- nklrta 3y ago
  They were right. The quality of home cooking has gone down dramatically throughout the 20th century.
- AnimalMuppet 3y ago
  Fun fact: People didn't like cake mixes when they first came out, precisely because it wasn't "really cooking". Then someone (Betty Crocker?) changed the mix (and the instructions) so that the person had to add an egg, not just water. Then the humans felt like they were actually cooking, and cake mixes were more accepted. A really smart AI would leave enough for the humans to do that they feel like they're still in charge.
qumpis 3y ago
The dev gave the initial idea to the LLM. That's the creative process. Everything after that, arguably, is just technical details in order to realize the idea. Sure, implementation requires plenty of creativity, but of different kind.
- jvanderbot 3y ago
  Believe me, only a dev can get this working. Maybe in the future, LLM wizards will conjure all our technology, but at this point, having a working knowledge of all APIs from 2021 is an assistive technology, not a magical code-machine. I've used LLM to generate a lot of code recently on side projects. It's a 10x jump in productivity, but it can only reliably do 50-80% of the work, and the last tail needs editing, verification, setup with infrastructure, etc. It won't read your mind, you need to iterate, re-create, and guide. And each of those 3 relies on a working knowledge of software, libraries, tech, and user experience to get right. For now.
  - ixaxaar 3y ago
    Exactly this. I doubt a non-programmer would be able to produce similar output of similar quality and completeness. Like I said, I am not losing my job yet. Maybe next year...
  - baq 3y ago
    > [snip] This is mind blowing. ‘Only 80%’ is a bonkers thing to say. > For now. The really scary part, right here.
  - qumpis 3y ago
    I hope so, speaking from the perspective of someone who wants to keep their job. But at the same time I feel it's not trivial to bring good arguments against LLMs taking over without resorting to "they can't take into account all the context and might make tiny mistakes". But maybe people can be trained to be verifiers/testers as opposed to code writers.
- deleted 3y ago
  [deleted]
freehorse 3y ago
By this argument we should be writing in machine code. Is there no desire for creative process if we use compilers?
- nklrta 3y ago
  This is not even wrong. The level of analogies people use in the first few months of $hype_cycle is mind boggling.
  - dhoe 3y ago
    It's a spot on - moving up a level in the abstraction hierarchy.
    - nklrta 3y ago
      Human: ChatGPT, please devise an algorithm that solves the traveling salesman problem in polynomial time. A: Certainly, as an AI language model I'm happy to oblige. Here is the algorithm. Human: I used a higher abstraction level! I have solved P == NP!
      - freehorse 3y ago
        This is a non-example, because it is not how interaction with LLMs to write code works right now. You can check the linked page to see that this took the author several hours spanned within 3 weekends to implement, where they had lengthy back and forth discussions with chatGPT building the code. And you still need to have some model of the code structure, you need to understand what is going on to go back and forth with it. It takes care of a part of the work that is quite standard so you work in a different level, the analogy is with a compiler taking care of certain optimisation part so that you do not have to invent it every time. If you think that you can build anything meaningful with chatGPT with one line prompts I would suggest you try to engage yourself in the context of a language/paradigm you are only a novice with to see how it works and the parts that can be a good learning experience, engaging and entertaining. Do not use some obscure language and libraries because it will start hallucinating a lot though.
bravura 3y ago
TBH, programming has lacked creativity since complilers got within 90% as good as hand-rolled assembly. Hand-rolled assembly wasn't really fun because you could type it and get a response instantly, rather than the creative good old days of mailing in punch cards and waiting weeks for a result. Punch cards weren't fun either, because using a computer wasn't creative. Doing math by hand was. Ad nauseum. If you think that really fucking excellent portal-opening tools don't enable creativity, you just have a dim view of what creativity is.
bobsmooth 3y ago
Someone has to type the prompt.
flangola7 3y ago
Artists are way ahead of you on this.
ChatGTP 3y ago
My manager is pretty confused about what ChatGPT is, just in the same way the same way they’re confused about Python. So not too worried yet. They’re not going to be coding tomorrow.

zerop 3y ago

It's good at writing new code, with sufficient prompting. But the big open question as of now for engineering orgs is - can it edit existing code like developers, just by instructions. Is there any hands on experience anyone has on copilot-x?

jvanderbot 3y ago
Copilot without -x does this relatively well. But it's a hands-on process. I can't just give it some source files and say "go". But it can easily make you 10x. I often spend more time tab-completing than writing.
- deleted 3y ago
  [deleted]
- eurekin 3y ago
  Any publically available demos? I've only played around with faux pilot and it seemed that it was mostly limited by my prompting skill. Also, is copilot aware of anything outside of currently edited file?
  - throwuwu 3y ago
    60 day free trial not enough for you?
- davidktr 3y ago
  I'm still trying to figure out how to do this. Copilot is extremely helpful. But more often than not, it suggests completions which would steer the project in another direction. It is extremely good once the code has been organised and an overall structure exists. But until then it can be a distraction.
  - zackify 3y ago
    How are you guys enjoying copilot? For me, it wastes time so often. It blocked my editor auto importing functionality and times where I really have some easy suggestion that I need, maybe 50% of the time it doesn’t suggest the right thing. I’m hopeful for -x version though.

tobr 3y ago

You don’t mention it explicitly, but I assume you’ve manually copied and pasted all the code, as well as the various patches with updates? In my experience, that quickly makes new suggestions from ChatGPT go out of sync with the actual state of the code. Did you occasionally start the conversation over and pasted in all the code you currently had, or did this not turn out to be an issue for you?

andai 3y ago
I asked GPT to write a program which displays the skeleton of a project, i.e. folders, files, functions, classes and methods. I put that at the top of the prompt. I had a spooky experience with a project that was written almost entirely by GPT. I gave it the skeleton and one method and asked it for a modification. It gave it and also said "don't forget to update this other method", and showed me the updated code for that too. The spooky part is, I never told it the code for that method, but it was able to tell from context what it should be. (I told it that it itself had written it, but I don't know if that made any difference: does GPT know how it "would" have done things, i.e. can predict any code it knows that it wrote?)
- jvanderbot 3y ago
  It's very good at guessing from function names and context. This is very impressive the first few times, and very frustrating when working with existing codebases, because it assumes the existence of functions based on naming schemes. When those don't exist, you can go ask it to write them, but this is a rabbit hole that leads to more distractions than necessary (oh now we need that function, which needs this one). It starts to feel like too much work. Often, it will posit the existence of a function that is named slightly differently. This is great for helping you find corners of an API with functionality you didn't know existed, but insanely frustrating when you're just trying to get code working the first time. You end up manually verifying the API calls.
  - freehorse 3y ago
    It only works well with codebases that are probably quite well represented in its training data ime. For more obscure ones one is better off just doing it by themselves. Finetuning may be a way to overcome this, though.

amelius 3y ago

Is anyone letting an LLM code and run its code by itself, then iteratively fix any bugs in it without human intervention until it e.g. passes some black box tests? Would it be possible to significantly improve an LLM using such unsupervised sessions?

cube2222 3y ago
I did, works fine for short pieces of code, but the context window size quickly becomes prohibitive in a naive approach. Also, gpt-4 does this much better than gpt-3.5, but gpt-4 is really slow, so the iteration process can take tens of minutes.
ixaxaar 3y ago
Exactly! During the process, it seemed like if there were like two GPTs self-playing to both generate the proper prompts iteratively and the other generates like the output, all triggered by one concise command from a human - say write tests and dont stop iterating till the tests pass - basically automating the human out of the loop - could get rid of the loops fixing tests, but also take away control.
jiggywiggy 3y ago
Yeah at some point gpt4 loses track and just consistently is wrong. Lately I can't feed it too much info, the longer the context the more issues. With your suggestion, it doesn't know which part of the iteration is correct at the moment. For us iteration is logically but for chatgpt I think it's just more variables that make the chance of being wrong larger. So you need to build that in somehow that it can iteratively filter and prompt
- shmoogy 3y ago
  Isn't the issue that things fall out of context and then it starts hallucinating a lot more ? Sometimes it helps to just start a new prompt
  - jiggywiggy 3y ago

braindead_in 3y ago

Combine TDD and Self debugging into a workflow and you almost have a new paradigm of software development where entire applications can be developed with a series of prompts. Software programmers have finally programmed themselves out of jobs! It's kind of poetic justice that LLMs trained on open source code is replacing us. We should have never listened to Richard Stallman. /s

hklwrand 3y ago
I'm not writing new open source. First, web developers have profited from the work of others while locking down their own work on servers. They became rich and started looking down on those who wrote real complex code. Now "Open"-AI launders the output while keeping their IP safe. I wonder though if the whole thing is a scheme by Microsoft to kill open source.
- chaxor 3y ago
  In what world have web devs ever been able to look down on real software engineers?
gwoolhurme 3y ago
If you really do feel that way. Honest non troll question. Why not quit? There still seems plenty of software work even if there was a feedback loop in the workflow. However if you feel we’ve wrote our selves out why not do something more physical. Not to sound snarky.
enraged_camel 3y ago
Writing code is such a small part of software development though. It’s an even smaller portion of software engineering (which involves tons of other skills such as translating user requirements/complaints into features/bug reports, recognizing trade offs and knowing when to make which one, having historical knowledge about the project such as which approaches to a particular problem have been tried and discarded and why, and so on).

snorkel 3y ago

Good preview of the near future of software dev, but I’m also wondering of the trend of companies forbids use of AI generated code do to copyright ambiguity. I suppose pressure to reduce cost as and move faster will overcome that concern.

w1zard0f0z 3y ago
Besides copyright ambiguity, I think a big problem will be security: someone that knows the tech stack and the business model in detail of a bank app, for example, that was generated with ChatGpt would be able to use ChatGpt to generate for him the same (or similar) code. This would turn any ChatGtp app almost in an open source app, no? And once you know the details of the implementation it's easier to find the security holes in it.

pwillia7 3y ago

Cool -- I did something similar with the goal: Imagine and simulate an instrument that doesn't exist and ended up with this -- it even created the assets or prompts for other AIs to make assets where it couldn't, including the model https://pwillia7.github.io/echosculpt3/ https://pwillia7.github.io/echosculpt3/

bobsmooth 3y ago
A truly 3D instrument, interesting!
noman-land 3y ago
How is this an instrument? It looks like a rock.
- stevenhuang 3y ago
  It beeps with a certain tone depending where on the rock you poke it
  - ShamelessC 3y ago
    Not on safari it doesn’t.
  - noman-land 3y ago
    Oh I didn't notice that!
- olddustytrail 3y ago
  The second request produced a roll.
cristiancavalli 3y ago
Seems like this type of spherical instrument exists and this seems like a mash up of this https://images.app.goo.gl/oW38YzoYMQUndH7Z9 https://images.app.goo.gl/oW38YzoYMQUndH7Z9 And a hang https://images.app.goo.gl/TTJ9uLiiRYoNZNtX8 https://images.app.goo.gl/TTJ9uLiiRYoNZNtX8

bobsmooth 3y ago

>It was more like handholding a fresh grad who had absorbed all of human knowledge but needed someone to tie various parts of that knowledge to create something useful. 10 INT but 0 WIS, that's a good mental model for LLMs.

ineedasername 3y ago

This is not really the same, but may be interesting to some: I subscribe to ChatGPT plus for a month to check out GPT-4. The rate limits were cumbersome though and it can be easy to waste a prompt, so I started to bootstrap: I would explain my problem to 3.5 and ask it to suggest comprehensive prompts to use with 4 to maximize my limited quota. It worked very well. In the long years to come the most advance AIs may become so far removed from us that the best intermediaries will be their less advanced brethren.

muffles 3y ago
Thats a great idea. I'm going to start doing this. For me it also seems GPT-4 just prints out slower. I find I can get most done with 3.5 and its faster to achieve what Im looking for. Then when I'm not satisfied with the 3.5 response I can clean it up and feed over into 4.
- visarga 3y ago
  I kick off with the 4 as there's no time to waste, and solely utilize 3.5 in API mode for my apps. It's way speedier, and if you're certain the task is doable, it's a no-brainer to employ it. Scripted uses are often like that.
hnlmorg 3y ago
I see this as an example of the reverse: AI is still stupid enough that it takes humans a degree of skill to craft a request which generates the desired output.
- srcreigh 3y ago
  Kolmogorov complexity would like a word! It seems intractable for AI to read minds, there should always be some degree of skill involved in prompt writing.
  - ineedasername 3y ago
    Agreed. But I think we may reach a point where it is difficult to prompt the most advance AI (which may are may note be an LLM, though maybe contain an LLM mode) in a productive way, especially due to computational demands on resources. And so AI's with lesser, cheaper capabilities may be reasonably competent in "understanding"-- a term I user very loosely-- the problem of dealing with more advanced but resource constrained systems and assist in the best practices in prompting them.

sentrysapper 3y ago

Why the random forgetfulness?

ixaxaar 3y ago
I thought of that as a kind of dropout for memory, so that the network also learns to build resilience around what it stores. I was also unsure of how to manage the memory - should I reset it every iteration? resulting in it behaving more like working memory maybe - should I reset it every epoch? would that pollute the memory from previous iterations and what would happen if it got full? - finally, why not maybe delete say 0.01% of the memory or maybe 1 cell per iteration randomly, this would imply the memory would not be that reliable as biological memory behaves and the neural net has to build resilience to use it effectively (by storing multiple copies of what is really useful (hyppocampal novelty detection type?)).

c7b 3y ago

The technological singularity is approaching fast. It's pretty clear that, given terminal access and an appropriate outer loop, GPT models can iteratively create new GPT models (either by writing and executing Pyhton code, or later versions trained on LLM weights may even be able to output new weights directly). If the inner workings of the loop are sufficiently obfuscated (in the code-based version), it wouldn't necessarily be clear to us what had changed, and the model weights/architecture on their own are not interpretable. That's very close to a singularity definition (machines that self-improve faster than what our understanding can keep up with), the only open question is whether the new versions would actually be improvements. But that sounds solvable too (except that it wouldn't be easy for us to tell for sure).

gwoolhurme 3y ago
Could you explain in detail how that causes the singularity? I’m lost. I saw a cool tool used to make another cool tool. You saw the singularity. Where is this logic coming from? Why would terminal access which I’m fairly certain I’ve seen in autogpt, change much.
- jimbokun 3y ago
  The cool tool makes another cool tool, which in turn makes another cool tool, faster and faster, until we really don't understand at all what the latest cool tool is doing. But it just keeps getting smarter/faster/more effective/whatever it's optimizing for. That's the basic definition of "the singularity".
  - gwoolhurme 3y ago
    I don’t see that in this specific case for one thing or how we are closer from before. Code generation and auto correction getting closer is very cool, the singularity needs to be spelled out for me how a tool making a tool will get us there. How does that solve all of the hurdles that go into AGI or the singularity. How does it change a large language model.
  - monkpit 3y ago

piqufoh 3y ago

Great article! It reminds me a lot of becoming a manager, especially this line > Finally, it was also tiring. Imagine being reduced to giving only instructions and doing code review. Reading and understanding code is tiring!

RedTwoOne 3y ago

I really appreciate the well documented, well commented code. It's usually hard to find energy to do this on personal projects, if nothing else that aspect makes GBT coding worth it to me Especially in this domain, most of the pytorch projects are impossible to read with the lack of comments and obscure variable names

durdn 3y ago

I liked this metaphor: >It was more like handholding a fresh grad who had absorbed all of human knowledge but needed someone to tie various parts of that knowledge to create something useful.

leetbulb 3y ago
This has definitely been my experience. I experiment every so often with ChatGPT, usually having it create a simple multiplayer browser-based app with a server-side backend. The most recent being a collaborative pixel art app similar to /r/place. Usually by the time ChatGPT generates something that actually works, after some guidance and generally minimal code edits (usually due to its context loss), I could've written a far more optimized version myself. Its capabilities are still extremely impressive nonetheless and I look forward to future iterations of this technology. Super nice tool to have for mundane code generation and it'll only get better. Really wish I could use anything like this at work to generate tests... it's really good at that.

hoangbao0201 3y ago

[flagged]

anotherpaulg 3y ago

I had chat gpt 3.5 build a small web app for me too. I have since been building some tooling for this sort of GPT-assisted programming. https://github.com/paul-gauthier/easy-chat https://github.com/paul-gauthier/easy-chat

samstave 3y ago
This deserves its own post, but that aside, please expand. And take it from the perspective of ELI5 where one needs to basically set this up on a fresh install, what does it take?
- anotherpaulg 3y ago
  The README in that repo tells the story of building that particular app. I recently got access to gpt-4, and the tooling I've built has become much more reliable. I will likely polish it up and put it onto GitHub sometime soon.
drdaeman 3y ago
How good was the process? I've had GPT3.5 teach me how to make a watchOS application to control a BLE device, and after a couple evenings I have one working on my watch right now. On the bright side, it gave some concrete hands-on guidance that I just wasn't been able to get from Apple documentation. It quickly gave me enough pointers to make things click in my head. While I've never wrote a single line for any Apple ecosystem before, I have a fair amount of experiences in various other niches, so I just needed a crash-course introduction showing me the ropes until I start to see the similarities with things I already know about - and it gave me exactly what I wanted. Unlike beginner "write your first app" blog articles, this crash course was tailored specifically to my requirements, which is incredibly helpful. However, on the downside, the code it wrote was awful, lacking any design. It gave me enough to get familiar with Swift syntax and how to setup CoreBluetooth, but the overall architecture was non-existent - just some spaghetti that kind-of worked (save for some bugs that were easier to fix on my own than explain to GPT) in my described happy scenario. It was like a junior developer would've grabbed some pieces from StackOverflow. In my attempts to bring things to order, I've hit a knowledge cutoff barrier (it has no clue about latest Swift with new its async syntax or Xcode 14 specifics - as a few things were moved around) and heavy hallucinations (ChatGPT actively trying to refer to various classes and methods that never existed). Still, I'm impressed. Not a replacement for a developer, but definitely a huge boon. Wonder what others' experiences and opinions are.

rapiz 3y ago

Every post that claimed using ChatGPT to achieve non-trivial tasks turned out to have non-trivial human intervention. > (from the original article) In fact, I found it better to let ChatGPT generate a toy-ish version of the code first, then let it add things to it step-by-step. This resulted in much better output than, say, asking ChatGPT to generate production-quality code with all features in the first go. This also gave me a way to break down my requirements and feed them one at a time - as I was also acting as a code-reviewer for the generated output, and so this method was also easier for me to work with. It takes a human who really knows the area to instruct ChatGPT and review the output, point out silly mistakes in the generated non-sense, and start next iteration. This kind of curated posts always cut off the most part of the conversations and the failed attempts, and then concatenate successful attempts with outputs of quality. Sure, it will be helpful as a super-IntelliSense. But not as helpful as the post suggested. I've tried to do something like in the post, but I was quickly bored with waiting output, reviewing, all the iterations. One important aspect about programming is that reading code may not be easier than writing code. And in my case, it's more painful.

MattRix 3y ago
Ok but this is extremely new tech, all of that stuff will get better over time, and the AI will require less and less intervention.
- rapiz 3y ago
  I don't think so. Ultimately there's not enough information in prompts to produce "correct" code. And any attempt to deliver more information will result in a worse programming language, or as it is now, more iterations.
  - janeway 3y ago
    Seems like if it can eventually test that the output meets the criteria then it will excel.
    - 3y ago

franze 3y ago

I directed chatGPT to code https://chrome.google.com/webstore/detail/franz-ai-text-rewriter/lclcncbdhfpolppomniofbkiaaebaefg?hl=en https://chrome.google.com/webstore/detail/franz-ai-text-rewr... a chrome extension which rewrites any content in place on any website. not a single line of code was written by me by hand. but damn many prompts.

rapiz 3y ago
Sometimes it's really hard to say "damn many prompts" is better than writing on your own.
- overbytecode 3y ago
  One of the most frustrating about this approach is that it feels asymptotic, you’re always approaching but never arriving at a solution. You start seeing diminishing returns on further prompting. It’s great for scaffolding, but not that great for non-trivial end-to-end projects.

akiselev 3y ago

I've found the best way to pair program with ChatGPT is with GPT4 API through a VSCode extension by @jakear [1] that uses the Notebook interface. Instead of setting a language for each cell, you set roles like "system", "user", or "assistant" and when you run a cell it sends the cells as chat messages. A huge benefit of this format is that you can delete cells, edit the responses from GPT4 to incorporate changes from future queries, and even rearrange or add mock assistant messages to prime the conversation. As ChatGPT suggests changes, I incorporate them into the main code cells and replace the old queries/feedback with new queries feedback. Since the old changes are incorporated into the parent cells, it loses track a lot less and I can also touch it up to use the right file paths, APIs, etc when it messes up. You can go a step further and monitor the llm file with inotify and extract assistant messages, infer the file path from the responses, and automatically write them to file as you update the notebook. That eliminates the back and forth copy pasting. It'd be nice to extend that interface to include Jupyter notebook cells so we can use ChatGPT to generate notebook cells that can be parsed and executed in the interface directly. Edit to add another tip: I use a variation of the below system prompt for working on larger sessions. Each user message begins with a file path and contains a code block with the contents of the file. After each user message containing a file, I manually add an assistant message that just says "continue", which allows adding several files at different paths. The last user message, the one I actually execute, contains the <request> tokens and the description of the modifications I want in the code. I incorporate the suggested changes into the messages then rinse and repeat. Prompt (sadly I forgot to record where I found it): You are a Rust AI programming assistant. The user will send you the relevant code over several requests. Please reply "continue" until you receive a message from the user starting with the tokens "<request>". Upon receiving a message from the user starting with the tokens "<request>" please carry out the request with reference to the code that the user previously sent. Assume the user is a senior software engineer who needs minimal instruction. Limit your commentary as much as possible. Under ideal circumstances, your response should just be code with no commentary. In some cases, commentary may be necessary: for example, to correct a faulty assumption of the user or to indicate into which file the code should be placed. [1] https://news.ycombinator.com/item?id=35605159 https://news.ycombinator.com/item?id=35605159

savbell 3y ago

I've also made six small apps completely coded by ChatGPT (with GitHub Copilot contributing a bit as well). Here are the two largest: PlaylistGPT (https://github.com/savbell/playlist-gpt https://github.com/savbell/playlist-gpt): A fun little web app that allows you to ask questions about your Spotify playlists and receive answers from Python code generated by OpenAI's models. I even added a feature where if the code written by GPT runs into errors, it can send the code and the error back to the model and ask it to fix it. It actually can debug itself quite often! One of the most impressive things for me was how it was able to model the UI after the Spotify app with little more than me asking it to do exactly that. WhisperWriter (https://github.com/savbell/whisper-writer https://github.com/savbell/whisper-writer): A small speech-to-text app that uses OpenAI's Whisper API to auto-transcribe recordings from a user's microphone. It waits for a keyboard shortcut to be pressed, then records from the user's microphone until it detects a pause in their speech, and then types out the Whisper transcription to the active window. It only took me two hours to get a working prototype up and running, with additions such as graphic indicators taking a few more hours to implement. I created the first for fun and the second to help me overcome a disability that impacts my ability to use a keyboard. I now use WhisperWriter literally every day (I'm even typing part of this comment with it), and I used it to prompt ChatGPT to write the code for a few additional personal projects that improve my quality-of-life in small ways. If people are interested, I may write up more about the prompting and pair programming process, since I definitely learned a lot as I worked through these, including some similar lessons to the article! Personally, I am super excited about the possibilities these AI technologies open up for people like me, who may be facing small challenges that could be easily solved with a tiny app written in a few hours tailored specifically to their problem. I had been struggling to use my desktop computer because the Windows Dictation tool was very broken for me, but now I feel like I can use it to my full capacity again because I can type with WhisperWriter. Coding now takes a minimal amount of keyboard use thanks to these AI coding assistants -- and I am super grateful for that!

daqhris 3y ago
Spot on, right! Glad you achieved all of the above. By design, tech advances to enhance human's ability to create. In your case, the AI tech (LLMs) truly augment your own capabilities. Therefore reaching the comfort that others enjoy freely. Hope to see more use cases, like yours, brought forward to inspire some anxious humans who are terrified by the rapid advancement of AI tech.

vrglvrglvrgl 3y ago

[dead]

m3kw9 3y ago

When you actually work with a high level engineer they can do a lot automouusly and can cut through ambiguous instructions based on experience, but they also require interactions that clarify important decision points and there are many. Gpt-x is miles away from this outcome

stuckkeys 3y ago

As someone who always wanted to work on multiple projects but was lacking the time and manpower. GPT has truly made that possible. Looking forward to GPT5.

1024core 3y ago

I just asked ChatGPT (v4) to write an app (a trivial one), and it refused: As an AI language model, I'm unable to write an entire app for you. I guess they've shut down that capability...

SamPatt 3y ago
It would be trivially easy to get it to do this, just not in one prompt. Ask it for help in making your app instead of asking it for the whole thing.

MuffinFlavored 3y ago

> Also ChatGPT is bad at dealing with abstractions beyond 2 layers. I wonder if this will be true in 5 years.