5 ms·

This is more interesting than the deluge of posts that say "I created an iOS app in 30 minutes using ChatGPT!" Which doesn't mean much because it could've done

This is more interesting than the deluge of posts that say "I created an iOS app in 30 minutes using ChatGPT!" Which doesn't mean much because it could've done nothing more than create a simple hello world. This one at least shows the finished product, which is indeed pretty impressive. Some details I'd need to know are (a) how long did it take, (b) how many prompts, (c) how many course-corrections were required, and (d) how competent this individual was with the technologies in question. I've personally found ChatGPT extremely empowering in lots of scenarios, but code generation was not among them.

cryptoz 3y ago
I’ve been playing with ChatGPT code generation to make entire sites with flask, python, html+js+css, backed with SQLite db and it’s amazing. I’ve had it write like 5k lines that are all live in prod and working (not much traffic lol but still). A huge huge factor is knowing the limitations and getting better at prompting. And identifying likely hallucinations and asking for risks etc. I’ve found it best with tech I don’t know well (I’m an android dev using it to make websites, something I haven’t done myself in like 15 years). Most of the coolest stuff for me is help with sysadmin and running the server. The ability to debug gunicorn errors is great. I do have to modify the code it outputs as the project grows and it loses context, but honestly the context limits are the biggest hurdle for bigger projects and those will be lifted soon. Edit: Most recent site I made with like 95% code from ChatGPT is https://cosmictrip.space/ https://cosmictrip.space/ which generates prompts with GPT-4 that are then used to generate space images with DALL-E. It's a simple site but there is a secret adventure game I'm working on (GPT+Dall-E) that is open-ended image+text AI-driven game. I'm hoping to launch before Nov 6 with DALL-E 3 API (hopefully...!). The adventure game is also written like 95%+ by ChatGPT. I've had such great success with it coding that I'm using the GPT-4 API with an agent I'm making (everyone is huh). I have function calling hooked up to generate structured subtasks that the agent can then write the code for, and support for files to include context, chat with your code, etc. It's not ready to show but the GPT-4 code generation abilities are really incredible - but you have to be experienced at prompting. Your first prompts aren't likely to be great, which is why I'm hoping my agent can have success. The idea of the agent I'm writing is a Jira/kanban style board where you have AI coders assigned to tasks that you can approve and modify etc. The tickets should automatically move across the columns as the AI checks the work etc.
- maxwelljoslyn 3y ago
  +1 for its suitability in helping with systems administration. One responsibility at my current job is administering a Windows server and trying to get it to do things that are easy on a Unix -- that should be easy anywhere -- but, on Windows, seem to inevitably degrade into nightmares. ChadGPT has given me huge amounts of blessed ammo to shoot at the nightmares, and there's no way I could do that portion of the job in a feasible time frame without it.
  - ChatGTP 3y ago
    ChadGPT ?
    - visarga 3y ago
      https://www.youtube.com/channel/UCR2JJPxRRWvxy6naePMpsAQ https://www.youtube.com/channel/UCR2JJPxRRWvxy6naePMpsAQ
- isoprophlex 3y ago
  I feel that trapping your AI agents in a kanban board isn't going to do your survival chances a lot of good when the robot apocalypse inevitably comes for us meatbags.
  - meiraleal 3y ago
    Parents know what's better for they kids, they will understand
- abdullahkhalids 3y ago
  Is it at all possible for you (or someone reading who has done something similar) to share their chat? Reading good prompting is probably one of the better ways of learning how to do it.
  - cryptoz 3y ago
    That is a great point and I will definitely share my prompting experience and some real prompts in a blog post this week. I'll come back here and link to it when ready.
strombofulous 3y ago
https://twitter.com/javilopen/status/1719363669685916095 https://twitter.com/javilopen/status/1719363669685916095 is relevant > Although the game is just 600 lines of which I haven't written ANY, [coding the game] was the most challenging part Not quite hello world, but not too much more difficult than a shopping list. The really impressive thing to me is you can make angry birds with just 600 loc (and a couple libraries)
- samspenc 3y ago
  My guess is that the main parts of the game are physics (collisions etc) and the scoring system, so that part wasn't too surprising to me. I was pleasantly surprised at the visual quality, I knew Midjourney could produce quality graphics assets, but I guess I didn't realize how easy it was to pull into a game.
  - gowld 3y ago
    2D sprite Graphics are just files. Game engines are able to load files.
croes 3y ago
There are lots of open source Angry bird clones, so it's not quite as impressive as it seems. Programming a new game without dozens of existing templates would be a better litmus test.
- fauria 3y ago
  Sumplete would be a good example: https://sumplete.com/ https://sumplete.com/
  - realharo 3y ago
    That's not an entirely new game (though the other versions of it are pretty obscure), but super interesting https://www.digitaltrends.com/gaming/sumplete-chatgpt-ai-game-design-ethics/ https://www.digitaltrends.com/gaming/sumplete-chatgpt-ai-gam...
- realharo 3y ago
  Not only are there tons of Angry Birds clones (Angry Birds itself is kind of a clone of earlier games), there are also tons of step-by-step tutorials for making them, which were no doubt included in the training data.
wokwokwok 3y ago
Honestly, it’s kind of difficult to believe. GPT4 is great at this stuff, but iterative refinement doesn’t work in my experience. As the conversation increases, the previous context is lost and the generated code deviates from its previous behaviour. For example, “fix this bug…” can easily result in a solution that breaks some other thing. You can also see code generated in thread (1) that does exist in the final result (2), suggesting that (since this is the very top of the code), they were getting chatGPT to iteratively generate 600+ line segments. I severely doubt this. Creating a new Slingshot on line 20 after it is defined on line 500? That is extraordinarily unlikely unless you specifically prompted it to do that. “loadImage('stone2.png');”, it just happened to pick the right file names? The right sprite sizes? You provided all that in a prompt and it wrote the code? Come onnnn… show us the actual prompt you used. It seems much more likely they generated a set of class objects relatively independently, then manually assembled them into a larger file, copied the entire thing as input and then crafted a “code prompt” like “write a function that does such and such”. It’s not impossible they used prompts like they claim (3), but I feel they are (for likes and cred) vastly overstating the “it did all the coding” part of this project. I feel they probably hand wrote some of the code (or assembled it) and used it as input + a “now do this also” style prompt, so the output was “100% generated”, but not in the way people are assuming. This approach tends to make GPT4 rewrite the existing code, but unless you specifically ask for (or add) comments describing the intent through out the code (missing in most of the generated code), it will drift from the previous functionality. With no test suite to verify, you won’t notice this subtle drift and things just break. There’s no mention of either of these things being done by the author. Further more, this user has a vested interest (4) in selling training materials for AI, so it’s in their interest to appear to be an expert at this, and has provided (even when asked on X) no additional details, no “step by step” git repo with history, no actual prompts they’ve used. Given the lack of details and the frankly unbelievable results, I think there’s fair call to be sceptical in this case. You could generate this kind of thing from models such as codellama 34B, or GPT 3.5; but not using the method as described. I’m… not convinced you could do it with gpt4. The prompts seem too stupid to be real (5)… but I happy to be proved wrong with more details. GPT4 is good. [1] - https://nitter.net/pic/orig/media%2FF9xoI8mXgAAn7v9.jpg https://nitter.net/pic/orig/media%2FF9xoI8mXgAAn7v9.jpg [2] - https://bestaiprompts.art/angry-pumpkins/sketch.js https://bestaiprompts.art/angry-pumpkins/sketch.js [3] - https://nitter.net/javilopen/status/1719363669685916095#m https://nitter.net/javilopen/status/1719363669685916095#m [4] - https://javilopen.substack.com/ https://javilopen.substack.com/ [5] - “Now, make the monsters circular, and be very careful: apply the same technique that already exists for the rectangular ones regarding scaling and collision area, and don't mess it up like before. ”
low_tech_love 3y ago
One other interesting question is “how does it work in all the other cases not shown in the video?”

cryptoz 3y ago

I’ve been playing with ChatGPT code generation to make entire sites with flask, python, html+js+css, backed with SQLite db and it’s amazing. I’ve had it write like 5k lines that are all live in prod and working (not much traffic lol but still). A huge huge factor is knowing the limitations and getting better at prompting. And identifying likely hallucinations and asking for risks etc. I’ve found it best with tech I don’t know well (I’m an android dev using it to make websites, something I haven’t done myself in like 15 years). Most of the coolest stuff for me is help with sysadmin and running the server. The ability to debug gunicorn errors is great. I do have to modify the code it outputs as the project grows and it loses context, but honestly the context limits are the biggest hurdle for bigger projects and those will be lifted soon. Edit: Most recent site I made with like 95% code from ChatGPT is https://cosmictrip.space/ https://cosmictrip.space/ which generates prompts with GPT-4 that are then used to generate space images with DALL-E. It's a simple site but there is a secret adventure game I'm working on (GPT+Dall-E) that is open-ended image+text AI-driven game. I'm hoping to launch before Nov 6 with DALL-E 3 API (hopefully...!). The adventure game is also written like 95%+ by ChatGPT. I've had such great success with it coding that I'm using the GPT-4 API with an agent I'm making (everyone is huh). I have function calling hooked up to generate structured subtasks that the agent can then write the code for, and support for files to include context, chat with your code, etc. It's not ready to show but the GPT-4 code generation abilities are really incredible - but you have to be experienced at prompting. Your first prompts aren't likely to be great, which is why I'm hoping my agent can have success. The idea of the agent I'm writing is a Jira/kanban style board where you have AI coders assigned to tasks that you can approve and modify etc. The tickets should automatically move across the columns as the AI checks the work etc.

maxwelljoslyn 3y ago
+1 for its suitability in helping with systems administration. One responsibility at my current job is administering a Windows server and trying to get it to do things that are easy on a Unix -- that should be easy anywhere -- but, on Windows, seem to inevitably degrade into nightmares. ChadGPT has given me huge amounts of blessed ammo to shoot at the nightmares, and there's no way I could do that portion of the job in a feasible time frame without it.
- ChatGTP 3y ago
  ChadGPT ?
  - visarga 3y ago
    https://www.youtube.com/channel/UCR2JJPxRRWvxy6naePMpsAQ https://www.youtube.com/channel/UCR2JJPxRRWvxy6naePMpsAQ
isoprophlex 3y ago
I feel that trapping your AI agents in a kanban board isn't going to do your survival chances a lot of good when the robot apocalypse inevitably comes for us meatbags.
- meiraleal 3y ago
  Parents know what's better for they kids, they will understand
abdullahkhalids 3y ago
Is it at all possible for you (or someone reading who has done something similar) to share their chat? Reading good prompting is probably one of the better ways of learning how to do it.
- cryptoz 3y ago
  That is a great point and I will definitely share my prompting experience and some real prompts in a blog post this week. I'll come back here and link to it when ready.

strombofulous 3y ago

https://twitter.com/javilopen/status/1719363669685916095 https://twitter.com/javilopen/status/1719363669685916095 is relevant > Although the game is just 600 lines of which I haven't written ANY, [coding the game] was the most challenging part Not quite hello world, but not too much more difficult than a shopping list. The really impressive thing to me is you can make angry birds with just 600 loc (and a couple libraries)

samspenc 3y ago
My guess is that the main parts of the game are physics (collisions etc) and the scoring system, so that part wasn't too surprising to me. I was pleasantly surprised at the visual quality, I knew Midjourney could produce quality graphics assets, but I guess I didn't realize how easy it was to pull into a game.
- gowld 3y ago
  2D sprite Graphics are just files. Game engines are able to load files.

croes 3y ago

There are lots of open source Angry bird clones, so it's not quite as impressive as it seems. Programming a new game without dozens of existing templates would be a better litmus test.

fauria 3y ago
Sumplete would be a good example: https://sumplete.com/ https://sumplete.com/
- realharo 3y ago
  That's not an entirely new game (though the other versions of it are pretty obscure), but super interesting https://www.digitaltrends.com/gaming/sumplete-chatgpt-ai-game-design-ethics/ https://www.digitaltrends.com/gaming/sumplete-chatgpt-ai-gam...
realharo 3y ago
Not only are there tons of Angry Birds clones (Angry Birds itself is kind of a clone of earlier games), there are also tons of step-by-step tutorials for making them, which were no doubt included in the training data.

wokwokwok 3y ago

Honestly, it’s kind of difficult to believe. GPT4 is great at this stuff, but iterative refinement doesn’t work in my experience. As the conversation increases, the previous context is lost and the generated code deviates from its previous behaviour. For example, “fix this bug…” can easily result in a solution that breaks some other thing. You can also see code generated in thread (1) that does exist in the final result (2), suggesting that (since this is the very top of the code), they were getting chatGPT to iteratively generate 600+ line segments. I severely doubt this. Creating a new Slingshot on line 20 after it is defined on line 500? That is extraordinarily unlikely unless you specifically prompted it to do that. “loadImage('stone2.png');”, it just happened to pick the right file names? The right sprite sizes? You provided all that in a prompt and it wrote the code? Come onnnn… show us the actual prompt you used. It seems much more likely they generated a set of class objects relatively independently, then manually assembled them into a larger file, copied the entire thing as input and then crafted a “code prompt” like “write a function that does such and such”. It’s not impossible they used prompts like they claim (3), but I feel they are (for likes and cred) vastly overstating the “it did all the coding” part of this project. I feel they probably hand wrote some of the code (or assembled it) and used it as input + a “now do this also” style prompt, so the output was “100% generated”, but not in the way people are assuming. This approach tends to make GPT4 rewrite the existing code, but unless you specifically ask for (or add) comments describing the intent through out the code (missing in most of the generated code), it will drift from the previous functionality. With no test suite to verify, you won’t notice this subtle drift and things just break. There’s no mention of either of these things being done by the author. Further more, this user has a vested interest (4) in selling training materials for AI, so it’s in their interest to appear to be an expert at this, and has provided (even when asked on X) no additional details, no “step by step” git repo with history, no actual prompts they’ve used. Given the lack of details and the frankly unbelievable results, I think there’s fair call to be sceptical in this case. You could generate this kind of thing from models such as codellama 34B, or GPT 3.5; but not using the method as described. I’m… not convinced you could do it with gpt4. The prompts seem too stupid to be real (5)… but I happy to be proved wrong with more details. GPT4 is good. [1] - https://nitter.net/pic/orig/media%2FF9xoI8mXgAAn7v9.jpg https://nitter.net/pic/orig/media%2FF9xoI8mXgAAn7v9.jpg [2] - https://bestaiprompts.art/angry-pumpkins/sketch.js https://bestaiprompts.art/angry-pumpkins/sketch.js [3] - https://nitter.net/javilopen/status/1719363669685916095#m https://nitter.net/javilopen/status/1719363669685916095#m [4] - https://javilopen.substack.com/ https://javilopen.substack.com/ [5] - “Now, make the monsters circular, and be very careful: apply the same technique that already exists for the rectangular ones regarding scaling and collision area, and don't mess it up like before. ”

low_tech_love 3y ago

One other interesting question is “how does it work in all the other cases not shown in the video?”