3 ms·

The first few weeks of ChatGPT I had a goal to make it code. My preliminary conclusion is that if given a feedback loop it could code a lot of basic stuff just

The first few weeks of ChatGPT I had a goal to make it code. My preliminary conclusion is that if given a feedback loop it could code a lot of basic stuff just fine, my workflow went something like this: - Identify a component in a system you want to build - Give ChatGPT a list of bullets and ask it to write tests - Review tests - Run tests, give ChatGPT output of failing tests - Add the code to your codebase, run the tests... I developed extensions to my console and test suite so the output exceptions/failures ended up in my clipboard. I'm highly confident workflows that put the AI in a feedback loop will be able to code most backend stuff. UIs are trickier as giving it context is not trivial.

dbish 4y ago
There is also a context window problem where you can only pass in 4k tokens (including the prompt). This is not that large of a window for providing history and information of the entire workflow. How were you trying to solve that? Also, if you’re building something yourself, you can use davinci-003 directly through their APIs instead of trying to build on top of chatgpt
- jrvarela56 4y ago
  You are right, what I described was just a prototype. ChatGPT was just a gateway drug for me to dive into the OpenAI docs and look for other providers and existing solutions. I've been reading up on approaches to 'package' context (example: https://thakkarparth007.github.io/copilot-explorer/posts/copilot-internals.html#how-is-the-prompt-prepared-a-code-walkthrough https://thakkarparth007.github.io/copilot-explorer/posts/cop...). My take for now is that this kind of packaging is very context specific to the point where it's going to make sense for engineering teams to craft their own version of 'gather context before the prompt is created'. I'm testing out approaches using toy examples for now (how would you create a chatbot about a book?; podcasts, etc). I don't have an answer but I'm highly confident we're going to replace a lot of junior-level work in the next year or two.
dmix 4y ago
I wonder if you could build a dataset of UI components (React/HTML/CSS) with a simple description and taxonomy then feed it into an AI algo
- jrvarela56 4y ago
  You can but this is a trickier version of the 'package context' process (see my reply to a comment above). Speaking to others who are playing with this stuff, it seems people who use opinionated libraries (Antd, Mantine) get better results out of the box.
jlarocco 4y ago
I'm guessing it depends on the situation, but at what point does all of that become more work than just writing the tests and code yourself? Is it worth the "investment" to learn how to teach ChatGPT coding? Or will ChatGPT2 come out in six months, based on a different algorithm, and you'll back to square one teaching it over again? Personally, I find writing new code easier than editting existing code, regardless of how it's written. I'd much rather have an AI that scans the code I write and searches for bugs Writing code is easy, finding bugs is hard, and it's the finding of bugs that I'd like to see automated.
- jrvarela56 4y ago
  It's obviously not paying off right now, but I'm assuming these models will get there in 2023 so I'm getting ready for that :P To take into account future changes, my approach is to develop interactive models for the components that make up this system using Jupyter notebooks and https://natto.dev/ https://natto.dev/. The idea is that if some part changes you can sort of test it in isolation and compare results.

dbish 4y ago

There is also a context window problem where you can only pass in 4k tokens (including the prompt). This is not that large of a window for providing history and information of the entire workflow. How were you trying to solve that? Also, if you’re building something yourself, you can use davinci-003 directly through their APIs instead of trying to build on top of chatgpt

jrvarela56 4y ago
You are right, what I described was just a prototype. ChatGPT was just a gateway drug for me to dive into the OpenAI docs and look for other providers and existing solutions. I've been reading up on approaches to 'package' context (example: https://thakkarparth007.github.io/copilot-explorer/posts/copilot-internals.html#how-is-the-prompt-prepared-a-code-walkthrough https://thakkarparth007.github.io/copilot-explorer/posts/cop...). My take for now is that this kind of packaging is very context specific to the point where it's going to make sense for engineering teams to craft their own version of 'gather context before the prompt is created'. I'm testing out approaches using toy examples for now (how would you create a chatbot about a book?; podcasts, etc). I don't have an answer but I'm highly confident we're going to replace a lot of junior-level work in the next year or two.

dmix 4y ago

I wonder if you could build a dataset of UI components (React/HTML/CSS) with a simple description and taxonomy then feed it into an AI algo

jrvarela56 4y ago
You can but this is a trickier version of the 'package context' process (see my reply to a comment above). Speaking to others who are playing with this stuff, it seems people who use opinionated libraries (Antd, Mantine) get better results out of the box.

jlarocco 4y ago

I'm guessing it depends on the situation, but at what point does all of that become more work than just writing the tests and code yourself? Is it worth the "investment" to learn how to teach ChatGPT coding? Or will ChatGPT2 come out in six months, based on a different algorithm, and you'll back to square one teaching it over again? Personally, I find writing new code easier than editting existing code, regardless of how it's written. I'd much rather have an AI that scans the code I write and searches for bugs Writing code is easy, finding bugs is hard, and it's the finding of bugs that I'd like to see automated.

jrvarela56 4y ago
It's obviously not paying off right now, but I'm assuming these models will get there in 2023 so I'm getting ready for that :P To take into account future changes, my approach is to develop interactive models for the components that make up this system using Jupyter notebooks and https://natto.dev/ https://natto.dev/. The idea is that if some part changes you can sort of test it in isolation and compare results.