4 ms·

That’s exactly what I did when I’ve experimented with it teaching me how to make a watchOS app (with me having zero iOS and Swift experience), but the issue us

That’s exactly what I did when I’ve experimented with it teaching me how to make a watchOS app (with me having zero iOS and Swift experience), but the issue us that it started to “forget” what we have talked about. It helped me to bootstrap the project and pointed out at some APIs I needed, but then it was a constant fight for code quality. I wanted proper actor abstraction (because I was dealing with Bluetooth, even if I don’t know the language I can see the patterns) and it just loved to write code in style of some cheapest-grade “how to use CoreBluetooth in Swift for dummies” blog articles.

mlyle 3y ago
Were you using vanilla ChatGPT or GPT4? Vanilla ChatGPT uses a GPT-3.5turbo version with a context window of 4096 tokens. GPT-4 has twice the context window by default, and has a variant with a context window 8x as big. It's not "8x better" with regards to forgetting what it's doing, but it's substantially better.
- drdaeman 3y ago
  Yes, that was GPT-3.5. GPT-4's 25 messages/3 hours ratelimit wouldn't have led us anywhere, even with me doing my own research and compiling larger lists of change requests would've exhausted that very quickly.
  - mlyle 3y ago
    Yup, GPT-4 is still likely too expensive for this application at a dollar or two per interaction with the huge context window (or 30 cents-ish with an 8k window). Especially with the tooling not proven for strong interactive use in big codebases. But we don't need to make much improvement in tooling and economics for it to start to look pretty good.
vidarh 3y ago
That's why I have it iterate on a spec first, and then an outline, and then specifics. The point is to have it produce something that narrows the scope so I can feed it specific sections back in again and have it fill in the details.

4 ms·

That’s exactly what I did when I’ve experimented with it teaching me how to make a watchOS app (with me having zero iOS and Swift experience), but the issue us

by drdaeman 3y ago

mlyle 3y ago
Were you using vanilla ChatGPT or GPT4? Vanilla ChatGPT uses a GPT-3.5turbo version with a context window of 4096 tokens. GPT-4 has twice the context window by default, and has a variant with a context window 8x as big. It's not "8x better" with regards to forgetting what it's doing, but it's substantially better.
- drdaeman 3y ago
  Yes, that was GPT-3.5. GPT-4's 25 messages/3 hours ratelimit wouldn't have led us anywhere, even with me doing my own research and compiling larger lists of change requests would've exhausted that very quickly.
  - mlyle 3y ago
    Yup, GPT-4 is still likely too expensive for this application at a dollar or two per interaction with the huge context window (or 30 cents-ish with an 8k window). Especially with the tooling not proven for strong interactive use in big codebases. But we don't need to make much improvement in tooling and economics for it to start to look pretty good.
vidarh 3y ago
That's why I have it iterate on a spec first, and then an outline, and then specifics. The point is to have it produce something that narrows the scope so I can feed it specific sections back in again and have it fill in the details.