4 ms·

> playing a slot machine hoping I still believe the slot machine analogy holds to some extent, but I can honestly say my winning percentage is at least 90% for

by sdesol 4mo ago

> playing a slot machine hoping I still believe the slot machine analogy holds to some extent, but I can honestly say my winning percentage is at least 90% for one shot generated code now. I think if you know it's limitations (inlcluding your own), I don't think about hoping anymore. I should note that when I say AI, I mean the collective models from all the major providers. The most important lesson is, you need to ask around. > There’s definitely a skill to using AI but it just doesn’t generalize very well. This I agree with. The only way working with AI can really be benefical outside of dealing with AI is, we are visited by extremely intelligent beings that will fuck up in the weirdest ways.

Retric 4mo ago
When it works the first time you’re done, when it fails you spend more time on it. This biases the cognitive investment around failure. > I don't think about hoping anymore. Running the exact same prompt again that already failed doesn’t have a that high success rate, but it’s also very low effort. So IMO it’s often worth attempting.
ncruces 4mo ago
I wish I could've one shoted this 200 LoC PR: https://github.com/ncruces/wasm2go/pull/42/changes#diff-5181f8745abc6d98fdef013bc2eab391642e4af2b3e499e4baefb0f939134ce6 https://github.com/ncruces/wasm2go/pull/42/changes#diff-5181... I'd really like to know the prompt, because I did try and, this took me way too long to figure out.
- sdesol 4mo ago
  The AI agreed > There isn’t a magic prompt I could have written before knowing the solution. A retrospective prompt can reproduce the 200 lines only because it embeds the invariants you discovered. The useful AI task would have been systematically finding counterexamples and iterating toward those invariants, not generating the final file in one shot. For me the value with AI isn't the one shot, it is the fact that it can help you iterate faster. My biggest take awway so far with AI is it can help you fail fast. The faster you can fail, the faster you can iterate.
carterschonwald 4mo ago
whats your fave/best one shot code gen? mine def has to be the initial impl of my python to my personal fave compiler ir. i got claude chat to write it in two sentences. 4 turns helping it rmeemver where it put shit becsuse of transcript issues
- sdesol 4mo ago
  The code in http://github.com/gitsense/gsc-cli http://github.com/gitsense/gsc-cli contains over 300 go files and I would say 90% were one shots. I can even prove that they are AI generated since I attribute the AI in each file, as shown in the example below: https://github.com/gitsense/gsc-cli/blob/main/internal/cli/root.go https://github.com/gitsense/gsc-cli/blob/main/internal/cli/r... Since I don't know go, I can't say which is the best. I am also sure they would not live up to a go experts expectations. I just know that since I don't know go, I would not have been able to create the tool. I can also prove that you can easily maintain AI generated code since I have been expanding and evolving it for over 6 months with pretty much no issues.
pseudosavant 4mo ago
AI has definitely gotten good enough that its one shot success rate is better than most developers.

by sdesol 4mo ago

Retric 4mo ago

When it works the first time you’re done, when it fails you spend more time on it. This biases the cognitive investment around failure. > I don't think about hoping anymore. Running the exact same prompt again that already failed doesn’t have a that high success rate, but it’s also very low effort. So IMO it’s often worth attempting.

ncruces 4mo ago

I wish I could've one shoted this 200 LoC PR: https://github.com/ncruces/wasm2go/pull/42/changes#diff-5181f8745abc6d98fdef013bc2eab391642e4af2b3e499e4baefb0f939134ce6 https://github.com/ncruces/wasm2go/pull/42/changes#diff-5181... I'd really like to know the prompt, because I did try and, this took me way too long to figure out.

sdesol 4mo ago
The AI agreed > There isn’t a magic prompt I could have written before knowing the solution. A retrospective prompt can reproduce the 200 lines only because it embeds the invariants you discovered. The useful AI task would have been systematically finding counterexamples and iterating toward those invariants, not generating the final file in one shot. For me the value with AI isn't the one shot, it is the fact that it can help you iterate faster. My biggest take awway so far with AI is it can help you fail fast. The faster you can fail, the faster you can iterate.

carterschonwald 4mo ago

whats your fave/best one shot code gen? mine def has to be the initial impl of my python to my personal fave compiler ir. i got claude chat to write it in two sentences. 4 turns helping it rmeemver where it put shit becsuse of transcript issues

sdesol 4mo ago
The code in http://github.com/gitsense/gsc-cli http://github.com/gitsense/gsc-cli contains over 300 go files and I would say 90% were one shots. I can even prove that they are AI generated since I attribute the AI in each file, as shown in the example below: https://github.com/gitsense/gsc-cli/blob/main/internal/cli/root.go https://github.com/gitsense/gsc-cli/blob/main/internal/cli/r... Since I don't know go, I can't say which is the best. I am also sure they would not live up to a go experts expectations. I just know that since I don't know go, I would not have been able to create the tool. I can also prove that you can easily maintain AI generated code since I have been expanding and evolving it for over 6 months with pretty much no issues.

pseudosavant 4mo ago

AI has definitely gotten good enough that its one shot success rate is better than most developers.