3 ms·

I tried, not actually at work but for something open source, to get it to help me rework a terraform provider using the old SDK to use the new 'framework' - I'v

by OJFord 4y ago

I tried, not actually at work but for something open source, to get it to help me rework a terraform provider using the old SDK to use the new 'framework' - I've only ever written Go for terraform providers (and that not much) so I often have dumb confusions about how to do something that end up more about Go than the API itself, it's just not necessarily obvious initially, or what to search for etc., so I thought it was a great opportunity. It didn't really work though - it produced something extremely plausible looking that checking against the docs I realised had no chance of compiling. After a lot of back and forth, I began to suspect it was because it was trained while only pre-1.0 versions of the 'framework' were available. I tried to get it to confirm that, but it just apologised profusely while continuing to lie, claiming to have been trained with access to 1.x versions that to this day have not been released. At this point I was too frustrated to bother with it anyway really, but I could only confirm my suspicion by asking it for the date of its training, and checking release history myself. (Solved my problems with some good old 'ChatHuman' in the Hashicorp forum.)

vertis 4y ago
I've had both positive and negative experiences along the lines you described. It think a core skill with ChatGPT at the moment is knowing when to abandon an approach or task. I've had it do some things in brilliant ways. Asking it to create typescript types for given data (and multiple examples to do union types). I've had it help me create migrations for Prisma schema that doesn't lose data. I've asked it to convert one format (html to jsx) and it can get that right. Often you still have to correct it, but it gives amazing starting points. But if it's wrong arguing it with it is a mistake. It's not aware, and it can't learn on the fly (currently). If you close the chat and then repeat the prompts exactly it will make the same mistakes again.

by OJFord 4y ago

vertis 4y ago

I've had both positive and negative experiences along the lines you described. It think a core skill with ChatGPT at the moment is knowing when to abandon an approach or task. I've had it do some things in brilliant ways. Asking it to create typescript types for given data (and multiple examples to do union types). I've had it help me create migrations for Prisma schema that doesn't lose data. I've asked it to convert one format (html to jsx) and it can get that right. Often you still have to correct it, but it gives amazing starting points. But if it's wrong arguing it with it is a mistake. It's not aware, and it can't learn on the fly (currently). If you close the chat and then repeat the prompts exactly it will make the same mistakes again.