4 ms·

In one case I tried using it to help me find the right functions in `oauthlib` that can validate a two-legged OAuth 1.0 signed message. It couldn't really get i

In one case I tried using it to help me find the right functions in `oauthlib` that can validate a two-legged OAuth 1.0 signed message. It couldn't really get it and made up information about how OAuth works, but it did get me on the right track so I'm thankful in that regard. I've also been using it while writing Common Lisp, and I'm not sure if it's just because it's a more obscure language, but it commonly produces nonsensical code, recommends functions that don't exist or work differently than it thinks they do, or produces non-idiomatic weird stuff. It also falls apart when writing z80 assembly, again just generating nonsense. It really is a cool tool though, and I use it to get me steered in the correct direction. I've seen other people mention that it can identify unknown unknowns, and I concur it's an amazing tool for that. We'll see if with more specialized training on programming-related datasets if it can get better, I wouldn't write off the possibility. It won't be stealing your job in its current state, though.

dmux 4y ago
>I've also been using it while writing Common Lisp, and I'm not sure if it's just because it's a more obscure language, but it commonly produces nonsensical code, recommends functions that don't exist or work differently than it thinks they do, or produces non-idiomatic weird stuff. I've been using it for Common Lisp as well and have had the same experience. For example, when asking it to help me generate some simple CRUD operations using the cl-sqlite library, it just comes up with function names that simply do not exist in that library [0]: `sqlite3:execute-command` `sqlite3:with-query-results` `sqlite3:get-result` [0] https://cl-sqlite.common-lisp.dev/ https://cl-sqlite.common-lisp.dev/
whatyesaid 4y ago
It seems to really fall apart in anything niche, even if you're just asking information about a topic. Doesn't have enough training data on that topic.

dmux 4y ago

>I've also been using it while writing Common Lisp, and I'm not sure if it's just because it's a more obscure language, but it commonly produces nonsensical code, recommends functions that don't exist or work differently than it thinks they do, or produces non-idiomatic weird stuff. I've been using it for Common Lisp as well and have had the same experience. For example, when asking it to help me generate some simple CRUD operations using the cl-sqlite library, it just comes up with function names that simply do not exist in that library [0]: `sqlite3:execute-command` `sqlite3:with-query-results` `sqlite3:get-result` [0] https://cl-sqlite.common-lisp.dev/ https://cl-sqlite.common-lisp.dev/

whatyesaid 4y ago

It seems to really fall apart in anything niche, even if you're just asking information about a topic. Doesn't have enough training data on that topic.