4 ms·

I love those ChatGPT projects! Of course it's silly and nobody is really replacing somebody with a program that confidently get half it's answers wrong. But it'

I love those ChatGPT projects! Of course it's silly and nobody is really replacing somebody with a program that confidently get half it's answers wrong. But it's fun to just naively let ChatGPT solve the problem. But I wonder what it's going to look like in a few years. Currently, it's really just a demo that got surprisingly huge traction. I think the most pressing problem is not to get ChatGPT smarter but to get it more reliable. I think more realistic use-cases would emerge if we could build systems that have a better understanding when they are out of breath. I don't think this needs a revolutionary breakthrough just more science.

knodi123 4y ago
> nobody is really replacing somebody with a program that confidently get half it's answers wrong I gotta tell you, I use copilot to help with my coding, and it still sends a shiver down my spine when it writes the entire database migration based off of the filename, or writes 4 pages of automated tests that work right on the first try.
- ExxKA 4y ago
  Which programming languages do you use?
- layer8 4y ago
  > writes 4 pages of automated tests that work right on the first try Not sure that’s the right criterion to assess the quality and adequacy of tests.
  - knodi123 4y ago
    lol, fair enough, but I would never commit a line of code I hadn't carefully read myself. my new class was basic, so the new unit test was basic as well, but it was thorough and correct. And I exaggerated a bit; I did write 3 or 4 comments to give the ai some idea of where I was heading.
  - pphysch 4y ago
    But my GitHub badge says I have 105% test coverage! That means there are negative bugs, right...?
  - bcbrown 4y ago
    I'd be impressed if AI wrote a test suite that immediately found a bunch of valid bugs. I'd be nonplussed if it wrote a test suite that "passed" on the first try. The presence of passing tests does not imply an absence of bugs, and the hard part of developing a test suite is the test design, not the implementation.
- jeremyjh 4y ago
  I've had it freak me out with how prescient can be, but it also gets a lot of things subtly wrong, like its programming against an API similar to the one in my application, but not identical.
- burnished 4y ago
  Do you check that those tests can fail at all? That seems like a common human error, might be present here?

knodi123 4y ago

> nobody is really replacing somebody with a program that confidently get half it's answers wrong I gotta tell you, I use copilot to help with my coding, and it still sends a shiver down my spine when it writes the entire database migration based off of the filename, or writes 4 pages of automated tests that work right on the first try.

ExxKA 4y ago
Which programming languages do you use?
layer8 4y ago
> writes 4 pages of automated tests that work right on the first try Not sure that’s the right criterion to assess the quality and adequacy of tests.
- knodi123 4y ago
  lol, fair enough, but I would never commit a line of code I hadn't carefully read myself. my new class was basic, so the new unit test was basic as well, but it was thorough and correct. And I exaggerated a bit; I did write 3 or 4 comments to give the ai some idea of where I was heading.
- pphysch 4y ago
  But my GitHub badge says I have 105% test coverage! That means there are negative bugs, right...?
- bcbrown 4y ago
  I'd be impressed if AI wrote a test suite that immediately found a bunch of valid bugs. I'd be nonplussed if it wrote a test suite that "passed" on the first try. The presence of passing tests does not imply an absence of bugs, and the hard part of developing a test suite is the test design, not the implementation.
jeremyjh 4y ago
I've had it freak me out with how prescient can be, but it also gets a lot of things subtly wrong, like its programming against an API similar to the one in my application, but not identical.
burnished 4y ago
Do you check that those tests can fail at all? That seems like a common human error, might be present here?