4 ms·

Fake or not, people are incorporating ChatGPT into their work. We had a pipeline break today and when we looked at the PR that introduced the change it was in t

by kodah 3y ago

Fake or not, people are incorporating ChatGPT into their work. We had a pipeline break today and when we looked at the PR that introduced the change it was in the PR notes that the tooling configuration was derived from a ChatGPT prompt. What do you even say or do about that?

nevi-me 3y ago
I've been embarrassed by a Copilot code snippet that I proofread to seem correct, but was incorrect due to some subtle comparison logic. I can only imagine the risks in professions that aren't dealing with logic that has a very finite list of options or where the answers AI produce require researched effort to validate. It's probably worse in law if you're given a case reference as you're professionally supposed to read the document in substance to not be muddled by selective quoting, then if it's from a foreign jurisdiction you have to also check if it's applicable to you.
- licebmi__at__ 3y ago
  > I can only imagine the risks in professions that aren't dealing with logic that has a very finite list of options or where the answers AI produce require researched effort to validate This can be done the same way as it’s done in the non ai-assisted process. Mistakes are made then the trained professionals correct the course of action. The problem is with people dealing and selling the problem like it’s a simple logical procedure. Then the layman will think that since a machine crafted the answer, and since machines are logical, the answer must be right.
  - __loam 3y ago
    And of course, there's no way to verify the output of an llm without professional experience or training, so what's the point?
- deleted 3y ago
  [deleted]
- weatherlite 3y ago
  Some software mistakes can cost millions ... not saying it happens all the time but it can happen. We SHOULD try to be as accurate and bug free as we can in this business its no joke. Yeah, sometimes its just an html div that looks funny (that can also cost the company depending on how embarrassing it looked and how much traffic was impacted). But sometimes its backend logic that screws up important data or who knows what with very real reputation or financial consequences to the business.
Kwpolska 3y ago
You fire them for being an idiot and sharing details about your infrastructure with a third party.
weatherlite 3y ago
> What do you even say or do about that? Nothing, you fix it. Since its a pipeline it was probably non trivial to somehow test it locally. So a pipeline broke, no biggie I think.
- hammyhavoc 3y ago
  Is fixing it not doing something about it?
dmurray 3y ago
Did ChatGPT also perform the code review?
- kodah 3y ago
  This is a good question. There are no tests for tooling but the tooling can result in pipeline failures. Theoretically the pipeline should've failed outright when they went to merge. Instead it failed on specific conditions the developer didn't account for and weren't familiar with because the configuration wasn't something they were familiar with because they weren't building it from docs.

by kodah 3y ago

nevi-me 3y ago

I've been embarrassed by a Copilot code snippet that I proofread to seem correct, but was incorrect due to some subtle comparison logic. I can only imagine the risks in professions that aren't dealing with logic that has a very finite list of options or where the answers AI produce require researched effort to validate. It's probably worse in law if you're given a case reference as you're professionally supposed to read the document in substance to not be muddled by selective quoting, then if it's from a foreign jurisdiction you have to also check if it's applicable to you.

licebmi__at__ 3y ago
> I can only imagine the risks in professions that aren't dealing with logic that has a very finite list of options or where the answers AI produce require researched effort to validate This can be done the same way as it’s done in the non ai-assisted process. Mistakes are made then the trained professionals correct the course of action. The problem is with people dealing and selling the problem like it’s a simple logical procedure. Then the layman will think that since a machine crafted the answer, and since machines are logical, the answer must be right.
- __loam 3y ago
  And of course, there's no way to verify the output of an llm without professional experience or training, so what's the point?
deleted 3y ago
[deleted]
weatherlite 3y ago
Some software mistakes can cost millions ... not saying it happens all the time but it can happen. We SHOULD try to be as accurate and bug free as we can in this business its no joke. Yeah, sometimes its just an html div that looks funny (that can also cost the company depending on how embarrassing it looked and how much traffic was impacted). But sometimes its backend logic that screws up important data or who knows what with very real reputation or financial consequences to the business.

Kwpolska 3y ago

You fire them for being an idiot and sharing details about your infrastructure with a third party.

weatherlite 3y ago

> What do you even say or do about that? Nothing, you fix it. Since its a pipeline it was probably non trivial to somehow test it locally. So a pipeline broke, no biggie I think.

hammyhavoc 3y ago
Is fixing it not doing something about it?

dmurray 3y ago

Did ChatGPT also perform the code review?

kodah 3y ago
This is a good question. There are no tests for tooling but the tooling can result in pipeline failures. Theoretically the pipeline should've failed outright when they went to merge. Instead it failed on specific conditions the developer didn't account for and weren't familiar with because the configuration wasn't something they were familiar with because they weren't building it from docs.