5 ms·

GPT-4 is better at planning than at executing. Have you tried asking it to generate a regex to transform your list into a CSV?

keiferski 2y ago
No, I’ll give that a shot. I have just been asking it to convert output into a CSV, which used to work somewhat well. It stumbles when there is more complexity though.
- kohlerm 2y ago
  Humans also stumble with that as well. Problems being CSV not really being that well defined and it is not clear to people how quoting needs to be done. The training set might not contain enough complex examples (newlines in values?)
  - keiferski 2y ago
    No, the data is very well defined. For example, “name, date of birth, key ideas,” etc. The issue is with ChatGPT formatting a file.
    - porksoda 2y ago
      Even if you get it to work 100% of the time, it will only be 99.something%. That's just not what it's for I guess. I pushed a few million items through it for classification a while back and the creative ways it found to sometimes screwup, astounded me.
      - keiferski 2y ago
        Yeah and that's why I'm skeptical of the idea that AI tools will just replace people, in toto. Someone has to ultimately be responsible for the data, and "the AI said it was true" isn't going to hold up as an excuse. They will minimize and replace certain types of work, though, like generic illustrations.
        pixl97 2y ago
        Do you even work with humans now? I get "Computer says no" out of corporations all the time as it is, AI is just completing that loop.
        jodrellblank 2y ago
        > "Someone has to ultimately be responsible for the data" All you have to do is survive long enough as an unemployed criminal until the system gets round to exonerating you: https://en.wikipedia.org/wiki/British_Post_Office_scandal https://en.wikipedia.org/wiki/British_Post_Office_scandal "The British Post Office scandal, also called the Horizon IT scandal, involved Post Office Limited pursuing thousands of innocent subpostmasters for shortfalls in their accounts, which had in fact been caused by faults in Horizon, accounting software developed and maintained by Fujitsu. Between 1999 and 2015, more than 900 subpostmasters were convicted of theft, fraud and false accounting based on faulty Horizon data, with about 700 of these prosecutions carried out by the Post Office. Other subpostmasters were prosecuted but not convicted, forced to cover Horizon shortfalls with their own money, or had their contracts terminated. The court cases, criminal convictions, imprisonments, loss of livelihoods and homes, debts and bankruptcies, took a heavy toll on the victims and their families, leading to stress, illness, family breakdown, and at least four suicides. In 2024, Prime Minister Rishi Sunak described the scandal as one of the greatest miscarriages of justice in British history. Although many subpostmasters had reported problems with the new software, and Fujitsu was aware that Horizon contained software bugs as early as 1999, the Post Office insisted that Horizon was robust and failed to disclose knowledge of the faults in the system during criminal and civil cases. [...] challenge their convictions in the courts and, in 2020, led to the government establishing an independent inquiry into the scandal. This was upgraded into a statutory public inquiry the following year. As of May 2024, the public inquiry is ongoing and the Metropolitan Police are investigating executives from the Post Office and its software provider, Fujitsu. Courts began to quash convictions from December 2020. By February 2024, 100 of the subpostmasters' convictions had been overturned. Those wrongfully convicted became eligible for compensation, as did more than 2,750 subpostmasters who had been affected by the scandal but had not been convicted."
Etherlord87 2y ago
I remember when people used to argue about regex being bad or good, with a lot of low quality regex introducing bugs in codebases. Now we have devs asking AI to generate regex formulas and pasting it into code without much concern on its validity.
- mFixman 2y ago
  Regexes are easy to test. Bad developers do bad regexes, regardless of whether they used AI.
  - Etherlord87 2y ago
    How do you test a regex to be 100% sure it's valid? I don't think it's possible.
    - fkyoureadthedoc 2y ago
      True for most things I think
    - eutropia 2y ago
      If it's using classical regex, without backtracking or other extensions, a regular expression is isomorphic to a state machine. You can enumerate combinations doing something like this: https://stackoverflow.com/a/1248566 https://stackoverflow.com/a/1248566 kids these days and their lack of exposure to finite automata
      - Etherlord87 2y ago
        - Vehicle steering is easy to test. - How so? I don't think it's possible to test for all cases... - Well, it's easy, assuming a car on a non-branching track, moving with a constant speed and without any realistic external influences on it, you can simply calculate the distance traveled using the formula s = v/t. Ah, I wish I'd stop running into fools not knowing Newton's first law of motion... - ??? Are you well?

keiferski 2y ago

No, I’ll give that a shot. I have just been asking it to convert output into a CSV, which used to work somewhat well. It stumbles when there is more complexity though.

kohlerm 2y ago
Humans also stumble with that as well. Problems being CSV not really being that well defined and it is not clear to people how quoting needs to be done. The training set might not contain enough complex examples (newlines in values?)
- keiferski 2y ago
  No, the data is very well defined. For example, “name, date of birth, key ideas,” etc. The issue is with ChatGPT formatting a file.
  - porksoda 2y ago
    Even if you get it to work 100% of the time, it will only be 99.something%. That's just not what it's for I guess. I pushed a few million items through it for classification a while back and the creative ways it found to sometimes screwup, astounded me.
    - keiferski 2y ago
      Yeah and that's why I'm skeptical of the idea that AI tools will just replace people, in toto. Someone has to ultimately be responsible for the data, and "the AI said it was true" isn't going to hold up as an excuse. They will minimize and replace certain types of work, though, like generic illustrations.
      - pixl97 2y ago
        Do you even work with humans now? I get "Computer says no" out of corporations all the time as it is, AI is just completing that loop.
      - jodrellblank 2y ago
        > "Someone has to ultimately be responsible for the data" All you have to do is survive long enough as an unemployed criminal until the system gets round to exonerating you: https://en.wikipedia.org/wiki/British_Post_Office_scandal https://en.wikipedia.org/wiki/British_Post_Office_scandal "The British Post Office scandal, also called the Horizon IT scandal, involved Post Office Limited pursuing thousands of innocent subpostmasters for shortfalls in their accounts, which had in fact been caused by faults in Horizon, accounting software developed and maintained by Fujitsu. Between 1999 and 2015, more than 900 subpostmasters were convicted of theft, fraud and false accounting based on faulty Horizon data, with about 700 of these prosecutions carried out by the Post Office. Other subpostmasters were prosecuted but not convicted, forced to cover Horizon shortfalls with their own money, or had their contracts terminated. The court cases, criminal convictions, imprisonments, loss of livelihoods and homes, debts and bankruptcies, took a heavy toll on the victims and their families, leading to stress, illness, family breakdown, and at least four suicides. In 2024, Prime Minister Rishi Sunak described the scandal as one of the greatest miscarriages of justice in British history. Although many subpostmasters had reported problems with the new software, and Fujitsu was aware that Horizon contained software bugs as early as 1999, the Post Office insisted that Horizon was robust and failed to disclose knowledge of the faults in the system during criminal and civil cases. [...] challenge their convictions in the courts and, in 2020, led to the government establishing an independent inquiry into the scandal. This was upgraded into a statutory public inquiry the following year. As of May 2024, the public inquiry is ongoing and the Metropolitan Police are investigating executives from the Post Office and its software provider, Fujitsu. Courts began to quash convictions from December 2020. By February 2024, 100 of the subpostmasters' convictions had been overturned. Those wrongfully convicted became eligible for compensation, as did more than 2,750 subpostmasters who had been affected by the scandal but had not been convicted."

Etherlord87 2y ago

I remember when people used to argue about regex being bad or good, with a lot of low quality regex introducing bugs in codebases. Now we have devs asking AI to generate regex formulas and pasting it into code without much concern on its validity.

mFixman 2y ago
Regexes are easy to test. Bad developers do bad regexes, regardless of whether they used AI.
- Etherlord87 2y ago
  How do you test a regex to be 100% sure it's valid? I don't think it's possible.
  - fkyoureadthedoc 2y ago
    True for most things I think
  - eutropia 2y ago
    If it's using classical regex, without backtracking or other extensions, a regular expression is isomorphic to a state machine. You can enumerate combinations doing something like this: https://stackoverflow.com/a/1248566 https://stackoverflow.com/a/1248566 kids these days and their lack of exposure to finite automata
    - Etherlord87 2y ago
      - Vehicle steering is easy to test. - How so? I don't think it's possible to test for all cases... - Well, it's easy, assuming a car on a non-branching track, moving with a constant speed and without any realistic external influences on it, you can simply calculate the distance traveled using the formula s = v/t. Ah, I wish I'd stop running into fools not knowing Newton's first law of motion... - ??? Are you well?