5 ms·

> In my opinion, using AI tools for programming at the moment, unless in a sandboxed environment and on a toy project, is just ludicrous. Well put. The fundam

> In my opinion, using AI tools for programming at the moment, unless in a sandboxed environment and on a toy project, is just ludicrous. Well put. The fundamental flaw is in trying to employ nondeterministic content generation based on statistical relevance defined by an unknown training data set, which is what commercial LLM offerings are, in an effort to repeatably produce content satisfying a strict mathematical model (program source code).

mlyle 1y ago
Nearly as bad: trying to use systems made out of meat, evolved from a unrelated background and trained on an undocumented and chaotic corpus of data, to try and produce content satisfying a strict mathematical model.
- jdiff 1y ago
  Meaty feet can be held to a fire. To quote IBM, "A computer can never be held accountable."
  - MobiusHorizons 1y ago
    This is the question I keep asking leaders (I literally asked a VP this question once in an all hands). How do we approach the risk associated mistakes made by AI?(process, legal, security, insurance etc) We have process and legal agreements in place to deal with humans that work for a business making mistakes. We need analogs for AI if we want to use it in similar ways.
    - jpcosta 1y ago
      What was the answer? Asking for a vp friend
      - AdieuToLogic 1y ago
        >>> Meaty feet can be held to a fire. To quote IBM, "A computer can never be held accountable." >> This is the question I keep asking leaders (I literally asked a VP this question once in an all hands). How do we approach the risk associated mistakes made by AI? > What was the answer? Asking for a vp friend This is a difficult issue to tackle, no doubt. What follows drifts into the philosophical realm by necessity. Software exists to provide value to people. Malicious software qualifies as such due to the desires of the actors which produce same, but will no longer be considered here as this is not germane. AI is an umbrella term for numerous algorithms having wide ranging problem domain applicability and often can approximate near-optimal solutions using significantly less resources than other approaches. But they are still algorithms, capable of only one thing - execute their defined logic. Sometimes this logic can produce results similar to be what a person would in a similar situation. Sometimes the logic will produce wildly different results. Often there is significant value when the logic is used appropriately. In all cases AI algorithms do not possess the concept of understanding. This includes derivatives of understanding such as: - empathy - integrity - morals - right - wrong Which brings us back to part of the first quoted post: To quote IBM, "A computer can never be held accountable." Accountability requires justification of actions taken or lack thereof, which demands the ability to explain why said actions were undertaken relative to other options, and implies a potential consequence be imposed by an authority. Algorithms can partially "justify their output" via strategic logging, but that's about it. Which is why "a computer can never be held accountable." Because it is a machine, executing the instructions ultimately initiated by one or more persons whom can be held accountable.
      - MobiusHorizons 1y ago
        In the all hands I got an answer about techniques that would be used to reduce the likelihood of mistakes. Ie not an answer.
    - Covenant0028 1y ago
      I suspect the analog will be that the "human in the loop" will bear all the consequences. Perhaps even if they did nothing wrong and are in fact the victim in that situation. Take the case of Linda Yaccarino. Ordinarily, if a male employee publicly and sexually harassed his female CEO on Twitter, he would (and should) be fired immediately. When Grok did that though, it's the CEO who ended up quitting.
    - galaxyLogic 1y ago
      My question is if I get some code from AI, save it to a file, then modify it or add some functions to it, can I still claim the copyright for it at the top of the file? Do I need to give the AI any credit? I'm asking because I read somewhere that "AI produced output cannot be copyrighted". But what if I modify that output myself? I am then a co-creator, right, and I think I should have a right to some copyright protection.
      - moi2388 1y ago
        You do.
      - jdiff 1y ago
        First, a few disclaimers, I am not a lawyer and this is an actively evolving area. The answer that most aligns with current precedent to my knowledge is that the parts you modify are protected by your copyright, but the rest remains uncopyrightable. With the exception of any chunks generated that align with someone's existing copyrighted code, as long as those chunks are substantial and unique enough.
        tuple 1y ago
        This is how I understand it as well. There was a case of a monkey taking a selfie and the photog attempting to assert copyright. Courts determined (if I remember correctly :p ) that only human works are copyrightable and therefore the monkey's selfie could not be copyrighted at all.
        jdiff 1y ago
        I deleted a whole paragraph with exactly that example, so at least if we're wrong, we're wrong together.
  - zimpenfish 1y ago
    > "A computer can never be held accountable." cf the Post Office scandal in the UK which was partly helped along by the 1999 change in law[1] which repealed the 1984 stance that "computer evidence is not permissible unless it is shown to be working correctly at the time"[0]; i.e. that a computer was now presumed to be working correctly and it was up to the defence to prove otherwise. [0] https://www.legislation.gov.uk/ukpga/1984/60/section/69/1991-02-01 https://www.legislation.gov.uk/ukpga/1984/60/section/69/1991... [1] https://www.legislation.gov.uk/ukpga/1999/23/section/60/1999-07-27 https://www.legislation.gov.uk/ukpga/1999/23/section/60/1999...
- AdieuToLogic 1y ago
  Except that the "systems made out of meat" are the entities which both define the problem needing to be solved and are the sole determiners if said problem has been solved. Of note too is that the same "systems made out of meat" have been producing content satisfying the strict mathematical model for decades and continue to do so beyond the capabilities of the aforementioned algorithms.
  - ygritte 1y ago
    +1. Also, those systems made out of meat were the ones that discovered the strict mathematical models in the first place and the rules by which they work.
  - mlyle 1y ago
    It's usually not the same pile of meat defining the problem and solving the problem. Yes, humans exceed the capability of machines, until they don't. Machines exceed humans in more and more domains. The style of argument you made about the nature of the machinery used applies just as well (maybe better) to humans. To get a valid argument, we'll need to be more nuanced.
    - AdieuToLogic 1y ago
      >> Except that the "systems made out of meat" are the entities which both define the problem needing to be solved and are the sole determiners if said problem has been solved. > It's usually not the same pile of meat defining the problem and solving the problem. True, but this distinction is also irrelevant. The point is that problems capable of being solved by software systems are identified, reified, and then determined to be solved by people. Regardless of the tooling used to do so and the number of people involved. > Yes, humans exceed the capability of machines, until they don't. Machines exceed humans in more and more domains. But machines do not, and cannot, exceed humans in the domain of "understanding what a human wants" because this type of understanding is intrinsic to people by definition. Machines can do a lot of things, things which can be amazing and are truly beneficial to mankind, but they cannot understand as people colloquially use this term since they are not people. I believe a decent analogy for this situation is how people will never completely understand the communication whales use with each other the way whales do themselves. There may someday exist the ability to translate their communication into a semblance of human language, but that would be only what we think is correct and not the same as being a whale.
- cookiengineer 1y ago
  The difference: Meatbags created something like an education system, where the track record in it functions as a ledger for hiring companies. There is no such thing for AI. No ledger, no track record, no reproducibility.
  - ozim 1y ago
    Batter. Meatbags created liability system where if meatbag harms others they go to jail.
  - Gud 1y ago
    On the other hand, some of the most capable meat bags said fuck you to your record keeping system and dropped out.
    - cookiengineer 1y ago
      Those meatbags usually have a different form of ledger. Be it open source contributions, built prototypes, apps or games, or other things they've shared before to create a track record of their skillset. I never claimed that there is one type of ultrageneric ledger that works for all areas of research. But somehow, the LLM world still thinks that is the case for whatever reason.
- jp0d 1y ago
  Countless bodies consisting of the said meat have been responsible for the advancement of technology so far. If these meat brains don't contribute to any new advancements then the corpus of data will stay stagnant!
- bcrosby95 1y ago
  Those systems of meat were trained over hundreds of millions of years compared to mere months or years.
- nyc_data_geek 1y ago
  Where do you think training data comes from
- ygritte 1y ago
  I'm getting so tired of this dumb kind of non-argument. You can't defend LLMs on their own merits, so you try to make them look smarter by throwing shade on humans. That's a non sequitur and whataboutism.
  - mlyle 1y ago
    Nah-- I feel like I have my eyes pretty wide open about the shortcomings of LLMs (but still find them useful often). But any argument seeking to dunk on LLMs needs to not also apply equally to the alternative (humans).
    - ygritte 1y ago
      And, wouldn't you know it, it actually does not also apply equally to the alternative (humans).
      - mlyle 1y ago
        How much of it doesn't? We're deterministic? (aren't we less deterministic than LLMs?) All of our training is auditable? (there's a wealth of unknown experiences in each person writing code, to say nothing of the unknown and irrelevant experiences in our evolutionary background). Maybe you can argue we don't use statistical completion and prediction as a heavy underpinning to our reasoning, but that's hardly settled. Nah-- you will have to try harder to make an argument that really focuses on how LLMs are different from the alternative.
- hitarpetar 1y ago
  I find this take to be purely misanthropic. we are more than stochastic parrots
ozim 1y ago
Sounds like we are in the end game for „move fast and break things”. Doesn’t feel like we can invent something that moves even faster and breaks more.
Finbel 1y ago
Have you not used commercial LLMs to generate program source code? You describe it as if it's an almost unsolvable problem which might have been reasonable 2 years ago but I just used gpt-5 to generate a complete NextJS application for flashcards. I've literally been employing nondeterministic content generation based on statistical relevance defined by an unknown training data, to repeatably produce content satisfying a strict mathematical model for months now.
- anticodon 1y ago
  I'm pretty sure there're thousands of blog posts and books describing creation of a complete flashcards application in all popular programming languages and on all popular frameworks.
  - Finbel 1y ago
    There're thousands of blog posts and books describing the vast majority of code most software developers write on a day to day basis. 99.99% of the code in that B2B SaaS for finding the cheapest industrial shipping option isn't novel.
    - AdieuToLogic 1y ago
      > 99.99% of the code in that B2B SaaS for finding the cheapest industrial shipping option isn't novel. That's like saying 99.99% of the food people eat consists of protein, carbohydrates, fats, and/or vegetables and therefore isn't novel. The implication being a McDonald's Big Mac and fries is the same as a spinach salad. The only way someone could believe all food is the same as a Big Mac and fries is if this is all they ate and knew nothing else. Hyperbole never ends well and neither does assuming novelty requires rarity or uniqueness, as distinct combinations of programmatic operations which deliver value in a problem domain is the very definition "new in an interesting way." Just like how Thai noodles have proteins, carbohydrates, fats, and/or vegetables, yet are nothing like a Big Mac and fries.
      - Finbel 1y ago
pmxi 1y ago
Humans are far less deterministic than LLMs, yet presumably are acceptable for writing program source code?

mlyle 1y ago

Nearly as bad: trying to use systems made out of meat, evolved from a unrelated background and trained on an undocumented and chaotic corpus of data, to try and produce content satisfying a strict mathematical model.

jdiff 1y ago
Meaty feet can be held to a fire. To quote IBM, "A computer can never be held accountable."
- MobiusHorizons 1y ago
  This is the question I keep asking leaders (I literally asked a VP this question once in an all hands). How do we approach the risk associated mistakes made by AI?(process, legal, security, insurance etc) We have process and legal agreements in place to deal with humans that work for a business making mistakes. We need analogs for AI if we want to use it in similar ways.
  - jpcosta 1y ago
    What was the answer? Asking for a vp friend
    - AdieuToLogic 1y ago
      >>> Meaty feet can be held to a fire. To quote IBM, "A computer can never be held accountable." >> This is the question I keep asking leaders (I literally asked a VP this question once in an all hands). How do we approach the risk associated mistakes made by AI? > What was the answer? Asking for a vp friend This is a difficult issue to tackle, no doubt. What follows drifts into the philosophical realm by necessity. Software exists to provide value to people. Malicious software qualifies as such due to the desires of the actors which produce same, but will no longer be considered here as this is not germane. AI is an umbrella term for numerous algorithms having wide ranging problem domain applicability and often can approximate near-optimal solutions using significantly less resources than other approaches. But they are still algorithms, capable of only one thing - execute their defined logic. Sometimes this logic can produce results similar to be what a person would in a similar situation. Sometimes the logic will produce wildly different results. Often there is significant value when the logic is used appropriately. In all cases AI algorithms do not possess the concept of understanding. This includes derivatives of understanding such as: - empathy - integrity - morals - right - wrong Which brings us back to part of the first quoted post: To quote IBM, "A computer can never be held accountable." Accountability requires justification of actions taken or lack thereof, which demands the ability to explain why said actions were undertaken relative to other options, and implies a potential consequence be imposed by an authority. Algorithms can partially "justify their output" via strategic logging, but that's about it. Which is why "a computer can never be held accountable." Because it is a machine, executing the instructions ultimately initiated by one or more persons whom can be held accountable.
    - MobiusHorizons 1y ago
      In the all hands I got an answer about techniques that would be used to reduce the likelihood of mistakes. Ie not an answer.
  - Covenant0028 1y ago
    I suspect the analog will be that the "human in the loop" will bear all the consequences. Perhaps even if they did nothing wrong and are in fact the victim in that situation. Take the case of Linda Yaccarino. Ordinarily, if a male employee publicly and sexually harassed his female CEO on Twitter, he would (and should) be fired immediately. When Grok did that though, it's the CEO who ended up quitting.
  - galaxyLogic 1y ago
    My question is if I get some code from AI, save it to a file, then modify it or add some functions to it, can I still claim the copyright for it at the top of the file? Do I need to give the AI any credit? I'm asking because I read somewhere that "AI produced output cannot be copyrighted". But what if I modify that output myself? I am then a co-creator, right, and I think I should have a right to some copyright protection.
    - moi2388 1y ago
      You do.
    - jdiff 1y ago
      First, a few disclaimers, I am not a lawyer and this is an actively evolving area. The answer that most aligns with current precedent to my knowledge is that the parts you modify are protected by your copyright, but the rest remains uncopyrightable. With the exception of any chunks generated that align with someone's existing copyrighted code, as long as those chunks are substantial and unique enough.
      - tuple 1y ago
        This is how I understand it as well. There was a case of a monkey taking a selfie and the photog attempting to assert copyright. Courts determined (if I remember correctly :p ) that only human works are copyrightable and therefore the monkey's selfie could not be copyrighted at all.
        jdiff 1y ago
        I deleted a whole paragraph with exactly that example, so at least if we're wrong, we're wrong together.
- zimpenfish 1y ago
  > "A computer can never be held accountable." cf the Post Office scandal in the UK which was partly helped along by the 1999 change in law[1] which repealed the 1984 stance that "computer evidence is not permissible unless it is shown to be working correctly at the time"[0]; i.e. that a computer was now presumed to be working correctly and it was up to the defence to prove otherwise. [0] https://www.legislation.gov.uk/ukpga/1984/60/section/69/1991-02-01 https://www.legislation.gov.uk/ukpga/1984/60/section/69/1991... [1] https://www.legislation.gov.uk/ukpga/1999/23/section/60/1999-07-27 https://www.legislation.gov.uk/ukpga/1999/23/section/60/1999...
AdieuToLogic 1y ago
Except that the "systems made out of meat" are the entities which both define the problem needing to be solved and are the sole determiners if said problem has been solved. Of note too is that the same "systems made out of meat" have been producing content satisfying the strict mathematical model for decades and continue to do so beyond the capabilities of the aforementioned algorithms.
- ygritte 1y ago
  +1. Also, those systems made out of meat were the ones that discovered the strict mathematical models in the first place and the rules by which they work.
- mlyle 1y ago
  It's usually not the same pile of meat defining the problem and solving the problem. Yes, humans exceed the capability of machines, until they don't. Machines exceed humans in more and more domains. The style of argument you made about the nature of the machinery used applies just as well (maybe better) to humans. To get a valid argument, we'll need to be more nuanced.
  - AdieuToLogic 1y ago
    >> Except that the "systems made out of meat" are the entities which both define the problem needing to be solved and are the sole determiners if said problem has been solved. > It's usually not the same pile of meat defining the problem and solving the problem. True, but this distinction is also irrelevant. The point is that problems capable of being solved by software systems are identified, reified, and then determined to be solved by people. Regardless of the tooling used to do so and the number of people involved. > Yes, humans exceed the capability of machines, until they don't. Machines exceed humans in more and more domains. But machines do not, and cannot, exceed humans in the domain of "understanding what a human wants" because this type of understanding is intrinsic to people by definition. Machines can do a lot of things, things which can be amazing and are truly beneficial to mankind, but they cannot understand as people colloquially use this term since they are not people. I believe a decent analogy for this situation is how people will never completely understand the communication whales use with each other the way whales do themselves. There may someday exist the ability to translate their communication into a semblance of human language, but that would be only what we think is correct and not the same as being a whale.
cookiengineer 1y ago
The difference: Meatbags created something like an education system, where the track record in it functions as a ledger for hiring companies. There is no such thing for AI. No ledger, no track record, no reproducibility.
- ozim 1y ago
  Batter. Meatbags created liability system where if meatbag harms others they go to jail.
- Gud 1y ago
  On the other hand, some of the most capable meat bags said fuck you to your record keeping system and dropped out.
  - cookiengineer 1y ago
    Those meatbags usually have a different form of ledger. Be it open source contributions, built prototypes, apps or games, or other things they've shared before to create a track record of their skillset. I never claimed that there is one type of ultrageneric ledger that works for all areas of research. But somehow, the LLM world still thinks that is the case for whatever reason.
jp0d 1y ago
Countless bodies consisting of the said meat have been responsible for the advancement of technology so far. If these meat brains don't contribute to any new advancements then the corpus of data will stay stagnant!
bcrosby95 1y ago
Those systems of meat were trained over hundreds of millions of years compared to mere months or years.
nyc_data_geek 1y ago
Where do you think training data comes from
ygritte 1y ago
I'm getting so tired of this dumb kind of non-argument. You can't defend LLMs on their own merits, so you try to make them look smarter by throwing shade on humans. That's a non sequitur and whataboutism.
- mlyle 1y ago
  Nah-- I feel like I have my eyes pretty wide open about the shortcomings of LLMs (but still find them useful often). But any argument seeking to dunk on LLMs needs to not also apply equally to the alternative (humans).
  - ygritte 1y ago
    And, wouldn't you know it, it actually does not also apply equally to the alternative (humans).
    - mlyle 1y ago
      How much of it doesn't? We're deterministic? (aren't we less deterministic than LLMs?) All of our training is auditable? (there's a wealth of unknown experiences in each person writing code, to say nothing of the unknown and irrelevant experiences in our evolutionary background). Maybe you can argue we don't use statistical completion and prediction as a heavy underpinning to our reasoning, but that's hardly settled. Nah-- you will have to try harder to make an argument that really focuses on how LLMs are different from the alternative.
hitarpetar 1y ago
I find this take to be purely misanthropic. we are more than stochastic parrots

ozim 1y ago

Sounds like we are in the end game for „move fast and break things”. Doesn’t feel like we can invent something that moves even faster and breaks more.

Finbel 1y ago

Have you not used commercial LLMs to generate program source code? You describe it as if it's an almost unsolvable problem which might have been reasonable 2 years ago but I just used gpt-5 to generate a complete NextJS application for flashcards. I've literally been employing nondeterministic content generation based on statistical relevance defined by an unknown training data, to repeatably produce content satisfying a strict mathematical model for months now.

anticodon 1y ago
I'm pretty sure there're thousands of blog posts and books describing creation of a complete flashcards application in all popular programming languages and on all popular frameworks.
- Finbel 1y ago
  There're thousands of blog posts and books describing the vast majority of code most software developers write on a day to day basis. 99.99% of the code in that B2B SaaS for finding the cheapest industrial shipping option isn't novel.
  - AdieuToLogic 1y ago
    > 99.99% of the code in that B2B SaaS for finding the cheapest industrial shipping option isn't novel. That's like saying 99.99% of the food people eat consists of protein, carbohydrates, fats, and/or vegetables and therefore isn't novel. The implication being a McDonald's Big Mac and fries is the same as a spinach salad. The only way someone could believe all food is the same as a Big Mac and fries is if this is all they ate and knew nothing else. Hyperbole never ends well and neither does assuming novelty requires rarity or uniqueness, as distinct combinations of programmatic operations which deliver value in a problem domain is the very definition "new in an interesting way." Just like how Thai noodles have proteins, carbohydrates, fats, and/or vegetables, yet are nothing like a Big Mac and fries.
    - Finbel 1y ago

pmxi 1y ago

Humans are far less deterministic than LLMs, yet presumably are acceptable for writing program source code?