5 ms·

The fun part is that specs already are non-deterministic. If you spend time to write out requirements in English in a way that cannot be misinterpreted in any

by ozim 1y ago

The fun part is that specs already are non-deterministic. If you spend time to write out requirements in English in a way that cannot be misinterpreted in any way you end up with programming language.

diavolodeejay 1y ago
So… COBOL?
jlawson 1y ago
Specs are ambiguous but not necessarily non-deterministic. The same entity interpreting the spec in exactly the same way will resolve the ambiguities the same way each time. Human and current AI interpretation of specs is non-deterministic process. But, if we wanted to build a deterministic AI we could.
- oblio 1y ago
  > But, if we wanted to build a deterministic AI we could. Is this bold proposal backed by any theory?
  - reverius42 1y ago
    Given that they all use pseudo-random (and not actually random) numbers, they are "deterministic" in the sense that given a fixed seed, they will produce a fixed result... But perhaps that's not what was meant by deterministic. Something like an understandable process producing an answer rather than a pile of linear algebra?
    - catoc 1y ago
      I was thinking the exact same thing: if you don’t change the weights, use identical “temperature” etc, the same prompt will yield the same output. Under the hood it’s still deterministic code running on a deterministic machine
      - block_dagger 1y ago
        This is incorrect. Temperature would need to be zero to get same result.
        catoc 1y ago
        You’re right - TIL
        jlawson 1y ago
        This is not correct. If the algorithm is deterministic and the random seed is the same, temperature can be anything and get the same result. Same as using a seed to get the same map generated in Dwarf Fortress.
        reverius42 1y ago
        Thanks for replying, this is what I thought as well.
  - globular-toast 1y ago
    You can just change your definition of "AI". Back in the 60s the pinnacle of AI was things like automatic symbolic integration and these would certainly be completely deterministic. Nowadays people associate "AI" with stuff like LLMs and diffusion etc. that have randomness included in to make them seem "organic", but it doesn't have to be that way. I actually think a large part of people's amazement with the current breed of AI is the random aspect. It's long been known that random numbers are cool (see Knuth volume 2, in particular where he says randomness make computer-generated graphics and music more "appealing"). Unfortunately being amazed by graphics and music (and now text) output is one thing, making logical decisions with real consequences is quite another.
    - oblio 1y ago
      In 2025, 99% of people are talking about LLMs or stable diffusion.
      - globular-toast 1y ago
        So then your question boils down to "how could X, which I've defined as Y, be Z?"
        oblio 1y ago
        The cutting edge of AI research is LLMs. Those aren't deterministic. You can build an "AI" with whatever you want, but context matters and we live in 2025, not 1985.
- andruby 1y ago
  > if we wanted to build a deterministic AI we could What's holding us back from building a deterministic generative AI? I probably don't understand enough, but I assumed that the non-determinism was "inherit" in the current LLM technology
jimbo808 1y ago
Humans don't make mistakes nearly as much, the mistakes they do make are way more predictable (they're easier to spot in code review), and they don't tend to make the kinds of catastrophic mistakes that could sink a business. They also tend to cause codebases to rapidly deteriorate, since even very disciplined reviewers can miss the kinds of strange and unpredictable stuff an LLM will do. Redundant code isn't evident in a diff, and things like tautological tests, or useless tests where they're mocking everything and only actually testing the mocks. Or they'll write a bunch of redundant code because they really just aggressively avoid code re-use unless you are very specific. The real problem is just that they don't have brains, and can't think. They generate text that is optimized to look the most right, but not to be the most right. That means they're deceptive right off the bat. When a human is wrong, it usually looks wrong. When an LLM is wrong, it's generating the most correct looking thing it possibly could while still being wrong, with no consideration for actual correctness. It has no idea what "correctness" even means, or any ideas at all, because it's a computer doing matmul. They are text summarization/regurgitation, pattern matching machines. They regurgitate summaries of things seen in their training data, and that training data was written by humans who can think. We just let ourselves get duped into believing the machine is the where the thinking is coming from and not the (likely uncompensated) author(s) whose work was regurgitated for you.
- ACCount37 1y ago
  >The real problem is just that they don't have brains, and can't think. That would have had more weight if you haven't just described junior developer behavior beforehand. "LLMs can't think" is anthropocentric cope. It's the old AI effect all over again - people would rather die than admit that there's very little practical difference between their own "thinking" and that of an AI chatbot.
  - gnulinux996 1y ago
    > That would have had more weight if you haven't just described junior developer behavior beforehand. Effectively telling that junior developers "don't have brains" is in very bad taste and offensively wrong. > people would rather die than admit that there's very little practical difference between their own "thinking" and that of an AI chatbot. Would you like to elaborate on this? I was told that McDonalds employees would have been replaced by now, self-driving cars will be driving the streets and new medicines would have been discovered. It's been a couple of years that "AI" is out, and no singularity yet.
qcnguy 1y ago
Not really, code even in high level languages is always lower level than English just for computer nonsense reasons. Example: "read a CSV file and add a column containing the multiple of the price and quantity columns". That's about 20 words. Show me the programming language that can express that entire feature in 20 words. Even very English-like languages like Python or Kotlin might just about do it, if you're working in something else like C++ then no. In practice, this spec will expand to changes to your dependency lists (and therefore you must know what library is used for CSV parsing in your language, the AI knows this stuff better than you), then there's some file handling, error handling if the file doesn't exist, maybe some UI like flags or other configuration, working out what the column names are, writing the loop, saving it back out, writing unit tests. Any reasonable programmer will produce a very similar PR given this spec but the diff will be much larger than the spec.
- cess11 1y ago
  Iverson languages could do that quite succinctly.
- sarchertech 1y ago
  >"read a CSV file and add a column containing the multiple of the price and quantity columns" This is an underspecification if you want to reliably repeatably produce similar code. The biggest difference is that some developers will read the whole CSV into memory before doing the computations. In practice the difference between those implementation is huge. Another big difference is how you represent the price field. If you parse them as floats and the quantity is big enough, you'll end up with errors. Even if quantity is small, you'll have to deal with rounding in your new column. You didn't even specific the name of the new column, so the name is going to be different every time you run the LLM. What happens if you run this on a file the program has already been ran on? And these are just a few of the reasonable ways of fitting that spec but producing wildly different programs. Making a spec that has a good chance of producing a reasonably similar program each time looks more like: “Read input.csv (UTF-8, comma-delimited, header row). Read it line by line, do not load the entire file into memory. Parse the price and quantity columns as numbers, stripping currency symbols and thousands separators; interpret decimals using a dot (.). Treat blanks as null and leave the result null for such rows. Compute per-row line_total = round(Decimal(price) * Decimal(quantity), 2). Append line_total as the last column (name the column "Total") without reordering existing columns, and write to output.csv, preserving quoting and delimiter. Do not overwrite existing columns. Do not evaluate or emit spreadsheet formulas.” And even then you couldn't just check this in and expect the same code to be generated each time, you'd need a large test suite--just to constraint the LLM. And even then the LLM would still occasionally find ways to generate code that passes the tests but does thing you don't want it to.

by ozim 1y ago

diavolodeejay 1y ago

So… COBOL?

jlawson 1y ago

Specs are ambiguous but not necessarily non-deterministic. The same entity interpreting the spec in exactly the same way will resolve the ambiguities the same way each time. Human and current AI interpretation of specs is non-deterministic process. But, if we wanted to build a deterministic AI we could.

oblio 1y ago
> But, if we wanted to build a deterministic AI we could. Is this bold proposal backed by any theory?
- reverius42 1y ago
  Given that they all use pseudo-random (and not actually random) numbers, they are "deterministic" in the sense that given a fixed seed, they will produce a fixed result... But perhaps that's not what was meant by deterministic. Something like an understandable process producing an answer rather than a pile of linear algebra?
  - catoc 1y ago
    I was thinking the exact same thing: if you don’t change the weights, use identical “temperature” etc, the same prompt will yield the same output. Under the hood it’s still deterministic code running on a deterministic machine
    - block_dagger 1y ago
      This is incorrect. Temperature would need to be zero to get same result.
      - catoc 1y ago
        You’re right - TIL
      - jlawson 1y ago
        This is not correct. If the algorithm is deterministic and the random seed is the same, temperature can be anything and get the same result. Same as using a seed to get the same map generated in Dwarf Fortress.
        reverius42 1y ago
        Thanks for replying, this is what I thought as well.
- globular-toast 1y ago
  You can just change your definition of "AI". Back in the 60s the pinnacle of AI was things like automatic symbolic integration and these would certainly be completely deterministic. Nowadays people associate "AI" with stuff like LLMs and diffusion etc. that have randomness included in to make them seem "organic", but it doesn't have to be that way. I actually think a large part of people's amazement with the current breed of AI is the random aspect. It's long been known that random numbers are cool (see Knuth volume 2, in particular where he says randomness make computer-generated graphics and music more "appealing"). Unfortunately being amazed by graphics and music (and now text) output is one thing, making logical decisions with real consequences is quite another.
  - oblio 1y ago
    In 2025, 99% of people are talking about LLMs or stable diffusion.
    - globular-toast 1y ago
      So then your question boils down to "how could X, which I've defined as Y, be Z?"
      - oblio 1y ago
        The cutting edge of AI research is LLMs. Those aren't deterministic. You can build an "AI" with whatever you want, but context matters and we live in 2025, not 1985.
andruby 1y ago
> if we wanted to build a deterministic AI we could What's holding us back from building a deterministic generative AI? I probably don't understand enough, but I assumed that the non-determinism was "inherit" in the current LLM technology

jimbo808 1y ago

Humans don't make mistakes nearly as much, the mistakes they do make are way more predictable (they're easier to spot in code review), and they don't tend to make the kinds of catastrophic mistakes that could sink a business. They also tend to cause codebases to rapidly deteriorate, since even very disciplined reviewers can miss the kinds of strange and unpredictable stuff an LLM will do. Redundant code isn't evident in a diff, and things like tautological tests, or useless tests where they're mocking everything and only actually testing the mocks. Or they'll write a bunch of redundant code because they really just aggressively avoid code re-use unless you are very specific. The real problem is just that they don't have brains, and can't think. They generate text that is optimized to look the most right, but not to be the most right. That means they're deceptive right off the bat. When a human is wrong, it usually looks wrong. When an LLM is wrong, it's generating the most correct looking thing it possibly could while still being wrong, with no consideration for actual correctness. It has no idea what "correctness" even means, or any ideas at all, because it's a computer doing matmul. They are text summarization/regurgitation, pattern matching machines. They regurgitate summaries of things seen in their training data, and that training data was written by humans who can think. We just let ourselves get duped into believing the machine is the where the thinking is coming from and not the (likely uncompensated) author(s) whose work was regurgitated for you.

ACCount37 1y ago
>The real problem is just that they don't have brains, and can't think. That would have had more weight if you haven't just described junior developer behavior beforehand. "LLMs can't think" is anthropocentric cope. It's the old AI effect all over again - people would rather die than admit that there's very little practical difference between their own "thinking" and that of an AI chatbot.
- gnulinux996 1y ago
  > That would have had more weight if you haven't just described junior developer behavior beforehand. Effectively telling that junior developers "don't have brains" is in very bad taste and offensively wrong. > people would rather die than admit that there's very little practical difference between their own "thinking" and that of an AI chatbot. Would you like to elaborate on this? I was told that McDonalds employees would have been replaced by now, self-driving cars will be driving the streets and new medicines would have been discovered. It's been a couple of years that "AI" is out, and no singularity yet.

qcnguy 1y ago

Not really, code even in high level languages is always lower level than English just for computer nonsense reasons. Example: "read a CSV file and add a column containing the multiple of the price and quantity columns". That's about 20 words. Show me the programming language that can express that entire feature in 20 words. Even very English-like languages like Python or Kotlin might just about do it, if you're working in something else like C++ then no. In practice, this spec will expand to changes to your dependency lists (and therefore you must know what library is used for CSV parsing in your language, the AI knows this stuff better than you), then there's some file handling, error handling if the file doesn't exist, maybe some UI like flags or other configuration, working out what the column names are, writing the loop, saving it back out, writing unit tests. Any reasonable programmer will produce a very similar PR given this spec but the diff will be much larger than the spec.

cess11 1y ago
Iverson languages could do that quite succinctly.
sarchertech 1y ago
>"read a CSV file and add a column containing the multiple of the price and quantity columns" This is an underspecification if you want to reliably repeatably produce similar code. The biggest difference is that some developers will read the whole CSV into memory before doing the computations. In practice the difference between those implementation is huge. Another big difference is how you represent the price field. If you parse them as floats and the quantity is big enough, you'll end up with errors. Even if quantity is small, you'll have to deal with rounding in your new column. You didn't even specific the name of the new column, so the name is going to be different every time you run the LLM. What happens if you run this on a file the program has already been ran on? And these are just a few of the reasonable ways of fitting that spec but producing wildly different programs. Making a spec that has a good chance of producing a reasonably similar program each time looks more like: “Read input.csv (UTF-8, comma-delimited, header row). Read it line by line, do not load the entire file into memory. Parse the price and quantity columns as numbers, stripping currency symbols and thousands separators; interpret decimals using a dot (.). Treat blanks as null and leave the result null for such rows. Compute per-row line_total = round(Decimal(price) * Decimal(quantity), 2). Append line_total as the last column (name the column "Total") without reordering existing columns, and write to output.csv, preserving quoting and delimiter. Do not overwrite existing columns. Do not evaluate or emit spreadsheet formulas.” And even then you couldn't just check this in and expect the same code to be generated each time, you'd need a large test suite--just to constraint the LLM. And even then the LLM would still occasionally find ways to generate code that passes the tests but does thing you don't want it to.