4 ms·

I think that's too easy an analogy, though. Calculators are deterministically correct given the right input. It does not require expert judgement on whether an

by lukev 6mo ago

I think that's too easy an analogy, though. Calculators are deterministically correct given the right input. It does not require expert judgement on whether an answer they gave is reasonable or not. As someone who uses LLMs all day for coding, and who regularly bumps against the boundaries of what they're capable of, that's very much not the case. The only reason I can use them effectively is because I know what good software looks like and when to drop down to more explicit instructions.

yunwal 6mo ago
Determinism just means you don't have to use statistics to approach the right answer. It's not some silver bullet that magically makes things understandable and it's not true that if it's missing from a system you can't possibly understand it.
- lukev 6mo ago
  That's not what I mean. If I use a calculator to find a logarithm, and I know what a logarithm is, then the answer the calculator gives me is perfectly useful and 100% substitutable for what I would have found if I'd calculated the logarithm myself. If I use Claude to "build a login page", it will definitely build me a login page. But there's a very real chance that what it generated contains a security issue. If I'm an experienced engineer I can take a quick look and validate whether it does or whether it doesn't, but if I'm not, I've introduced real risk to my application.
  - threatofrain 6mo ago
    Those two tasks are just very different. In one world you have provided a complete specification, such as 1 + 1, for which the calculator responds with some answer and both you and the machine have a decidable procedure for judging answers. In another world you have engaged in a declaration for which the are many right and wrong answers, and thus even the boundaries of error are in question. It's equivalent to asking your friend to pick you up, and they arrive in a big vs small car. Maybe you needed a big car because you were going to move furniture, or maybe you don't care, oops either way.
    - lukev 6mo ago
      Yes. That is the point I was making. Calculators provide a deterministic solution to a well-defined task. LLMs don't.
      - bluefirebrand 6mo ago
        Furthermore, it is possible to build a precise mathematical formula to produce a desired solution It is not possible to be nearly as precise when describing a desired solution to an LLM, because natural languages are simply not capable of that level of precision... Which is the entire reason coding languages exist in the first place
II2II 6mo ago
> Calculators are deterministically correct Calculators are deterministic, but they are not necessarily correct. Consider 32-bit integer arithmetic: 30000000 * 1000 / 1000 30000000 / 1000 * 1000 Mathematically, they are identical. Computationally, the results are deterministic. On the other hand, the computer will produce different results. There are many other cases where the expected result is different from what a computer calculates.
- wongarsu 6mo ago
  A good calculator will however do this correctly (as in: the way anyone would expect). Small cheap calculators revert to confusing syntax, but if you pay $30 for a decent handheld calculator or use something decent like wolframalpha on your phone/laptop/desktop you won't run into precision issues for reasonable numbers.
  - Ifkaluva 6mo ago
    He’s not talking about order of operations, he’s talking about floating point error, which will accumulate in different ways in each case, because floating point is an imperfect representation of real numbers
    - II2II 6mo ago
      Yeap, the specific example wasn't important. I choose an example involving the order of operations and an integer overflow simply because it would be easy to discuss. (I have been out of the field for nearly 20 years now.) Your example of floating point errors is another. I also encountered artifacts from approximations for transcendental functions. Choosing a "better" language was not always an option, at least at the time. I was working with grad students who were managing huge datasets, sometimes for large simulations and sometimes from large surveys. They were using C. Some of the faculty may have used Fortran. C exposes you the vulgarities of the hardware, and I'm fairly certain Fortran does as well. They weren't going to use a calculator for those tasks, nor an interpreted language. Even if they wanted to choose another language, the choice of languages was limited by the machines they used. I've long since forgotten what the high performance cluster was running, but it wasn't Linux and it wasn't on Intel. They may have been able to license something like Mathematica for it, but that wasn't the type of computation they were doing.
bsder 6mo ago
> Calculators are deterministically correct given the right input. It does not require expert judgement on whether an answer they gave is reasonable or not. That's not actually true. The HP-12C calculator is still the dominant calculator in business schools 45 years later precisely because it did take expert judgement to determine whether certain interest and amortization calculations were reasonable.

by lukev 6mo ago

yunwal 6mo ago

Determinism just means you don't have to use statistics to approach the right answer. It's not some silver bullet that magically makes things understandable and it's not true that if it's missing from a system you can't possibly understand it.

lukev 6mo ago
That's not what I mean. If I use a calculator to find a logarithm, and I know what a logarithm is, then the answer the calculator gives me is perfectly useful and 100% substitutable for what I would have found if I'd calculated the logarithm myself. If I use Claude to "build a login page", it will definitely build me a login page. But there's a very real chance that what it generated contains a security issue. If I'm an experienced engineer I can take a quick look and validate whether it does or whether it doesn't, but if I'm not, I've introduced real risk to my application.
- threatofrain 6mo ago
  Those two tasks are just very different. In one world you have provided a complete specification, such as 1 + 1, for which the calculator responds with some answer and both you and the machine have a decidable procedure for judging answers. In another world you have engaged in a declaration for which the are many right and wrong answers, and thus even the boundaries of error are in question. It's equivalent to asking your friend to pick you up, and they arrive in a big vs small car. Maybe you needed a big car because you were going to move furniture, or maybe you don't care, oops either way.
  - lukev 6mo ago
    Yes. That is the point I was making. Calculators provide a deterministic solution to a well-defined task. LLMs don't.
    - bluefirebrand 6mo ago
      Furthermore, it is possible to build a precise mathematical formula to produce a desired solution It is not possible to be nearly as precise when describing a desired solution to an LLM, because natural languages are simply not capable of that level of precision... Which is the entire reason coding languages exist in the first place

II2II 6mo ago

> Calculators are deterministically correct Calculators are deterministic, but they are not necessarily correct. Consider 32-bit integer arithmetic: 30000000 * 1000 / 1000 30000000 / 1000 * 1000 Mathematically, they are identical. Computationally, the results are deterministic. On the other hand, the computer will produce different results. There are many other cases where the expected result is different from what a computer calculates.

wongarsu 6mo ago
A good calculator will however do this correctly (as in: the way anyone would expect). Small cheap calculators revert to confusing syntax, but if you pay $30 for a decent handheld calculator or use something decent like wolframalpha on your phone/laptop/desktop you won't run into precision issues for reasonable numbers.
- Ifkaluva 6mo ago
  He’s not talking about order of operations, he’s talking about floating point error, which will accumulate in different ways in each case, because floating point is an imperfect representation of real numbers
  - II2II 6mo ago
    Yeap, the specific example wasn't important. I choose an example involving the order of operations and an integer overflow simply because it would be easy to discuss. (I have been out of the field for nearly 20 years now.) Your example of floating point errors is another. I also encountered artifacts from approximations for transcendental functions. Choosing a "better" language was not always an option, at least at the time. I was working with grad students who were managing huge datasets, sometimes for large simulations and sometimes from large surveys. They were using C. Some of the faculty may have used Fortran. C exposes you the vulgarities of the hardware, and I'm fairly certain Fortran does as well. They weren't going to use a calculator for those tasks, nor an interpreted language. Even if they wanted to choose another language, the choice of languages was limited by the machines they used. I've long since forgotten what the high performance cluster was running, but it wasn't Linux and it wasn't on Intel. They may have been able to license something like Mathematica for it, but that wasn't the type of computation they were doing.

bsder 6mo ago

> Calculators are deterministically correct given the right input. It does not require expert judgement on whether an answer they gave is reasonable or not. That's not actually true. The HP-12C calculator is still the dominant calculator in business schools 45 years later precisely because it did take expert judgement to determine whether certain interest and amortization calculations were reasonable.