6 ms·

You find it hard to believe that non-deterministic black boxes at the core of complex systems are eager to put non-deterministic black boxes at the core of comp

You find it hard to believe that non-deterministic black boxes at the core of complex systems are eager to put non-deterministic black boxes at the core of complex systems?

beepbooptheory 2y ago
Can you actually like follow through with this line? I know there are literally tens of thousands of comments just like this at this point, but if you have chance, could you explain what you think this means? What should we take from it? Just unpack it a little bit for us.
- dangsux 2y ago
  [dead]
- therealcamino 2y ago
  I took it to be a joke that the description "slow, expensive, non-deterministic black boxes" can apply to the engineers themselves. The engineers would be the ones who would have to place LLMs at the core of the system. To anyone outside, the work of the engineers is as opaque as the operation of LLMs.
- croshan 2y ago
  An interpretation that makes sense to me: humans are non-deterministic black boxes already at the core of complex systems. So in that sense, replacing a human with AI is not unreasonable. I’d disagree, though: humans are still easier to predict and understand (and trust) than AI, typically.
  - sdesol 2y ago
    With humans we have a decent understanding of what they are capable of. I trust a medical professional to provide me with medical advice and an engineer to provide me with engineering advice. With LLM, it can be unpredictable at times, and they can make errors in ways that you would not imagine. Take the following examples from my tool, which shows how GPT-4o and Claude 3.5 Sonnet can screw up. In this example, GPT-4o cannot tell that GitHub is spelled correctly: https://app.gitsense.com/?doc=6c9bada92&model=GPT-4o&samples=5 https://app.gitsense.com/?doc=6c9bada92&model=GPT-4o&samples... In this example, Claude cannot tell that GitHub is spelled correctly: https://app.gitsense.com/?doc=905f4a9af74c25f&model=Claude+3.5+Sonnet&samples=5 https://app.gitsense.com/?doc=905f4a9af74c25f&model=Claude+3... I still believe LLM is a game changer and I'm currently working on what I call a "Yes/No" tool which I believe will make trusting LLMs a lot easier (for certain things of course). The basic idea is the "Yes/No" tool will let you combine models, samples and prompts to come to a Yes or No answer. Based on what I've seen so far, a model can easily screw up, but it is unlikely that all will screw up at the same time.
    - visarga 2y ago
      It's actually a great topic - both humans and LLMs are black boxes. And both rely on patterns and abstractions that are leaky. And in the end it's a matter of trust, like going to the doctor. But we have had extensive experience with humans, it is normal to have better defined trust, LLMs will be better understood as well. There is no central understander or truth, that is the interesting part, it's a "Blind men and the elephant" situation.
      - sdesol 2y ago
        We are entering the nondeterministic programming era in my opinion. LLM applications will be designed with the idea that we can't be 100% sure and what ever solution can provide the most safe guards, will probably be the winner.
  - deleted 2y ago
    [deleted]
- brookst 2y ago
  Sure. I mean, humans are very good at building businesses and technologies that are resilient to human fallibility. So when we think of applications where LLMs might replace or augment humans, it’s unsurprising that their fallible nature isn’t a showstopper. Sure, EDA tools are deterministic, but the humans who apply them are not. Introducing LLMs to these processes is not some radical and scary departure, it’s an iterative evolution.
  - beepbooptheory 2y ago
    Ok yeah. I think the thing that trips me up with this argument then is just, yes, when you regard humans in a certain neuroscientific frame and consider things like consciousness or language or will, they are fundamentally nondeterministic. But that isn't the frame of mind of the human engineer who does the work or even validates it. When the engineer is working, they aren't seeing themselves as some black box which they must feed input and get output, they are thinking about the things in themselves, justifying to themselves and others their work. Just because you can place yourself in some hypothetical third person here, one that oversees the model and the human and says "huh yeah they are pretty much the same, huh?", doesn't actually tell us anything about whats happening on the ground in either case, if you will. At the very least, this same logic would imply fallibility is one dimensional and always statistical; "the patient may be dead, but at least they got a new heart." Like isn't in important to be in love, not just be married? To borrow some Kant, shouldn't we still value what we can do when we think as if we aren't just some organic black box machines? Is there even a question there? How could it be otherwise? Its really just that the "in principle" part of the overall implication with your comment and so many others just doesn't make sense. Its very much cutting off your nose to spite your face. How could science itself be possible, much less engineering, if this is how we decided things? If we regarded ourselves always from the outside? How could even be motivated to debate whether we get the computers to design their own chips? When would something actually happen? At some point, people do have ideas, in a full, if false, transparency to themselves, that they can write down and share and explain. This is not only the thing that has gotten us this far, it is the very essence of why these models are so impressive in the certain ways that they are. It doesn't make sense to argue for the fundamental cheapness of the very thing you are ultimately trying to defend. And it imposes this strange perspective where we are not even living inside our own (phenomenal) minds anymore, that it fundamentally never matters what we think, no matter our justification. Its weird! I'm sure you have a lot of good points and stuff, I just am simply pointing out that this particular argument is maybe not the strongest.
- famouswaffles 2y ago
  Because people are not saying "let's replace Casio Calculators with interfaces to GPT!" By and large, the processes people are scrambling to place LLMs in are ones that typical machines struggle or fail and humans excel or do decently (and that LLMs are making some headway in). There's no point comparing LLM performance to some hypothetical perfect understanding machine that doesn't exist. It's nonsensical actually. You compare it to the performance of the beings it's meant to replace or augment - humans. Replacing non-deterministic black boxes with potentially better performing non-deterministic black boxes is not some crazy idea.
klabb3 2y ago
Yes I do! Is that some sort of gotcha? If I can choose between having a script that queries the db and generates a report and “Dave in marketing” who “has done it for years”, I’m going to pick the script. Who wouldn’t? Until machines can reliably understand, operate and self-correct independently, I’d rather not give up debuggability and understandability.
- famouswaffles 2y ago
  >If I can choose between having a script that queries the db and generates a report and “Dave in marketing” who “has done it for years” If you could that would be nice wouldn't it? And if you couldn't? If people were saying, "let's replace Casio Calculators with interfaces to GPT" then that would be crazy and I would wholly agree with you but by and large, the processes people are scrambling to place LLMs in are ones that typical machines struggle or fail and humans excel or do decently (and that LLMs are making some headway in). You're making the wrong distinction here. It's not Dave vs your nifty script. It's Dave or nothing at all. There's no point comparing LLM performance to some hypothetical perfect understanding machine that doesn't exist. You compare to the things its meant to replace - humans. How well can the LLM do this compared to Dave ?
  - kuhewa 2y ago
    > by and large, the processes people are scrambling to place LLMs in are ones that typical machines struggle or fail I'm pretty sure they are scrambling to put them absolutely anywhere it might save or make a buck (or convince an investor that it could)
    - blincoln 2y ago
      100%, and a lot of them are truly terrible use cases for LLMs. For example, using a LLM to transform structured data into JSON, and doing it with two LLMs in parallel to try to catch the inevitable failures, instead of just writing code that outputs JSON.
      - 2y ago
talldayo 2y ago
In a reductive sense, this passage might as well read "You find it hard to believe that entropy is the source of other entropic reactions?" No, I'm just disappointed in the decision of Black Box A and am bound to be even more disappointed by Black Box B. If we continue removing thoughtful design from our systems because thoughtlessness is the default, nobody's life will improve.
ithkuil 2y ago
I'm a non-deterministic black box who teaches complex deterministic machines to do stuff and leverages other deterministic machines as tools to do the job. I like my job. My job also involves cooperating with other non-deterministic black boxes (colleagues). I can totally see how artificial non-deterministic black boxes (artificial colleagues) may be useful to replace/augment the biological ones. For one, artificial colleagues don't get tired and I don't accidentally hurt their feelings or whatnot. In any case, I'm not looking forward to replacing my deterministic tools with the fuzzy AI stuff. Intuitively at least it seems to me that these non-deterministic black boxes could really benefit from using the deterministic tools for pretty much the same reasons we do as well.
xg15 2y ago
Yes. One does not have to do with the other.
snowwrestler 2y ago
One great thing about humans is that we have developed ways to be deterministic when we want to. That’s what math is for. Does an LLM know math? Not like we do. There’s no deductive logic in there; it’s all statistical inferences from language. An LLM doesn’t “work through” a circuit diagram systematically the way a physics student would. It observes the entire diagram at once, and then guesses the most likely next token.
staticman2 2y ago
>You find it hard to believe that non-deterministic black boxes at the core of complex systems are eager to put non-deterministic black boxes at the core of complex systems? Hello, fellow tech enthusiasts, just stopping by to announce I performatively can't tell the difference between "Latest big tech product (TM)" and Homo Sapiens Sapiens!!! I'll be seeing you in the next LLM related message thread with the same exact comment!!! As you were!!!

You find it hard to believe that non-deterministic black boxes at the core of complex systems are eager to put non-deterministic black boxes at the core of complex systems?

beepbooptheory 2y ago

Can you actually like follow through with this line? I know there are literally tens of thousands of comments just like this at this point, but if you have chance, could you explain what you think this means? What should we take from it? Just unpack it a little bit for us.

dangsux 2y ago
[dead]
therealcamino 2y ago
I took it to be a joke that the description "slow, expensive, non-deterministic black boxes" can apply to the engineers themselves. The engineers would be the ones who would have to place LLMs at the core of the system. To anyone outside, the work of the engineers is as opaque as the operation of LLMs.
croshan 2y ago
An interpretation that makes sense to me: humans are non-deterministic black boxes already at the core of complex systems. So in that sense, replacing a human with AI is not unreasonable. I’d disagree, though: humans are still easier to predict and understand (and trust) than AI, typically.
- sdesol 2y ago
  With humans we have a decent understanding of what they are capable of. I trust a medical professional to provide me with medical advice and an engineer to provide me with engineering advice. With LLM, it can be unpredictable at times, and they can make errors in ways that you would not imagine. Take the following examples from my tool, which shows how GPT-4o and Claude 3.5 Sonnet can screw up. In this example, GPT-4o cannot tell that GitHub is spelled correctly: https://app.gitsense.com/?doc=6c9bada92&model=GPT-4o&samples=5 https://app.gitsense.com/?doc=6c9bada92&model=GPT-4o&samples... In this example, Claude cannot tell that GitHub is spelled correctly: https://app.gitsense.com/?doc=905f4a9af74c25f&model=Claude+3.5+Sonnet&samples=5 https://app.gitsense.com/?doc=905f4a9af74c25f&model=Claude+3... I still believe LLM is a game changer and I'm currently working on what I call a "Yes/No" tool which I believe will make trusting LLMs a lot easier (for certain things of course). The basic idea is the "Yes/No" tool will let you combine models, samples and prompts to come to a Yes or No answer. Based on what I've seen so far, a model can easily screw up, but it is unlikely that all will screw up at the same time.
  - visarga 2y ago
    It's actually a great topic - both humans and LLMs are black boxes. And both rely on patterns and abstractions that are leaky. And in the end it's a matter of trust, like going to the doctor. But we have had extensive experience with humans, it is normal to have better defined trust, LLMs will be better understood as well. There is no central understander or truth, that is the interesting part, it's a "Blind men and the elephant" situation.
    - sdesol 2y ago
      We are entering the nondeterministic programming era in my opinion. LLM applications will be designed with the idea that we can't be 100% sure and what ever solution can provide the most safe guards, will probably be the winner.
- deleted 2y ago
  [deleted]
brookst 2y ago
Sure. I mean, humans are very good at building businesses and technologies that are resilient to human fallibility. So when we think of applications where LLMs might replace or augment humans, it’s unsurprising that their fallible nature isn’t a showstopper. Sure, EDA tools are deterministic, but the humans who apply them are not. Introducing LLMs to these processes is not some radical and scary departure, it’s an iterative evolution.
- beepbooptheory 2y ago
  Ok yeah. I think the thing that trips me up with this argument then is just, yes, when you regard humans in a certain neuroscientific frame and consider things like consciousness or language or will, they are fundamentally nondeterministic. But that isn't the frame of mind of the human engineer who does the work or even validates it. When the engineer is working, they aren't seeing themselves as some black box which they must feed input and get output, they are thinking about the things in themselves, justifying to themselves and others their work. Just because you can place yourself in some hypothetical third person here, one that oversees the model and the human and says "huh yeah they are pretty much the same, huh?", doesn't actually tell us anything about whats happening on the ground in either case, if you will. At the very least, this same logic would imply fallibility is one dimensional and always statistical; "the patient may be dead, but at least they got a new heart." Like isn't in important to be in love, not just be married? To borrow some Kant, shouldn't we still value what we can do when we think as if we aren't just some organic black box machines? Is there even a question there? How could it be otherwise? Its really just that the "in principle" part of the overall implication with your comment and so many others just doesn't make sense. Its very much cutting off your nose to spite your face. How could science itself be possible, much less engineering, if this is how we decided things? If we regarded ourselves always from the outside? How could even be motivated to debate whether we get the computers to design their own chips? When would something actually happen? At some point, people do have ideas, in a full, if false, transparency to themselves, that they can write down and share and explain. This is not only the thing that has gotten us this far, it is the very essence of why these models are so impressive in the certain ways that they are. It doesn't make sense to argue for the fundamental cheapness of the very thing you are ultimately trying to defend. And it imposes this strange perspective where we are not even living inside our own (phenomenal) minds anymore, that it fundamentally never matters what we think, no matter our justification. Its weird! I'm sure you have a lot of good points and stuff, I just am simply pointing out that this particular argument is maybe not the strongest.
famouswaffles 2y ago
Because people are not saying "let's replace Casio Calculators with interfaces to GPT!" By and large, the processes people are scrambling to place LLMs in are ones that typical machines struggle or fail and humans excel or do decently (and that LLMs are making some headway in). There's no point comparing LLM performance to some hypothetical perfect understanding machine that doesn't exist. It's nonsensical actually. You compare it to the performance of the beings it's meant to replace or augment - humans. Replacing non-deterministic black boxes with potentially better performing non-deterministic black boxes is not some crazy idea.

klabb3 2y ago

Yes I do! Is that some sort of gotcha? If I can choose between having a script that queries the db and generates a report and “Dave in marketing” who “has done it for years”, I’m going to pick the script. Who wouldn’t? Until machines can reliably understand, operate and self-correct independently, I’d rather not give up debuggability and understandability.

famouswaffles 2y ago
>If I can choose between having a script that queries the db and generates a report and “Dave in marketing” who “has done it for years” If you could that would be nice wouldn't it? And if you couldn't? If people were saying, "let's replace Casio Calculators with interfaces to GPT" then that would be crazy and I would wholly agree with you but by and large, the processes people are scrambling to place LLMs in are ones that typical machines struggle or fail and humans excel or do decently (and that LLMs are making some headway in). You're making the wrong distinction here. It's not Dave vs your nifty script. It's Dave or nothing at all. There's no point comparing LLM performance to some hypothetical perfect understanding machine that doesn't exist. You compare to the things its meant to replace - humans. How well can the LLM do this compared to Dave ?
- kuhewa 2y ago
  > by and large, the processes people are scrambling to place LLMs in are ones that typical machines struggle or fail I'm pretty sure they are scrambling to put them absolutely anywhere it might save or make a buck (or convince an investor that it could)
  - blincoln 2y ago
    100%, and a lot of them are truly terrible use cases for LLMs. For example, using a LLM to transform structured data into JSON, and doing it with two LLMs in parallel to try to catch the inevitable failures, instead of just writing code that outputs JSON.
    - 2y ago

talldayo 2y ago

In a reductive sense, this passage might as well read "You find it hard to believe that entropy is the source of other entropic reactions?" No, I'm just disappointed in the decision of Black Box A and am bound to be even more disappointed by Black Box B. If we continue removing thoughtful design from our systems because thoughtlessness is the default, nobody's life will improve.

ithkuil 2y ago

I'm a non-deterministic black box who teaches complex deterministic machines to do stuff and leverages other deterministic machines as tools to do the job. I like my job. My job also involves cooperating with other non-deterministic black boxes (colleagues). I can totally see how artificial non-deterministic black boxes (artificial colleagues) may be useful to replace/augment the biological ones. For one, artificial colleagues don't get tired and I don't accidentally hurt their feelings or whatnot. In any case, I'm not looking forward to replacing my deterministic tools with the fuzzy AI stuff. Intuitively at least it seems to me that these non-deterministic black boxes could really benefit from using the deterministic tools for pretty much the same reasons we do as well.

xg15 2y ago

Yes. One does not have to do with the other.

snowwrestler 2y ago

One great thing about humans is that we have developed ways to be deterministic when we want to. That’s what math is for. Does an LLM know math? Not like we do. There’s no deductive logic in there; it’s all statistical inferences from language. An LLM doesn’t “work through” a circuit diagram systematically the way a physics student would. It observes the entire diagram at once, and then guesses the most likely next token.

staticman2 2y ago

>You find it hard to believe that non-deterministic black boxes at the core of complex systems are eager to put non-deterministic black boxes at the core of complex systems? Hello, fellow tech enthusiasts, just stopping by to announce I performatively can't tell the difference between "Latest big tech product (TM)" and Homo Sapiens Sapiens!!! I'll be seeing you in the next LLM related message thread with the same exact comment!!! As you were!!!