2 ms·

The part that does count is that we can verify that we does what we want it to do. We just don't know how. Same with the black box part of AI. What the weights

The part that does count is that we can verify that we does what we want it to do. We just don't know how. Same with the black box part of AI. What the weights represent? Arcane dark magic if you ask me. What do they do? Well with LLMs we're all experiencing it.

amelius 5d ago
> The part that does count is that we can verify that we does what we want it to do. We just don't know how. How do we know the person getting anesthesia doesn't die and a new soul/consciousness replaces them? (We can ask the same question about going to sleep, er even walking through a door, but it's still something to think about).
- no_input 5d ago
  This was my existential horror for most of my life as I almost never remembered dreaming or going to sleep - until I got a CPAP. Once I started using it, there was continuity to my life, I vividly dream and remember those dreams now. I never scored too badly on the apnea tests but just enough to qualify for one so I said what the hell and got it. Even though I don't use it regularly, something about my sleep has changed for the better.
CrimsonRain 5d ago
you can verify it does what you want it to? How? Everything is a probability and so is your verification.
- zeta0134 5d ago
  You're correct that verification *reduces* risk. It does not eliminate risk. Humans are imperfect. That does not mean that we should stop seeking understanding on principle though, because the journey towards that understanding improves outcomes. Recall also that LLMs are not actually entities. In their current form, there is no sentience, there is no agency. They are tools. Therefore, their output must benefit the user that requested it. Right now, that's humans (and, ideally, the planet at large; we don't exist in a vacuum) and so it makes sense that humans should verify that output and try to ensure that it aligns with their goals. That's not to say that the output of an LLM is useless, far from it. But we should still *try* to understand its output. It gives us at least some chance to notice flaws, and an even greater chance to appreciate the implications and tradeoffs of the solution it picked.
  - CrimsonRain 5d ago
    I don't disagree with anything you said. But the thread is about... LLM outputs, even if it cures cancer and solves world hunger, it is useless (it does not count for anything) if we don't understand it.
    - mettamage 5d ago
      Interesting, I really don’t see your point. It would definitely be a lot more useful if we also understand it. But extra lives saved definitely counts for a whole lot and is definitely useful. One comparison I see: there are so many religious people that simply wish they are saved from their cancer (or whatever else). They pray for it. They don’t care how it happens. They simply want to live longer, etc. It’s a sketch of an argument, I hope you know what I am getting at.

amelius 5d ago

> The part that does count is that we can verify that we does what we want it to do. We just don't know how. How do we know the person getting anesthesia doesn't die and a new soul/consciousness replaces them? (We can ask the same question about going to sleep, er even walking through a door, but it's still something to think about).

no_input 5d ago
This was my existential horror for most of my life as I almost never remembered dreaming or going to sleep - until I got a CPAP. Once I started using it, there was continuity to my life, I vividly dream and remember those dreams now. I never scored too badly on the apnea tests but just enough to qualify for one so I said what the hell and got it. Even though I don't use it regularly, something about my sleep has changed for the better.

CrimsonRain 5d ago

you can verify it does what you want it to? How? Everything is a probability and so is your verification.

zeta0134 5d ago
You're correct that verification *reduces* risk. It does not eliminate risk. Humans are imperfect. That does not mean that we should stop seeking understanding on principle though, because the journey towards that understanding improves outcomes. Recall also that LLMs are not actually entities. In their current form, there is no sentience, there is no agency. They are tools. Therefore, their output must benefit the user that requested it. Right now, that's humans (and, ideally, the planet at large; we don't exist in a vacuum) and so it makes sense that humans should verify that output and try to ensure that it aligns with their goals. That's not to say that the output of an LLM is useless, far from it. But we should still *try* to understand its output. It gives us at least some chance to notice flaws, and an even greater chance to appreciate the implications and tradeoffs of the solution it picked.
- CrimsonRain 5d ago
  I don't disagree with anything you said. But the thread is about... LLM outputs, even if it cures cancer and solves world hunger, it is useless (it does not count for anything) if we don't understand it.
  - mettamage 5d ago
    Interesting, I really don’t see your point. It would definitely be a lot more useful if we also understand it. But extra lives saved definitely counts for a whole lot and is definitely useful. One comparison I see: there are so many religious people that simply wish they are saved from their cancer (or whatever else). They pray for it. They don’t care how it happens. They simply want to live longer, etc. It’s a sketch of an argument, I hope you know what I am getting at.