red75prime

6 ms·

91.
by red75prime 2mo ago
> We didn’t replicate the human brain. We built systems that can statistically approximate some of what the human brain might output in certain limited situations. Which is equivalent to "We didn't replicate the human brain. We
92.
by red75prime 2mo ago
> “should I add the frog’s eye or leg into the cauldron” This is surely a homebrew witchery. An engineering approach would be to A/B-test batches of potions with eyes and legs, add quality control by testing potions on model organis
93.
by red75prime 2mo ago
Its state is fully defined by its context window. So, yes, if the context is reset, then its state is reset too.
94.
by red75prime 2mo ago
> It might be boring but it is the truth. The inability of people to reflect on their own beliefs amazes me to no end. How can you know that LLMs have no phenomenal states? For example, I believe that rocks (ordinary rocks, not silicon c
95.
by red75prime 2mo ago
> You absolutely, undoubtedly know that. First things first. You are wrong. I don't have a gut feeling that LLMs have no phenomenal states. Not anymore. I was skeptical that LLMs can form complex internal representations. Mechanis
96.
by red75prime 2mo ago
I wasn't going solely for "we don't know and we can't know." There are other relevant observations. A matmul (or a fixed finite sequence of matmuls) is not Turing-complete. An LLM that produces answer in the number
97.
by red75prime 2mo ago
> I know it doesn't experience joy or surprise or empathy. You don't know. You strongly feel that you know. There's no standard procedure to assess the existence of phenomenal consciousness.
98.
by red75prime 2mo ago
Philosophers are still considering it possible to infer things about aesthetic taste a priory ? Good luck, I guess.
99.
by red75prime 2mo ago
Shouldn't this 236-year-old paper be updated in accordance with, say, neuroeconomics research on how provenance influences aesthetic taste?
100.
by red75prime 2mo ago
I think it's obvious that repeatedly noting that the absolute perfection isn't achieved yet is an absolutely useless way of tracking progress. No? BTW, when was the last time Tesla driver died due to FSD fault?
101.
by red75prime 2mo ago
> the LLMs were trained to mimic how people speak (write) It's a foundational model (fresh from autoregressive pretraining) that approximates the probability distribution of human texts. And, no, it's not the statistical averag
102.
by red75prime 2mo ago
Then robots begin cleaning dishes too. And the other members aren't happy, because what's the point, we are here not for the dishes, but for the process.
103.
by red75prime 2mo ago
I believe that the decentralization of manipulation (taken in a very broad sense as an effort to modify people's views) spells trouble for democracies. The mass media of old, with all its flaws, created a shared pool of information man
104.
by red75prime 2mo ago
I'm trying to understand what's going on in the head of people who are certain that LLMs can't be intelligent. And it proves to be a daunting task. Have you tried to reflect on your certainty? Where it comes from?
105.
by red75prime 2mo ago
Why do you think that intelligence of a Turing-complete system trained on human data can be likened to a Russell's teapot? At least, don't you think that the recent mathematical results of LLMs are a bit like a glimpse of somethin
106.
by red75prime 2mo ago
It's a negative in your worldview where "intelligence can't be a computation" is an axiom or something close. To me this worldview is sloppy thinking.
107.
by red75prime 2mo ago
There is no reason to think that teapots were sent into orbit or spontaneously formed there. Likewise, there is no reason to think the brain employs super-Turing or quantum computations that cannot be approximated by LLMs.
108.
by red75prime 2mo ago
OK. A more pointed question. What do you know about intelligence that allows you to exclude LLMs with CoT from the category of intelligent systems with certainty?
109.
by red75prime 2mo ago
Apophatic intelligence? "We don't know what intelligence is, but LLMs with CoT are certainly not it despite being Turing-complete." Watching for unexpected failure modes is surely worth it.
110.
by red75prime 2mo ago
> AI is not deterministic, it will be making tons of mistakes. Just set the sampling temperature to zero and remove any unintended non-determinism during the parallel computation of the token probability distribution. The problem is solv
111.
by red75prime 3mo ago
Someone says "cars are so comfortable now." Someone else responds "In my experience they are no better than in the 80s." Isn't it prudent to point that the second person drives Mirage or something like that? The fir
112.
by red75prime 3mo ago
The AI you are talking about is a low-inference-cost model optimized for massive throughput. It's not like every model has the same amount of those shortcomings.
113.
by red75prime 3mo ago
Yeah, I should have said RL, not RLVR. The point is RL interacts with external world, while autoregressive pretraining is limited to the passive ingestion, and RLHF relies on a model of human preferences that has no access to truth sources
114.
by red75prime 3mo ago
Here's another technical point: > They're word-association mechanisms with no embodiment and no way to associate the text vectors they manipulate with real-world phenomena. This is wrong too. RLVR grounds foundational models in
115.
by red75prime 3mo ago
"Repeating what they've seen" could be, depending on their generalization abilities, something like low-n-gram Markov chain, repeating of surface-level speech patterns, repeating reasoning patterns, repeating mind-theory-leve
116.
by red75prime 3mo ago
Capability or outcome...or internal structure. What does "glorified autocomplete" say in an ontological sense exactly? Nah. It's just a lazy dismissal. BTW, autoregressive pretraining (autocomplete) is a part of training.
117.
by red75prime 3mo ago
They probably can't order tofu. It would be B12-fortified oatmeal or something like this.
118.
by red75prime 3mo ago
> where and when the LLM use is justified and where it is not while being bombarded with articles like "AI makes things worse", "AI consumes all the water" and the like
119.
by red75prime 3mo ago
> So what if your image classifier thinks a cat is an impala? For starters, the artificial neural network is the only instrument we have for sufficiently robust image classification. If it doesn't work nothing else will. We can only
120.
by red75prime 3mo ago
> https://youtu.be/C4NQNeSO2vs?si=epkxhVXpypOCppGW "VLAs fine-tuned on human demonstrations overfit. Here's how to mitigate it." Er, OK, I guess. But I think that Waymo uses mitigations or a different appr

More ›