5 ms·
This is proposed as a way to measure "true" reasoning by asking a certain type of trick questions, but I don't quite see how this could be a basis of a sustaina
by doe_eyes 2y ago
This is proposed as a way to measure "true" reasoning by asking a certain type of trick questions, but I don't quite see how this could be a basis of a sustainable benchmark.
If this gets attention, the next generation of LLMs will be trained on this paper, and then fine-tuned by using this exact form of questions to appear strong on this benchmark, and... we're back to square one.
- jcoc611 2y agoFrom an external perspective, is there a way to distinguish between simulation of consciousness and the real thing? If the answer is no, could you make an argument that they are the same?
- chx 2y agoThrowing all the paintings made prior 1937 into an LLM would never get Guernica out of it. As long as it's an LLM this stands, not just today but all the way to the future. This empty sophistry of presuming automated bullshit generators somehow can mimic a human brain is laughable. Please please read https://aeon.co/essays/your-brain-does-not-process-information-and-it-is-not-a-computer https://aeon.co/essays/your-brain-does-not-process-informati...
- deleted 2y ago[deleted]
- altruios 2y agoThe author fails to provide any argument other than one of incredulity and some bad reasoning with bad faith examples. The dollar bill copying example is a faulty metaphor. His claim of humans not being information processors and he tries to demonstrate this by having a human process information (drawing from reference is processing an image and giving an output)... His argument sounds like one from 'it's always sunny'. As if metaphors never improve or get more accurate over time, and that this latest metaphor isn't the most accurate metaphor we have. It is. When we have something better: we'll all start talking about the brain in that frame of reference. This is an idiot that can write in a way that masks some deep bigotries (in favor of the mythical 'human spirit'). I do not take this person seriously. I'm glossing over all casual incorrectness of his statements - a good number of them just aren't true. the ones I just scrolled to statements like... 'the brain keeps functioning or we disappear' or 'This might sound complicated, but it is actually incredibly simple, and completely free of computations, representations and algorithms' in the description of the 'linear optical trajectory' ALGORTHIM (a set of simple steps to follow - in this case - visual pattern matching). Where is the sense in what I just read?
- chx 2y agoAI bros never take anything seriously aside from the bullshit they are drunk on https://disconnect.blog/what-comes-after-the-ai-crash/ https://disconnect.blog/what-comes-after-the-ai-crash/ this is your future
- __loam 2y agoWhy are you bringing up metaphysics when the concern is that the student has seen the exam, so to speak.
- aflukasz 2y agoI like this observation. And it fascinates me each time I see some self proclaimed conscious entity arguing that this just simply cannot be.
- cscurmudgeon 2y agoBecause you don’t see X is not a proof that X doesn’t exist. Here X may or not exist. X = difference between simulated and real consciousness Black holes were posited before they were detected empirically. We don't declare them to be non-existent when their theory came out just because we couldn't detect them.
- altruios 2y ago> self proclaimed conscious entity Well, I do not proclaim consciousness: only the subjective feeling of consciousness. I really 'feel' conscious: but I can't prove or 'know' that in fact I am 'conscious' and making choices... to be conscious is to 'make choices'... Instead of just obeying the rules of chemistry and physics... which YOU HAVE TO BREAK in order to be conscious at all (how can you make a choice at all if you are fully obeying the rules of chemistry {which have no choice}). A choice does not apply to chemistry or physics: from where does choice come from - I suspect from our fantasies and nothing from objective reality (for I do not see humans consistently breaking the way chemistry works in their brains) - it probably comes from nowhere. If you can explain the lack of choice available in chemistry first (and how that doesn't interfere with us being able to make a choice): then I'll entertain the idea that we are conscious creatures. But if choice doesn't exist at the chemical level, it can't magically emerge from following deterministic rules. And chemistry is deterministic not probabilistic (h2 + o doesn't magically make neon ever, or 2 water molecules instead of one).
- jrflowers 2y agoYou could make the argument that two things that we don’t understand are the same thing because we’re equally ignorant of both in the same way that you could make the argument that Jimmy Hoffa and Genghis Khan are probably buried in the same place, since we have equal knowledge of their locations.
- gorjusborg 2y agoLike the original Mechanical Turk. Clearly there is a difference between a small person hidden within playing chess and a fully mechanical chess automaton, but as the observer we might not be able to tell the difference. The observer's perception of the facts doesn't change the actual facts, and the implications of those facts.
- mannykannot 2y agoThe Mechanical Turk, however, was not a simulation of human consciousness, reasoning, chess-playing or any other human ability: it was the real thing, somewhat artfully dressed-up as to appear otherwise. Is it meaningful to say that Alphago Zero does not play Go, it just simulates something that does?
- mensetmanusman 2y agoThere might not be an external perspective, just someone else’s internal perspective of the external.
- layer8 2y agoConsciousness and reasoning are orthogonal to each other.
- deleted 2y ago[deleted]
- ben_w 2y agoI suspect that depends on which of the 200 definitions of "consciousness" you're using. And some other broad range of definitions of "reasoning".
- andrewla 2y agoThere's an interesting paper [1] that discusses this very possibility. [1] https://academic.oup.com/mind/article/LIX/236/433/986238?login=false https://academic.oup.com/mind/article/LIX/236/433/986238?log...
- imtringued 2y agoYou can shuffle the choices in multiple choice based benchmarks and models that memorize the benchmark tend to do really badly, almost worse than random guessing.
- slashdave 2y agoEasily overcome by training models on randomized questions. Trivial to implement too.
- tsukikage 2y agoIf the model works correctly on way more questions than there is room to store a giant list of recorded answers for, some kind of deduction of generalised rules must have taken place. "there is room to represent recorded answers for" is doing a lot of work, of course; it might e.g. have invented compression mechanisms better than known ones instead.
- slashdave 2y agoYou cannot create generalized questions using randomization alone
- Jensson 2y agoIt does system 1 thinking, it doesn't do system 2 thinking. That makes it really dumb, but can still answer a very wide range of questions it hasn't seen exact matches of since system 1 thinking can pattern match in complex ways. > it might e.g. have invented compression mechanisms better than known ones instead. You mean humans? Humans invented the transformer architecture, that is what compresses the human text to this form where semantics of text gets encoded instead of the raw words.
- altruios 2y agoMaybe: there is no measurable difference between 'real' reasoning, and 'fake' reasoning. And if there is no measurable difference... we can't measure 'realness', we just have to measure something different (and more useful) 'soundness'. Regardless of it is reasoning or not internally, if it produces a sound and logical argument: who cares? I agree: I don't think any measure tested linguistically can prove it is internally reasoning... in the same way we haven't truly proven other sentient people aren't in fact zombies (we just politely assume the most likely case that they aren't).
- doe_eyes 2y ago> Maybe: there is no measurable difference between 'real' reasoning, and 'fake' reasoning. The point is, it's easier to teach an LLM to fake it than to make it - for example, they get good at answering questions that overlap with their training data set long before they start generalizing. So on some epistemological level, your point is worth pondering; but more simply, it actually matters if an LLM has learned to game a benchmark vs approximate human cognition. If it's the former, it might fail in weird ways when we least expect it.
- visarga 2y agoIt's like students learning for the test, not really understanding. Or like regular people who don't always understand, just follow the memorized steps. How many times do we really really understand and how many do we just imitate? I have a suspicion that humans often use abstractions or methods they don't understand. We frequently rely on heuristics, mental shortcuts, and received wisdom without grasping the underlying principles. To understand has many meanings: to predict, control, use, explain, discover, model and generalize. Some also add "to feel". In one extreme we could say only a PhD in their area of expertise really understands, the rest of us just fumble concepts. I am sure rigorous causal reasoning is only possible by extended education, it is not the natural mode of operation of the brain.
- Jensson 2y ago> I am sure rigorous causal reasoning is only possible by extended education, it is not the natural mode of operation of the brain. I'd say the other way around, education teaches you to not reason and instead just follow the patterns you learned in the book. Most people do reason a ton before they go to school, but then school beats that out of them.
- mannykannot 2y agoI don't think we can just assume that training with this exact form of questioning will lead to a strong performance on such questions. For one thing, given the LLM propensity for hallucinating, I do not think we can be confident that an LLM, after this training, will reliably employ the correct model to answer a given question.