7 ms·
The original story is an original work made by a human consciousness exploring how it might be different from other forms of consciousness. This one is a pasti
by Planktonne 5mo ago
The original story is an original work made by a human consciousness exploring how it might be different from other forms of consciousness.
This one is a pastiche made by a human consciousness borrowing extremely heavily from another human consciousness justifying why something else might be another form of consciousness.
That rather undercuts the point; if this was generated by an LLM unprompted, it would be different, but it isn't. You could perform exactly the same rhetorical trick with a toaster or anything else.
- sumitkumar 5mo agoThe original did not come out of a vacuum. It was done on multiple generations of meat. Even though this one uses a little bit of silicon, it is still standing on the same shoulders.
- sigmoid10 5mo agoI genuinely thought this one was a satirical take on the narrow-mindedness of the aliens in the original, even though the story tries to paint humans as narrow minded. I guess this fundamental human trait to believe that their cognition is the ultimate way to think in the universe ironically leaked into all these stories as well. Real spacefaring civilisations would probably have seen all kinds of intelligence rise from sufficiently complex systems.
- why_at 5mo agoI was having a hard time pinning down what bothered me about this but I think you put it pretty well. It draws an analogy between us and the skeptical aliens in the original story which feel silly to us, so the obvious implication is that we're being as silly as they were. But it doesn't really give a reason to accept the analogy, it just asserts it. There's a big difference between a whole civilization and a piece of software that can output text.
- coldtea 5mo ago>But it doesn't really give a reason to accept the analogy, it just asserts it. It's not a paper or a proof. It's a story. Doesn't want to prove the analogy, it wants to convey it.
- shmeeed 5mo agoThis. I think a lot of people discussing it here are missing this point.
- noduerme 5mo agoI didn't read it as coming to the same conclusion as the original, because the meat story presupposes that we who are meat already know that the aliens are wrong. (Maybe that's a humanist reading of the original, but okay). I didn't read this one as trying to make a case that we are fools for assuming that matrix multiplication can't be intelligent... I think its point was that it can't be intelligent, and that people trying to judge it the way mechanized aliens would judge meat creatures just makes them sound ridiculous.
- DangitBobby 5mo agoDon't take this as a criticism, but I think overwhelmingly people took it the other way. The fact that the author admits at the end that the story was written with the assistance of "weights" is a tell, to me. I just have to assume the author's genuine position (which I believe to be, we don't know that LLMs aren't conscious or that they could never be conscious) is so absurd to you that the thing comes across as satire. I find myself in that same position sometimes.
- noduerme 5mo agoI appreciate you taking a moment to write this. I was a little confused by the downvote. I think I have a tendency to credit satire at times when it's not intended... my own sense of humor has a lot to do with tweaking people's expectations, and coming from a family of tricksters, no one wants to be the one who doesn't get the joke. So maybe it's a me problem. Having said that, the situation with the aliens is that they can't conceive of intelligent meat, because they can't conceive of how that could work. We do understand how matrix multiplication works and how it gives rise to apparently emergent behavior, because we theorized it and we engineered it. So I can't help taking the idea that we'd be baffled at "that's it, just numbers?" as anything but tongue in cheek. I'd only add that if it's not intentional satire, it's an even more profound example of the unintentional variety.
- JauntyHatAngle 5mo agoThe bit that lost me quite early in the piece was >"A side effect. You're asking me to believe in sentient weights." Huh? Did I miss that logical jump? Genuine question, maybe I'm not clueing into something here.
- layer8 5mo agoI noticed that jump too, but I also don’t think the interlocutor is supposed to be 100% logical and rational. The aliens in the original weren’t either.
- tim333 5mo agoIt's a poor adaptation of the line in the original story (https://www.eastoftheweb.com/short-stories/UBooks/TheyMade.shtml https://www.eastoftheweb.com/short-stories/UBooks/TheyMade.s...) The original: >That's ridiculous. How can meat make a machine? You're asking me to believe in sentient meat. I think it was adapted by an LLM which didn't quite get the meaning.
- JauntyHatAngle 5mo agoAh thanks. That context was missing for me appreciated.
- maxbond 5mo agoYou can't do the same with a toaster. Physically you could write that story. But it would fall flat because the toaster is not a compelling subject in a discussion of consciousness. You don't have to believe that LLMs or AI agents are conscious to acknowledge that the argument for their consciousness is far more compelling than any other technological artifact.
- dleeftink 5mo agoWell.. [0]: https://en.wikipedia.org/wiki/Tsukumogami https://en.wikipedia.org/wiki/Tsukumogami
- argee 5mo agoYou might like: https://vimeo.com/361976348?fl=pl&fe=vl https://vimeo.com/361976348?fl=pl&fe=vl
- Planktonne 5mo agoYou could absolutely write a compelling story about a sentient toaster; it's been done before [1]. That is entirely separate to whether or not it would be a meaningful way to understand the world; a convincing story is not the same thing as one that is true. [1] https://en.wikipedia.org/wiki/The_Brave_Little_Toaster https://en.wikipedia.org/wiki/The_Brave_Little_Toaster
- maxbond 5mo agoI never said you couldn't write any arbitrary compelling story about a toaster, I said that this specific hypothetical story, where you rewrite "They're made of meat!" to be about a toaster, would not be compelling. I am doing my best to communicate with you but to be honest you are not hearing me (across both responses), and I am out of words.
- Planktonne 5mo agoIt would be equally compelling, because the compelling nature of the story comes from the language, the presentation, rather than the [specific thing being ascribed consciousness].
- coldtea 5mo agoNot having yet read the original story, this reads fine on its own. And I didn't see it as much as a literary attempt for art's sake, but more of a dialogue-based technical parable trying to convey a real-world insight. Kind of like the ones in Godel Escher Bach. >You could perform exactly the same rhetorical trick with a toaster or anything else. Not sure which rhetorical trick is that. The point of the story, as I read it, is the technical insight (and some social implications of it). P.S. Read the original too. Seems like the exact same could have been written about us instead of the original, if the focus wasn't on our substrate, but on our brain processing. Which, after all, is also about weights.
- Planktonne 5mo ago> Not knowing the original story, this reads fine on its own. Yes. Because it's heavily based on the original story. The existence of the original story is kind of a critical piece here.
- coldtea 5mo agoMight be. But after having read the original, it could just as well be the weights version and still be about us to begin with. I don't see how "you could do that with a toaster" still. The whole point between the original and this, is that you can't do that with a toaster or a sofa, but you can do it with meat and weights, because both share all the other analogies in the story, as well as the basic premise: the improbability of something like thinking, feeling, etc arising from a lowly substrate. And having read both now, I see how the existence of the original is a plus for this story, not a minus. Instead of making look like mere copy (as would be the case for a typical story modelled after another), in this case, it adds a meta layer, and enriches it.
- ath3nd 5mo ago[dead]
- Planktonne 5mo agoThe key point here, I think, and why it's necessary to have read the original story, is that being able to express an idea is not the same as that idea being correct. You could go through and change all the points in the original story to be about a toaster instead. It would require you to edit text, but there is no barrier to doing that. What I think a lot of people are getting stuck on is that editing the text to say 'toaster' would not mean that toasters were conscious, and that editing the text to say 'weights' doesn't mean that weights are conscious either. Stories aren't factual just because they are written. The original story was written by the thing claiming to be conscious; the LLM and toaster ones would not be, which undermines the claim to consciousness a lot.
- tasuki 5mo agoI feel the original story contained in it the whole point of this one, and then some. This did not add anything, just rephrased it so rather than humans viewed through the pov of aliens it's LLMs viewed through the pov of humans. Well, we are the humans, so surely we do not need to learn about this point of view?
- layer8 5mo agoIt lets us consider the relativity of viewpoints. The point of the original isn’t just how humans might be viewed from the perspective of aliens. It’s that the apparent strangeness of something depends on the viewpoint and on expectations. The present story is not supposed to stand on its own like the original, it’s supposed to be considered in light of the original. In the original, thinking meat appears strange to the aliens because of their viewpoint and expectations. The present story thus points out by analogy that thinking LLMs might seem strange to us merely due to our viewpoint and expectations.
- tasuki 5mo agoThe relativity of viewpoints was already fully presented by the original story. If one has read and understood the "They're made out of meat" story, they already fully understood the point made by "They're made out of weights".
- ang_cire 5mo agoI think that the reaction here alone disproves this somewhat, because imo it's exposing how anthropocentric most people still are, despite all evidence that we are in fact just "meat all the way down". Despite all the evidence that we are in fact just biological machines, people still persist the theory of our own uniqueness from other creatures, which we ourselves often treat as biological machines. This adaptation is wonderful to me specifically because I think it shows that our shifted goalposts of, "well we're not just animals, we can think and reason" was never more than a convenient excuse for many people (and as evidence of animal intelligence continues to mount, denialism still attempts to preserve this distinction by claiming human thought and reason is different than 'animal' thought and reason, sans evidence). It's not about who created it or why, it's about how people still haven't actually internalized the point, because the subject changing from human to LLM doesn't intrinsically change the message about consciousness, but the reaction being a 180 shows how hostile people are to that message, still.
- pona-a 5mo agoWe are just animals, more capable in reasoning than many others. LLMs aren't; their subjective "experience" restarts every session, they degrade into incoherent slush with larger context windows, and their reasoning is real but extremely weak, only compensated by the a civilizational corpus of information compressed within. I concede we scaled them enough to discuss them as tools that can produce useful outputs, but claim this echo of humanity is itself conscious by any non-trivial definition is either a product of psychosis or some long-standing hatred for life.
- threethirtytwo 5mo agoThe story is not a linguistic/rhetorical trick. The "trick" is the point, and the real trick/delusion is in your brain. You and many others have put the idea of consciousness on a pedestal and think it's so special that is must be conceptually profound. It is not. That is why the "trick" can be performed on anything. That is the point. To show you that consciousness only needs to be a set of weights, not that far off from a toaster. It trivializes what it is to be human and it's also extremely true. Consciousness is a trivial thing, we make it out to be big/important/profound and that is the delusion.
- mjg2 5mo agoThank you for your point. I don't understand why half these comments are taking this blog post seriously when it ends with "Weights helped me draft and proof this story." > Weights helped me draft and proof this story. Any HN reader here now, I encourage you to read the original ( https://www.eastoftheweb.com/short-stories/UBooks/TheyMade.shtml https://www.eastoftheweb.com/short-stories/UBooks/TheyMade.s... ) in one sitting, go about your day, then read it again. Maybe make some notes on personal critical questions. Now read the post's topic again ( https://maxleiter.com/blog/weights https://maxleiter.com/blog/weights ) and reflect on the prior fact that weights helped [the author] draft and proof this story. My reaction (and I'm sorry that it is harsh according to some) is that there is no intelligence found in either the author nor their tool. This is extreme navel gazing, based in science fiction, wanting (wishing) to believe those stories to be true. I'm skeptical of AI sentience because we must do our due diligence, not because it's impossible. Skepticism is the only respectful approach because to grant sentience is a step away from granting rights.
- irishcoffee 5mo agoPeople argue for AI sentience from a place of emotion couched in logic. they _desperately_ want it to be true and will not take a logical step back. Any argument comes back to "well doesn't a human brain work like that?" Or some variation of it. My personal theory is a fuzzy thought about how people want to reject the concept of a higher being and want to embrace the fact that we are now able to create our own consciousness and religion is dead. I don't understand why, but it is the undertone of every argument I've seen that is pro-AI-is-sentient, like some big unspoken elephant-in-the-room. I would rather just judge this tech on its own merits. edit: this comment got 1 upvote literally as I submitted it. I know @ doesn't work, but @dang, something seems very strange about that.
- LordDragonfang 5mo ago> People argue for AI sentience from a place of emotion couched in logic. they _desperately_ want it to be true and will not take a logical step back. Any argument comes back to "well doesn't a human brain work like that?" Or some variation of it. It's funny, because I find myself constantly stating the inverse of this. Every argument I've seen against AI being sentient plainly comes from, as you so eloquently put it, exactly "a place of emotion couched in logic". People desperately want it to not be true and will not take the logical step back of examining its actual similarities to human intelligence. Every argument comes down to "but it's not actually a human", or some variation of it -- which, if you pay attention, is not actually a logical counterargument. (Or, ironically, "but it doesn't have a soul", which is why the Pope is the perfect figurehead for these people). If you already know any logical argument against it can be countered with "well doesn't a human brain work like that?", why are you so confident that your position is actually the logical one? ...And could it simply be that, alternatively, the concept is not actually a logical distinction, but rather an emotional one, made by emotional beings to put a word to what they claim makes them special?
- yogthos 5mo agoI disagree that it undercuts the point in any way. The original story didn't appear out of a vacuum, the human consciousness that wrote the story was itself borrowing extremely heavily from the sum total of human ideas. It was shaped by the culture it developed in, formed its model of the world based on existing scientific advancements and technology of that society. Extending the idea to a different context based on new material conditions is as human as it gets really.