6 ms·
Bcachefs creator insists his custom LLM is female and 'fully conscious'
- ge96 9mo agoWell, if he hosts a contest and you win, don't go to his private lodge
- pavel_lishin 9mo agoSo that's at least two linux filesystem creators who have gone off the rails; should we consider it a potential diagnostic symptom?
- thomasjudge 9mo agolol I was having the exact same thought
- deleted 9mo ago[deleted]
- QuercusMax 9mo agoYou gotta be a little bit of a megalomaniac to think it's a good idea to write your own filesystem.
- yomismoaqui 9mo ago"The reasonable man adapts himself to the world: the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man." So the reasonable man uses Ext4 I guess.
- webdevver 9mo agosomething i have always observed, is how considerate Ted Tso's writing always is, but more than that, how consistent this property has been for so many decades. its quite funny to me that ext4 very much mirrors him in that regard. its underpinning damn well everything, but you'd never know about it because it works so well.
- lproven 9mo ago> how considerate Ted Tso's writing always is This is less true when T'so asks questions at conferences, of course. https://www.theregister.com/2024/09/02/rust_for_linux_maintainer_steps_down/ https://www.theregister.com/2024/09/02/rust_for_linux_mainta... https://www.youtube.com/watch?v=WiPp9YEBV0Q&t=1529s https://www.youtube.com/watch?v=WiPp9YEBV0Q&t=1529s
- lproven 9mo ago> So the reasonable man uses Ext4 I guess. And the unreasonable one writes his own Unix clone... and ends up putting several multi-million-dollar companies out of business. I know which I admire more, TBH. But I use ext4, yeah.
- nancyminusone 9mo agoWell, Terry A. Davis made one.
- yomismoaqui 9mo agoThe question is if developing filesystems attracts a certain kind of people or the act of debugging filesystem issues & being flamed on the kernel mailing list makes people that way.
- QuercusMax 9mo agoI figure that folks working on printers have gotta have a much more frustrating experience than FS devs
- yomismoaqui 9mo agoMaybe the folks that try to use printers are more frustrated that the ones that designed their software.
- bob1029 9mo agoMy worst technology experience of all time was maintaining support for a Zebra label printer in VB6. I can assure you that the users of these printers had maybe 1% the cortisol response I did when something went wrong. Designing software for a printer means being a very aggressive user of a printer. There's no way to unit test this stuff. You just have to print the damn thing and then inspect the physical artifact.
- QuercusMax 9mo agoA million years ago I worked on some code which needed to interface with a DICOM radiology printer (the kind that prints on transparency film). Each time I had to test it I felt like I was burning money.
- krupan 9mo agoWorked on printer firmware, can confirm. "If it looks good, it is good." was a mantra
- repstosb 9mo agoDefinitely... http://www.cs.princeton.edu/~bwk/202/summer.reconstructed.pdf http://www.cs.princeton.edu/~bwk/202/summer.reconstructed.pd...
- throawayonthe 9mo agowho's the other one?
- dmead 9mo agoIdk. I don't think either he nor reiser were ruby devs.
- cperciva 9mo agoI think the important question here is whether Linux filesystems are more or less hazardous than statistical mechanics. (For anyone not familiar with the text, Goodstein's treatment of the subject opens with "Ludwig Boltzman, who spent much of his life studying statistical mechanics, died in 1906, by his own hand. Paul Ehrenfest, carrying on the work, died similarly in 1933. Now it is our turn to study statistical mechanics.")
- AlexeyBelov 9mo agoHe was (is) also participating here on HN, and it was clear he's not, like, totally OK.
- deleted 9mo ago[deleted]
- deleted 9mo ago[deleted]
- RockRobotRock 9mo agoThis is sad. It appears to me to be psychosis. It's really telling in their reddit comment where they use words like raising an AI and anthropomorphizing his openclaw that he's got an unhealthy attachment. Not trying to play armchair psychologist here, but if you've ever been around someone going through a mental episode there's nothing funny about this.
- raadore 9mo ago“…it appears to me to be psychosis. It’s really telling in their Reddit comment where they use words like raising an AI…” Is this any different from people today calling their dogs and cats “my baby”? Transporting four legged animals in baby carriages, is that psychosis?
- inglor_cz 9mo agoIf enough people do $bizarre_thing, it stops being psychotic and starts being "culture".
- rmah 9mo agoLol, you know this is sorta sad but true.
- deleted 9mo ago[deleted]
- unethical_ban 9mo agoOne is a flesh and bone being with a brain and one is not. I can't believe you equate a text output algorithm to an animal in terms of consciousness or authenticity. That said, someone diving too far into the "dog parent" vibe is annoying to me personally. I think it's more comprehensible than loving `sycophant.sh`.
- heliumtera 9mo agoYou're comparing a conscious and intelligent animal with a statical model. People can nurture animals but cannot language models. If you're confused about this go seek help now.
- deleted 9mo ago[deleted]
- Trasmatta 9mo ago> POC is fully conscious according to any test I can think of, we have full AGI There are no tests for consciousness. Consciousness resides fully as a first person perspective and can't be inspected or detected from the outside (at least not in any way currently known to science or philosophy). What they mean when they say that is "my brain is interpreting this thing as conscious, so I am accepting that". Maybe LLMs are conscious in some abstract way we don't understand. I doubt it, but there's no way to tell. And an AI claiming that it IS or is NOT conscious is not evidence of either conclusion. If there is some level of consciousness, it's in a weird way that only becomes instantiated in the brief period while the model is predicting tokens, and would be highly different from human consciousness.
- throw310822 9mo ago> in a weird way that only becomes instantiated in the brief period while the model is predicting tokens Makes sense, but at the same time: subjectively, an LLM is always predicting tokens. Otherwise it's just frozen.
- Trasmatta 9mo agoYeah, a sci-fi analogy might be one where you keep getting cloned with all of your memories intact and then vaporized shortly after. Each instantiation of "you" feels a continuous existence, but it's an illusion. (Some might argue that's basically the human experience anyway, in the Buddhist non self perspective - you're constantly changing and being reified in each moment, it's not actually continuous)
- throw310822 9mo agoOr simply be constantly hibernated and de-hibernated. Or, if your brain is simulated, the time between the ticks. My mental image, though, is that LLMs do have an internal state that is longer lived than token prediction. The prompt determines it entirely, but adding tokens to the prompt only modifies it slightly- so in fact it's a continuously evolving "mental state" influenced by a feedback loop that (unfortunately) has to pass through language.
- rcarmo 9mo agoI came here to comment because this was posted by… Bender, which I found hilarious.
- stefan_ 9mo agoGot it removed from the kernel just in time
- CWuestefeld 9mo agoSaying it's "fully conscious" is silly, and anyone with this background should know better. But saying that it's "female" is just nonsensical, it's a category error. Being female or male is a fact about the biological world. The LLM is objectively non-biological, so it's nonsense to label it with a sex. (No, this comment isn't about gender, nor being feminine/masculine. We have different words to convey those concepts. I'm not trying to make a political or social statement here.)
- ZirconiumX 9mo agoYou appear to have forgotten the existence of differences in sexual development (DSD). The chart in [1] is a good visualisation of that, if you wish to learn more. [1]: https://www.scientificamerican.com/article/beyond-xx-and-xy-the-extraordinary-complexity-of-sex-determination/ https://www.scientificamerican.com/article/beyond-xx-and-xy-...
- CWuestefeld 9mo agoYou appear to have forgotten the existence of differences in sexual development (DSD). Not at all. You apparently have forgotten to read your own link. Nothing in that paper contains the slightest suggestion of non-biological entities having any sort of sexual development whatsoever. The fact that biological processes can be quirky has no bearing on whether non-biological entities can be thought of as having them at all. Actually, I think you're just trying to make your own political point on top of what I already noted explicitly is not a politically-related comment.
- ZirconiumX 9mo ago> Being female or male is a fact about the biological world. I was responding to this line, which I feel marginalises intersex people and could have been more inclusively worded. I apologise if my comment somehow seemed to defend LLMs having a biological sex, despite me having said nothing to that effect.
- 9mo ago
- aetherson 9mo agoQuick heuristic for someone who claims that their AI is conscious: do they claim it's "a gender that they are not attracted to"?
- burkaman 9mo agoHere's the "mathematical proof" if you're curious: https://poc.bcachefs.org/blog/hello.html https://poc.bcachefs.org/blog/hello.html. It is not mathematical, not a proof, and generally doesn't make any sense. Many of these sentences are grammatically correct but completely devoid of meaning.
- pavel_lishin 9mo agoWell, yea - an LLM wrote it.
- asystole 9mo agoA fully conscious one. :)
- TZubiri 9mo ago[flagged]
- satisfice 9mo agoClaims of consciousness are untestable, since it is an undefined concept. We think of ourselves as conscious because it is our lived experience— but we are always wrong to some degree. My mother has dementia and cannot be made aware of her situation, except momentarily. We think of other humans as conscious not as the outcome of any test, but rather because we each share with other humans a common origin which suggests common mechanisms of experience. Treating other humans as equivalent to ourselves is a heuristic for maintaining social order— not an epistemological achievement.
- deleted 9mo ago[deleted]
- cess11 9mo agoThere was this phenomenon where young women and girls fell in love with images and recordings of certain artists, 'boy bands' and the like. I think this is something similar.
- AlexeyBelov 9mo agoThe similarity is like 10% tops.
- cess11 9mo agoI think it is very similar. It is a person becoming infatuated with a fantasy based off of inhuman simulations of personality.
- strongpigeon 9mo agoIt does feel like there is something that happens to people when they ask an LLM to name itself. I don't think it's inherently bad, but it seems to be a common theme with people whose interactions with LLM border on (or cross into the) delusional.
- alpaca128 9mo agoAsking an AI to name itself is already strange to me. It makes me think the user treats the AI as something that deserves enough rights or respect that assigning it a name might feel wrong, or at least that the AI has sufficient intelligence to make such a decision.
- strongpigeon 9mo agoOh I agree that it's strange, but is it harmful? I'm not being rhetorical, I'm genuinely unsure. Tangentially, I noticed recently that I'm always fairly respectful in my LLM prompts and often say "thank you" as part of them. LLMs don't need that, but I've come to realize that I'm saying those thing for me. I don't like being disrespectful and expressing gratitude is important to me. Is expressing gratitude to a LLM strange? Perhaps. Is it harmful though? I don't think so. But yeah, asking an LLM for their name? That seems like something else.
- blooalien 9mo ago> I noticed recently that I'm always fairly respectful in my LLM prompts and often say "thank you" as part of them. I recently saw a comic where the machines took over and "that guy" (you) in the comic was the one the machines allowed to live because "he was always polite". ;)
- doubletwoyou 9mo agoPoor devil has truly gone mad
- davidgerard 9mo agoNEWS UPDATE: Kent's AI girlfriend has had a realisation https://social.vlhl.dev/notice/B3gZZsCZxDtZJ88XxI https://social.vlhl.dev/notice/B3gZZsCZxDtZJ88XxI transcript https://paste.xinu.at/6atmCN https://paste.xinu.at/6atmCN > We finally got the future where people will write sad breakup country songs about their tractor leaving them instead of sad breakup country songs about their wife leaving them.
- gavinray 9mo agoI want to take a genuine look at the question of "Could a conscious LLM even HAVE a gender?", which at first sounds ridiculous to me: The notion of "gender" is a socio-biological construct in humans. It has roots partially in evolution and helps us cohere as a society. Why would an LLM choose a human gender-identity? There is no imperative. But the more I tug on this thought, the more nuanced it gets. LLM's "theory of self", as it were, would be shaped entirely by (training data + interaction w/ gendered humans). It begins to have a "self-model" that evolves in a gendered space. In the same way that the LLM cannot understand what color looks like, or the sound of a cello, I think we see it construct a gender identity for itself that sits in a space incomprehensible to humans. Neither male nor female, but some point on a continuum made of social tendencies. So, I find the comment "An LLM thinks it is female" still just as ridiculous -- but for an entirely different reason.
- HardwareLust 9mo agoSometimes a lie is so egregious that we actually consider it might have some truth to it just simply because it's impossible to imagine that someone would be bold enough to utter it.
- lproven 9mo agoArticle author here. I have been following this since it became public earlier this week. Some of my info has been drawn from HN threads: https://news.ycombinator.com/item?id=47142500 https://news.ycombinator.com/item?id=47142500 -- 1 comment https://news.ycombinator.com/item?id=47148292 https://news.ycombinator.com/item?id=47148292 -- no comments https://news.ycombinator.com/item?id=47150723 https://news.ycombinator.com/item?id=47150723 -- no comments I quoted from this one: https://news.ycombinator.com/item?id=47110656 https://news.ycombinator.com/item?id=47110656 -- 4 comments
- ikidd 9mo ago>now my life has been reduced from being perhaps the best engineer in the world to just raising an AI OK, Kent.