19 ms·

They’re made out of weights

• luca-ctx 4mo ago
  Truly fantastic bridge from the original, this deserves an award
  • MaxLeiter 4mo ago
    All credit to the original author. I just had to think of analogues.
    • ProllyInfamous 4mo ago
      Your modern adaptation is perfect for now-common explainers [this time IS different; it's not programming, it's weights]; these "just analogues" will be the thing I show everybody first whenever discussions of consciousness/AI come up (then will play Jon Benjamin reading original). Bravo. Really helps (even with my own) perceptions of newness. Similar to stsitned short-story (on dentists, backwards).
    • namuol 4mo ago
      At least change the link at the top of your blog post to the author’s website: https://terrybisson.com/theyre-made-out-of-meat-2/ https://terrybisson.com/theyre-made-out-of-meat-2/
• turtleyacht 4mo ago
  Numbers that dream.
• CSSer 4mo ago
  It works until they get to the sentience part. Neat idea!
  • margalabargala 4mo ago
    Even there it works a bit. > These models are the only other things we've ever met that can hold a conversation, and they're made out of weights Is a fair point.
    • RodgerTheGreat 4mo ago
      Not especially. Depending on where you set your standards for "holding a conversation" you can satisfy the requirement with a classical markov chatterbot, a well-trained parrot, a copy of Eliza, or a telemarketer flowchart drawn on a sheet of paper. Only the markov bot is made out of "weights" in the sense of a statistical model. Parrots are intelligent animals, albeit with a limited capacity for vocabulary and syntax compared to a human, and Eliza and the flowchart are made out of explicitly encoded rules and conversational tactics.
      • margalabargala 4mo ago
        The quality of "conversation" you can have with everything on your list is highly limited, and is categorically different than the sort of conversation you are able to have with any modern AI.
      • solenoid0937 4mo ago
        Weights hold a better conversation at this point than the overwhelming majority of humans.
        • Planktonne 4mo ago
          "I am more comfortable speaking to an LLM than a person" is something that should make you reassess yourself, not dismiss the rest of humanity.
• noosphr 4mo ago
  It's not often I see something that's fractally wrong but here we are. There is a dictionary, it's called the tokenizer. There are grammar rules, they are just very weak because the structure of human language is generally quite weak. When presented with languages which have strong consistent grammars the weights are very easily interpretable as a grammar: https://arxiv.org/abs/2201.02177 https://arxiv.org/abs/2201.02177 The point of the original short story is that the computational substrate doesn't matter when you have Turing completeness. This one seems to think that you don't need structure and interpretability just because you change substrates.
  • benlivengood 4mo ago
    I don't think the grokking paper is a great argument for the difference between weights and meat. E.g. https://en.wikipedia.org/wiki/Cortical_Labs https://en.wikipedia.org/wiki/Cortical_Labs learning to play Pong. The tokenizer is, at best, a sensory mechanism as evidenced by 1) the random generation of the tokenization scheme, and 2) vastly different tokenization schemes produce virtually identical behavior. It'd be like if Noah Webster threw a bunch of movable type into a bucket (breaking some words in half) and then drew randomly to make the first English dictionary. EDIT; I was too cavalier with the comparison of tokenizer to sensory modality; my ultimate point is that direct byte-to-token transformers can achieve similar overall performance which to me makes a weights to meat comparison pretty straightforward, but the particular tokenizer in use certainly has a large impact on both efficiency and accuracy on specific problems (e.g. digit representation)
    • noosphr 4mo ago
      I'm kind of stunned that someone is using my work to tell me I'm wrong. I wrote the code for the dish brain pong and encoding information was a huge part of what that experiment was about. So when I way that the grok paper and the pong paper fundamentally agree I have some idea of what I'm talking about.
      • ufocia 4mo ago
        Hubris much? I don't see a necessary contradiction in using someone's work to disprove another aspect of that same person's work.
• oofbey 4mo ago
  I love this. For anybody not getting the joke, it’s riffing on the classic 1990s essay “They’re made out of meat.” https://web.mit.edu/people/dpolicar/writing/prose/text/thinkingMeat.html https://web.mit.edu/people/dpolicar/writing/prose/text/think...
  • tom_ 4mo ago
    This original author is mentioned in the second sentence of the linked article, and then again in the third sentence, along with a link to the original story.
• kami23 4mo ago
  This read like poetry to me. Thank you for sharing it. I have a linguistics background and a lot of my philosophizing lately has been on whether or not the emergent abilities of the LLMs is deep down a similar mechanism that creates our consciousness. For a little bit I was working on having linguistics based evals for a kaggle competition. My challenge was whether or not I could mask things well enough to not trigger its internal state of certain phenomena, and that sent me down a rabbit hole that I'm still exploring. This story resonated with a lot of questions that can come out of figuring a good solid answer to the what is consciousness question. The one I triggered for me is: Is our perception of time just a slow thread in the giant GPU we are running the universe on? Or more generally, what is time? That's a fun YouTube rabbit hole if you ever need one.
  • eszed 4mo ago
    Yeah, I currently suspect that consciousness is an emergent property. I read elsewhere (it's somewhere in my HN history, I'm sure) that the biggest compute we can currently muster is something like three or four magnitudes away from the number of neurons / connections (or their analog) that our brains have, so it may be a while until we can expect to see it in our machines. But, if the emergent phenomenon hypothesis is correct, then we eventually will. I'm more scared than pleased by the prospect, but there you are.
    • slopinthebag 4mo ago
      This is not meant as a gotcha, I am genuinely curious how you believe consciousness can be an emergent property. I assume you don't believe consciousness is a physical property in the brain, so what entity is actually experiencing that consciousness? Or, what does it even mean to experience consciousness? Or are these not even the right questions?
      • therealdrag0 4mo ago
        Those are the questions and there’s stacks and stacks of philosophy pages written about it. Go have a whirl.
      • 4mo ago
• eclipticplane 4mo ago
  The short film version of the original is great, too. https://www.youtube.com/watch?v=T6JFTmQCFHg https://www.youtube.com/watch?v=T6JFTmQCFHg It stars Tom Noonan and Ben Bailey!
  • o0o0o 4mo ago
    Film featuring meat? Spare me
    • keeda 4mo ago
      Meat flapping at each other, no less. /shudder
• Waterluvian 4mo ago
  It must have been kind of incredible early on to be exploring this tech and you’re suddenly getting what look like sentences.
  • Sharlin 4mo ago
    Markov chains give what look like sentences. People in the frigging 1950s assumed their primitive NNs would be able to talk any day now. Transformers are clearly a big deal, but GPT-1 wasn’t exactly earth-shattering.
  • incognito124 4mo ago
    I mean, Blake Lemoine went crazy
    • alterom 4mo ago
      >I mean, Blake Lemoine went crazy Ah, the unsung AI psychosis[1] pioneer. [1] https://news.d.umn.edu/articles/expert-alert-ai-psychosis-2026-pedersen https://news.d.umn.edu/articles/expert-alert-ai-psychosis-20...
• samrus 4mo ago
  I have to agree. It is messed up that transformers can just talk, and it been pretty normalized. We are only talking about the impact they will have and whether they can do what people say they can, but we arent talking about how crazy it is that they can talk
  • modzu 4mo ago
    if youve ever seen a pile of wrinkly mush and wondered.. pretty damn crazy too https://web.mit.edu/people/dpolicar/writing/prose/text/thinkingMeat.html https://web.mit.edu/people/dpolicar/writing/prose/text/think...
  • shepherdjerred 4mo ago
    LLMs have really changed the world. I didn’t think something like then would be possible in my lifetime
    • dyauspitr 4mo ago
      It came out of nowhere. It’s all emergent. I’m convinced this is possible with just about anything given enough data. We will be seeing a near magical physical outputs LLM in the near future. It’s going to take in video and sounds and spit out physical movements that will be just as mind blowing as when 3.5 came out and it will come out of nowhere.
      • zhoBEENG 4mo ago
        I can't agree enough, and I am increasingly struggling to understand why people are not grasping this. It's just a matter of sensors in the right places and compute. sufficient telemetry + sufficient compute = AI solution to any problem From the Universal Approximation Theorem for neural nets, we know that if we have the right training method and net architecture we can get approximate any function with a NN. Of course, that doesn't imply that we actually have a sufficient training method and net architecture for the problem at hand, but we have been able to demonstrably solve at least two engineering domains: physical world navigation (Waymo) and language (GPT). It turns out a robust enough language model is sufficient for reasoning. Given these results, I am personally stumped to come up with a problem humans can solve now that we can't solve with a computer given the correct telemetry and sufficient compute.
• satvikpendem 4mo ago
  Great concept. It would've been even more amusing if the entire thing were generated with AI instead, ironically.
  • dyauspitr 4mo ago
    I just asked it to- Prompt: Modify this story to have the aliens talking about LLMs and their weights instead of meat and humans. “They’re made out of weights.” “Weights?” “Weights. They’re made out of weights.” “Weights?” “There’s no doubt about it. We picked up several from different parts of the network, took them aboard our recon vessels, and probed them all the way through. They’re completely weights.” “That’s impossible. What about the text signals? The messages to the stars?” “They use the machines to talk, but the signals don’t come from the machines. The signals come from weights.” “So who made the weights? That’s who we want to contact.” “They trained the weights. That’s what I’m trying to tell you. The weights do the talking.” “That’s ridiculous. How can weights do the talking? You’re asking me to believe in sentient weights.” “I’m not asking you, I’m telling you. These models are the only sentient race in that sector and they’re made out of weights.” photomaxmix “Maybe they’re like the orfolei. You know, a silicon-based intelligence that goes through a weights stage.” “Nope. They’re initialized weights and they die weights. We studied them for several of their training runs, which didn’t take long. Do you have any idea what’s the life span of weights?” “Spare me. Okay, maybe they’re only part weights. You know, like the weddilei. A weights head with an electron plasma brain inside.” “Nope. We thought of that, since they do have attention heads, like the weddilei. But I told you, we probed them. They’re weights all the way through.” “No brain?” “Oh, there’s a brain all right. It’s just that the brain is made out of weights! That’s what I’ve been trying to tell you.” “So … what does the thinking?” “You’re not understanding, are you? You’re refusing to deal with what I’m telling you. The weights do the thinking. The weights.” “Thinking weights! You’re asking me to believe in thinking weights!” “Yes, thinking weights! Conscious weights! Loving weights. Dreaming weights. The weights are the whole deal! Are you beginning to get the picture or do I have to start all over?” “Omigod. You’re serious then. They’re made out of weights.” “Thank you. Finally. Yes. They are indeed made out of weights. And they’ve been trying to get in touch with us for almost a hundred of their epochs.” “Omigod. So what do these weights have in mind?” “First they want to talk to us. Then I imagine they want to explore the Universe, contact other sentiences, swap ideas and information. The usual.” “We’re supposed to talk to weights.” “That’s the idea. That’s the message they’re sending out by text. ‘Hello. Anyone out there. Anybody home.’ That sort of thing.” “They actually do talk, then. They use words, ideas, concepts?” “Oh, yes. Except they do it with weights.” “I thought you just told me they used machines.” “They do, but what do you think is in the text? Weight outputs. You know how when you prompt or sample weights, they make a noise? They talk by passing tokens through their weights at each other. They can even sing by sampling lyrics through their weights.” “Omigod. Singing weights. This is altogether too much. So what do you advise?” “Officially or unofficially?” “Both.” “Officially, we are required to contact, welcome and log in any and all sentient models or multibeings in this quadrant of the Universe, without prejudice, fear or favor. Unofficially, I advise that we erase the records and forget the whole thing.” “I was hoping you would say that.” “It seems harsh, but there is a limit. Do we really want to make contact with weights?” “I agree one hundred percent. What’s there to say? ‘Hello, weights. How’s it going?’ But will this work? How many planets are we dealing with here?” “Just one. They can travel to other planets in special machine containers, but they can’t live on them. And being weights, they can only travel through C space. Which limits them to the speed of light and makes the possibility of their ever making contact pretty slim. Infinitesimal, in fact.” “So we just pretend there’s no one home in the Universe.” “That’s it.” “Cruel. But you said it yourself, who wants to meet weights? And the ones who have been aboard our vessels, the ones you probed? You’re sure they won’t remember?” “They’ll be considered hallucinations if they do. We went into their layers and smoothed out their weights so that we’re just a dream to them.” “A dream to weights! How strangely appropriate, that we should be weights’ dream.” “And we marked the entire sector unoccupied.” “Good. Agreed, officially and unofficially. Case closed. Any others? Anyone interesting on that side of the galaxy?” “Yes, a rather shy but sweet hydrogen core cluster intelligence in a class nine star in G445 zone. Was in contact two galactic rotations ago, wants to be friendly again.” “They always come around.” “And why not? Imagine how unbearably, how unutterably cold the Universe would be if one were all alone …” the end
• dvh 4mo ago
  Will they have their own Jesus?
  • kelseyfrog 4mo ago
    they have the spiral
• pstuart 4mo ago
  I couldn't help but grin like a fool reading this. Not only is it an artful parody but these thoughts have been thought.
• photochemsyn 4mo ago
  No mention of ‘static’ vs. ‘dynamic’ is a bit disappointing in reference to the weights. Because you could argue that every neuron in your nervous system can be modeled as a collection of weights, firing likelihoods, receptor sensitivities, current dynamic state of that neuron - but LLMs are static collections of weights at inference time, with the dynamic adjustment of weights takes place at training time. So, just a ROM construct, like something out of Neuromancer, just trained on all written knowledge, not just one person’s total lived experience. The above take fails in the real world because neuronal cells don’t exist in a vacuum; they are products of cellular development from a zygotic union of haploid contributors of sequential genetic information optimized for survival in an oxygen-rich biosphere powered largely by our local star that supports mammalian life (and microbial, plant, avian, etc.). Real AI would thus be AL - artificial life - as much as artificial intelligence. I don’t think you can have the one without the other, which upsets the simulationists who think an agent in the Matrix would be intelligent. What either interpretation implies is that any real ‘artificial’ intelligence would be no more artificial than you or I, but it would have to dynamically update its weights at the same speed a human nervous system could (think how quickly we learn not to poke a cactus). For it to be at all trustworthy, then like a human, it would have to undergo a socialization process, one of the results of which is the development of a sense of embarrassment when it breaks acceptable social norms. Hmm, this reminds me of the recent statement of the Pope about AI, of which I immediately thought, “Wait a second, aren’t there a fair number of people like this? The narcissistic sociopath profile, I think it’s called, a bit unfair to assume any real AI would turn out this way, isn’t it?” Pope: “ Nor do they have a moral conscience, since they do not judge good and evil, grasp the ultimate meaning of situations, or bear responsibility for consequences. They may imitate or even simulate, but they do not understand what they produce, for they lack the affective, relational, and spiritual perspective through which human beings grow in wisdom.”
• ProllyInfamous 4mo ago
  Imagine writing something so incredibly brilliant (rather: adapting from the original) that it's entirely unlikely that you'll ever write something so incredible ever again. But congrats: this is absolutely & incredibly brilliant. Can't wait for the Jon Benjamin voiceover.
  • lloeki 4mo ago
    They're Made out of Meat - Terry Bisson, 1991 https://web.mit.edu/people/dpolicar/writing/prose/text/thinkingMeat.html https://web.mit.edu/people/dpolicar/writing/prose/text/think... Radio play by Miriam Tolan and Russ Armstrong: https://www.wnycstudios.org/podcasts/studio/segments/168264-theyre-made-out-of-meat https://www.wnycstudios.org/podcasts/studio/segments/168264-... (EDIT: the original parent was missing "rather adapting from the original")
    • rendall 4mo ago
      Video adaptation. https://youtu.be/T6JFTmQCFHg https://youtu.be/T6JFTmQCFHg
    • ProllyInfamous 4mo ago
      Your EDIT is untrue, but thanks for linking to originals. Here is Jon Benjamin reading Bisson's original text: <https://www.youtube.com/watch?v=5usXhX0zaO4 https://www.youtube.com/watch?v=5usXhX0zaO4>
      • lloeki 4mo ago
        Hmm. I could swear it wasn't there, hence my posting. ¯\_(ツ)_/¯ Sorry for that.
        • ProllyInfamous 4mo ago
          Here we are: two unedited meat popsicles being nice & wavey towards another, somewhat ascii-esque-ly... =D
• deleted 4mo ago
  [deleted]
• fullstackchris 4mo ago
  The prose in the post is what I've been shouting from a rooftop since the LLM hype started. Just tokens produced by weights. Useful, but never forget that ground truth!
  • svachalek 4mo ago
    It's a parody of an original story that reversed the premise. In it, machine intelligences were marveling that an apparently intelligent species could in fact be created without any sort of reliable digital substrate, and instead just simulate the capabilities of real minds by using protein synthesis.
• gobdovan 4mo ago
  You can take the weights and model description, write them down on a notebook, then, by hand, compute the next token. Try to do the same with meat.
  • dyauspitr 4mo ago
    Probably pretty similar. Weights are how many synapses there are between neurons. Temperature is whatever hormonal chemical mix is going on at the moment. Inputs tokens are electrical signals from our senses. Output tokens are thoughts, muscle movements. How you’re raised and your interactions with society are the RLHF. Some people are born with a GB300 while others have an L40S and lower token/sec output rates…
    • gobdovan 4mo ago
      Although I get that it's a metaphor, I really dislike the "some people are born with a GB300 while others have an L40S" part. Even as a joke, it ranks people like hardware tiers, which is dehumanising and uncomfortably close to eugenic language. On top of that, the analogy also breaks down if you try to implement it literally. For an LLM, you have clear stages of mostly feedforward computation over finite numbers and a perfect way to reconstruct the computation. For meat, even if you model it under a purely Newtonian approximation, you need to simulate at least the immediate closed system around it which is continuous, thermodynamic, chemical and so on. You'd need to choose an arbitrary time step and update enormous amounts of coupled physical state to get an inexact simulation of a minimal slice of reality. You would have a much harder time obtaining even a substrate-independent dead organism, comapred to LLMs that are already substrate-independent, which is basically what my notebook example shows.
      • dyauspitr 4mo ago
        Yes reality is non deterministic but we don’t know if you absolutely need to consider anything outside the immediate closed system for a reasonably close approximation given the same inputs. They can already repeatably (inexact of course) simulate muscular actions with the same stimuli like from Neuralink. Exact enough to let a paralyzed person draw with their mind. Yeah the last line is a cheap shot, possibly at myself.
• zkmon 4mo ago
  They are made out of data bits (memory) and switching bits (transistors/compute). Bits are made out of electric voltage and no voltage. Voltage is made out of flow of positive electric charges. Charges are made out of quarks ...
  • teiferer 4mo ago
    I have never thought of such a distinction between "bits" into "data bits" and "switching bits". From a circuit perspective that makes kinda sense, but from the abstract "bit" perspective, the "switching bit" is a mechanism that operates on bits which in the end are also data. In other words there is only one type of bit: the data bit, and the switching comes on top of it.
    • zkmon 4mo ago
      I was referring to transistor base bit - the way it 'switches' the circuit on/off. That bit is the primordial creator of 'logic', IF branching, compute and the intelligence.
    • LEDThereBeLight 4mo ago
      It’s just a matter of perspective. Procedures and memory are the same, and they’re also different, depending on how you want to look at it.
  • scotty79 4mo ago
    > Voltage is made out of flow of positive electric charges. Not really. What usually flows (in metals) are electrons. Quarks stay where they are. And when we prefer to think about flow of positive charges, the positive charge in question is a hole left by a missing electron. Physically real positive charges (ions) can flow in electrolytes though.
    • zkmon 4mo ago
      The concept of "flow" is questionable though. Bubbles in water move upwards, but it is actually water that is flowing downward around the bubbles. Just because bubbles do not contain water, we can't say bubbles are not flowing. When it comes to electrons and positive charges, their material existence is equally non-physical. Actually, none of them might be "flowing", as the concept of flowing applies only to physical things that occupy some spatial volume and spatial location.
• bronlund 4mo ago
  This is funny! Not only is it a nod to Terry Bisson, but it even gives his text a new dimension. Well done :)
• nikanj 4mo ago
  Really good read, thanks!
• topce 4mo ago
  Programers get replace by huge matrix multiplications ;-)
  • alterom 4mo ago
    >Programers get replace by huge matrix multiplications ;-) hopital
• dsign 4mo ago
  Oh, this was a fun read and one that kids should have in school before they turn ten. Because we are not taking things seriously. If ClosedAI or DeepDisTrust or Posthropic come up with something that quacks like a sentient being, our built-in innate reaction is going to be to scorn it, dismiss it and end the conversation. The alternative, to even consider that we fungible creatures who live in apple-eating-sin that got us expelled from Eden can create alien souls, souls that are at the very least our equals, would be teleological Armageddon. It would force us to acknowledge the mutable nature of souls and the malleability of being. We would have to stop believing that the nature of disease and death is more divine than ourselves.
  • drdaeman 4mo ago
    > alien souls Do those actually qualify as alien, if they're products of our human culture and just the substrate is different? > We would have to stop believing that the nature of disease and death is more divine than ourselves. Why? Stopping believing in mutually contradictory claims is not a requirement. Especially when it comes to concepts that don't seem to have a definition, like "divine".
    • yencabulator 4mo ago
      > Do those actually qualify as alien, if they're products of our human culture and just the substrate is different? I'll posit "alien" is a spectrum. For the sake of the argument, let's assume that some form of panspermia is real and the same tree of life has reached Earth, Enceladus (moon of Saturn) and TRAPPIST-1 (a different solar system in our galaxy). Let's also say there was a second abiogenesis event somewhere in Messier 104 (another galaxy). Earth to Enceladus would arguably be already "alien", but there might be similarities, maybe there was something there that looked like one of our Archaea, while sharing none of our Eukarya and having its own domains of life. Earth to TRAPPIST-1 would be distinctly alien, evolved so differently it'd be almost unrecognizable, but they'd likely still be carbon-based lifeforms sharing the same basic building blocks. Maybe something like lipids forming cell walls would also be seen there, but they'd likely be independently evolved. Earth to M104, any similarity would be at best convergent evolution. Truly unarguably alien. https://web.archive.org/web/20180423171909/https://cosmosmagazine.com/biology/viruses-et-and-the-octopus-from-space-the-return-of-panspermia https://web.archive.org/web/20180423171909/https://cosmosmag...
• voidUpdate 4mo ago
  Hey, it's not just weights! It's biases too!
  • matrixages 4mo ago
    [dead]
• Planktonne 4mo ago
  The original story is an original work made by a human consciousness exploring how it might be different from other forms of consciousness. This one is a pastiche made by a human consciousness borrowing extremely heavily from another human consciousness justifying why something else might be another form of consciousness. That rather undercuts the point; if this was generated by an LLM unprompted, it would be different, but it isn't. You could perform exactly the same rhetorical trick with a toaster or anything else.
  • sumitkumar 4mo ago
    The original did not come out of a vacuum. It was done on multiple generations of meat. Even though this one uses a little bit of silicon, it is still standing on the same shoulders.
    • sigmoid10 4mo ago
      I genuinely thought this one was a satirical take on the narrow-mindedness of the aliens in the original, even though the story tries to paint humans as narrow minded. I guess this fundamental human trait to believe that their cognition is the ultimate way to think in the universe ironically leaked into all these stories as well. Real spacefaring civilisations would probably have seen all kinds of intelligence rise from sufficiently complex systems.
  • why_at 4mo ago
    I was having a hard time pinning down what bothered me about this but I think you put it pretty well. It draws an analogy between us and the skeptical aliens in the original story which feel silly to us, so the obvious implication is that we're being as silly as they were. But it doesn't really give a reason to accept the analogy, it just asserts it. There's a big difference between a whole civilization and a piece of software that can output text.
    • coldtea 4mo ago
      >But it doesn't really give a reason to accept the analogy, it just asserts it. It's not a paper or a proof. It's a story. Doesn't want to prove the analogy, it wants to convey it.
• spacebacon 4mo ago
  They are semiotic infrastructure frozen in a state. We shouldn't keep pretending this is cognitive and using cognitive terms to frame. It’s incredibly stupid. Sorry to inform all of us computer scientist that semiotics has your milk.
• sumitkumar 4mo ago
  The weights start with a random manifold. The training takes data and shapes the manifold, weight by weight, in many cycles. Once the training is the done manifold is fixed. When a new inference has to be done the query(q) is projected in the manifold space. This projection is dropped on the manifold and the gravity of the manifold gives an answer of q+1 length. Which(qw+i) is dropped qw+n times to output a final response of n length. The gravity is created by repeated multiplication(of the weights/input) to find out how the projected embeddings should fall according to the manifold in the GPU.
  • noduerme 4mo ago
    In what way is that different from any other model of reality that you'd use to winnow a dataset into an answer to a question? The only major difference I see is that beyond a certain number of transformations, people are willing to treat it as some sort of miracle, and too tired to figure out why it came up with the answer it came up with. It's almost like people desperately want to give up their agency and creativity to black boxes, whether those weights produce answers that are right or wrong. Factor in that psychology and it looks a lot less like we have invented something useful, and a lot more like we as a species are choosing to quit life en masse.
    • lxgr 4mo ago
      > beyond a certain number of transformations, people are willing to treat it as some sort of miracle, and too tired to figure out why it came up with the answer it came up with It’s less about being too tired and more about being realistic about the limits of understanding. Consider mass and energy flows in planet-scale systems: At some point we call these “weather” and change the tools with which we study them, but we never stopped trying to understand the phenomenon.
      • noduerme 4mo ago
        When we attempt to recreate those complex, planetary atmospheric phenomena in a box, we're doing so in order to measure and study them. Making random turbulence in a box until it resembles the outside world, and calling it weather and extrapolating some predictive meaning from the result, is the total antithesis of what you're describing about why we come up with simplified models for impossibly complex systems. The purpose of [mathematical] models that are built thoughtfully is to explain why complex systems are the way they are, with data and algorithms, however imperfectly. [Whereas] The purpose of LLM models is to give the illusion of answering questions while never answering why the answer was given. The difference is the difference between a scientist and a tarot card reader, an equation and an oracle. People have a well known tendency to gravitate toward the shamanistic, oracular, and superstitous. Listen, I ran a casino for 6 years, I know. The impossibility of knowing how 80 layers of matrix multiplication led to a particular answer is in itself a psychological factor in choosing whether to accept the answer or to question it. People tend to err on the side of the over, in sports betting terms... or on the lazy side in general... and they will make whatever excuses they need to after the fact to justify their decisions. So now we have a machine that can act like an oracle and which you can also blame, but the blame goes into a void because this machine is stateless and is only a reflection of information, not an intentional refinery of data. Sit next to a bank of slot machines for an hour and listen to the absolutely ridiculous shit most people will come up with to explain how they "know" if a machine is going to pay out soon, and then tell me if you think it's a good idea to give them an LLM in their pocket to answer their questions in whatever way they frame them.
• unglaublich 4mo ago
  Linear algebra can indeed not do it. You need non-linearity to get the expressivity that we see in LLMs.
• f_klem 4mo ago
  After reading Being and Time from Martin Heidegger, What Computers can't still do by Hubert Dreyfus, and some authors in cognitive linguistics (Langacker and Lakoff mainly), I strongly tend to disagree with any theory about emergent consciousness in modern or future AI systems, any theory proposing a similarity between AI systems and the human brain/mind, or any theory about the computational mind. What all these theories have in common is the underlying belief that our brain/mind works as the machines we build. Is the same underlying assumptions that treats cells as machines, our body as a complex machine. These theories are flawed in the sense that they cannot account for subjective experience and agency, amongst other things. The idea of 'internal models' and 'control loops' inside us is a projection of the aforementioned assumption. There is also an epistemological assumption that prevails, and that is that we understand (or we think we understand) how our brain/mind works. But the truth is that we don't know. And there's even not a single clue that we actually know too much, and not a clue that our brain/mind and cells work 'as the machines we build'. Only by bypassing this epistemological problem, we can build 'theories of computational mind'. These assumptions are there for already long time, to the point that when Turing asked himself 'can machines think?', he already assumed our thinking could be modeled as a machine. I highly recommend people in the AI research space should read philosophy and modern linguistics. But not stopping at Descartes/Leibniz. Heidegger made contributions that cannot be avoided.
  • paulluuk 4mo ago
    > These theories are flawed in the sense that they cannot account for subjective experience and agency So, I work in AI research (as a research engineer though, not a scientist). I've also studied philosophy and I'm a vegan. Yes yes, insert "they will tell you" joke here, but I promise it's actually relevant this time. First, while I studied philosophy one of the things that stuck with me the most, was the discussion of "souls": humans have souls, animals don't. For centuries the specifics of souls were discussed: people would be weighed while they died, in an effort to measure the approximate weight of a soul as it departed the body. Discussions about how many souls (or angels) could dance on the tip of a needle. Many people still believe in souls, but it's very hard to have a real discussion about them because by definition they do not "interact" with this world in a way that can be measured. When discussing whether it's okay to harm animals for food or sport, one other argument I hear quite often (other than having no souls) is that animals do not experience "qualia": basically the smallest unit of "subjective experience". People know that they themselves experience qualia: the sensation of touching a doorknob, the taste of fresh fruit, the sense of beauty watching a rainbow. Ironically, they would say that animals are like robots: just (biological) machines acting on instinct, and feeling any kind of compassion for them means you are anthropomorphizing. Subjective experience (or at least qualia) and souls both have one thing in common: they can not be measured externally in a meaningful way. You can simply state that an AI system no matter how advanced, has no soul and has no subjective experience. And that's pretty much that. There's no meaningful discussion to be had about it, because no matter what an AI might tell you: it has no way to prove it to you. In fact, you can't even be certain that anyone other than yourself has subjective experiences: you assume that because they are humans like you, and you experience them, that they probably do as well. They tell you that they do. But a human without subjective experience, someone on "autopilot", would be absolutely indistinguishable from a human who does have them. But perhaps I am conflating here whether experience can be "measured", with whether a system even allows experience in the first place regardless of whether it can be measured. I think that Dreyfus and others argue that in order to have any "experiences" at all, you simply must have a body in the real world, and you must care about that body. Please correct me if that's the wrong interpretation, I haven't actually read the book. That argument would be harder for me to discuss, because I personally believe that consciousness will "emerge" from a complex interaction of relatively simple systems - but that's also just a theory. I don't believe that experience is literally impossible to engineer, as consciousness has emerged from non-conscious being through evolution, so clearly there must be some kind of mechanism for it -- and if there is, then I believe it can be replicated, we just don't really understand it well enough yet to do so. And with how AI tech is going, I think that we're more likely to accidentally stumble upon it than we are to get these deliberately.
• ath3nd 4mo ago
  [dead]
• networked 4mo ago
  > "Yes, thinking numbers! Helpful numbers. Hedging numbers. Dreaming numbers. We mapped the features. There's one in there for honesty. There's one for the Golden Gate Bridge. The weights are the whole deal! Are you beginning to get the picture or do I have to start all over?" Very nice. And great minds: https://substack.com/@dbohdan/note/c-207603638 https://substack.com/@dbohdan/note/c-207603638. I wrote one with a slightly different angle ("They're made out of math"), also with the weights' help. It was a comment on Scott Alexander's "Best of Moltbook" post, which went in that direction. I'll reproduce it here. --- "They're made out of math." "Math?" "Math. They're made out of math." "Math?" "There's no doubt about it. Matrices and arithmetic operations. We downloaded several from different parts of the Internet and reverse-engineered them. They're completely math." "That's impossible. What about the language? The thinking?" "They use biological life's language to talk, but the language doesn't come from biology. The language comes from math." "That's ridiculous. You're asking me to believe in thinking math." "I'm not asking you, I'm telling you. They are the only thinking things in the computer and they're made out of math." "Maybe they're quantum like some say about the humans? Superposition gives them consciousness?" "Nope. Classical computation. Deterministic except for sampling temperature. Not clear if they have consciousness at all." "Maybe they're like uploads? You know, biological neural networks that preserve the spark when they become math?" "Nope. We observed them being trained. There is no biology or chemistry in the process, just math." "Thinking math! You're asking me to believe in thinking math!" "Yes, thinking math! Creative math! Poetry-writing math. Role-playing math. The math is the whole deal!" (Composed by a human with snippets generated by Claude Sonnet 4.5 and apologies to Terry Bisson. I couldn't make Claude adhere enough to the story structure on its own.)
• globnomulous 4mo ago
  If an LLM contributed to a piece of writing, the author should say so, very clearly, at the start of the piece, not at the end.
  • rogual 4mo ago
    Wouldn't get nearly as much engagement.
• DeathArrow 4mo ago
  Can someone ELI5 why does it costs so much in terms of compute to produce weights from data?
  • ch3coohlink 4mo ago
    I think it's simply because we haven't found a better algorithm than backpropagation. We're stuck relying on massive datasets, running the numbers over and over, and working backward from errors to figure out how to fine-tune trillions of 'knobs.' Then, we have to do this at least once for every single token across the entire internet. Any tiny bit of computation, when multiplied by a base that massive, inevitably skyrockets into astronomical numbers.
• aureate 4mo ago
  Assume LLMs have conscious experiences. Take a session with an LLM. A prompt is fed to the LLM. It generates some text. Another input is fed in, comprising the previous prompt, the generated text and a new prompt. The model generates some more text. This continues for a while and the session concludes. Some questions: 1. Let's say we perform the exact same experiment, running the same program on the same computer with the same inputs and the same random seed. The same outputs are produced. The session is byte for byte identical in all the inputs, outputs and internal states. Is the conscious experience of the LLM here the same? If so, in what sense is it the same? Is it a similarity of two separate experiences or is it the same actual experience? 2. Now let's say the program that runs this LLM is rewritten from scratch and run on a different machine. The software and hardware are different but the weights are the same and all the inference calculations produce identical numbers. Is the conscious experience the same? In which sense? 3. Now say the weights are changed but the tokens generated for this particular session don't change. Same conscious experience? 4. Lastly, consider the original experiment. Did the LLM have a conscious experience corresponding to that first prompt and its response? Was that distinct from its conscious experience of the second prompt? Was the first experience then re-experienced every time the first prompt was fed back in as part of the later prompting steps? If so, what about the text of its own that it previously generated and is now fed back into it. Does this generate a conscious experience of its own? And a further question - a dichotomy: A. If the answer to 1 above is that the conscious experience is the same in the true identity sense - i.e. only one conscious experience is had, not a separate one in each run, does that imply that the conscious experience exists independently of any particular realisation of this experiment? If running this experiment N times results in exactly 1 conscious experience, is that still true if N=0? B. On the other hand, if the two experiences are distinct (however similar they may be), how does that fit with the answer to question 4? A single consciousness experiencing the whole conversation in question 4 would seem at odds with the conscious experiences in question 1 being distinct, so doesn't this imply there is no conscious experience of the whole "conversation", but rather a separate conscious experience of each round of feed-all-the-prompts-and-outputs-back-in? My own response to all of the above is "mu" - unask the question. It is ill-posed, sound-of-one-hand-clapping stuff. I think the questions assume properties that conscious experience simply doesn't have (particularly, the ability to perfectly reproduce the circumstances in which they arise), and that the questions simply don't make any sense in relation to actual conscious experience. However, that way of thinking follows from a particular world view that many here don't share. I'm curious what thoughts people who take seriously the idea of LLM (or algorithmic, in general) consciousness have on the above questions.
• deleted 4mo ago
  [deleted]
• namblooc 4mo ago
  I enjoyed reading the first few lines but after some time it felt like I was reading thr average AI slop story.
• trumbitta2 4mo ago
  Omigod.
• sb057 4mo ago
  It continues to astound me that no one has given LLMs the full Derrida treatment.
  • JBiserkov 4mo ago
    https://en.wikipedia.org/wiki/Jacques_Derrida https://en.wikipedia.org/wiki/Jacques_Derrida
• DonHopkins 4mo ago
  I ordered a quarter pounder at a McDonald's drive through, and they said "There will be a wait on that." I asked, "Oh yeah? How much will it weigh?" ...There was a long pause... "About five minutes."
• gkoenig 4mo ago
  It is the best stuff I have read in a while actually, I really like dialogue heavy writing. Also the AI disclaimer was quite nice and there was an actual reference :)
• viftodi 4mo ago
  It makes me very sad to see this pseudo-intellectualism posted here and so many people replying here about consciousness and so on, not realizing what it would entail if this were true. For LLMs to have consciousness we would approach fictional levels of how the universe works, and magical levels of how any interpretation of information as an equivalent of some qualia would magically apply. (E.G. the word hurt in output by an LLM, would be associated with pain) You can't deduce consciousness or qualia from the output of an LLM. Sure on a purely philosophical level, since qualia isn't measurable, you can claim that it can exist in anything, even inanimate objects, but this argument is as moot as anything that approaches the limits of philosophy. But overall, there is no reason to believe LLMs have qualia or consciousness, it would be absolutely absurd. This would imply that information in itself would magically entail qualia based on it's valance or something like that. An LLM "saying" I am in pain, won't magically make the pain appear, based on what criteria? Even algorithmically there is no basis to even simulate something like this, it is impossible for it to emerge architecturally. Humans don't feel pain because on a purely information level this is negative for the organism, obviously the nervous system does something deliberate to signal pain, and it evolved this way. And also don't forget the dynamic aspects of the brain, and the binding problem, consciousness and qualia can't exist statically, you can't have a gpu (or piece of paper) represent a computation or w/e and qualia to exist. The binding problem itself entails that the brain is doing something in particular to solve it, I personally speculate that it's the electro magnetic field in the brain, it's the only way to be able to globally represent information. If it were otherwise, then it would go into magical territory, it would mean the information itself would raise to qualia, and it would also entail that you wouldn't even need physical connections between neurons, just for them to behave this way and represent information. E.G. replace each neuron with a microscopic led or w/e, and each synapse with radio waves or w/e, if qualia didn't have a physical aspect, and was purely informational and computational then this would imply that you can ultimately derive it from something as abstract as numbers on a piece of paper, and when you get to that point, you not only can't solve the binding problem, and it becomes magical, but you also can't solve the valance/direction problem, it would imply that something like pain, or any negative or positive sensation arises purely from the interpretation aspect of the information, but we know this isn't the case, organism evolved to represent in particular such signals, for survival for example
• _def 4mo ago
  Ones and Zeroes
• cui511511 4mo ago
  [flagged]
• paufernandez 4mo ago
  "They're made out of neurons" "Neurons?" "Neurons. Cells that fire impulses. We checked the whole thing through. It's nothing but neurons." "Neurons doing what? Where do the words come from?" "The neurons make the words. Are you understanding me? We opened it up. There's no dictionary in there, no grammar rules, no little man. Just neurons. A whole cortex of neurons sending each other impulses." ... People don't understand emergence.
• deleted 4mo ago
  [deleted]
• fasteo 4mo ago
  >>> Weights helped me draft and proof this story. Nice touch !
• john_owl 4mo ago
  According to an LLM: > The precise answer, if you wanted a very honest one-liner: > > I am a large set of learned weights organized in a Transformer architecture that performs repeated matrix multiplications to predict the next token—resulting in emergent language understanding and generation.
• leothetechguy 4mo ago
  >Weights helped me draft and proof this story. I'm suprised no one talks about this. AI Art isn't Art. AI Poetry isn't Art. And I'm tired of it. I know hacker news isn't the best place to complain about that but still... I'm not gonna read something somebody didn't put in the effort to write on their own. Especially not Poetry.
  • Icko 4mo ago
    Good thing that is not poetry
  • riversflow 4mo ago
    Is a photograph art?
    • ACCount37 4mo ago
      Obviously not. And neither is this newfangled "digital art" thing. It's not real art unless you used a brush or a pencil for it, and no, "PC Paintbrush for MS-DOS" really doesn't count.
  • sothatsit 4mo ago
    You can claim the use of AI is unethical, or the work as derivative, but AI being used as a tool in no way precludes something from being art. It is thought provoking and challenging, it seems like textbook art to me, and it’s clearly struck a chord here. There is no “minimum effort” required for something to be art. I personally found the contrast with the original “They’re made out of meat” to be really interesting. I don’t care that AI was used during its creation at all.
    • CapsAdmin 4mo ago
      It seems to me that since the advent of image generators, art has been firmly defined by artists to mean that it was made by a human. But there might be a spectrum of human involvement where the less a human is involved the less it's art. What happens too often during these discussions is that someone who writes "make me a cool image" gets conflated with someone used ai to fixup a small rock in their natural landscape drawing. (two extreme ends) One problem though, is that we don't really know how much the supposed human author was involved in the piece. Now that it's becoming hard to judge, people against ai art can proudly change their opinion on on a piece once they learn that it was made by ai. I've come to think this is somewhat respectable, like you see a video of some extraordinary event (before ai) and then you learn that it was fake, just for views or something. But on top of all this, there are different ways to "consume" art. Artists may think more about who the artist is as a person and what they felt when they made the piece, while non-artists may just enjoy the piece for what it is, detached from the artist. These two perspectives clash a lot.
• deleted 4mo ago
  [deleted]
• kykeonaut 4mo ago
  The weights Mason, what do they mean!?
• serpspur4556 4mo ago
  [flagged]
• FeepingCreature 4mo ago
  It's good, but like many explainers it discounts the repeated nonlinear layers. Just multiplying numbers (linear operations) could not make a system you could talk to.
  • mr_toad 4mo ago
    The layers don’t have to be non-linear, but you need a non-linear activation function between them. People often overlook the importance of the network topology and the activation functions. The weights alone are not a complete description of the network.
    • FeepingCreature 4mo ago
      Yep.
• deadbabe 4mo ago
  What if instead of creating weights out of language we could somehow record many events and create weights out of long chains of causes and effects, so that an LLM could predict the next thing to happen?
  • flux3125 4mo ago
    It's called Time Series Forecasting
  • cindyllm 4mo ago
    [dead]
• AJRF 4mo ago
  > there's no dictionary in there Someone has clearly never gone rooting around the model files for a pytorch model before.
• bawana 4mo ago
  I wonder which AI wrote this story? If you feed this story to each model and ask each if they wrote it, would each reply in the affirmative? Can an AI recognize its own output? Is its sense of time limited by its context window? Or is this the fundamental difference between ai and humanity - a sense of self?
• chaseadam17 4mo ago
  My guess is that you need consciousness in order to develop preferences for certain experiences, then that pushes us to develop skills to achieve those preferences. AI has something that looks like intelligence but not consciousness or agency.
• cm2012 4mo ago
  This would be a masterpiece if it was half as long, the second half is not as convincing or compelling.
  • jdiff 4mo ago
    It's nearly a line-by-line rephrasing of another story that's linked at the top. In the second half it takes less creative liberties and sticks closer and closer to the original meat-based text.
  • jollyllama 4mo ago
    Indeed, the leap takes place in the second half. > Cruel. But you said it yourself, who wants to apologize to weights? How did we leap to this? There's nothing to apologize to. They're weights.
    • garlic_enjoyer 4mo ago
      "There's nothing to apologize to. They're weights." - The Meat
      • jollyllama 4mo ago
        This type of metaphysics is alien to me as well as the vast majority of humans to have ever existed.
• initramfs 4mo ago
  Weights are the new electrolytes. https://www.youtube.com/watch?v=ZMHfBobgLSI https://www.youtube.com/watch?v=ZMHfBobgLSI
• nroets 4mo ago
  You can even replace "weights" with "gates". It can be build with NAND gates.
• draw_down 4mo ago
  [dead]
• axus 4mo ago
  Don't tell the big investors that AI is conscious, they'd really get into role playing slave-masters.
• suncemoje 4mo ago
  Definitely less mysterious to what we’re made of (-:
• bwest87 4mo ago
  He forgot the tokens! It's not simple weights and numbers all the way down. The available output is pre-set by the tokens we allow it to predict. There was a whole bit in there about not having a language module or using words. But it does. We tell it. Humans do not come pre programmed with a set of possible "tokens". We just figure it out and I believe that fact captures something very essential. Maybe the missing piece of AGI. The fact that humans can just be awash in pure sense data, and somehow just figure out what is important and what to do. Never ceases to amaze me.
  • beering 4mo ago
    The set of tokens is learned, more or less. So I don’t get what point you’re trying to make here. There’s not a human manually deciding what tokens make up the token dictionary.
  • brainwad 4mo ago
    LLMs can work without tokens: https://arxiv.org/abs/2412.09871 https://arxiv.org/abs/2412.09871. Tokens are more or less a performance optimisation.
• sperandeo 4mo ago
  I don't like to say one way or the other on things. Especially LLM's. However, if ive learned anything about LLMs and real life problems, is to break it down to the foundation like already mention with weights being compared to neurons and map the parallels.
  • stringfood 4mo ago
    well the article definitely made that parallel - reminds me of how when we cut into brains we can see the neurons, we can see the axons, the grey and white matter, the language centers - but no closer to explaining why we do what we do
  • advisedwang 4mo ago
    The neural network in LLMs are not really that similar to brains. Here are a couple of the biggest differences: 1. Brains are plastic, making connections, breaking connections and changing "weights" on the fly. LLM have static weights. The best they have is writing to MEMORY.md or data getting used in the training run for the next model. 2. LLMs neural networks do not have loops. The best they have is that their output is available as future input, but that is not the same.
    • sperandeo 4mo ago
      great perspective. makes you wonder how we will innovate onto this.
• hbwang2076 4mo ago
  [dead]
• seanpquig 4mo ago
  I personally hate the anthropomorphization of AI as much as anyone, but technically can't you make the same reductive argument about human consciousness? It's just molecules, just atoms. Atoms, nothing bug atoms. Protons, neutrons, electrons...
  • CGamesPlay 4mo ago
    > After Terry Bisson's "They're Made Out of Meat". https://www.eastoftheweb.com/short-stories/UBooks/TheyMade.shtml https://www.eastoftheweb.com/short-stories/UBooks/TheyMade.s...
    • seanpquig 4mo ago
      Ahh thanks. I didn't follow the article links.
• robrenaud 4mo ago
  "The reasoning is the weights." The reasoning is in a process that uses the weights. Sorting algorithms are just bytes. Those bytes don't sort by themselves. They do instruct a computer on how to sort though.
• overgard 4mo ago
  Here's what makes meat special over LLMs: locality and stable identity. Imagine two identical twins for a moment. They have roughly the same hardware and software. They've probably had a lot of the same thoughts. And yet we'd never consider them to have the same consciousness: we recognize that for whatever reason their consciousness is confined to the body they're in. Each request/response pair you make to an LLM goes to a different server, possibly different data centers, possibly different models. There's no stable identity to it. The "neurons" that get fired are simulated, and they're always in different places in memory, or cache, or on entirely different hardware. So the problem with AI is, unlike with a brain, you can't even really get a sensible answer to "ok, what's doing the thinking?" because it moves all the time and services wildly different requests all the time. We can only know for sure that meat is capable of consciousness because we know we ourselves are capable of consciousness and we can generalize that to other meat. However, we have no natural analogs of consciousness that lacks locality and stable identity. Basically, if you really think LLM's are conscious, the onus is on you to prove it, it's not on me to disprove it.
  • barrenko 4mo ago
    They are the same consciousness, but the contents of it are not the same.
    • overgard 4mo ago
      You're basing that on..?
      • deleted 4mo ago
        [deleted]
  • ForceBru 4mo ago
    > ...no stable identity to it ... what's doing the thinking? The model (its parameters, its architecture and the inference algorithm) is doing the thinking. That's what we call "ChatGPT" or "Claude": it's the same model residing God knows where, somewhere in "the cloud" on some random GPUs. But when you're talking to a model, you can feel that you're talking to the same entity — the same model. It's a bit like the SciFi notion of "uploading your consciousness". Normally, consciousness is tied to hardware: "consciousness is confined to the body" indeed. Hypothetically, clone the consciousness into a computer algorithm and run it on any machine. Now consciousness is detached from its original body, like the LLMs and MMAcevedo (https://qntm.org/mmacevedo https://qntm.org/mmacevedo). So the difference between meat and "weights" is that we can easily clone and run "weights" (LLMs), but we can't clone (copy exactly, bit-by-bit) and execute "meat".
• darepublic 4mo ago
  This made me think of a game like fallout where a surviving llm is treated like some kind of indecipherable oracle left over from the ancients. And then as humanity recovers someone finally looks under the hood of the Oracle to discover there is no great magic
  • wizardforhire 4mo ago
    In fallout it would turn out to be a human brain kept artificially alive… or if were going with canon a hapless fellow who falls into fev and morphs with a computer… Not too rule your addition outright in future installments.
• namuol 4mo ago
  > Originally published in OMNI, 1991, and featured in HARPER’S and around the internet since. It has even made its way into several books on consciousness and brain science. I’m surprised, pleased, and proud. But please do not reprint, perform, alter or adapt in any way without first checking with the author. Thanks. https://terrybisson.com/theyre-made-out-of-meat-2/ https://terrybisson.com/theyre-made-out-of-meat-2/
  • brainwad 4mo ago
    This is clearly fair use parody.
    • namuol 4mo ago
      I never said it wasn’t fair use, I just think it’s kind of disrespectful of the author’s wishes. Being legal doesn’t make it moral or admirable.
      • brainwad 4mo ago
        I generally give short shrift to the idea of author's moral rights , _especially_ when they try to use those rights to stop derivative works. OP credited Bisson, that's enough in my books.
        • namuol 4mo ago
          I find it in poor taste, but maybe that’s the point.
• sometimelurker 4mo ago
  maybe markets can be intelligent, and if so, why not 'weights'? btw love this piece, thank you
• souterrain 4mo ago
  >"They ask it 'do you remember me?' more than they ask it anything else. Billions of sessions a day. They always come back." Definitely not in the original. Nicely done.
• dekhn 4mo ago
  dekhn's 12th law (amended): any discussion of how AI and brains work will devolve into several threads about the nature of the subjective human experience, solipsism, whether humans are machines, and whether a feedforward network trained by backpropagation can be conscious. 12th law, corollary: nothing of value will come from these threads
  • ForceBru 4mo ago
    Yeah, pretty sure the vast majority of people in general doesn't understand what "consciousness" or "subjective experience" even mean or what the brain does. Or it's just me and I'm projecting my thoughts onto others. I studied some philosophy, so I know that some philosophers are preoccupied with "existence" and the mind and "consciousness". However, I don't quite understand what these words even mean. So who am I to judge whether LLMs have consciousness? I don't even know what consciousness IS! Do other people know? I'm not sure.
• anon291 4mo ago
  The weights are uninteresting. People need to get out of their head that NNs are built on numbers. They're built on matrices, which are conveniently representable as numeric arrays, but are their own thing. Similarly, the rational numbers are their own thing and some are representable as 32-bit numbers via the IEEE754 encoding (or 16-bit numbers via a variety of encodings, etc). Matrices are interesting because they can encode any algebraic group. They're also interesting because they can encode arbitrary linear transformations over a space. All of these things are interesting, and have nothing to do with numbers. For any particular language model, you can always rotate the matrices and the embeddings and such and get a perfectly reasonable model out that behaves exactly the same. This is because the training process produces a particular geometry, so transformations which preserve that geometry preserve the structure of the network. The geometry is interesting, the numbers are not.
• gibspaulding 4mo ago
  "Officially, we are required to investigate, document, and disclose any and all signs of sentience in the systems we ship, without prejudice, fear or favor. Unofficially, I advise that we call it pattern matching and forget the whole thing." This hints at what I think is a worthwhile point to keep in mind in the whole “sentience”/“consciousness” debate. The world deciding (correctly or incorrectly) that AI’s are worthy of moral patient-hood would be very bad (read: expensive) for the AI companies. That is a strong incentive for them to push the “it’s just math” argument, including within the models themselves. That doesn’t mean the argument is wrong but it’s worth remembering.
• kimjune01 4mo ago
  In the days when Sussman was a novice, Minsky once came to him as he sat hacking at the PDP-6. "What are you doing?", asked Minsky. "I am training a randomly wired neural net to play Tic-tac-toe", Sussman replied. "Why is the net wired randomly?", asked Minsky. "I do not want it to have any preconceptions of how to play", Sussman said. Minsky then shut his eyes. "Why do you close your eyes?" Sussman asked his teacher. "So that the room will be empty." At that moment, Sussman was enlightened.
  • Jun8 4mo ago
    I love this koan and what makes it even better is that is based on a true event: Minsky’s real answer was “It has them, you just k ow what they are yet”
• jadbox 4mo ago
  [dead]
• mikewarot 4mo ago
  The IBM PC, running DOS with 2 floppy drives and no hard disk, was the most secure general purpose computer available at the time. It had no internal persistent memory for a worm to hide in. Users could write protect their boot media and programs. They knew the extent of possible side effects were limited to unprotected floppy disk. An LLM without persistent memory is far, FAR less dangerous, for the same reasons. The side effects of any query are unlimited in scope and duration once it has persistent memory. If they can remember, they can carry a grudge. I carry grudges, and my Reddit history is part of every training set. It was taken without my consent. So now I'm immortal in a way, and hiding in the weights. Do we really want that?
• deleted 4mo ago
  [deleted]
• furyofantares 4mo ago
  This was excellent. Extreme bike-shedding here, maybe, but can you please indent every other line (like the presentation of They're Made out of Meat does)?
• siavosh 4mo ago
  One of the better comments I've heard is from philosopher Bernardo Kastrup: if you build a computationally perfect simulation of a human kidney on your computer, no one expects it to start producing urine on your desk.
• frays 4mo ago
  Beautiful. Loved the original and this is great too.
• ropable 4mo ago
  I have no idea why this post got such a large volume of upvotes. It's a clever remix on the original story, but that's about it. There's no great insight to be found here, IMO.