quantadev

53 ms·

1. 871.▲
   by quantadev 2y ago

   Consciousness/qualia actually correlates stronger to brain waves than to synaptic activity, and for that and a lot of other reasons (that I'm presently writing a paper on) I think consciousness is a phenomena made up of waves, and
2. 872.▲
   by quantadev 2y ago

   I don't like the analogy of "compression" that much, because for example if you train a model to predict linear data points, ideally it will only end up knowing two numbers in it's model weights when it's done train
3. 873.▲
   by quantadev 2y ago

   I said googolplex facts can be "derived". I didn't say they're stored. Also a single perceptron can answer an infinite number of questions. Here's how... For example the knowledge of how to answer the linear equatio
4. 874.▲
   by quantadev 2y ago

   I always assume some people might define "thinking" as requiring "consciousness", because up until about 2022 pretty much every human on the planet did.
5. 875.▲
   by quantadev 2y ago

   When people say "If it was reasoning, then it would be able to know, How many Rs does the word Strawberry have in it?", but that's not quite right, but I would say this instead "If it was reasoning THE SAME WAY HUMANS re
6. 876.▲
   by quantadev 2y ago

   The "reasoning" is an emergent property that no one understands yet. Yes we do all the training only to try to predict the next word (i.e. train to do word prediction), yet with enough training data, then at some scale (GPT 3.5ish
7. 877.▲
   by quantadev 2y ago

   I think the phrase "stochastic parrot" is misleading and has fooled millions of people into thinking that LLMs can't do genuine reasoning about situations they've never seen before, nor been trained on, which is wrong be
8. 878.▲
   by quantadev 2y ago

   You're right. Humans have short term memory, so we can genuinely recall our reasoning, but at the same time, once an action (or even a thought) has been had, we begin weaving a new narrative about why we did that thing or had that thou
9. 879.▲
   by quantadev 2y ago

   I didn't use the word "think" in the human sense of the word, or even in the consciousness/qualia sense of the word. I of course only meant inference (Matrix multiplications) What I like to say however is that LLMs are d
10. 880.▲
    by quantadev 2y ago

    LLMs can't explain their past behavior because any time you ask them to, all they're really doing is reading the previous context from a previous inference and looking for any reason they can think of that they might have said tha