3 ms·

This and parent are both approaching toward what I see as the main obstacle, that we as a species don't know how in its entirety a human mind thinks (and it var

This and parent are both approaching toward what I see as the main obstacle, that we as a species don't know how in its entirety a human mind thinks (and it varies among people), so trying to "model" it and reproduce it is reduced to a game of black-boxing. We black box the mind in terms of what situations its been seen to be in and how it has performed, the millions of correlative inputs/outputs are the training data. Yet, since we don't know the fullness of the interior we can only see its outputs it becomes somewhat of a Plato's cave situation. We believe it 'thinks' this way but again we cannot empirically say it performed a task a certain way, so unlike most other engineering problems, we are grasping at straws while trying to reconstruct it. This doesn't not mean that a human mind's inner-workings can't ever be %100 reproduced, but not until we know it further.

tempodox 1y ago
And there is another important difference: Our environments have oodles of details that inform us, while LLM training data is just “everything humans have ever written”. Those are completely different things. And LLMs have no concept of facts, only statements about facts in their training data that may or may not be true.
BobbyTables2 1y ago
It’s kinda interesting that a simple generator that probabilistically generates a word based on the previous one (less than 100 lines of simple python code), when trained on a book or two, will fairly consistently capitalize the first word after a period and form somewhat reasonable sentences. It’s not that it knows grammar, it just was trained on a dataset that applied proper capitalization. Humans learn from seeing patterns. I suspect AI only repeats them, more like a parrot.
- kragen 1y ago
  I share your concern about the Eliza effect. However, the kind of first-order Markov-chain model you're describing does not form "somewhat reasonable" sentences, except very rarely. A second-order Markov model seems to do much better at first, but that's because it's copying big chunks out of its training set. When the training set gets big enough, it stops doing that. Here's what the first-order model you described looks like trained on Moby Dick and the King James Bible: > The Project Gutenberg-tm License terms of God, into a prince of the last days, and what things that he had an altar of the camp of the rest content, I laughed us into the sand which were upon the hope and made a snare shall continually flitting through their pains ye offer sacrifice. 50:6 And he saw that the forest of the threshingfloors. 23:2 Son of the fire, and came to anger of Israel. 48:20 And they cannot survive without money. 5:20 Despise not on fire. 1:5 And when he healed of legerdemain in the pitiless jaw; spilled it a chain work, and blood upon by letters. 10:10 And the chief of the Antothite. 12:4 And he hath given them he sprinkled her that the LORD, and hold of Bigthana and hill in the patterns of man, is your God. 3:4 And at the sword. 8:25 And the priest: and cast lots. 23:35 Hezrai the other rings of them competent to, these coffin-canoes were departed from the lower jaw; you to a vast bodies have I will not of which sat still in controversy and my people go. 16:7 And the pledge again with sea-water; which is to the glory, and make ye say, and carried to his disciples said unto the Philistines, The only grammatical sentence in the bunch is "And they cannot survive without money." Trained on my reproducible corpus of 49 megs of RFCs, I get: > RFC 1213, defines a prototype algorithm, since the remote networks to have keys as during a standard is used by the 16-bit words, our first fragment. While designing X.500 and a CR NUL ::= "Community: " {<time> "-" (also known to that databases are not contain the causes the Internet community name and especially since in the data element of decrease in a few protocols are specified in their experimentation. 6. REFERENCES [1] is given in which point several kinds of the destination file is a request arrives, the abstract information present state of cfdpcln and error 38 0[0] Message Indication Not supported. If the textual reference shall be of 10.2.0.0 path down the interface's Area Networks Graphics - Size: The meanings for publication date information Agreements on the SNMPv2 entity and eases the header in datagram shortest-path trees are the Digest Authentication Value ----- ----------- ---------- Receive a list of this command to construct a sequence number of an error in the end-system to unblock the basic monitoring map well, and found and root dispersion <$Ephi tau>, where that would be sent by the array of This contains zero grammatical sentences. Here's the source code, so you can see if I fucked up the algorithm: #!/usr/bin/python import random, sys model, last = {}, None for line in sys.stdin: for word in line.split(): model.setdefault(last, []).append(word) last = word words = list(model.keys()) while True: last = random.choice(model.get(last) or words) print(last, end=' ') (11 lines of code is indeed "less than 100," but maybe you had in mind a more memory-efficient implementation.) I also suspect that current AI only repeats patterns without understanding them, as you evidently did in your comment about Markov-chain models, and as people evidently do almost all of the time. But I know I don't know, as your earlier comments claim to. Moreover, I think it's obviously foolish to extrapolate from my current experience with current LLMs to the ultimate limits of the Transformer architecture, much less whatever DeepSeek and Anthropic come up with three years from now. I haven't even implemented a Transformer! Have you? I may seem a little harsh in this comment, but I think it's important that you stop claiming to know things you don't actually know.

tempodox 1y ago

And there is another important difference: Our environments have oodles of details that inform us, while LLM training data is just “everything humans have ever written”. Those are completely different things. And LLMs have no concept of facts, only statements about facts in their training data that may or may not be true.

BobbyTables2 1y ago

It’s kinda interesting that a simple generator that probabilistically generates a word based on the previous one (less than 100 lines of simple python code), when trained on a book or two, will fairly consistently capitalize the first word after a period and form somewhat reasonable sentences. It’s not that it knows grammar, it just was trained on a dataset that applied proper capitalization. Humans learn from seeing patterns. I suspect AI only repeats them, more like a parrot.

kragen 1y ago
I share your concern about the Eliza effect. However, the kind of first-order Markov-chain model you're describing does not form "somewhat reasonable" sentences, except very rarely. A second-order Markov model seems to do much better at first, but that's because it's copying big chunks out of its training set. When the training set gets big enough, it stops doing that. Here's what the first-order model you described looks like trained on Moby Dick and the King James Bible: > The Project Gutenberg-tm License terms of God, into a prince of the last days, and what things that he had an altar of the camp of the rest content, I laughed us into the sand which were upon the hope and made a snare shall continually flitting through their pains ye offer sacrifice. 50:6 And he saw that the forest of the threshingfloors. 23:2 Son of the fire, and came to anger of Israel. 48:20 And they cannot survive without money. 5:20 Despise not on fire. 1:5 And when he healed of legerdemain in the pitiless jaw; spilled it a chain work, and blood upon by letters. 10:10 And the chief of the Antothite. 12:4 And he hath given them he sprinkled her that the LORD, and hold of Bigthana and hill in the patterns of man, is your God. 3:4 And at the sword. 8:25 And the priest: and cast lots. 23:35 Hezrai the other rings of them competent to, these coffin-canoes were departed from the lower jaw; you to a vast bodies have I will not of which sat still in controversy and my people go. 16:7 And the pledge again with sea-water; which is to the glory, and make ye say, and carried to his disciples said unto the Philistines, The only grammatical sentence in the bunch is "And they cannot survive without money." Trained on my reproducible corpus of 49 megs of RFCs, I get: > RFC 1213, defines a prototype algorithm, since the remote networks to have keys as during a standard is used by the 16-bit words, our first fragment. While designing X.500 and a CR NUL ::= "Community: " {<time> "-" (also known to that databases are not contain the causes the Internet community name and especially since in the data element of decrease in a few protocols are specified in their experimentation. 6. REFERENCES [1] is given in which point several kinds of the destination file is a request arrives, the abstract information present state of cfdpcln and error 38 0[0] Message Indication Not supported. If the textual reference shall be of 10.2.0.0 path down the interface's Area Networks Graphics - Size: The meanings for publication date information Agreements on the SNMPv2 entity and eases the header in datagram shortest-path trees are the Digest Authentication Value ----- ----------- ---------- Receive a list of this command to construct a sequence number of an error in the end-system to unblock the basic monitoring map well, and found and root dispersion <$Ephi tau>, where that would be sent by the array of This contains zero grammatical sentences. Here's the source code, so you can see if I fucked up the algorithm: #!/usr/bin/python import random, sys model, last = {}, None for line in sys.stdin: for word in line.split(): model.setdefault(last, []).append(word) last = word words = list(model.keys()) while True: last = random.choice(model.get(last) or words) print(last, end=' ') (11 lines of code is indeed "less than 100," but maybe you had in mind a more memory-efficient implementation.) I also suspect that current AI only repeats patterns without understanding them, as you evidently did in your comment about Markov-chain models, and as people evidently do almost all of the time. But I know I don't know, as your earlier comments claim to. Moreover, I think it's obviously foolish to extrapolate from my current experience with current LLMs to the ultimate limits of the Transformer architecture, much less whatever DeepSeek and Anthropic come up with three years from now. I haven't even implemented a Transformer! Have you? I may seem a little harsh in this comment, but I think it's important that you stop claiming to know things you don't actually know.