I share your concern about the Eliza effect. However, the kind of first-order Markov-chain model you're describing does not form "somewhat reasonable" sentence

I share your concern about the Eliza effect. However, the kind of first-order Markov-chain model you're describing does not form "somewhat reasonable" sentences, except very rarely. A second-order Markov model seems to do much better at first, but that's because it's copying big chunks out of its training set. When the training set gets big enough, it stops doing that. Here's what the first-order model you described looks like trained on Moby Dick and the King James Bible: > The Project Gutenberg-tm License terms of God, into a prince of the last days, and what things that he had an altar of the camp of the rest content, I laughed us into the sand which were upon the hope and made a snare shall continually flitting through their pains ye offer sacrifice. 50:6 And he saw that the forest of the threshingfloors. 23:2 Son of the fire, and came to anger of Israel. 48:20 And they cannot survive without money. 5:20 Despise not on fire. 1:5 And when he healed of legerdemain in the pitiless jaw; spilled it a chain work, and blood upon by letters. 10:10 And the chief of the Antothite. 12:4 And he hath given them he sprinkled her that the LORD, and hold of Bigthana and hill in the patterns of man, is your God. 3:4 And at the sword. 8:25 And the priest: and cast lots. 23:35 Hezrai the other rings of them competent to, these coffin-canoes were departed from the lower jaw; you to a vast bodies have I will not of which sat still in controversy and my people go. 16:7 And the pledge again with sea-water; which is to the glory, and make ye say, and carried to his disciples said unto the Philistines, The only grammatical sentence in the bunch is "And they cannot survive without money." Trained on my reproducible corpus of 49 megs of RFCs, I get: > RFC 1213, defines a prototype algorithm, since the remote networks to have keys as during a standard is used by the 16-bit words, our first fragment. While designing X.500 and a CR NUL ::= "Community: " {<time> "-" (also known to that databases are not contain the causes the Internet community name and especially since in the data element of decrease in a few protocols are specified in their experimentation. 6. REFERENCES [1] is given in which point several kinds of the destination file is a request arrives, the abstract information present state of cfdpcln and error 38 0[0] Message Indication Not supported. If the textual reference shall be of 10.2.0.0 path down the interface's Area Networks Graphics - Size: The meanings for publication date information Agreements on the SNMPv2 entity and eases the header in datagram shortest-path trees are the Digest Authentication Value ----- ----------- ---------- Receive a list of this command to construct a sequence number of an error in the end-system to unblock the basic monitoring map well, and found and root dispersion <$Ephi tau>, where that would be sent by the array of This contains zero grammatical sentences. Here's the source code, so you can see if I fucked up the algorithm: #!/usr/bin/python import random, sys model, last = {}, None for line in sys.stdin: for word in line.split(): model.setdefault(last, []).append(word) last = word words = list(model.keys()) while True: last = random.choice(model.get(last) or words) print(last, end=' ') (11 lines of code is indeed "less than 100," but maybe you had in mind a more memory-efficient implementation.) I also suspect that current AI only repeats patterns without understanding them, as you evidently did in your comment about Markov-chain models, and as people evidently do almost all of the time. But I know I don't know, as your earlier comments claim to. Moreover, I think it's obviously foolish to extrapolate from my current experience with current LLMs to the ultimate limits of the Transformer architecture, much less whatever DeepSeek and Anthropic come up with three years from now. I haven't even implemented a Transformer! Have you? I may seem a little harsh in this comment, but I think it's important that you stop claiming to know things you don't actually know.