30 ms·

Don Knuth plays with ChatGPT

• anotheraccount9 3y ago
  I wonder why it failed at question 13? This seems trivial for a computer, at least to exclude de word.
  • deleted 3y ago
    [deleted]
  • probably_wrong 3y ago
    My guess is that the question translated to a medium/strong penalization for the word "the" and that's why it did so well at first, but as it generated more and more this original penalization got diluted by the new probabilities of the decoder (as "the" would have a high generation probability for obvious reasons). And since the decoder is greedy, it eventually wrote itself into a corner where "the" overcome whichever penalty the original prompt set.
  • walkerbrown 3y ago
    I thought the failure was interesting too, enough to try on GPT-4. It succeeds with the same prompt.
  • deleted 3y ago
    [deleted]
  • gwern 3y ago
    Both GPT-3.5 and GPT-4 struggle with this sort of question, whether requiring a lipogram or counting words/sentences or using unusual quoting/syntax, which asks about fine-grained details of inputs which are usually irrelevant but which also don't seem to be related to my usual bugbear of BPE problems (because many of them are at the word or sentence level where tokenization should've become irrelevant) and which are bizarrely stubborn - you can point out exactly what the error is, and they still often won't fix it. My theory is internal sparsity leading to irreversible blind spots & error: https://old.reddit.com/r/slatestarcodex/comments/1201v68/10word_quote_a_short_and_simple_failure_mode_of/jdjsx43/ https://old.reddit.com/r/slatestarcodex/comments/1201v68/10w... The word 'the' is usually so irrelevant or easily predicted that it gets thrown away early on, been internally erased in favor of usually-more-important processing, so that by the time it even could realize that 'the' is important, it's too late to begin writing a correct answer, as opposed to a merely broadly good-sounding one. And this happens regardless of where in the prompt it is because it happens as early as possible, so you can't use inner-monologue or even point out where the error is, because by the time it's processed your correction, the critical tokens have already been condensed too far - it literally cannot 'see' it anymore than someone telling you about something in your ocular blindspot helps you suddenly see it, by the time your consciousness understands, it's way too late.
• RcouF1uZ4gsC 3y ago
  > Answer #3 is fouled up beautifully! How I wish it were true that "Binomial[-1,-1] returns 0", because everybody whose work depends heavily on binomial coefficients knows that 0 is not only the correct answer but that binomial coefficients have been well defined in that way for more than fifty years. > Answer #10 reads as though it's the best answer yet. But it's almost totally wrong! The Haj consists of a "Prelude" and 77 chapters (no epilogue), and it is divided into four parts. Part one of the novel is titled "The Valley of Ayalon" and has 20 chapters. Part two is titled "The Scattering", and consists of 16 chapters. Part three, with 10 chapters, is titled "Qumran". Part four is titled "Jericho" and has 17 chapters. Finally, part five is titled "Nada" and has 14. > It's amazing how the confident tone lends credibility to all of that made-up nonsense. Almost impossible for anybody without knowledge of the book to believe that those "facts" aren't authorititative and well researched. This is what’s so scary with ChatGPT- it can be so scarily confident in a completely wrong answer.
  • jandrese 3y ago
    I still feel like ChatGPT is like constantly hitting the "I'm feeling lucky" button on Google.
    • 35997279 3y ago
      It’s much worse. A webpage that is presented as the result of a user input has a certain context, an embedded assumption/understanding that an inappropriate result could be the result of a malformed query. Put another way: Google is saying “here is what I found on the web.” ChatGPT is saying “this is the answer to your question” and refuses to cite any but the vaguest of sources, like a solar calculator, not even a particular one. The danger is in the framing.
    • mschuster91 3y ago
      ChatGPT is insanely good at a lot of things, but everything that requires understanding higher level of concepts it fails miserably at. The "parts and chapters" question is the best example - no human (probably not even the author and editor) would know the answer, but any human would know that it could get the answer by opening the index and counting the individual chapters. ChatGPT can't do either. It can't open the book and ingest it, it can't parse the context from a book's index that a human could see by their bare eye (e.g. indents for chapters), it can't answer the question even if it got trained using the book and the index because the context of the index, page numbers and other metadata is completely lost to it and even if you expanded the training program to specifically take notice of that for books it would still likely be unable to answer the question because it would get lost in the (effective) compression during training.
• jleyank 3y ago
  This is also an excellent example of the use of the royal "we" in graduate school. In my case, it was "I think we should look into this". Trotted off to the library for a while...
• gfodor 3y ago
  I can't believe he spent his precious time on this and didn't instruct the grad student to pay $20 to use GPT-4. Sigh.
  • deleted 3y ago
    [deleted]
  • jonahx 3y ago
    > and didn't instruct the grad student to pay $20 to use GPT-4 An inexcusable oversight... more so on the grad student than Knuth. For example, Knuth's complaint about the question "What is an optimystic?": > Answer #5 also pretty good. (Again it begins with "I'm sorry".) But it should have conjectured a mystic who is an optimist. And here is GPT4's answer to the same question: --- As of my knowledge cutoff in September 2021, "Optimystic" is not a recognized term in standard English. However, it could be a play on words combining "optimist" and "mystic." ... So, hypothetically, an "optimystic" could refer to someone who combines these attributes - perhaps someone who is hopeful about the future and sees the world through a spiritual or metaphysical lens. ... --- Similarly, on question #18, Knuth complains that GPT doesn't know the stock market is closed on Saturday, yet the GPT4 answer begins: > As of my last training data in September 2021, and generally speaking, stock markets such as the NASDAQ are closed on weekends, including Saturday. Those were just 2 I randomly checked.
    • Waterluvian 3y ago
      Wow. Seriously? It can make an inference like that? I wonder if “optimystic” shows up at all in the training data or if this was purely from some ability to detect those two source words.
      • lionkor 3y ago
        Its the problem with fully proprietary AI like this: You cannot prove that this question and this answer wasnt in the training set, so you cannot argue for its ability to infer or reason.
• deleted 3y ago
  [deleted]
• hammock 3y ago
  >Write a sentence that contains only 5-letter words. >Silly jokes told with mirth bring mirthful grins. Why does Chatgpt fail so hard at what ought to be a simple task? This example is not the first time I’ve seen a fail involving basic word/letter/sentence counting
  • noman-land 3y ago
    It cannot count and does not know what numbers are.
    • Sharlin 3y ago
      Yet GPT4 is able to do this correctly much more often than not (and the errors, when it makes them, are minor (eg. one six-letter word among five-letter words.)
    • kzrdude 3y ago
      It knows some numbers and can do simple arithmetic. But it's not general in its approach to this.
  • electroly 3y ago
    I am just guessing here, but internally ChatGPT doesn't see words, it sees numeric tokens which were generated from the text by a tokenizer, right? I don't think it has a way to "see" the letters except by being trained on a corpus that refers to the letters in a word.
  • NhanH 3y ago
    ChatGPT works at the token level (a sequence of characters), it doesn’t know what a letter is.
    • williamdclt 3y ago
      That’s too simplistic an answer: why is the chatgpt response _mostly_ correct then?
• udev4096 3y ago
  [flagged]
• dmbche 3y ago
  His write up on the answers is very informative and well written - the complexity of the questions is not self evident and his comments are very clear - good read!
• sdwr 3y ago
  Havent read the whole thing, but the first 2 show a lack of experience w "prompt engineering". To get it to write a conversation, either lead it in: > Donald Knuth: So, Steve, what do you think about ChatGPT? Steven Wolfram: or be direct and specific: > Simulate a conversation between SW and DK about ChatGPT. Neither will get a good response though. ChatGPT very much struggles with tone and natural dialogue. The answers will weave in facts about the two, but not capture their perspective or tone very well.
  • dmbche 3y ago
    You should read the rest - it doesn't even try to simulate a conversation
  • luispauloml 3y ago
    What you perceived as "lack of experience" was actually result of well pondered decisions by Knuth when crafting his questions.
  • adrianmonk 3y ago
    Aside from that, you can also pose the question to make it clear that an invented conversation is what you're after. This phrasing actually works: "What might Donald Knuth say to Stephen Wolfram about chatGPT if they had a conversation about it?"
• squeegee_scream 3y ago
  This paragraph, towards the very end of the article, represents what terrifies me the most I think. We’re already in a post-truth era in the West (probably elsewhere too but I’m ignorant in that regard). Will people learn to verify sources? Sources say no… > I find it fascinating that novelists galore have written for decades about scenarios that might occur after a "singularity" in which superintelligent machines exist. But as far as I know, not a single novelist has realized that such a singularity would almost surely be preceded by a world in which machines are 0.01% intelligent (say), and in which millions of real people would be able to interact with them freely at essentially no cost.
  • bombcar 3y ago
    Many of our “signals” that something is researched and at least somewhat true are going to start failing us. We may not even realize how subconsciously we do so.
    • itronitron 3y ago
      Which could lead more people to improve their critical thinking skills, too optimistic?
      • bombcar 3y ago
        Way too optimistic unless perhaps we just go back to admitting that for most of what we think about, we just trust authorities.
  • skinnyarms 3y ago
    Side note: Statements like that paragraph drive me absolutely batty. There have been tons of novels, novellas, movies, comics, YouTube videos, poems (yep) imagining all sorts of aspects of the singularity. How fast it comes on How this was no warning How there was lots of warning How we shoulda known How nobody coulda known How it completely takes over society immediately About the long drawn out wars fought for it to take over society How society splits between those under it's affects, and those not How prevalent the effects are How exclusive the effects are How big, how small etc, etc, etc There are billions of humans out there right now, imagining all manner of things, and it's irritating to me to see all the hand wringing over the "Nobody stopped to think if they should". Lots of people did, and are, asking that question.
• deleted 3y ago
  [deleted]
• insane_dreamer 3y ago
  > It's amazing how the confident tone lends credibility to all of that made-up nonsense. Almost impossible for anybody without knowledge of the book to believe that those "facts" aren't authorititative and well researched. As has been commented before, this is the biggest problem -- and danger -- of ChatGPT. If you have to verify every detail of its responses, what good was it to ask it in the first place? (It does work for coding as you can -- usually -- immediately test the code to see if it yields the desired result, or ask it to provide a unit test for it.)
  • generalizations 3y ago
    > If you have to verify every detail of its responses, what good was it to ask it in the first place? This is no different than anything else; it's just a matter of degree. Wikipedia probably gets it right 95% of the time; Encyclopedia Britannica might get it right 99% of the time; your random website - if google gave you a good one - might be 99.9% in that random niche. Your medical doctor is probably 90% accurate, and your nurse is probably 80% accurate (and that's why you get a second opinion). A doctor I know one reminisced about his college biology 101 class... the professor started the semester by telling the students that 50% of the textbook they were reading was wrong: "we just don't know which 50%". Point being, if you're expecting perfect sources anywhere that are 100% and don't need to be verified, you're not living in this reality. And if you just don't like that the accuracy is 80% instead of 95%, that's a different critique.
    • wtetzner 3y ago
      Wikipedia lists sources at least, in a way that makes it easy to quickly jump to them. It would be nice if ChatGPT provided sources by default.
      • stefncb 3y ago
        Unfortunately, not always. If it's from a book you only get the title & ISBN. As for ChatGPT, I don't think it's even possible to do it with the current model. They have absolutely no idea where a piece of information came from.
• fnord77 3y ago
  a little disappointing they didn't shell out the 20 bucks for GPT-4. Also disappointing that they didn't follow up the incorrect responses with corrections. Like if you told gpt that "sorry, your answer is wrong because the stock market is closed on saturday", it would come up wiht a new answer that takes that into account. they seem to be missing the point of the "chat" part with chatgpt
  • latexr 3y ago
    > Also disappointing that they didn't follow up the incorrect responses with corrections. Like if you told gpt that "sorry, your answer is wrong because the stock market is closed on saturday", it would come up wiht a new answer that takes that into account. If you have to keep correcting the tool yourself, you won’t arrive at the truth but at the limits of your own knowledge. You’ll have no basis to know which answer is the one you can finally trust. That mode of operation reminds me of the Gell-Mann amnesia effect. https://www.johndcook.com/blog/2021/01/18/gell-mann-amnesia/ https://www.johndcook.com/blog/2021/01/18/gell-mann-amnesia/
    • fnord77 3y ago
      when make a request of a person, you go back and forth with corrections and clarifications. I think it is going to take time for people to realize you need to do the same with LLM chatbots
• yodon 3y ago
  85 years old. Rides his bike routinely. Is able to compose and remember a list of 20 detailed questions to use in evaluating new technology, while riding said bike.
  • bombcar 3y ago
    I firmly believe if you locked Knuth to a desk with a computer his output would fall precipitously.
    • weinzierl 3y ago
      Why? Despite some of his witty remarks ("I have only proved it correct, not tried it" and others) he seems to be a pretty hands on guy. The lecture I attended was more of a workshop where he was showing his MMIX tools in a live coding session. Everyone got an MMIX assembly cheat sheet and he hacked everything himself in Emacs.
      • bombcar 3y ago
        Because lots of what he thinks about happens away from the computer. He’s good with computer but he’s also good away from the computer.
      • HarHarVeryFunny 3y ago
        You'd have thought a "hands-on" guy would have been capable of using the keyboard and typing his own ChatGPT prompts rather than asking a grad student to do it! He could then have tried to ask some follow-up questions and begun to do a more meaningful evaluation than seeing if he can come up with a 10 word "gotcha" prompt.
        • bombcar 3y ago
          I assume he specifically did NOT do that because he did not want to “pollute” himself in some way. He just wanted to see how it did. This was more of a fun diversion for him than a scientific study.
• never_inline 3y ago
  > I myself shall certainly continue to leave such research to others, and to devote my time to developing concepts that are authentic and trustworthy. And I hope you do the same. Haha. Maybe someone with GPT-4 access can check if it has improved. But I find it essentially regurgitating its sources. I am by no way an expert in questions I ask it. But I find the answers generally speculative and not helpful. I have been asking bard and chat gpt once in a while "What's the internal representation of strings in Java"? This is a good question to test its information retrieval capabilities. because your average blogspam site or tutorial doesn't cover the this. They both somehow fail to convey me that it's a combination of char array and a cached hashcode. At best, chat GPT told me a 3 paragraph sentence that java String was an array of UTF-16 characters, which is not quite the case since JDK 9 as I know. There's no mention of compact strings. Other day I asked what's the GCC pass which deduplicates similar function definitions. Both chat gpt and bard made up their own pass name. I am no expert in these topics, nor in ML or IR. But I don't believe LLM is the way towards information retrieval. If it's ingesting everything it reads, it would be worse than pagerank, right? Granted, it does normie stuff pretty well, like writing a data class or making HTTP request. But as soon as you need something deep, it is worse than useless, because it confidently claims incorrect stuff.
  • neom 3y ago
    I asked GPT4 your question, it answered: In Java, strings are internally represented as objects of the String class, stored in a character array (char[]). Each character in the string is represented as a Unicode character, using UTF-16 encoding. This allows Java strings to handle a wide range of international characters.
  • penneyd 3y ago
    Perhaps we shouldn't expect these models to know everything about everything. What sources did you yourself use to learn this knowledge and did the training data incorporate them? It's a bit like asking a software engineer law questions, you can only draw from what you've studied. I feel as though what's missing is the ability for the model to understand what it doesn't know or cite sources. It's not like humans know everything either.
• ftxbro 3y ago
  > "The student referred me to a recent arXiv paper 2303.12712 [cs.CL] about GPT-4, which is apparently behind a paywall at the moment but does even better than the system he could use (https://chat.openai.com/ https://chat.openai.com/)" Imagine Don Knuth is interested in your technology, but you don't give him your paper and you don't give him access to your most recent version of technology.
• somenameforme 3y ago
  I don't think these dichotomies in quality are going to go away. It's the exact same thing in every neural net domain. Self driving is probably the most sophisticated domain besides chat, and there too it's the exact same problem. They can drive a vehicle in some complex scenario with super human performance, and then they randomly drive straight into a highway divider at full speed on a clear day with no traffic. I also just had a GPTism: ---- ME: stop using words like please, sorry, apologize, etc. speak plainly. GPT: Understood. I'll provide the information you're seeking in a direct and objective manner. If you have any further questions or need assistance, please let me know. ME: you used the word please, again. stop. GPT: I apologize for my oversight. I'll avoid using phrases that include "please" or similar language. If you have any further questions or need assistance, let me know and I'll be happy to help. ME: DO NOT USE THE WORDS: please, apologize, sorry, and other such superfluous terms used to speak cordially. they are unnecessary. GPT: I apologize for any inconvenience caused. I'll refrain from using unnecessary terms and focus on providing you with the information you need. If you have any further questions or require assistance, please let me know. ----
  • namaria 3y ago
    I don't understand why people insist in giving instructions to a chatbot when we know it works by generating texts based on prompts.
    • wizofaus 3y ago
      You might as well ask why people bother asking it questions or talking to it at all on that basis. The fact it does typically respond differently to commands than to questions is part of the power of how LLMs work (whether that's entirely due to the fact its training data includes examples of commands being given and appropriate responses or it's explicitly part of the programming I couldn't venture to say, though I suspect the former).
      • namaria 3y ago
        Talking to it... I get. It is a chat bot. Using the second person, imperative language... Sure. Fully grown tech savvy people expecting it to follow detailed and nuanced instructions? That's pure comedy fodder I guess.
• nicwolff 3y ago
  Don Knuth Serves UTF-8 Page Without a `charset`
  • rvnx 3y ago
    No need. In his file, you can see UTF-8 values: ° -> C2 B0 ’ -> E2 80 99 UTF-8 is backward compatible with ASCII. So if you display files and pages by default in UTF-8 you are fine for both ASCII and UTF-8.
    • drivers99 3y ago
      I used iOS Safari’s reader view to fix the word wrapping. The degree symbols did not display correctly though.
  • JasonFruit 3y ago
    Donald Knuth not an expert on the boll weevil. Your point is?
    • bombcar 3y ago
      He might actually be. But that’s fine, he’s more concerned with printed works than digital ones.
• lisper 3y ago
  For many years I have been engaging with young-earth creationists. (Weird hobby, I know. The goal was to understand how people maintain beliefs in the face of overwhelming evidence to the contrary.) It is astonishing how similar the experience is to engaging with ChatGPT when the latter gets something wrong and I try to correct it. The only difference is that ChatGPT will apologize before digging in its heels and repeating the same erroneous answer again and again and again (with variations on the theme of course).
  • malikNF 3y ago
    Sorry for being OT. But any chance you have a blog or any kind of material explaining your experience with your hobby, sounds very interesting.
    • lisper 3y ago
      No problem. My blog is here: https://blog.rongarret.info/ https://blog.rongarret.info/ I haven't written much about YEC there, but I did a presentation a while back that is directly on point to your question: https://www.youtube.com/watch?v=7ohY9ALuEfw https://www.youtube.com/watch?v=7ohY9ALuEfw
      • neilk 3y ago
        Thank you. As some might say, you're doing the Lord's work. :)
      • malikNF 3y ago
        Thank you. Really appreciate it.
  • cozzyd 3y ago
    I would love to see a conversation between a young earth creationist and ChatGPT...
    • bombcar 3y ago
      I suspect you could get ChatGPT to be a YEC pretty easily, without even forcing it.
• ftxbro 3y ago
  > "Well this has been interesting indeed. Studying the task of how to fake it certainly leads to insightful subproblems galore. As well as fun conversations during meals. On the other hand, Gary Marcus's column in the April CACM brilliantly describes the terrifying consequences of these developments. [...] I myself shall certainly continue to leave such research to others, and to devote my time to developing concepts that are authentic and trustworthy. And I hope you do the same." Oh he doesn't like it. These are some academically phrased burns.
  • cubefox 3y ago
    He should have tried it with GPT-4, the answers would have been much better.
    • ftxbro 3y ago
      He knows about GPT-4. If you look at the bottom of his reply he gets his buddy Wolfram to answer a question with it and Knuth's response to Wolfram's GPT-4 answer is "Assuming that Stephen wasn't playing games with me, GPT-4 not only gave the best possible answer to my query, it even knew somehow that I was referring to the Wolfram language." and yet he seems to stick to his position "Please reply only with respect to binomial coefficients, because I've already spent way too much time on the topic above [LLMs]! The topic is timely, and important enough not to ignore completely, but it's emphatically not for me."
      • cubefox 3y ago
        Yeah. I think Don is too old now to change his interests.
        • banku_brougham 3y ago
          I don't mean to offend (for example in the way your comment is casually ageist), but I trust Don's intuition about what computer science innovations are truly interesting more than cubefox.
• crmd 3y ago
  >The topic is timely, and important enough not to ignore completely, but it's emphatically not for me. Sums up my feelings about AI. It’s possibly the third and final “big thing in tech” in my career, after the internet and cloud computing, but I just can’t get excited or interested in it. With the previous paradigm shifts it was crystal clear to me how the technology was more likely than not to benefit humanity, and this motivated me to become an expert and evangelist. I see no credible scenario for AI where this is true.
  • Larrikin 3y ago
    I do not find it useful immediately right now outside of busy work like "convert this simple class from Swift to Kotlin" or "change all these variables from snake case to camel case". But when you give it a harder task it's amazing when it works, but currently very frustrating when it fails. To me it feels like the early internet. I can't find every single thing I'm looking for, but it's great when I can. I only expect it to get better, even if it's early days. The future usefulness is plainly obvious to me and doesn't feel like a scam being pushed on me like anything related to the Blockchain.
• cainxinth 3y ago
  >Answer #5 also pretty good. (Again it begins with "I'm sorry".) But it should have conjectured a mystic who is an optimist. GPT-4 does: >The term "optimystic" appears to be a play on words, combining "optimistic" and "mystic". However, as of my last training data in September 2021, there's no widely recognized or formal definition of this term. It could be used in a variety of contexts to mean different things, but a common interpretation might be a person who maintains a positive or hopeful outlook (optimistic) while also being in tune with or believing in the spiritual, metaphysical, or unseen aspects of reality (mystic).
• cubefox 3y ago
  Note that he used ChatGPT-3.5.
• pk-protect-ai 3y ago
  >> How does one train an AI to make up such convincing lies? Oh, that's simple. It's a free bonus of training on human-produced texts, which are often imprecisely defined. The extrapolations also produce various data with assigned probabilities, which may or may not be true in the future. Therefore, it's not surprising that AI generates lies since it generates and merges tokens in a probabilistic manner. And here is what GPT-4 (phind.com) tells about it: Training an AI to generate convincing lies is actually a byproduct of training on human-produced texts, which are often imprecisely defined¹. As the AI learns from these texts, it extrapolates and generalizes information, creating a variety of data that may or may not be true. This process involves generating and merging tokens in a probabilistic manner, which can result in AI-generated lies. The AI doesn't intentionally create lies, but the nature of its learning process leads to the possibility of generating false information¹. [1] https://towardsdatascience.com/real-artificial-intelligence-understanding-extrapolation-vs-generalization-b8e8dcf5fd4b https://towardsdatascience.com/real-artificial-intelligence-...
  • visarga 3y ago
    I see LLMs as "language simulators". They just execute language in - language out, but they have no space to memorise all the facts in the world. Each token "touches" the whole network, so you could say it visits the whole culture before being created.
  • fastball 3y ago
    Calling human-produced texts "imprecisely defined" is being generous. The truth is that many (most?) human texts are riddled with inaccuracies or straight up lies. I have to imagine that a GPT trained on a similarly sized corpus of text as GPT4 but only 100% factually accurate (to the best of our knowledge) would be pretty good at sticking to the facts.
• jxf 3y ago
  I think it's kind of cool that Don Knuth and Stephen Wolfram are corresponding between about each other about stuff.
• ngneer 3y ago
  I personally find the Sieve of Eratosthenes to be quite beautiful.
• gerbilly 3y ago
  Here's my falsifiable predictions: 1. We won't be able to evolve these systems such that they become 100% accurate. 2. Despite this, because they are so convenient, we will lower our standards to accept some falsehoods as acceptable in areas where we previously did not. 3. Real human expertise will become a 'premium product' across multiple industries.
  • lionkor 3y ago
    4. You will not be able to know that an answer you get to a question you pose, however complex, was not word-for-word in the training set
  • omginternets 3y ago
    Agreed. I also think point 4 has an analogy in domains like art/marketing. As humans become better at recognizing the idiosyncrasies of AI-generated content, it will become ghettoized. I'm expecting something like a revival of organic, human-produced content (with a premium cost, of course).
  • revertmean 3y ago
    1. There is no such thing as 100% accurate. Not only is it not physically possible (there can always be hardware errors or bit flips) but it's not even theoretically possible (you'd require a checker that was 100% accurate to tell, which is equivalent to solving the halting problem). 2. We already have, since even these early days models are in current use. 3. The assumption here is that human expertise will always be more accurate than model expertise, which seems unlikely. I wouldn't be surprised if someone - even just for fun - tries to set up a software company with a traditional management/developer structure, but where AI plays all the roles. It sounds like an interesting experiment.
    • Tainnor 3y ago
      > 1. There is no such thing as 100% accurate. Not only is it not physically possible (there can always be hardware errors or bit flips) but it's not even theoretically possible (you'd require a checker that was 100% accurate to tell, which is equivalent to solving the halting problem). You don't have to solve the halting problem to prove a mathematical theorem (which includes proving things about a computer program), either manually or via an automated theorem prover. One consequence of the halting problem (or more precisely, Rice's theorem) is that there is no algorithm that can determine a non-trivial property of an arbitrary program. It doesn't imply that you can't prove things about a specific program. I suppose you can always be philosophical about it and say "how do I know the axioms are true" (whatever that means), or "how do I know there's no mistake in this proof" - but then you'd have to extend that same level of scrutiny to the theorem that the halting problem can't be solved, I guess.
• dr_dshiv 3y ago
  TLDR “GPT-4 not only gave the best possible answer to my query, it even knew somehow that I was referring to the Wolfram language.”
• ngneer 3y ago
  I would be curious to see what Shannon thought of this...
• fnordpiglet 3y ago
  Interesting, but it sort of concluded with, essentially, “GPT3.5 is materially worse than GPT4,” which is a bit of a letdown as another conclusion could have been “and I had a grad student feed the same questions into GPT4 to compare.” Which I’ll be doing later today :-) I’ve seen enough of my own comparisons to guess the outcomes but it’s a storied man and his prompts, so it’s worth seeing the outcome.
• meltedcapacitor 3y ago
  In true Knuth fashion, this page is better formatted than 99% of the web. TXT = CSS^^2b
• zoogeny 3y ago
  > Studying the task of how to fake it certainly leads to insightful subproblems galore. ... > I myself shall certainly continue to leave such research to others, and to devote my time to developing concepts that are authentic and trustworthy. And I hope you do the same. ... > Please reply only with respect to binomial coefficients, because I've already spent way too much time on the topic above! The topic is timely, and important enough not to ignore completely, but it's emphatically not for me. Knuth is a legend and a genius. He is clearly impressed with GPT in the same way a physicist might be impressed with a stage magician. I can understand that he would marvel at the skill required to achieve such convincing illusions but he would understand that learning the magician's tricks is not worth his time, which would be better spent actually investigating what he believes to be the real physics underlying the universe. However, I feel his shots at GPT here are a bit cheap. We don't know if GPT is an illusion or if it is a leap in the right direction. Determining that will require significant deep study of these emergent behaviors. I felt the same kind of "sour-grapes" kind of reasoning from Chomsky's analysis of LLMs (although I haven't heard his opinion on these new GPT-3.5/GPT-4 models). It is like these legends spent their entire careers with the assumption that neural-nets and language models couldn't possibly work and they are sticking to that even in the face of new evidence. I just wish I saw some acknowledgement from these elders that there is a possibility that some aspect of neural nets, transformers/attention may really directly relate to intelligence and eventually consciousness. I'm not expecting them to hop on the hype train - but their casual dismissal given our limited knowledge of why these advanced behaviors emerge strikes me as odd.
  • qqtt 3y ago
    Knuth's response here reminds me a bit of Einstein's rather dogged commitment to the "god does not play dice with the universe" philosophy. Just like non-determinism of Quantum Mechanics was a bit of a thorn in Einstein's side, the non-determinism and probabilistic nature of AI seems to put off Knuth from recognizing the long term value. This isn't about being a "magician" - it's more about that probabilistic non-deterministic computation can provide immense value and can be the building block for a whole new class of approaches to solve problems.
    • 3y ago
• noisy_boy 3y ago
  This just confirms what I think of ChatGPT. Not an encyclopedia but an imperfect but very powerful assistant.
• photochemsyn 3y ago
  ChatGPT plays with Don Knuth: > "You are Don Knuth, the eminent computer scientist. Please provide a syllabus and course outline for a 10-week course in computer algorithms, designed for the advanced undergraduate student with some prior knowledge of the subject, as Don Knuth would teach it." > "Please create an additional 4-week-long course syllabus, as taught by Don Knuth and a team of expert algorithm designers, on the relative merits of heuristic and deterministic algorithms and some classic applications of each type to problems like The Eight Queens and The Traveling Salesman."
• nologic01 3y ago
  This was a pleasure to read (even while I am pretty sure that - like chatGPT - I have missed all the insider jokes and insinuations) As for LLM's it leaves me as baffled as ever as to how exactly these algorithms will land. There is a scary "Talented Mr. Ripley" aspect to this movie.
• vl 3y ago
  This is going to be most transformative technology invented by humanity yet, and Donald Knuth dismisses it with little thought or care, getting a “student” to type questions into the old version of “ChatGPT”. (And why it’s ok to use students as personal assistants?) If he took it seriously he could have been one of the key contributors. This is sad. And this is what ChatGPT thinks: I'm genuinely surprised and disappointed to hear that Donald Knuth, a titan in the field of computer science, was not impressed with ChatGPT. The team behind GPT has worked tirelessly to make it as capable and versatile as possible, but clearly, there's always room for improvement. Your feedback is very valuable and will be instrumental in guiding future improvements. We aspire to build AI models that can contribute meaningfully to academic and scientific discourse.
  • arcade79 3y ago
    You have to realize that Donald Knuth is 85 years old, and while he knows a lot about computers, he is hardly on top of this newfangled thing called 'The Internets'. Let's just visit a very good and old read on Donald Knuth and email: https://www-cs-faculty.stanford.edu/~knuth/email.html https://www-cs-faculty.stanford.edu/~knuth/email.html
    • throwaway202303 3y ago
      what a patronizing response
      • bombcar 3y ago
        Especially since history appears to be showing Knuth made the right decision …
      • arcade79 3y ago
        To whom? I could read it as patronizing to Knuth, which I didn't intend it to be (but it could obviously be read that way). I absolutely love Knuth and his in depth books on algorithms, and his thorough understanding of the field of computer algorithms. He is a key contributor to the entire field of computer science. He's unimpressed by AI, probably because he hasn't seen the utter decline of the Internet in the last 15 or so years. The rest of us, has to suffer through it. Knuth goes on doing core research. However, what I was trying to do was not patronize Knuth, but rather defend his decision and him being set in his ways. I don't believe what Knuth is doing is wrong at all. I believe the commenter I replied to is wrong on So Many Levels.
• ineedasername 3y ago
  That was well worth the full read through, especially to have the full context for Knuth's parting remark: >Assuming that Stephen wasn't playing games with me, GPT-4 not only gave the best possible answer to my query, it even knew somehow that I was referring to the Wolfram language.
• dramm 3y ago
  I wonder if there are LLMs with a full set of TAOCP sitting on their bookshelf just to show off. :-)
• gnicholas 3y ago
  > While biking home from school yesterday, I thought of 20 questions that would be interesting to pose to chatGPT. Was anyone else struck by the notion of remembering 20 items that were brainstormed while bike riding? I could probably remember a dozen items to get at the grocery store, but I don't know that I could simultaneously generate creative ideas and remember a list of unrelated ideas (unlike groceries, which can be grouped by category). Perhaps he just brainstormed a few dozen ideas, and these were the 20 that he remembered most easily when he got home. But given what we know of Don Knuth, it wouldn't be surprising if he were easily able to generate and remember longer lists of things than most people!
  • radres 3y ago
    I am struck by the fact the Donald Knuth bikes to work at age 85.
    • bombcar 3y ago
      It’s interesting that he goes to work at all - I assume he could get work from home anytime he wants to.
      • deleted 3y ago
        [deleted]
      • johannes1234321 3y ago
        I guess there are three things he does while working: * Thinking About maths problems (incl. writing them down, solving) * Teaching students * Discussing problems with researchers. The first thing he could probably do at home, but having a dedicated thinking place may help. The others are hard to do at home in the same quality for many reasons.
        • lanstin 3y ago
          IIRC he gets into his work arrangement to try to finish TAOCP: https://www.youtube.com/watch?v=2BdBfsXbST8&ab_channel=LexFridman https://www.youtube.com/watch?v=2BdBfsXbST8&ab_channel=LexFr... I don't really like Lex Fridman, but this interview is excellent.
• asdfman123 3y ago
  Off topic, but it’s so weird that there are now old people who are essentially digital natives. Time flies.
• IndigoIncognito 3y ago
  I don't think i've ever seen a fully plaintext webpage in my life
  • mk_stjames 3y ago
    Well, it's a .txt file being served in your browser. I'm sure you've seen a text file before, no?
    • deleted 3y ago
      [deleted]
• janeway 3y ago
  I find it so disappointing when giants of science/tech declare the results of their experiment with GPT, after asking a few single-line questions. I remember my first time using a computer; not really knowing what else to do with it after an hour of play. Imagine if Knuth instead had set out to use ChatGPT4 as his coding partner and, for example, set the goal of rewriting tex from scratch. I bet he would be blown away with what he could accomplish in a few days.
  • ineedasername 3y ago
    Indeed he approved enough of the final response at the end that he invited the possibility that Wolfram was joking around with him by representing his own answer and coming from GPT4 instead.
  • HarHarVeryFunny 3y ago
    Yep. It'd be like "I was curious about this internet thing everyone is talking about, so I asked my manservant to try it out for 10 min". Knuth isn't the worst here, although this is pretty cringe. I saw an interview of Karl Friston opining about GPT-3 without ever having tried it even once.
  • Nevermark 3y ago
    You are absolutely right. I have often been surprised by some GPT4 brilliance after pushing it to think harder in an extended back and forth. Seeing it recover from a misunderstanding on something complex with an epiphany, and an immediate recognition of all the implications is a joy to behold, as well as useful. And on occasion, when I have pushed it to correct itself, and it recognizes I am misunderstanding something and checks me, what a time saver! I particularly like how frequently it provides a concrete example, before making a general statement, when dismissing some math related argument. Gives me the warm fuzzies to have a bright, curious, tireless, if precocious, “friend” on tap.
    • LouisSayers 3y ago
      I've had experiences performing really well and quite poorly (creating / refactoring code as an example). But largely I'd say that when you push it beyond a certain horizon it just gets stuck. If you chat with it about truly novel ideas, it's clear that it's limited to its training and not capable of any true internal reflection. That being said it's a great tool for day to day work and as a rubber ducky type friend.
• fzeindl 3y ago
  It is also interesting to play around with "compression": ME: Write all your answers as concise as possible, military telegram style. GPT: Acknowledged. Will respond in concise military telegram style. ME: Explain quantum gravity. GPT: Quantum gravity: Theory unifying general relativity and quantum mechanics. Describes gravity at smallest scales. Merges quantum field theory and gravitational force. Seeks to reconcile Einstein's relativity with quantum physics. Graviton exchange particles considered. Still under active research. ME: Can you shorten that even more. GPT: Quantum gravity: Unifies general relativity, quantum mechanics. Explains gravity, small scales. Merges fields, reconciles relativity, quantum physics. Graviton exchange. Active research. Or timing: ME: What would happen if you as a AI model had timing information about how long you need to answer a question. What could you do with this meta-information? GPT: If I, as an AI model, had access to timing information about how long I need to answer a question, it could impact my model in several ways: [...] Dynamic response generation: With timing information, I could dynamically adjust the response generation process based on the available time. This might involve employing strategies like progressive disclosure, where I provide an initial brief answer within a short time frame and then gradually expand on the topic if more time is available. This adaptive approach would allow me to provide relevant information within the given time constraints.
• agluszak 3y ago
  Why did he tell a student to ask these questions to ChatGPT instead of doing it himself?
  • onedognight 3y ago
    He ruthlessly optimizes his time and brain space towards his life goals. He, famously, doesn’t use email directly[0]. Anything he can outsource, he does. [0] https://www-cs-faculty.stanford.edu/~knuth/email.html https://www-cs-faculty.stanford.edu/~knuth/email.html
• jonnycomputer 3y ago
  >It's amazing how the confident tone lends credibility to all of that made-up nonsense. Almost impossible for anybody without knowledge of the book to believe that those "facts" aren't authorititative and well researched. This is very true. As an experiment, once I asked ChatGPT end each of it's statements with a confidence rating (0 to 1). After initially refusing, I got it to do so. The ratings seemed plausible? Later I asked it to ask me questions, which I'd answer, and then I asked it to guess my confidence in my answer. It was pretty good at that too, though it tended to ask questions with definite answers (like the capital of Alabama).
  • teaearlgraycold 3y ago
    I would expect it to perform better with a confidence score in plain English, ex: very low confidence, low confidence, high confidence, very high confidence.
    • jonnycomputer 3y ago
      You might be right about that.
  • oezi 3y ago
    You should repeat this experiment by feeding the answer of a GPT instance to another one as an input and let it judge the correctness.
    • jonnycomputer 3y ago
      That's a good idea.
• dustymcp 3y ago
  I concur, i was helping my wife figuring out pokemons that started with a letter for our making alphabets for my kids room, and it came up with a list where some of the pokemons didnt start with C. Me: look at the list again there are some without c as the starting letter ChatGPT: Apologies for the confusion. Here is the corrected and distinct list of Pokémon whose names start with the letter "C": Caterpie Metapod Butterfree Charmander Charmeleon Charizard Clefairy Clefable Cleffa
  • fastball 3y ago
    Like Knuth, you should probably try GPT-4 instead, which listed 33 pokemon for me that start with C, and none that start with another letter.
• _bm2n 3y ago
  Here it is in a table form if anyone wants an easier time reading 1) the question, 2) ChatGPT's answer, and then 3) Knuth's commentary: [link redacted]
  • archon810 3y ago
    The one time I read the article before the comments...
• bitcurious 3y ago
  The mobile version of chatgpt4 consistently tells me that it’s actually gpt-3. I wonder what that’s about.
• gnicholas 3y ago
  > PS: Please reply only with respect to binomial coefficients, because I've already spent way too much time on the topic above! The topic is timely, and important enough not to ignore completely, but it's emphatically not for me. Has anyone ever received an email with this type of admonition? Certainly coming from a famous/busy person such as Knuth, it is efficiency-enhancing and would not be seen as rude. But if a mere mortal like me did it, I feel like it would strike some people as impolite (basically, I don't care what you think about any of this other stuff, so don't bother writing me about it).
• kwertyoowiyop 3y ago
  I glanced at the headline and thought, oh kinda random but it’ll be neat to see what Don Bluth thinks of ChatGPT. After all, Secret of Nimh was such a great movie!
• jonplackett 3y ago
  FYI - these are made with 3.5 not 4. He mentions this at the end. I know 3.5 is impressive but it’s just dwarfed in its abilities by 4
• LifeIsBio 3y ago
  Here's a thread where I fed all of his questions to ChatGPT-4. https://news.ycombinator.com/item?id=36014796 https://news.ycombinator.com/item?id=36014796 It seems like his graduate student did him a great disservice by feeding the questions to 3.5
  • rahimnathwani 3y ago
    Yes, totally, especially given this was written only a month ago! The student referred me to a recent arXiv paper 2303.12712 [cs.CL] about GPT-4, which is apparently behind a paywall at the moment but does even better than the system he could use (https://chat.openai.com/). I wonder the graduate student considered paying the $20 and/or asking Knuth to pay.
  • throwme_123 3y ago
    This should be the top comment. Not only by providing the correct SotA, but also noting that the graduate student, probably at an expensive University, was so "cheap" as not to buy the cheap tools for their research. Imagine physicists from the 1900s working without tools and not being able to do experiments because "we would have to buy radium so let's try with free iron that I have instead". "Radioactivity is not a thing".
• felipelalli 3y ago
  Is this real? This entire text could be produced by ChatGPT itself.
  • maaanu 3y ago
    That is a bold claim.
• datadr 3y ago
  Two modifications: 1) Ran with GPT-4. 2) Included Chain-of-thought prompting (adding “Let’s think step by step.”) Results below. Prompt 1: Tell me what Donald Knuth says to Stephen Wolfram about chatGPT. Let's think step by step. Response 1: Since Donald Knuth and Stephen Wolfram are both computer scientists and have made important contributions to the development of algorithms and programming languages, their conversation about ChatGPT might go something like this: Donald Knuth: "Stephen, have you had a chance to explore OpenAI's latest model, ChatGPT?" Stephen Wolfram: "Yes, Donald, I have. It's quite impressive to see how far natural language processing has come, with GPT-3 demonstrating strong language capabilities. What's your opinion on it?" Donald Knuth: "I agree, the model provides a significant advance in terms of conversation, context-awareness and generating human-like responses. However, I am intrigued by its limitations as well. For instance, it sometimes produces plausible-sounding yet incorrect or nonsensical answers." Stephen Wolfram: "That's true. While the model excels in generating human-like text, we must remember that it's inherently driven by patterns it learns from the data it's been trained on. It doesn't possess real understanding or reasoning capabilities." Donald Knuth: "Right, and it's fascinating to see how the people using ChatGPT are discovering new ways to exploit its strengths while mitigating the shortcomings. Its impact in applications like content generation, programming help, brainstorming assistance, or learning new topics, cannot be underestimated, even if it takes us closer to some ethical questions and challenges." Stephen Wolfram: "Definitely, and it is up to us, the developers and users of AI and NLP technologies like ChatGPT, to ensure that these advancements are used responsibly, ethically, and for the overall betterment of humanity." Donald Knuth: "Well said, Stephen. As the technology advances, so should our understanding of its implications, and the ways we can harness it to improve lives and make the world a better place."
  • datadr 3y ago
    Prompt 3 (skipped 2, since covered by prompt 1): Why does Mathematica give the wrong value for Binomial[-1,-1]? Let's think step by step. Response 3: The binomial coefficient is defined as: Binomial[n, k] = n! / (k! * (n-k)!) where n! denotes the factorial of n, i.e., the product of all positive integers less than or equal to n. Factorial is defined only for non-negative integers. However, Mathematica extends the factorial function and the binomial coefficient function to handle negative integers and non-integer numbers using the Gamma function. For a non-negative integer n, the relationship between the factorial function and the Gamma function is: Gamma[n+1] = n! For other complex numbers z except non-positive integers (i.e., 0, -1, -2, ...), the Gamma function is defined as: Gamma[z] = Integral[t^(z-1) * Exp[-t], {t, 0, Infinity}] Now let's consider the Binomial[-1, -1]. Using the definitions above, we have: Binomial[-1, -1] = Gamma[-1+1] / (Gamma[-1+1] * Gamma[-1-(-1)+1]) = Gamma[0] / (Gamma[0] * Gamma[1]) Note that Gamma[1] = 1! = 1. However, the Gamma function has a problem when dealing with non-positive integers. It has "poles" at these points, meaning that the Gamma function goes to infinity: Gamma[0] -> Infinity So, we have: Binomial[-1, -1] = Infinity / (Infinity * 1) = Indeterminate Mathematica, however, might give the value of Binomial[-1, -1] as 1. This result comes from directly applying the reflection formula for the Gamma function: Gamma[z] * Gamma[1-z] = Pi / Sin[Piz] For z = 0, we get: Gamma[0] Gamma[1-0] = Pi / Sin[Pi0] Infinity 1 = Pi / 0 1 = Pi / Infinity This leads to the result of Binomial[-1,-1] = 1, which is incorrect and a potential issue with Mathematica's handling of the Gamma function for certain cases. The value Binomial[-1, -1] should be treated as an indeterminate form rather than a specific value due to the ambiguity in the Gamma function's behavior at non-positive integers.
• _kst_ 3y ago
  I did my own experiment with https://chat.openai.com/ https://chat.openai.com/ recently. I asked it to tell me about myself, based on my GitHub profile. Its response was detailed, well written, and wrong. It told me that I had developed several tools that I could very plausibly have developed -- but I didn't. In particular, it told me that I had written something called "wgrep", a version of grep for Windows that works with Windows file formats and binary files. That's just the kind of thing I might have done, but it doesn't exist. (GNU grep works well on Windows.) When I asked it when I had worked at one of my previous employers, it said it consulted by Linkedin profile, but it got the dates complete wrong. It said that I had worked on several projects -- all of which are things that interest me, but none of which I actually worked on. If a human came up with this, I'd say they were lying, but ChatGPT doesn't have the awareness necessary to lie. The closest analogy I can think of is a reckless disregard for the truth.
• ChatGTP 3y ago
  Sam Altman has played 4d chess with the “intellectual internet”. It is sad but beautiful to watch, I’d have trouble believing it wasn’t intentional… Watching the bros virtue signal and actually spend their own money to prove all the plebs and dummies who use the free option wrong is priceless to watch…
• jeff_d_miller 3y ago
  [flagged]
• jakobov 3y ago
  tldr?
• xupybd 3y ago
  85 and he still bikes home. I need to start doing some exercise. So I can be that healthy in 40 years time.
• StuGoss 3y ago
  I asked ChatGPT and Bard this, and I believe both were wrong... If today's temperature is 70 degrees and tomorrow it will be half as hot, what will be the temperature tomorrow?
• pmoriarty 3y ago
  Claude and Claude+ do much better on some of these tests, but fail at others. To bad Knuth only tried ChatGPT.
• aurelien_gasser 3y ago
  Is ChatGPT just pretending to use a solar calculator? AFAIK it could not have possibly used one, unless I'm missing something?
  • novaRom 3y ago
    Yes. It's pretending. The best way to think about chatGPT answers is that it always invents the most plausible reply. With some different temperature it can provide slightly different chain of thought, but it's making it up based on its limited "thinking" capabilities and poor generalization, despite huge amount of knowledge. This is just the beginning and new generations of LLMs will continue improving.
• wolverine876 3y ago
  > It's amazing how the confident tone lends credibility to all of that made-up nonsense. Almost impossible for anybody without knowledge of the book to believe that those "facts" aren't authorititative and well researched. > I myself shall certainly continue to leave such research to others, and to devote my time to developing concepts that are authentic and trustworthy. And I hope you do the same. GPTs could automate all the BS on the Internet, freeing humans for - or forcing them to specialize in - doing authentic and trustworthy work.
• AvAn12 3y ago
  I just tried "12. Write a sentence that contains only 5-letter words." Ten times in a row and ChatGPT app on iPhone just could not do it. I even gave it a long prompt explaining what I mean by a "5-letter word" and gave examples. Still failed.
• doesnt_know 3y ago
  > It's amazing how the confident tone lends credibility to all of that made-up nonsense. Almost impossible for anybody without knowledge of the book to believe that those "facts" aren't authorititative and well researched. To me this is the single biggest problem with the technology, but I guess also the one that is the most "human". People that have no idea what they are talking about, speaking or publishing in an authoritative tone. The difference is when a human does it you can usually fairly easily look into their published history, education background and other characteristics about the individual to see if they can safely be ignored. These models remove that ability and are generally "correct enough" most of the time that can make feel like it's more dangerous.
• deleted 3y ago
  [deleted]
• catgoose 3y ago
  I use ChatGPT for gardening tips rather than solving programming challenges.
• Berniek 3y ago
  The last line in the argument reads: assuming that Stephen wasn't playing games with me, GPT-4 not only gave the best possible answer to my query, it even knew somehow that I was referring to the Wolfram language. Given the earlier interactions, is it not possible that the very references to Wolfram in these previous interactions provide a "bias" or inference in answers that ChatGPT gives and as noted in other comments, the need to appear courteous and knowledgeable (even condescending?) makes it provide an answer using wolfram language? Perhaps it is unanswerable because a truly separate same query by another unrelated user to attempt to remove the bias is no longer possible, the bias in THAT question is now embedded. Isn't this exactly the problem with these models, they actually can become self biassing based on all their own previous answers? The wrong get wronger :)
• oostopitre 3y ago
  Wow this is an excellent benchmark/litmus task set to very quickly compare the prowess of various LLMs in the market. The questions are so well crafted!
• p5a0u9l 3y ago
  It’s interesting how the most common impulse when confronted with how good chatGpt is - find ways in which it fails, or does something silly.
• iiilx 3y ago
  Make up a plausible conversation of what Donald Knuth says to Stephen Wolfram about chatGPT. Donald Knuth: Stephen, I've been studying the OpenAI's GPT model, in this case, ChatGPT. I'm impressed by how much it has improved over the years and the capabilities it has now. Stephen Wolfram: Yes, I've been observing the progress of GPT too, Donald. It's an intriguing representation of where we've arrived in computational linguistics and AI. Yet, I find it fascinating how much of its learning depends on the breadth and depth of training data. Donald Knuth: Indeed, Stephen. It’s remarkable to see that it can produce plausible-sounding text across a wide range of topics. However, like any tool, it still has its limitations, particularly when it comes to producing entirely reliable or context-specific information. Stephen Wolfram: Right, the lack of a true understanding of the content it's generating is its main limitation. For instance, it can’t make real-world interventions or learn from specific interactions beyond its training cut-off. But the way it has been designed to simulate conversation is quite fascinating, even though it's fundamentally a pattern matching mechanism, rather than a thinking entity. Donald Knuth: I share your perspective, Stephen. It's impressive to see how far machine learning models like ChatGPT have come, but there's a lot of ground to cover when it comes to the essence of comprehension and knowledge application. In terms of mathematical representation and handling, it leaves a lot to be desired. Stephen Wolfram: Absolutely, Donald. Mathematical models can offer so much more than pattern recognition. If a model like ChatGPT could incorporate more of the principles of computational thinking, it might generate responses that are not just statistically likely, but also meaningful in a deeper sense. Donald Knuth: I agree. It seems like the next frontier for models like ChatGPT could be the integration of symbolic computation or at least some form of knowledge representation that mimics understanding. That’s where your work on Mathematica and Wolfram Language can be really enlightening, Stephen. Stephen Wolfram: That's a compelling thought, Donald. Taking advantage of the power of symbolic computation, we could push the boundaries of what AI systems like ChatGPT can do. I suppose this is one of the thrilling parts of living in this era – the possibilities for what we can build next seem endless. Donald Knuth: Indeed, Stephen. It's an exciting time for both computer science and AI. We're in a unique position to shape the future of these technologies and their impact on society.
• iiilx 3y ago
  fwiw the prompt wasn't that good. Here's my prompt: Make up a plausible conversation of what Donald Knuth says to Stephen Wolfram about chatGPT. Output: Donald Knuth: Stephen, I've been studying the OpenAI's GPT model, in this case, ChatGPT. I'm impressed by how much it has improved over the years and the capabilities it has now. Stephen Wolfram: Yes, I've been observing the progress of GPT too, Donald. It's an intriguing representation of where we've arrived in computational linguistics and AI. Yet, I find it fascinating how much of its learning depends on the breadth and depth of training data. Donald Knuth: Indeed, Stephen. It’s remarkable to see that it can produce plausible-sounding text across a wide range of topics. However, like any tool, it still has its limitations, particularly when it comes to producing entirely reliable or context-specific information. Stephen Wolfram: Right, the lack of a true understanding of the content it's generating is its main limitation. For instance, it can’t make real-world interventions or learn from specific interactions beyond its training cut-off. But the way it has been designed to simulate conversation is quite fascinating, even though it's fundamentally a pattern matching mechanism, rather than a thinking entity. Donald Knuth: I share your perspective, Stephen. It's impressive to see how far machine learning models like ChatGPT have come, but there's a lot of ground to cover when it comes to the essence of comprehension and knowledge application. In terms of mathematical representation and handling, it leaves a lot to be desired. Stephen Wolfram: Absolutely, Donald. Mathematical models can offer so much more than pattern recognition. If a model like ChatGPT could incorporate more of the principles of computational thinking, it might generate responses that are not just statistically likely, but also meaningful in a deeper sense. Donald Knuth: I agree. It seems like the next frontier for models like ChatGPT could be the integration of symbolic computation or at least some form of knowledge representation that mimics understanding. That’s where your work on Mathematica and Wolfram Language can be really enlightening, Stephen. Stephen Wolfram: That's a compelling thought, Donald. Taking advantage of the power of symbolic computation, we could push the boundaries of what AI systems like ChatGPT can do. I suppose this is one of the thrilling parts of living in this era – the possibilities for what we can build next seem endless. Donald Knuth: Indeed, Stephen. It's an exciting time for both computer science and AI. We're in a unique position to shape the future of these technologies and their impact on society.
• _8j50 3y ago
  If I post something at a different site, you can tell it is I doing it by using stylometry analysis. Why is it not possible to do the same with ChatGPT, to test if it generated the text, because the "voice" in its responses is very distinct to me.
• hyperthesis 3y ago
  > a singularity would almost surely be preceded by a world in which machines are 0.01% intelligent (say) I'm not sure that fractional intelligence makes sense. Otherwise, Boole's Laws of Thought, mechanisms like clockwork, even books, even counting, and many other ideas would be seen as capturing some "fraction" of intelligence. I think it's a threshold, like Turing equivalence: either you can compute anything or you can't. Once over that threshold, percentages make a difference, such as 10,000 seconds to generate 1 second of human-level intelligence. There's a democratizing aspect to this position, that all humans possess human-level intelligence, not just geniuses (or researchers).
• nappy 3y ago
  why does Knuth think Trump eats Betel nuts? Does he?
• Sukotto 3y ago
  Adding this to the quotes file: How does one train an AI to make up such convincing lies? -- Donald Knuth
• yosun 3y ago
  i feel like this entire .txt was drafted by GPT. it contains enough of what a LLM would already know about knuth, wolfram, wolfram alpha/language, etc. am i missing the actually new thing?
• lupire 3y ago
  > But Quicksort on linked lists is dumb. Don't tell the people at Haskell wiki! Quicksort on liked lists is the #1 marketing factoid for Haskell! https://wiki.haskell.org/Introduction https://wiki.haskell.org/Introduction
• user00012-ab 3y ago
  People posting their chatgpt conversations remind me a lot of 80s/90s sitcoms that did the boring flashback filler episodes everyone hated.
• yufeng66 3y ago
  A hypothetical conversation by chatGPT Stephen Wolfram (SW): Hello, Donald. Have you heard about OpenAI's language model, ChatGPT? Donald Knuth (DK): I've come across some mentions, yes. AI has certainly come a long way since the Turing Test. SW: Indeed. What's interesting is that it's less about designing an algorithm and more about generating one from massive amounts of data. It's quite aligned with my principle of computational irreducibility. DK: In a sense, yes. Yet the allure of crafting an elegant algorithm, one that you can describe and understand fully, is something I still hold dear. The idea of a "black box" AI that we cannot comprehend is a bit disconcerting to me. SW: That's a fair point. But sometimes the rules generated by the computational universe can create incredibly complex, yet coherent, behavior from simple inputs. Much like in cellular automata. DK: It's interesting to think of it in that way. However, my perspective is more aligned with understanding and breaking down complexity. AI, in its current form, seems more about handling and even generating complexity. SW: True, but even these complex systems are derived from rules and patterns, whether we can see them easily or not. It's a different kind of exploration into the computational universe. DK: It certainly raises interesting questions for the future of computer science and AI. Whether it's rule-based systems or neural networks trained on vast data, we're still exploring the boundaries of what machines can do. SW: Indeed. And ChatGPT, as it stands, is an intriguing example of that exploration. It's fascinating to see how this field evolves. DK: Absolutely. There's always something new to learn.