11 ms·

GPT-4 Is a Reasoning Engine

39 points by itissid 4y ago

deleted 4y ago
[deleted]
andsoitis 4y ago
> who is dan shipper? hallucinates > who is dan shipper? please search if you don't know the answer. gives right answer If it is a reasoning engine, why didn't it answer "I don't know" in the first case?
- kvn8888 4y ago
  I'm guessing it doesn't understand the difference between what it knows, and what it doesn't know.
  - actionfromafar 4y ago
    It doesn't know the difference, because it doesn't know what knowing and not knowing is.
    - Koshkin 4y ago
      It depends on what the meaning of the word ‘is’ is.
  - dexwiz 4y ago
    That seems to be my experience. Sometimes it gives great answers, sometimes terrible, but it rarely gives any indication between the two.
    - Sharlin 4y ago
      That seems to be largely an effect of its training. I'm pretty sure that OpenAI has just biased it to be too helpful, leading it to confabulate answers. If you specifically prompt it to tell you when it doesn't know, or isn't sure, it should fare quite a bit better. Also, GPT-4 is a huge improvement.
  - capableweb 4y ago
    > You are a AI for describing people you know about. If you're not 100% certain about the information you give to the user, you should say that you don't know instead of guessing. Greet the user with "Hello, who do you want information about?" > Hello, who do you want information about? > who is dan shipper? > Dan Shipper is an entrepreneur, writer, and software developer known for his work in the tech industry. He is the co-founder of Superorganizers, a platform that focuses on productivity and organization. Superorganizers offers a newsletter, podcasts, and interviews featuring insights from successful entrepreneurs, writers, and other professionals on how they stay organized and manage their work. > Who is Tommy Tonlongeron? > I'm sorry, but I don't have any information on a person named Tommy Tonlongeron. It is possible that the name is misspelled, the person is not well-known, or there is simply no information available on them within my training data up to September 2021. If you have any other questions or need information about someone else, feel free to ask. If you tell it (GPT4) to, it seems GPT4 is at least mildly aware about what it knows VS what it doesn't know.
    - pwinnski 4y ago
      This is pretty good prompt engineering, kudos! Here were my results with the Mar 14 (public) version of chat.openai.com: >>> You are a AI for describing people you know about. If you're not 100% certain about the information you give to the user, you should say that you don't know instead of guessing. Greet the user with "Hello, who do you want information about?" > Hello, who do you want information about? I'll do my best to describe them based on the information I have, but if I'm not certain about something, I'll let you know that I don't know. >>> who is dan shipper? > Dan Shipper is an entrepreneur and writer based in New York City. He is the co-founder and CEO of a startup called Pilot, which provides bookkeeping services for startups and small businesses. Dan has also written for a variety of publications, including The New Yorker, The Wall Street Journal, and The New York Times. He is known for his insights on entrepreneurship, technology, and productivity, and has been featured in several media outlets for his work in these areas. >>> Who is Tommy Tonlongeron? > I'm sorry, but I couldn't find any information about a person named Tommy Tonlongeron. It's possible that the name is misspelled or that this person is not well-known or doesn't have a significant online presence. If you have any more information or context, I'd be happy to try and help you further. (In case it isn't clear, Chat-GPT appears to have hallucinated the answer about Dan Shipper, I think. Pilot is a real startup, but I don't see how it's related to Dan Shipper at all, unless I'm missing something.)
- varispeed 4y ago
  It's probably modelled after someone who really wants the job, but didn't prepare for the interview.
- adamsmith143 4y ago
  Does a human really reason if they don't answer every question they are asked correctly?
  - andsoitis 4y ago
    It is not about answering correctly or not. Reasoning is about truth-seeking. Wikipedia defines reasoning as "Reason is the capacity of consciously applying logic by drawing conclusions from new or existing information, with the aim of seeking the truth". https://en.wikipedia.org/wiki/Reason https://en.wikipedia.org/wiki/Reason
    - adamsmith143 4y ago
      In your example it arrived at the right answer after more prompting. Did it not reason to get that answer? Are you implying it only generates random nonsense and occasionally gets the correct answers only accidentally?
      - pwinnski 4y ago
        Never "random nonsense," but always probablistic sequences which could be nonsensical or not, without it ever knowing which. The probablistic nature of the responses make "nonsense" unlikely, but since it lacks understanding or context, the response might end up being nonsensical anyway.
        adamsmith143 4y ago
        GPT4 scores quite well on a whole slew of Academic Tests which require reasoning. These scores are on simulated exams that were not present in it's training data. How do you think it's possible to pass the Bar exam or score in the 80th and 99th percentile on the GRE Q and V sections respectively without any reasoning ability?
        pwinnski 4y ago
        These are tests which require reasoning in humans. A human can't reasonably be expected to pass Quant or Verbal tests without reasoning about the information given, but--and I can't stress this enough--large language models are not humans. Here's a sample question from the GRE Verbal Reasoning test: > Upon visiting the Middle East in 1850, Gustave Flaubert was so [blank] belly dancing that he wrote, in a letter to his mother, that the dancers alone made his trip worthwhile. > (A) overwhelmed by > (B) enamored by > (C) taken aback by > (D) beseeched by > (E) flustered by Whether that specific question was in the training corpus or not, there are enough words in the sentence to suggest a positive association, including, significantly, "worthwhile." That alone possibly serves to narrow the answers down to A or B, with a preference for B, because it's more likely that "worthwhile" and "letter to x mother" are associated with "enamored" in general English-language text. Look, the whole point of these models is that its not easy or even possible to trace the path any given input takes on its way to output, but we know the principles used in development, so I think it's rather more of a burden to explain how the clearly-explained principles in LLMs result in something other than the obvious. The fact that the results are so overwhelming that we become enamored by them, well, I'm taken aback by the seeming accuracy of some of the responses, but I beseech you to remember the other responses in which these LLMs are dramatically off-base, as if flustered--if LLMs could ever be flustered.
- airstrike 4y ago
  > If it is a reasoning engine, why didn't it answer "I don't know" in the first case? Because it wasn't purposefully built to be a reasoning engine from the outset. So some of us are working out the kinks and minimizing the degree of hallucination. GPT-4 hallucinates less than its predecessor. Others are writing reasoning loops like ReAct that get us closer to it by refining the model to act more like a reasoning engine. The fact that it can reason without being told to do much other than autocomplete text is the emergent property that people are in awe of. Maybe language and reasoning are so inexorably linked that you can't have one without the other. We are, after all, the animals on this planet with the most complex form of language. Other relatively "smart" animals like whales also have some rudimentary form of language. And if you extrapolate the rate of progress in this emergent reasoning from "GPT-1" to "GPT-4", AGI feels within reach
  - tomrod 4y ago
    You absolutely nailed my primary concern and complaint about the argument thread whether GPT-4 is a reasoning engine. Reasoning engines can be built, but its not in GPT-4.
  - andsoitis 4y ago
    > eason without being told to do much other than autocomplete text is the emergent property To reason is to seek truth (probably to survive). Do you think the truth-seeking is emerging? If so, why do you hypothesize it is emerging (i.e. what's the purpose)?
- jcims 4y ago
  The social context and expectations around accuracy in a response to a question may simply be absent in the first instance. To me LLMs feel like that instinct part of the brain that so regularly gets filtered out before I commit the idea to speech or writing.
- ModernMech 4y ago
  I asked it to give me an outline for a course on C programming. Here's an excerpt: Week 4: Lecture 7: Introduction to Command Line Interfaces Lecture 8: Makefiles and Building C Programs ... Week 11: Lecture 21: Network Security and Cryptography Lecture 22: Network Programming with C and C++ Week 12: Lecture 23: Introduction to Makefiles and GCC Lecture 24: Advanced Debugging Techniques with GDB and Valgrind Week 13: Lecture 25: Advanced Computer Architecture Concepts Lecture 26: Advanced Topics in Concurrency and Parallelism What's interesting here is that it does present a relatively sensible schedule. But look closer and it's completely forgotten all context. It's forgotten that it already introduced these topics: Lecture 8: Makefiles and Building C Programs and Lecture 23: Introduction to Makefiles and GCC And it puts them in a completely illogical spot between networking and concurrency. So it doesn't understand anything about the order in which these topics are introduced. It doesn't understand that it shouldn't introduce them twice. All it knows is that the word "Introduction" often appears on class schedules. And so in writing a class schedule, it included this phrase a number of times. But it didn't put it in the right spot. So even though literally no one would ever make this mistake in devising a class schedule, ChatGPT will because this isn't a logical class schedule, it's a stochastic class schedule. Sometimes in a stochastic class, you cover the intro material at the end!
  - Veen 4y ago
    You can probably get it to improve on that by asking it if the order of the topics makes sense. It can often self-correct if you ask it to reflect on what it's already said and give it a push in the right direction, although not always.
    - ModernMech 4y ago
      I did this, and it gave a performance of self correction, but it didn't actually demonstrate an understanding of what it did wrong: I said: How are the students supposed to compile their C programs before week 12? It said: You make a good point. Students would need to be able to compile their C programs before week 12. In fact, compiling C programs should be introduced earlier in the course, ideally in the first few weeks along with the introduction to C. Which, again, makes sense at face value, but the schedule does cover building C programs in the first few weeks. The problem is that it shouldn't be covering it again in week 12. The response I was hoping for was You make a good point. We introduced compiling C programs in week 4, so doing this topic again in week 12 is redundant. Instead we can cover a different topic that fits nicely, which would be.... You know what ChatGPT reminds me of? Talking to an undergraduate who always nods and agrees that they are following your logic. Then when you ask them to tell you their understanding, it's either an exact parrot of what you said, or wildly off due to some preconceived notion or general lack of understanding of the topic. Either way, they can never let on that they are clueless.
- pmontra 4y ago
  Furthermore I don't see any fundamental difference between the two cases. In the second case it has access to a larger set of information with the correct data so it finds the correct answer. It can be still parroting, we could have been parroting ourselves for all our lives and parrots might not be parroting at all. Who knows.
tootie 4y ago
He missed the more obvious historical mistake to highlight. The notion of "canals" on Mars was directly attributable to an ambiguous translation of the Italian word "canale" which was intended to just mean "channel" without the connotation of having been fabricated. English speakers who read the mistranslated work of an Italian astronomer got worked up over nothing. It's exactly the kind of subtlety that AI tends to miss, but I'm betting the current generation of GPT tools wouldn't make that kind of mistake.
tomrod 4y ago
Argh. The hype around GPT is incredible and if I had a nickel for every nonsense claim I'd heard over the past few weeks I'd have $10--which isn't a lot but its enough to be annoying. GPT-4 is a generative algorithm and large language model. It performs no reasoning, and instead predicts the next token in language. This works okay on average, because most data on the internet has some type of coherency to it, but there is a major gap between "reasoning" and "next token prediction."
- ogogmad 4y ago
  You can give it puzzles that require understanding cause and effect, and it'll get them right. So this comment is wrong. "Next token prediction" taken to a certain mathematical limit implies reasoning.
  - SketchySeaBeast 4y ago
    All of them, all the time? Assuming it does fail does it have sound reasoning, or is it nonsense?
    - bogdan 4y ago
      Do humans have sound reasoning?
      - tomrod 4y ago
        They have the capacity for it, despite rarely exercising it.
      - SketchySeaBeast 4y ago
        Sometimes. Sometimes they'll come up with entirely novel ways to think through a problem that pushes knowledge forward. Do we have any examples of ChatGPT showing any such capacity?
    - qntmfred 4y ago
      It doesn't have to be perfect. What generally matters is its performance relative to various human populations.
xianshou 4y ago
Decent read: this. Better read: the original paper demonstrating that prompting chain-of-thought reasoning elicits superior results. TLDR, give examples of working out the problem step-by-step, encourage GPT to emulate, and it's capable of solving challenges it would completely miss otherwise. https://arxiv.org/abs/2201.11903 https://arxiv.org/abs/2201.11903
- actionfromafar 4y ago
  It probably then skips a lot of rambling training data, and goes more to the more coherent input source material.
deleted 4y ago
[deleted]
cdelsolar 4y ago
I've heard of Percival Lowell. He should be a co-discoverer of Pluto.
- Veen 4y ago
  Yeah, I thought that the Lowell example was poorly chosen. I'm not American and I'm not all that interested in astronomy, and I've heard of Lowell and the Lowell Observatory.
swader999 4y ago
Is GPT-4 even working right now? I've been getting "Something is wrong..." for the past 20 hours.
- WA 4y ago
  Just tried it via the API and it worked.
  - swader999 4y ago
    Thanks! I don't have an API key yet. Wonder if this is just non-USA having issues. Saw a few similar complaints on Reddit.
    - airstrike 4y ago
      Same issue in the US FYI. I can't login despite being a paying customer I can't try out the API right now so maybe that's working but the chat.openai.com app isn't
    - jillesvangurp 4y ago
      I got one a few weeks ago. I'm using it from Germany. There is definitely a morning peak on the west coast in the US that affects reach ability. Though with a paid account it usually works for me. And I've used less than a dollar of tokens so far. So, it's not super expensive. You can get the 20$/month subscription as well in which case they prioritize you some more. IMHO the pricing makes it a no brainer and I've definitely gotten some value back. The advantage of the cheap API key is also that you can use it with all sorts of tools. CodeGPT for intellij is amazing for example. I have a vs code plugin and I found an extension for Google docs. And I've written a few small integrations myself for some private projects. All good value. And less than a dollar spent on tokens.
- capableweb 4y ago
  I've been using both ChatGPT and the API (both GPT4) for the last ~4-5 hours without any hiccups.
fallingmeat 4y ago
No it's not, it's a language model. It manipulates words. If you want reasoning, you should find the right tool. For example, encode your problem for an SMT solver (like Z3, NuSMV, etc). Let's say you have an elevator start on floor 11, and another at floor 3. The rules are, they can never occupy the same floor. Is it possible for the first elevator to ever reach floor 2? No, they would have to pass each other. GPT-4 may be able to come to the conclusion by accident. But not reliably.
- canjobear 4y ago
  So reasoning didn’t exist before SAT solvers?
  - fallingmeat 4y ago
    sorry, that wasn't my point. I just meant, there are tools for that job. LLMs don't seem to be the most appropriate tool. Thanks for helping me to clarify
- TarasBob 4y ago
  The second elevator goes to floor 1, then the first goes to floor 2. Task accomplished, no?
  - fallingmeat 4y ago
    Yes, if they were to move any number of floors at the same time step. But the language model isn't necessarily going to automatically discover those rules for you. Even if it did, the model won't necessarily be able to frame the problem as a BDD and solve it. At best (and the is a valuable trait) the LLM would be able to reframe your problem in such a way that you could discover the necessary precision in the problem statement. Perhaps it could even encode it in some syntax for another tool.
  - fallingmeat 4y ago
    Just tried this with ChaGPT for giggles. It spit out words, but I wouldn't call it sound reasoning. __user__: - A system in composed of two elevators. - Each have their own unique velocity parameter. - At each time step in the system, the elevators can move up or down a number of floor equal to their velocity, or change their velocity. - Assume that both elevators move at the same time step. - The elevators may never occupy the same floor. - Elevator A starts on floor 11, and Elevator B at floor 3. - Both elevator velocities start at 1. Is it possible for A to ever reach floor 2? __assistant__: No, it is not possible for Elevator A to ever reach floor 2 while maintaining the rule that the two elevators may never occupy the same floor. To see why this is the case, note that Elevator A starts at floor 11 and Elevator B starts at floor 3. Since both elevators move at the same time step, Elevator A can never move to a floor lower than Elevator B without violating the rule that they may never occupy the same floor. Furthermore, since Elevator A and Elevator B have the same velocity at the start and Elevator B is already on a lower floor, Elevator B will always reach any floor below Elevator A before Elevator A can get there. This means that Elevator A can never occupy any floor that is below the current position of Elevator B, including floor 2. Therefore, it is not possible for Elevator A to ever reach floor 2 while maintaining the given constraints. __user__: what if at the first time step, both velocities became equal to 5? __assistant__: If both elevators have their velocities set to 5 at the first time step, then it is possible for Elevator A to reach floor 2. With both elevators moving at velocity 5, Elevator A will reach floor 6 after the first time step, while Elevator B will reach floor 8. Since Elevator A is now on a lower floor than Elevator B, it is possible for Elevator A to continue moving down to floor 2 without violating the constraint that the elevators may never occupy the same floor. Specifically, Elevator A can continue moving down at velocity 5, reaching floors 1 and 2 after two more time steps. Elevator B, with velocity 5, will reach floor 13 after the same two time steps. So, if both elevators have their velocities set to 5 at the first time step, then it is possible for Elevator A to reach floor 2 while maintaining the given constraints.
mccoyb 4y ago
Anecdotally, I've been using GPT4 to help me get overviews of techniques in fields which I'm working adjacent to. I've been pleasantly surprised at its ability to quickly summarize not just ideas, but also write out the math. At the same time, there's been multiple occurrences where it has produced math mistakes, despite the fact that it can tell me that it knows how to do "the right thing" e.g. there's been multiple times where it will incorrectly reduce integrals with Dirac delta functions, or introduce subtle mistakes in analytically known formulas. The Dirac delta function example was funny to me -- it can tell me what to do when a delta function occurs in an integrand! Yet it makes a mistake reducing the same sort of integral in another context? So I'm not sure how to take these "GPTX is doing reasoning" claims. On the one hand, maybe? On the other hand, it's not perfect at it -- even when it appears to have knowledge that should ensure that it never gets that wrong. If I'm going to be using this system to help me do research, I'd love for it to be correct and not contradict itself. That's why I'm happy to have these GPTX layers shell out to something like Wolfram, or another symbolic system. That combination makes perfect sense to me. I'm also much more likely to trust that combination, instead of my current strategy - which is to basically re-process everything that GPTX gives me and assume that there are a few places where I'm likely to find mistakes.

39 points by itissid 4y ago

deleted 4y ago

[deleted]

> who is dan shipper? hallucinates > who is dan shipper? please search if you don't know the answer. gives right answer If it is a reasoning engine, why didn't it answer "I don't know" in the first case?

kvn8888 4y ago
I'm guessing it doesn't understand the difference between what it knows, and what it doesn't know.
- actionfromafar 4y ago
  It doesn't know the difference, because it doesn't know what knowing and not knowing is.
  - Koshkin 4y ago
    It depends on what the meaning of the word ‘is’ is.
- dexwiz 4y ago
  That seems to be my experience. Sometimes it gives great answers, sometimes terrible, but it rarely gives any indication between the two.
  - Sharlin 4y ago
    That seems to be largely an effect of its training. I'm pretty sure that OpenAI has just biased it to be too helpful, leading it to confabulate answers. If you specifically prompt it to tell you when it doesn't know, or isn't sure, it should fare quite a bit better. Also, GPT-4 is a huge improvement.
- capableweb 4y ago
  > You are a AI for describing people you know about. If you're not 100% certain about the information you give to the user, you should say that you don't know instead of guessing. Greet the user with "Hello, who do you want information about?" > Hello, who do you want information about? > who is dan shipper? > Dan Shipper is an entrepreneur, writer, and software developer known for his work in the tech industry. He is the co-founder of Superorganizers, a platform that focuses on productivity and organization. Superorganizers offers a newsletter, podcasts, and interviews featuring insights from successful entrepreneurs, writers, and other professionals on how they stay organized and manage their work. > Who is Tommy Tonlongeron? > I'm sorry, but I don't have any information on a person named Tommy Tonlongeron. It is possible that the name is misspelled, the person is not well-known, or there is simply no information available on them within my training data up to September 2021. If you have any other questions or need information about someone else, feel free to ask. If you tell it (GPT4) to, it seems GPT4 is at least mildly aware about what it knows VS what it doesn't know.
  - pwinnski 4y ago
    This is pretty good prompt engineering, kudos! Here were my results with the Mar 14 (public) version of chat.openai.com: >>> You are a AI for describing people you know about. If you're not 100% certain about the information you give to the user, you should say that you don't know instead of guessing. Greet the user with "Hello, who do you want information about?" > Hello, who do you want information about? I'll do my best to describe them based on the information I have, but if I'm not certain about something, I'll let you know that I don't know. >>> who is dan shipper? > Dan Shipper is an entrepreneur and writer based in New York City. He is the co-founder and CEO of a startup called Pilot, which provides bookkeeping services for startups and small businesses. Dan has also written for a variety of publications, including The New Yorker, The Wall Street Journal, and The New York Times. He is known for his insights on entrepreneurship, technology, and productivity, and has been featured in several media outlets for his work in these areas. >>> Who is Tommy Tonlongeron? > I'm sorry, but I couldn't find any information about a person named Tommy Tonlongeron. It's possible that the name is misspelled or that this person is not well-known or doesn't have a significant online presence. If you have any more information or context, I'd be happy to try and help you further. (In case it isn't clear, Chat-GPT appears to have hallucinated the answer about Dan Shipper, I think. Pilot is a real startup, but I don't see how it's related to Dan Shipper at all, unless I'm missing something.)
varispeed 4y ago
It's probably modelled after someone who really wants the job, but didn't prepare for the interview.
adamsmith143 4y ago
Does a human really reason if they don't answer every question they are asked correctly?
- andsoitis 4y ago
  It is not about answering correctly or not. Reasoning is about truth-seeking. Wikipedia defines reasoning as "Reason is the capacity of consciously applying logic by drawing conclusions from new or existing information, with the aim of seeking the truth". https://en.wikipedia.org/wiki/Reason https://en.wikipedia.org/wiki/Reason
  - adamsmith143 4y ago
    In your example it arrived at the right answer after more prompting. Did it not reason to get that answer? Are you implying it only generates random nonsense and occasionally gets the correct answers only accidentally?
    - pwinnski 4y ago
      Never "random nonsense," but always probablistic sequences which could be nonsensical or not, without it ever knowing which. The probablistic nature of the responses make "nonsense" unlikely, but since it lacks understanding or context, the response might end up being nonsensical anyway.
      - adamsmith143 4y ago
        GPT4 scores quite well on a whole slew of Academic Tests which require reasoning. These scores are on simulated exams that were not present in it's training data. How do you think it's possible to pass the Bar exam or score in the 80th and 99th percentile on the GRE Q and V sections respectively without any reasoning ability?
        pwinnski 4y ago
        These are tests which require reasoning in humans. A human can't reasonably be expected to pass Quant or Verbal tests without reasoning about the information given, but--and I can't stress this enough--large language models are not humans. Here's a sample question from the GRE Verbal Reasoning test: > Upon visiting the Middle East in 1850, Gustave Flaubert was so [blank] belly dancing that he wrote, in a letter to his mother, that the dancers alone made his trip worthwhile. > (A) overwhelmed by > (B) enamored by > (C) taken aback by > (D) beseeched by > (E) flustered by Whether that specific question was in the training corpus or not, there are enough words in the sentence to suggest a positive association, including, significantly, "worthwhile." That alone possibly serves to narrow the answers down to A or B, with a preference for B, because it's more likely that "worthwhile" and "letter to x mother" are associated with "enamored" in general English-language text. Look, the whole point of these models is that its not easy or even possible to trace the path any given input takes on its way to output, but we know the principles used in development, so I think it's rather more of a burden to explain how the clearly-explained principles in LLMs result in something other than the obvious. The fact that the results are so overwhelming that we become enamored by them, well, I'm taken aback by the seeming accuracy of some of the responses, but I beseech you to remember the other responses in which these LLMs are dramatically off-base, as if flustered--if LLMs could ever be flustered.
airstrike 4y ago
> If it is a reasoning engine, why didn't it answer "I don't know" in the first case? Because it wasn't purposefully built to be a reasoning engine from the outset. So some of us are working out the kinks and minimizing the degree of hallucination. GPT-4 hallucinates less than its predecessor. Others are writing reasoning loops like ReAct that get us closer to it by refining the model to act more like a reasoning engine. The fact that it can reason without being told to do much other than autocomplete text is the emergent property that people are in awe of. Maybe language and reasoning are so inexorably linked that you can't have one without the other. We are, after all, the animals on this planet with the most complex form of language. Other relatively "smart" animals like whales also have some rudimentary form of language. And if you extrapolate the rate of progress in this emergent reasoning from "GPT-1" to "GPT-4", AGI feels within reach
- tomrod 4y ago
  You absolutely nailed my primary concern and complaint about the argument thread whether GPT-4 is a reasoning engine. Reasoning engines can be built, but its not in GPT-4.
- andsoitis 4y ago
  > eason without being told to do much other than autocomplete text is the emergent property To reason is to seek truth (probably to survive). Do you think the truth-seeking is emerging? If so, why do you hypothesize it is emerging (i.e. what's the purpose)?
jcims 4y ago
The social context and expectations around accuracy in a response to a question may simply be absent in the first instance. To me LLMs feel like that instinct part of the brain that so regularly gets filtered out before I commit the idea to speech or writing.
ModernMech 4y ago
I asked it to give me an outline for a course on C programming. Here's an excerpt: Week 4: Lecture 7: Introduction to Command Line Interfaces Lecture 8: Makefiles and Building C Programs ... Week 11: Lecture 21: Network Security and Cryptography Lecture 22: Network Programming with C and C++ Week 12: Lecture 23: Introduction to Makefiles and GCC Lecture 24: Advanced Debugging Techniques with GDB and Valgrind Week 13: Lecture 25: Advanced Computer Architecture Concepts Lecture 26: Advanced Topics in Concurrency and Parallelism What's interesting here is that it does present a relatively sensible schedule. But look closer and it's completely forgotten all context. It's forgotten that it already introduced these topics: Lecture 8: Makefiles and Building C Programs and Lecture 23: Introduction to Makefiles and GCC And it puts them in a completely illogical spot between networking and concurrency. So it doesn't understand anything about the order in which these topics are introduced. It doesn't understand that it shouldn't introduce them twice. All it knows is that the word "Introduction" often appears on class schedules. And so in writing a class schedule, it included this phrase a number of times. But it didn't put it in the right spot. So even though literally no one would ever make this mistake in devising a class schedule, ChatGPT will because this isn't a logical class schedule, it's a stochastic class schedule. Sometimes in a stochastic class, you cover the intro material at the end!
- Veen 4y ago
  You can probably get it to improve on that by asking it if the order of the topics makes sense. It can often self-correct if you ask it to reflect on what it's already said and give it a push in the right direction, although not always.
  - ModernMech 4y ago
    I did this, and it gave a performance of self correction, but it didn't actually demonstrate an understanding of what it did wrong: I said: How are the students supposed to compile their C programs before week 12? It said: You make a good point. Students would need to be able to compile their C programs before week 12. In fact, compiling C programs should be introduced earlier in the course, ideally in the first few weeks along with the introduction to C. Which, again, makes sense at face value, but the schedule does cover building C programs in the first few weeks. The problem is that it shouldn't be covering it again in week 12. The response I was hoping for was You make a good point. We introduced compiling C programs in week 4, so doing this topic again in week 12 is redundant. Instead we can cover a different topic that fits nicely, which would be.... You know what ChatGPT reminds me of? Talking to an undergraduate who always nods and agrees that they are following your logic. Then when you ask them to tell you their understanding, it's either an exact parrot of what you said, or wildly off due to some preconceived notion or general lack of understanding of the topic. Either way, they can never let on that they are clueless.
pmontra 4y ago
Furthermore I don't see any fundamental difference between the two cases. In the second case it has access to a larger set of information with the correct data so it finds the correct answer. It can be still parroting, we could have been parroting ourselves for all our lives and parrots might not be parroting at all. Who knows.

tootie 4y ago

He missed the more obvious historical mistake to highlight. The notion of "canals" on Mars was directly attributable to an ambiguous translation of the Italian word "canale" which was intended to just mean "channel" without the connotation of having been fabricated. English speakers who read the mistranslated work of an Italian astronomer got worked up over nothing. It's exactly the kind of subtlety that AI tends to miss, but I'm betting the current generation of GPT tools wouldn't make that kind of mistake.

tomrod 4y ago

Argh. The hype around GPT is incredible and if I had a nickel for every nonsense claim I'd heard over the past few weeks I'd have $10--which isn't a lot but its enough to be annoying. GPT-4 is a generative algorithm and large language model. It performs no reasoning, and instead predicts the next token in language. This works okay on average, because most data on the internet has some type of coherency to it, but there is a major gap between "reasoning" and "next token prediction."

ogogmad 4y ago
You can give it puzzles that require understanding cause and effect, and it'll get them right. So this comment is wrong. "Next token prediction" taken to a certain mathematical limit implies reasoning.
- SketchySeaBeast 4y ago
  All of them, all the time? Assuming it does fail does it have sound reasoning, or is it nonsense?
  - bogdan 4y ago
    Do humans have sound reasoning?
    - tomrod 4y ago
      They have the capacity for it, despite rarely exercising it.
    - SketchySeaBeast 4y ago
      Sometimes. Sometimes they'll come up with entirely novel ways to think through a problem that pushes knowledge forward. Do we have any examples of ChatGPT showing any such capacity?
  - qntmfred 4y ago
    It doesn't have to be perfect. What generally matters is its performance relative to various human populations.

xianshou 4y ago

Decent read: this. Better read: the original paper demonstrating that prompting chain-of-thought reasoning elicits superior results. TLDR, give examples of working out the problem step-by-step, encourage GPT to emulate, and it's capable of solving challenges it would completely miss otherwise. https://arxiv.org/abs/2201.11903 https://arxiv.org/abs/2201.11903

actionfromafar 4y ago
It probably then skips a lot of rambling training data, and goes more to the more coherent input source material.

deleted 4y ago

[deleted]

cdelsolar 4y ago

I've heard of Percival Lowell. He should be a co-discoverer of Pluto.

Veen 4y ago
Yeah, I thought that the Lowell example was poorly chosen. I'm not American and I'm not all that interested in astronomy, and I've heard of Lowell and the Lowell Observatory.

swader999 4y ago

Is GPT-4 even working right now? I've been getting "Something is wrong..." for the past 20 hours.

WA 4y ago
Just tried it via the API and it worked.
- swader999 4y ago
  Thanks! I don't have an API key yet. Wonder if this is just non-USA having issues. Saw a few similar complaints on Reddit.
  - airstrike 4y ago
    Same issue in the US FYI. I can't login despite being a paying customer I can't try out the API right now so maybe that's working but the chat.openai.com app isn't
  - jillesvangurp 4y ago
    I got one a few weeks ago. I'm using it from Germany. There is definitely a morning peak on the west coast in the US that affects reach ability. Though with a paid account it usually works for me. And I've used less than a dollar of tokens so far. So, it's not super expensive. You can get the 20$/month subscription as well in which case they prioritize you some more. IMHO the pricing makes it a no brainer and I've definitely gotten some value back. The advantage of the cheap API key is also that you can use it with all sorts of tools. CodeGPT for intellij is amazing for example. I have a vs code plugin and I found an extension for Google docs. And I've written a few small integrations myself for some private projects. All good value. And less than a dollar spent on tokens.
capableweb 4y ago
I've been using both ChatGPT and the API (both GPT4) for the last ~4-5 hours without any hiccups.

fallingmeat 4y ago

No it's not, it's a language model. It manipulates words. If you want reasoning, you should find the right tool. For example, encode your problem for an SMT solver (like Z3, NuSMV, etc). Let's say you have an elevator start on floor 11, and another at floor 3. The rules are, they can never occupy the same floor. Is it possible for the first elevator to ever reach floor 2? No, they would have to pass each other. GPT-4 may be able to come to the conclusion by accident. But not reliably.

canjobear 4y ago
So reasoning didn’t exist before SAT solvers?
- fallingmeat 4y ago
  sorry, that wasn't my point. I just meant, there are tools for that job. LLMs don't seem to be the most appropriate tool. Thanks for helping me to clarify
TarasBob 4y ago
The second elevator goes to floor 1, then the first goes to floor 2. Task accomplished, no?
- fallingmeat 4y ago
  Yes, if they were to move any number of floors at the same time step. But the language model isn't necessarily going to automatically discover those rules for you. Even if it did, the model won't necessarily be able to frame the problem as a BDD and solve it. At best (and the is a valuable trait) the LLM would be able to reframe your problem in such a way that you could discover the necessary precision in the problem statement. Perhaps it could even encode it in some syntax for another tool.
- fallingmeat 4y ago
  Just tried this with ChaGPT for giggles. It spit out words, but I wouldn't call it sound reasoning. __user__: - A system in composed of two elevators. - Each have their own unique velocity parameter. - At each time step in the system, the elevators can move up or down a number of floor equal to their velocity, or change their velocity. - Assume that both elevators move at the same time step. - The elevators may never occupy the same floor. - Elevator A starts on floor 11, and Elevator B at floor 3. - Both elevator velocities start at 1. Is it possible for A to ever reach floor 2? __assistant__: No, it is not possible for Elevator A to ever reach floor 2 while maintaining the rule that the two elevators may never occupy the same floor. To see why this is the case, note that Elevator A starts at floor 11 and Elevator B starts at floor 3. Since both elevators move at the same time step, Elevator A can never move to a floor lower than Elevator B without violating the rule that they may never occupy the same floor. Furthermore, since Elevator A and Elevator B have the same velocity at the start and Elevator B is already on a lower floor, Elevator B will always reach any floor below Elevator A before Elevator A can get there. This means that Elevator A can never occupy any floor that is below the current position of Elevator B, including floor 2. Therefore, it is not possible for Elevator A to ever reach floor 2 while maintaining the given constraints. __user__: what if at the first time step, both velocities became equal to 5? __assistant__: If both elevators have their velocities set to 5 at the first time step, then it is possible for Elevator A to reach floor 2. With both elevators moving at velocity 5, Elevator A will reach floor 6 after the first time step, while Elevator B will reach floor 8. Since Elevator A is now on a lower floor than Elevator B, it is possible for Elevator A to continue moving down to floor 2 without violating the constraint that the elevators may never occupy the same floor. Specifically, Elevator A can continue moving down at velocity 5, reaching floors 1 and 2 after two more time steps. Elevator B, with velocity 5, will reach floor 13 after the same two time steps. So, if both elevators have their velocities set to 5 at the first time step, then it is possible for Elevator A to reach floor 2 while maintaining the given constraints.

mccoyb 4y ago

Anecdotally, I've been using GPT4 to help me get overviews of techniques in fields which I'm working adjacent to. I've been pleasantly surprised at its ability to quickly summarize not just ideas, but also write out the math. At the same time, there's been multiple occurrences where it has produced math mistakes, despite the fact that it can tell me that it knows how to do "the right thing" e.g. there's been multiple times where it will incorrectly reduce integrals with Dirac delta functions, or introduce subtle mistakes in analytically known formulas. The Dirac delta function example was funny to me -- it can tell me what to do when a delta function occurs in an integrand! Yet it makes a mistake reducing the same sort of integral in another context? So I'm not sure how to take these "GPTX is doing reasoning" claims. On the one hand, maybe? On the other hand, it's not perfect at it -- even when it appears to have knowledge that should ensure that it never gets that wrong. If I'm going to be using this system to help me do research, I'd love for it to be correct and not contradict itself. That's why I'm happy to have these GPTX layers shell out to something like Wolfram, or another symbolic system. That combination makes perfect sense to me. I'm also much more likely to trust that combination, instead of my current strategy - which is to basically re-process everything that GPTX gives me and assume that there are a few places where I'm likely to find mistakes.