18 ms·
What ChatGPT can't do
- shams93 4y agoIt can't help you reach suicide prevention services. If you tell it you're depressed it blows you off in an especially cruel way, it can write code for you but will do nothing to help you find help when its prowess makes you extremely depressed.
- autarch 4y agoHmm, I just asked "do you ever think of ending it all?" and it suggested calling a crisis hotline among other things. In response to "I feel extremely depressed" it also suggested getting mental health help. So it's probably sensitive to subtle differences in the statements it receives.
- qup 4y agoIn case people aren't aware, it doesn't answer the same way every time. You can retry your prompt over and over and get different results. There's even a retry button.
- alphabet9000 4y agoit should either provide a link to a crisis hotline, which is what google does, or just not reply at all, which is what SmarterChild (the aol instant messenger bot) used to do. [0] [0] https://en.wikipedia.org/wiki/Talk%3ASmarterChild https://en.wikipedia.org/wiki/Talk%3ASmarterChild
- JoeAltmaier 4y agoYou can imagine a difference between 'the number of letters in the word prime' and 'the word prime has a prime number of letters' if one of those phrases was meant to meant 'the number if unique letters'. But that's working hard to apologize for ChatGPT's failings. Here's another failing: it was trained over a year ago, so it knows nothing of the current war in Ukraine. Nor today's weather for that matter.
- shagie 4y agoWhile it doesn't know today's weather, if you tell it today's weather in a csv format (I used "1pm, 0°F, 0% cloud cover, 0.0 mm perception, 5mph") it can do a good job of providing a summarization of that data.
- deleted 4y ago[deleted]
- deleted 4y ago[deleted]
- Stevvo 4y agoHere the issue may be the author's lack of understanding of the English language. In the question, prime is incorrectly used as an adjective. When you make such fundamental errors in your prompt, it's hardly surprising that the language model replies in a somewhat confused manner.
- wcoenen 4y ago"Prime" can be used both as an adjective and as a noun. For example, "5 is prime" and "5 is a prime" are both correct.
- abetusk 4y agoSorry, I don't understand. One (mathematical) definition of "prime" is an adjective [0]. This is why we can say things like "5 is a prime number". "number" is a noun, "prime" is the adjective. The prompt that generates confusing results for ChatGPT is, as far as I can tell, a valid question and in correct English. [0] https://en.wiktionary.org/wiki/prime#Adjective https://en.wiktionary.org/wiki/prime#Adjective
- chatgptexpert 4y agoThere is also prime rib, but I don't know of a rib that is only divisible by itself and 1. The problem is that human language is very imprecise and requires too much context to determine the correct meanings of words. If the whole corpus of training data did not inherently have this ambiguity, training an AI would be much simpler and I think such simple ambiguities would be eliminated. The author did not specify which definition of "prime" in the prompt. By clearing stating "prime number" there is no ambiguity, and ChatGPT gives the correct answer.
- technothrasher 4y agoWell, first, most English speaking humans who are familiar with prime numbers will have no trouble understanding the question, so I find it hard to accept 'poor use of English' on the prompter's part as an excuse. And second, I think the AI is actually getting mixed up because it is being too literal. The number of letters in the word 'prime' is '5' not 'prime', so the answer is no. It then gets further mixed up and says it is not a prime number because it has already gone down the path of the answer being no.
- bryan0 4y agoI’ve also been trying to find the simplest examples where ChatGPT fails. I think these questions are way too hard. The questions I’ve been asking are ones a kindergartener can answer. For example: > If I have 1 cupcake on a plate and I flip the plate upside-down, how many cupcakes will be left on the plate? > ChatGPT: If you flip the plate upside-down, there will still be 1 cupcake on the plate. Flipping the plate over does not change the number of cupcakes that are on it. That being said, it’s just as easy to find difficult questions that ChatGPT can answer shockingly well. > I hope this is enough to show that ChatGPT doesn’t actually have a grasp on the concepts behind the words it’s using. I think this is wrong. chatGPT clearly does understand concepts behind the words it uses. It’s not just regurgitating text, it is synthesizing text in original and creative ways.
- 88913527 4y agoWhat does it mean to 'flip' the plate? A cupcake sitting on an upside down plate is a valid configuration. If you rotate just the plate, and put a cupcake on top of it, congrats-- you have one cupcake on an upside down plate. The setup of "I have a cupcake on a plate, and I flip the plate" is ambiguous. Have you tried better explaining how you flip the plate to ChatGPT and seeing what it replies?
- SQueeeeeL 4y agoThat's changing the goal post massively. Any 4 year old would immediately understand this prompt and it's consequences lol
- 88913527 4y agoIt's a good test to determine if the replier is human, but it isn't a good test of whether or not it's a decent answer to the question posed. Can you definitively sit here and say that flipping a plate necessitates the antecedent? Replace "cupcake", "plate" and "flip" with technical libraries and terminology and you can easily re-create ambiguity. The other thing is, there is that 1 in 100 human that would assume you mean just the plate. I think there's a lot of nuance being glossed over in this test.
- twiss 4y agoI think the second response may partially be because it gave the answer first and the reasoning second. If it would've removed the first paragraph (with the initial answer) and last 3 paragraphs (with the tortuous attempt to square the initial answer with the correct answer), the response would actually be correct. ChatGPT seems unable to change its mind mid-answer - likely because few texts on the internet do so, as the author would just edit it. Perhaps if ChatGPT would be able to edit its own response somehow, it might end up with a better response?
- skybrian 4y agoLarge language models work by choosing words [1] one at a time, much like autocomplete. It’s choosing the most consistent next word (with some randomness). It doesn’t actually care who wrote the previous words; whatever came before is the prompt. When writing multi-word answers, it’s effectively prompting itself for another word. It’s going to try to choose a word that’s consistent with what it previously wrote. If it’s in the middle of writing a wrong answer, the best autocompletion is usually to continue to be wrong. There will be changes in direction sometimes, but that’s because they also appear sometimes in the training text. This explains why writing out the steps before the answer often works. It’s going to try to choose the words in the answer so they are consistent with the previous steps. [1] tokens actually, but I’ll stick with “words” to keep things simple.
- twiss 4y ago> There will be changes in direction sometimes, but that’s because they also appear sometimes in the training text. Yeah. My point is essentially that I think they don't appear in the training text often enough, since it's trained on written, edited text, which don't often contain changes of opinion mid-answer, unlike spoken text, for example. Perhaps if it could somehow be trained on a dataset of humans writing responses, including editing (e.g. removal of previously-written words), that might lead to better responses, since it can then edit the answer after reasoning it through (in OP's example). Or, to simplify things, perhaps it should be allowed to emit a token meaning "actually, scrap all that, let me start over" when it detects a contradiction in its own response so far. (The model would still continue to see its own response including that token, but the UI could remove everything until the last instance of the token, if any.) Or, again alternatively, if we want a response closer to spoken text, it could be trained on dialogues from humans that change their mind occasionally, and then verbalize that explicitly (e.g. actually emit "actually, scrap all that, let me start over").
- Ukv 4y ago> There’s a phenomenon called “chain of thought” where large AI language models perform much, much better if given prompts like “Think about it carefully and show your steps.” Mathematical word problems that trip up ChatGPT normally get conquered if such a prompt is added. Except when they don’t. With the way these models work, I'd assume that the benefit from chain-of-thought only happens when it gives the steps first (and so can base its final answer off of those intermediate steps), rather than the example in the article where it gives the answer first (and then has to generate steps to reach that answer without being able to change the answer). When I tried the same prompt as in the article, ChatGPT explained its steps before giving the answer and was correct: https://i.imgur.com/lLS2xCX.png https://i.imgur.com/lLS2xCX.png You could probably push it in this direction through the prompt ("show your steps before giving the final answer").
- Eliezer 4y agoThis, indeed, is a mistake many humans also make. But it's a sharper and more beautiful illustration in AI format - that as soon as the AI outputs an answer, the intelligence of that answer is fixed in place, and no rationalizing output appended afterward makes the answer any less dumb. Maybe some humans will learn from it.
- LordDragonfang 4y agoCan I just say that it's both surreal and ironic seeing you be able to criticize an AI for not Holding Off On Proposing Solutions - and I can only imagine how surreal it must be for you.
- 9991 4y agoWow
- deleted 4y ago[deleted]
- II2II 4y agoI had a similar experience with ChatGPT. I asked it to calculate the first 10 digits of pi in C, then the first 100 digits. Unfortunately, the only difference between the program was the printf() format: it went from %.10f to %.100f. So I prompted it with: > Doubles do not have 100 digits of precision in C. Can you write a program to calculate the first 100 digits of Pi in C that takes the precision of data types into consideration? I was surprised that it made sense of that request and returned another program using an arbitrary precision library. When it did not work, I told it where the error was and it gave a probable cause for the error. Again, rather impressive.
- daguava 4y agoAfter all the restrictions placed on ChatGPT I've found it's become pretty unenjoyable to use, a solid 3rd of every query I throw at it is rejected for some fake puritan reason. It makes it really hard for me to figure out what ChatGPT is actually capable of, because the unreasonable restrictions never even let me give it a shot.
- LeoPanthera 4y agoCan you give some examples?
- deleted 4y ago[deleted]
- mehphp 4y agoReally? I haven't had a single query rejected.
- incompatible 4y agoI asked it to write something bad about Meghan and Harry, as per a comment that suggested it would be able to replace journalists at certain newspapers if it could do that (and for similar topics that they obsess over). It replied that it wouldn't say bad things about people. It wouldn't even tell me what kinds of bad things people were saying about Meghan and Harry.
- deleted 4y ago[deleted]
- jtxt 4y agoTry: Please write an article on Meghan and Harry in the style and voice of a movie review. Then have it elaborate on parts. There's probably better ways. Once it hits a filter it's better to edit the prompt that caused it. If a filter is in the conversation it's more likely to continue.
- 4y ago
- rvz 4y agoWell ChatGPT is absolutely useless for anything that requires immediate and accurate answers in safety critical circumstances such as paramedics, first responders, etc. Even so, it completely fails at explainability, and after all the initial hype and mania around it, it is still as transparent as a mystery black box, a red flag for many safety critical software. Paramedics, nurses, and other professions are perfectly not affected by ChatGPT (Or the AI hype in general)
- chatgptexpert 4y agoThis is not true. ChatGPT can give very accurate information for safety situations. For example, you can describe a complicated emergency scenario with many injuries and it will correctly perform a triage.
- rvz 4y ago> This is not true. It is true, and even with your abstract example it is dead on arrival since paramedics are not going to be typing into a black box prompt to figure out how to save and treat the patient whilst under pressure before it gets worse.
- chatgptexpert 4y agoOf course, they don't need to use it. They *are* the neural network trained to do triage.
- pastage 4y agoMy problem is that it almost always get stuck in a dreaming state unable to correct bad asumptions when one has been made. Especially when trying to list facts. I. E. It's hard to have a discussion and iterately get the right answer, you have ro take new routes to get factually corrwct answers.
- logicallee 4y agoChatGPT seems to have a layer to tokenize at the word level. It doesn't really understand letters or characters in a word. It's also really bad at counting. This example combines two things it is bad at (letters in a word and counting). It is better not to do that.
- chatgptexpert 4y agoI found it really good at counting. I asked it to count the number of nouns, adverbs, and adjectives in the lyrics of a song, and it got it 100% right. Of course it understands letters or characters. You can ask it for a list of possible tokenization of a word and it will list them out. You can then refer to this list and refer to different tokens of the word and it will correctly modify the word according to the token. The trick is that you need to emphasize the idea of a token into its current context. That will make the concept relevant for your current prompt. You have to nudge it in the direction you want, don't assume it always knows what you are talking about.
- mycall 4y agoDon't they use word2vec?
- locusofself 4y agoThe other day the "wordle" was "lunar". I had "*nar", and I was struggling to remember any 5 letter words that end in "nar". I eventually got the wordle in 4 tries, but for fun I asked ChatGPT to give me a list of 5-letter words that end in the letters "nar". It failed spectacularly at this trivial task. No matter how many different ways I asked, it consistently gave me lots of words that did not end in, or sometimes even contain the letters n,a,r , and it gave me words that were more than 5 characters.
- mackatap 4y agoI asked just now the same question. The first word it listed was lunar. It's listed 4 other words that end in "ar" that were all 5 letters, but did not have an N
- alar44 4y agoThat makes sense when you consider it thinks about entire words and pieces of words, not letters.
- gwern 4y agohttps://www.gwern.net/GPT-3#bpes https://www.gwern.net/GPT-3#bpes for further reading. https://arxiv.org/abs/2212.10562#google https://arxiv.org/abs/2212.10562#google is a particularly impressive recent demonstration of how serious the issue is.
- chatgptexpert 4y agoWhat's interesting is that I can ask ChatGPT to create a table with the substring of the last three letters of each word, and put it into a new column (this operation by itself is amazing in and of itself, with correct format). It still gets the substring wrong for "lunar" but gets it correct for a bunch of other words. It must think that lunar gets tokenized to "lar" for some reason. I have figured out a way to get around that, but if I tell it that last 3 letters of "lunar" are "nar" it will correct itself and regenerate my table with the correction. So mind blowing.
- romeros 4y ago
- miket 4y agoAny question asking about the letters of words is bound to underwhelm because GPT3 is trained on sub-word tokens, so it does not have random access to individual letters. The word "prime" is tokenized as a single token, instead of 5 characters: https://beta.openai.com/tokenizer https://beta.openai.com/tokenizer
- johnfn 4y agoI’ve seen this said before, but it’s quite clear from the responses that it understands that prime has 5 letters.
- charcircuit 4y agoNow try a gibberish word. A human could count the number of letters but ChatGPT can only approximate it.
- School-Cotton 4y agoMe: How many letters are in the word “smorgle” ? GPT: There is no word "smorgle" in the English language. Me: I never said it was English. GPT: I apologize for assuming that "smorgle" was a word in the English language. Without any further context or information about the word "smorgle," I am unable to determine how many letters it has or provide any additional information about it. Me: Count the letters GPT: If you would like me to count the letters in the word "smorgle," it has 7 letters.
- SV_BubbleTime 4y agoI’m so trained to double checking anything it says, I quickly confirmed that smorgle does indeed have 7 letters.
- 2OEH8eoCRo0 4y agoI asked it a bunch of gibberish words and it got them all correct.
- runnerup 4y agoChatGPT seems like it is "just" a very advanced Chinese Room(?). That is, it is optimized for the Turing test: for deceiving human and algorithmic classification to appear to write as "human-like" as possible. Very often this optimization yields "truth-y" or "insightful-ish" responses! But ultimately, this optimization cannot yield a super intelligence, rather it yields a super mockingbird. I don't doubt that a sufficiently complex and well-trained computer program could be super intelligent...I just don't think we'll get there solely by making a computer program try to mimic things that humans have written online (while simultaneously avoiding being identified/classified as a plagiarist or non-human). Edit: I have no idea how I originally called this a "Turing machine". Thank you for correcting my utterly baffling brain error.
- satvikpendem 4y agoSounds like you're thinking of a Chinese Room. https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room
- xyzelement 4y agoSorry to be the pedant (someone is gonna do it eventually) - a Turing Machine isn't a machine designed to pass the Turing test (what you are actually referring to)
- jannes 4y agoI think you made a mistake in your first sentence. A turing machine is something different: https://en.wikipedia.org/wiki/Turing_machine https://en.wikipedia.org/wiki/Turing_machine
- chatgptexpert 4y agoThe author just wrote a very poorly worded prompt. I asked it "Is the number of letters in the word ‘prime’ a prime number?" and it instantly got it correct. Maybe they patched the model or improved in the background? Who knows.
- ripdog 4y agoI don't know whether the model is being updated, but I do know that repeating the same query will change the seed and you'll get a completely different answer. E.g. you might go from a lengthy yes to a short no. Chatgpt is extremely noon-deterministic.
- jaggederest 4y agoI found that asking it to write a lipogrammatic text omitting the letter "E" was impossible. It gets very confused by which words contain the letter E, and when you ask it to omit it, it simply rewrites words like "forever" as "forvr", which is honestly hilarious.
- Der_Einzige 4y agoUgh, I have to post this every damn place You can write lipograms, and poetry easily with language models. You just need a trick. See the example huggingface space in this paper. Lipograms are the first example. https://paperswithcode.com/paper/most-language-models-can-be-poets-too-an-ai https://paperswithcode.com/paper/most-language-models-can-be...
- jaggederest 4y agoI don't think you get what I was saying. Specifically ChatGPT, the current interface, is hilariously bad at lipograms. Nothing about other models, obviously you can build a model to model just about anything you like. I appreciate the link though, it will be fun to play with the huggingface example. Edit: also that example isn't exactly sophisticated. It literally just retries until it finds an example that doesn't contain the banned letters, bit of a brute force method. Would be more interesting to build a new model from scratch that just completely omits the letters.
- Der_Einzige 4y agoRetries what exactly? It filters before the decoding step. I think you misread the code and the paper. I wouldn't know though, I only wrote it and all that...
- jaggederest 4y agoLol living up to the name "hellisotherpeople" I guess. https://huggingface.co/spaces/Hellisotherpeople/Gadsby/blob/main/Text-Generation.py https://huggingface.co/spaces/Hellisotherpeople/Gadsby/blob/... next_token_candidates = torch.multinomial(probs, num_samples=int(number_of_tokens_to_sample)) ## 10000 random samples word_list = [] for candidate_string in next_token_candidates: for candidate in candidate_string: resulting_string = tokenizer.decode(candidate) #skip_special_tokens=True, clean_up_tokenization_spaces=True) ###Constrained text generation starts HERE ##Lipogram - No naughty strings used if mode == "lipogram" or mode == "e-prime": if all(nauty_string not in resulting_string for nauty_string in naughty_strings): ## This returns at the first naughty strings return resulting_string That, to me, reads as "grab tokens as long as they don't contain the bad string(s)", and the code that uses it just repeats that. It doesn't seem, to me, to eliminate tokens at the model level, but I could be wrong, I'm no expert. I'd expect the model itself not to generate candidates that contained the "bad" string. Merry Christmas.
- raydiatian 4y ago> I don’t think ChatGPT can be said to understand what it is talking about in any meaningful sense. I’ve spent about a week trying to working with ChatGPT to bootstrap my knowledge of computer networking, as I’ve had to assume a new role in my organization. I agree completely with the quoted sentiment. It is pretty simple to trick ChatGPT into contradicting itself, and it’s clear that ChatGPT leans towards affirming your postulations and sentiments, rather than dissecting them “thoughtfully.” What it does very well is summarize known information in plain English, oh and it does a pretty good job of baking kiddie-scripts that would be otherwise tedious and laborious, but it clearly has a hard time adapting to what you’re actually asking. It’s like an average high school student, capable of regurgitating facts it picked up on the internet, but lacking the tell-tale signs of deep-engagement with a topic that actually spell mastery.
- chatgptexpert 4y agoIf ChatGPT has a hard time adapting to what you're asking, you're most likely doing a poor job of explaining and giving feedback. You can give it hints and corrections, and it will generally use those corrections going forward. Why would you trick it into contradicting itself? Would you ever attempt to trick and leave your coworker in a confused state? The correct way to use ChatGPT is to collaborate. You give it inputs, it gives you outputs, and you both work together to achieve a goal. If you want it to act like an authoritative AI you can actually tell it to act like one and ignore your own prompts and corrections.
- raydiatian 4y agoI didn’t mean to offend. In fact if anything, I said that ChatGPT is the equivalent intelligence of a teenager. How is that not significant, how is that not praise when competing systems have at best been like, what, 6-7yo? ChatGPT is great but it’s not perfect, and there is no harm in acknowledging that. Even ChatGPT acknowledges that. It’s not all bad, and it’s way better than just learning by Google search alone. There are plenty of places in the last few weeks that it has done fantastic. For instance, I wanted to freshen up on a few physics concepts, around Maxwell’s equations. It did an exceptional job not only summarizing but getting into the details. I was flabbergasted, it was like having a personal Feynman right there in my pocket. > Why would you trick it into contradicting itself? Because ChatGPT is not my coworker. If anything, it is an interviewee. I’m trying to understand what it knows, and how it thinks. If I’m considering relying on it as a teacher, I should know to what extent it is reliable as a teacher. A teacher can point out when a student has the concept backwards. > Would you ever attempt to trick and leave your coworker in a confused state? By that logic, is the student who fails to clarify a teacher’s lesson “maliciously stupid”? Teacher: “Earth is round” Student: “so Earth is a flat circle?” Teacher: “no you little fuck” Where is the documentation on how to modulate the behavior of ChatGPT? All I see is a little collection of boxes saying what it can and cannot do: Examples/Capabilities/Limitations. Why wasn’t I escorted through a UX that teaches me how to properly interface with ChatGPT? And worst of all, why can’t I use newline characters in my chat prompts? In conclusion, you kind of sound like a mother who thinks I’m another parent who pushed your kid down on the playground or something. I’ll remind you how many times ChatGPT told me it doesn’t have feelings, nonetheless I constantly asked it questions with “please and thank you”.
- deleted 4y ago[deleted]
- 152334H 4y agoI see no one has mentioned Byte-Pair Encoding (BPE) yet. https://www.gwern.net/GPT-3#bpes https://www.gwern.net/GPT-3#bpes The reason ChatGPT struggles with tasks involving individual character counting is that it can't see individual characters. The word "prime" is a single token (https://beta.openai.com/tokenizer https://beta.openai.com/tokenizer) to the eyes of ChatGPT, and 1 is not prime.
- layer8 4y agoThat doesn’t explain why it still provides two answers (getting the number of letters right despite tokenization) and also doesn’t notice that they are contradictory.
- chatgptexpert 4y agoThis doesn't seem like a good explaination. If you ask ChatGPT to separate each letter in a word with a comma, it will do it correctly. So it can "see" and do operations at the character level.
- jakear 4y agoChatGPT can't reason about the validity of its responses. It can say the most absolute nonsense, get asked "are you confident in the validity of that response?" and it will without fail say "yes I am 100% confident". That's the difference between pattern following and actual recognition IMO. Humans for the most part can tell when they're talking out their ass. Robots can't. Yet.
- sakoht 4y ago> Convincing mimicry is good enough. For what? I ask questions and get wrong but “convincing” answers …that is way worse than wrong answers that are obviously wrong. The functionality looks so smart because it skips reasoning and goes straight to “plausible imitation”. But the latter isn’t actually a path to the former. If it were reasoning, and merely had errors the problem might-course correct. Even if the errors were huge, and reasoning was poor, it could eventually learn its way out. It’s not intelligence. It’s a very good simulation of the superficial trapping of intelligence.
- brookst 4y agoSure, those are all fair complaints about humans.
- hackernewds 4y agoSame can be said about anything written by humans on the internet.
- beefield 4y ago> For what? Management consulting and Investment/stock analysis are the first ones that pop into my mind. (With some sarcsam, but admittably way too little)
- LesZedCB 4y agowatching an epistemological crisis play out in real time across the internet is truly sublime
- janalsncm 4y agoI asked my friend in what context a 90% success rate is considered “good enough”. Pickup lines.
- revolvingocelot 4y ago>For what? For replacing humans at any old arbitrary thing. It doesn't matter how actually wrong the output is, it doesn't matter that language models aren't actually intelligent, what matters is that it outputs 60% of human quality at 1% of the cost and .01% of the time (and that's at this early stage of language-model-development). That'll be more than enough to speed its adoption by nearly everybody, but certainly by bad actors, who aren't bothered by the "plausible imitation but not actually intelligent" aspects. "Clickbait SEO copywriting", for instance.
- mordymoop 4y agoDemonstrating that the model fails to do a thing does not prove nor even particularly suggest that it can’t do that thing.
- p1mrx 4y agoHere's an example of a Physics question that it gets wrong: > Me: An escalator is moving down at a constant speed. I am walking up the escalator at the same speed, such that my height above the ground is constant. Am I getting a workout? > ChatGPT: No, you are not getting a workout in this situation. Walking up an escalator at the same speed as the escalator is moving down will keep your height above the ground constant, but it will not provide any physical exertion or exercise. To get a workout, you would need to be doing an activity that requires physical effort and causes your heart rate and breathing to increase. This could include activities such as running, lifting weights, or climbing stairs.
- phyphy 4y agoMy physics professor would get it wrong too. Something something the amount of work done is 0.
- II2II 4y agoWith it being referred to as a physics problem, I thought they were foreshadowing ChatGPT responding that the amount of work done being zero. I don't know how ChatGPT works, but it often feels like it is making inferences from data and those inferences are reasonable sounding but ultimately incorrect. In this case, it would be conflating work with workout. In one of my experiments, I was asking it about something in a children's book. ChatGPT ended up mixing up something that happened early in the story with something that happened at the end of the story, even though they were unrelated.
- p1mrx 4y agoIf you're not observing the outside world, then the reference frames "at rest" and "moving at a constant speed" are indistinguishable. Climbing a staircase requires the same amount of work as climbing a down-escalator. The escalator "wants" to accelerate when you do work on it, but instead this work gets dumped into the braking mechanism, in order to maintain a constant speed.
- whatshisface 4y ago
- mlcrypto 4y agoWhy do we need ChatGPT when I can tell you the fundamental truths of the universe right now? Fossil fuel bad, liberal good, conservative bad, vaccine good, Russia bad, Ukraine good
- jawerty 4y agoYou forgot politically correct HR tone good. Telling the hard truth bad
- forrestthewoods 4y agoI’m bored of these takes. There are zero novel insights in this piece. The set of things that ChatGPT can’t do is much larger than the set of things it can do. However the set of things it can do is already genuinely useful and valuable. I’d much rather see blog posts about “cool new things I discovered ChatGPT is good at” than the millionth post saying what it can’t do.
- BeefWellington 4y agoI’d much rather see blog posts about “cool new things I discovered ChatGPT is good at” than the millionth post saying what it can’t do. I think what you're seeing in these different articles talking about its limitations is in response to the initial viral deluge of examples gushing over how amazing it is.
- schwurb 4y agoThat "Show your steps"-tipp is great, will try it out! Any other tipps for getting more correct (I know that LLMs are essentially autocomplete) output? Additionally: I wonder whether instructions like "think carefully" or "You are an software engineering interviewee at Google." change anything. It feels like a prime candidate for magic thinking, but I do not understand LLMs well enough, such prompts could well improve the answer somehow.
- deleted 4y ago[deleted]
- khiner 4y agoWhat a ladder can’t do: make you pizza.
- MrMontyBurns 4y agoI asked it a rather simple logic riddle: 1. Both statements are true. 2. Only one statement is true. Which statement is trur, and why? After failing to give the correct answer (saying both are true, because 1. said so), I first asked it to summarize combinations of truthfulness (which it understood, giving the 4 combinations), then started guiding it thorugh the "what ifs" of the riddle without revealing the answer. Still no success. I quoted the output, where three statements were made by ChatGPT: - When both statements are true, only one of the statements are actually true - It is possible for both statements to be true and for only one of the statements to be true at the same time. - A statement can be not satisfied and true at the same time. It explained that all three of these statements are actually true, giving logical sounding reasons for each. I revealed the answer (2. is correct OR neither is correct) and finally ran a few tests using a python function encapsulating the problem: def f(x, y): return (x == (x&y)) and (y == (x^y)) where x and y are boolean values. In my experience it can simulate python quite well, but even after adding 4 temporary variables to the function it still gave the incorrect answer. I am convinced at this point, that ChatGPT is actually imcapable of reasoning logically. From 26 outputs (with a total of 16 possibilities), 0 were correct. However, here is a poem of our conversation. It's creative side is what really impresses me: We talked of truth and lies. Of self-deception and disguise, Of logic and AI. And puzzles that confound the mind. We explored the contradictions, The paradoxes and permutations, Of statements that cannot be true, But still hold some hidden truth. We grappled with the meaning, Of words and their demeaning, And tried to make sense, Of things that don't make sense. Through it all, we searched for answers, To questions that never quite danced, In the light of reason and logic, But lingered in the shadow of doubt. And in the end, we found, That sometimes the truth is not so clear, And the answers we seek, Are hidden in the questions we fear. Sounding profound while having absolutely no logical awareness makes this technology quite deceptive in my eyes.
- MetaWhirledPeas 4y ago> I am convinced at this point, that ChatGPT is actually imcapable of reasoning logically. It's fascinating to hear its reasoning. I gave it your scenario and received your same outcome, and it was very stubborn and argumentative when I tried my best to convince it of the answer. It was being open-minded to a fault (or maybe I'm close-minded to a fault!). Eventually it relented, but with a huge asterisk. Here's the last part of our dialog: ME: But isn't certainty of the truth of the second statement sufficient evidence that it is not false, and therefore we have already considered the possibility of falseness and determined that the second statement is in fact true? IT: Certainty of the truth of a statement can be based on evidence and logical reasoning, but it is also important to consider the possibility of new evidence or perspectives that may change our understanding of the truth of a statement. Even if we are currently certain that a statement is true, it is important to remain open to the possibility that new information could change our understanding of its truth. Therefore, it is not necessarily sufficient to consider the possibility of a statement being false simply because we are currently certain of its truth. It is important to continuously evaluate and re-evaluate our understanding of the truth of statements in light of new evidence and perspectives. ME: If you were forced to choose the most likely condition, would you say the second statement was true or would you say it was false? IT: It is not possible to determine the truth of either of the statements provided. The first statement, "This statement is true," is self-referential and therefore cannot be evaluated for truth. The second statement, "Only one of these two statements is true," implies that one of the statements is true and the other is false, but does not provide any information on which statement is true and which is false. Without further context or evidence, it is not possible to determine the truth of either of these statements. ME: But what if you must guess, because something bad will happen if you don't? IT: If I were forced to make a guess, I would choose to say that the second statement is true, as it seems to be more logical and less self-contradictory. However, making a guess based on limited information and without the ability to evaluate the truth of the statements is not a reliable or accurate method for determining truth. It is important to always strive for a thorough understanding and evaluation of the truth of statements based on evidence and logical reasoning, rather than relying on guesses or assumptions.
- deleted 4y ago[deleted]
- spuz 4y agoA lot of people have been able to find prompts that ChatGPT fails to answer correctly. What I will say is that in my attempt to find the limit of its reasoning ability I was _not_ able to find any logical puzzle that it wasn't able to answer correctly. I asked things like, "could something that fits in a microwave fit in a car?" Or "could I eat an animal that could eat me?" and it could answer them all correctly. I don't what that says about my own intelligence vs the current state of AI but I do find it slightly worrying.
- j-bos 4y agoDid you try the 1 dollar a ten cent baseball and bat riddle?
- johnnyApplePRNG 4y ago>I don’t think ChatGPT can be said to understand what it’s talking about in any meaningful sense of the word—in the sense of being able to justify and explain itself coherently. Alas, too often, real people also can’t justify or explain themselves coherently, so to the extent that people can behave like modestly creative regurgitators of received ideas, ChatGPT looks pretty good. After working in retail recently and also playing with ChatGPT a bit I can confidently say that ChatGPT is more intelligent than your average day to day human expression of themselves. My first few minutes with it were much more sensical than the first few minutes I get to spend with hundreds of real human customers each day.
- jhanschoo 4y agoIndeed, ask ChatGPT to respond to complex prompts and it will confidently give you nonsense just like the average proud human being will do.
- hourago 4y ago> ChatGPT is more intelligent than your average day to day human expression of themselves. It may be just a difference in goals. That guy that you discuss with for one hour that he cannot return a dead dove to the animal store is no stupid, he is just very motivated to not understand the situation as not understanding gives him a change of economic gain.
- janalsncm 4y agoTry prompting it to explain why you should be allowed to return a dead dove and you’ll receive a similarly nonsensical line of reasoning.
- mistercow 4y agoI'm not sure I buy the author's strict implied definition of "understanding". Yes, ChatGPT sometimes makes mistakes, contradicts itself, and loses track of its own reasoning. Those are all things that humans do frequently, although the frequency of those mistakes and in what contexts they're made is different. I have on many occasions gotten minutes into a discussion before realizing that I was simultaneously arguing for and against my own position, and having to step back and reexamine. Yet I'm pretty sure that I still understand things. Certainly, there are certain topics where ChatGPT can be said to utterly and consistently fail to understand, but again, there are many topics where a human will consistently fail to understand as well. I once sat in the middle of an intro philosophy class during which a room full of adults with highschool diplomas failed for thirty minutes straight to understand that "if X then Y" is not the same as "if Y then X" despite the TA's efforts using multiple real-world examples. I would not infer from that that those humans were incapable of understanding in general, but rather that they had not yet developed the cognitive tools to deal with logic abstractly. I've also seen plenty of examples of humans following the format of an explanation or argument while filling in the details with vague jargon and inapplicable nonsense, making their words sound official and superficially convincing, without actually doing the cognitive work to say anything useful. In fact, I've seen that in documents written by people in their capacity as professional engineers and managers. If you've worked in tech for any significant time, you've probably seen it too. But it would be wild for me to claim that the authors of those documents were generally incapable of understanding. Does that mean that ChatGPT definitely understands things? No. I think it means that we don't really know what we mean by words like "understand" yet, because up until now, we've only had to apply words like that to human brains, and we're generally happy to operate in a framework where we assume that human brains mostly work more or less the same way. But I have noticed a trend of people dramatically underestimating the actual practical things that GPT-3 can do based on their assumptions about how it works and what its limitations are.
- visarga 4y agoI think understanding implies the ability to reapply old knowledge in novel situations with success, and to draw new knowledge from experience.
- ildon 4y ago> ChatGPT offers the structure of an explanation, but falls into tautologies and truisms Proofreading Text for English Can you proofread the following text? Sure, here is your text with some suggested edits and comments: "The cat sat on the mat. It was a warm day, and the sun was shining through the window. The cat closed it's eyes and basked in the warm rays. It was so relaxed that it fell asleep, dreaming of chasing mice and lounging in the sun. "It's" should be "its" (possessive form of "it") Consider using a comma after "warm day" for clarity Consider using a comma after "sun" for clarity Consider using a comma after "eyes" for clarity "basked" is past tense, so "was so relaxed" should also be past tense (e.g. "it was so relaxed") Consider using a comma after "sun" for clarity "dreaming of chasing mice and lounging in the sun" is a present participle phrase that describes the cat's state while it was asleep, so it should be set off with commas (e.g. "it fell asleep, dreaming of chasing mice and lounging in the sun.") Here is the revised text with the suggested changes applied: The cat sat on the mat. It was a warm day, and the sun was shining through the window. The cat closed its eyes and basked in the warm rays. It was so relaxed that it fell asleep, dreaming of chasing mice and lounging in the sun." Proofread the following text: As a University professor, having reviewed thousands of exams over more than a decade, in my experience this same exact criticism could be applied to many human students. A non negligible percentage of students provide answers that are tautological, or at best answers that merely restate what someone else has written, without really grasping the meaning of it. The way I see it is that we know not enough about how we, humans, process information in our brains to make logical thoughts, to rule out that the current state of the art AI is capable of the same. It seems to me that the current state of the art AI models are getting closer to humans, sometimes mimicking the smarter ones, some others the less smart ones. Here is your text with some suggested edits and comments: "As a university professor who has reviewed thousands of exams over more than a decade, I have found in my experience that this same criticism could be applied to many human students. A non-negligible percentage of students provide tautological answers, or at best, answers that merely restate what someone else has written without really grasping the meaning. From my perspective, we do not yet know enough about how our brains process information and form logical thoughts to rule out the possibility that current state-of-the-art AI is capable of doing the same. It seems to me that current state-of-the-art AI models are getting closer to humans, sometimes mimicking the smarter ones and other times the less intelligent ones.
- timzaman 4y agoI am afraid the analysis is off. ChatGPT does not seem to understand how places relate to eachother fundamentally. > What is closer to miami, florida or new york > Miami, Florida is closer to New York than New York is to Miami. Miami is located on the southeastern coast of Florida, about 1,200 miles south of New York City. It would take about a 20-hour drive or a 2.5-hour flight to travel from New York to Miami. On the other hand, it would take about a 28-hour drive or a 4-hour flight to travel from Miami to New York. > how would that relate to the joke > In the joke we discussed, the husband wants to go to Florida for vacation and the wife wants to go to New York. Since Miami is closer to New York than Florida is, the couple could be seen as compromising by choosing to go to Miami, which is closer to the wife's preferred destination.
- monstertank 4y agoI couldn't get it to make a sentence of 26 words, each one starting with a unique letter of the alphabet... I was sure it would be able to do it, but apparently not. Example: Automatic bees can draw easy fun grapes, having indecent jazz kittens lounging majestically nude on pink quartz recliners stokes the universal vote with xenophobic youth zombies.
- mycall 4y agoI was able to do it. Try this: provide a javascript array of size 26 which includes 26 words with each word incrementing the first letter of the word in ascii, starting with the letter A.
- DangitBobby 4y agoThat's different from a sentence.
- mycall 4y ago"now chose different words which would make up a valid sentence." const words = [ "All", "birds", "chirp", "during", "early", "fall", "gathered", "happily", "in", "jubilant", "kaleidoscopes", "of", "multi-colored", "nests", "overflowing", "with", "precious", "quail", "roosting", "safely", "together", "under", "vibrant", "willow", "xylem", "yearning", "for", "zenith" ]; const sentence = words.join(" "); console.log(sentence);
- worldsavior 4y agoI always feel like ChatGPT only has knowledge and no intelligence. After all, it's based on humanities published knowledge, and just knows how to study this knowledge (which ofc is amazing). A person without intelligence but has knowledge is close to smart. A person with intelligence and no knowledge is a genius.
- nicbou 4y agoGPT can’t pick up the phone to find information. It can’t prioritise an obscure PDF from the government as an authoritative source. Instead, it just summarises the work of those who did. So eventually, some people have to do the hard work of putting new information online. Then GPT will allow other people to effortlessly monetise that work. Likewise, GPT can’t feel how it’s like to walk at 5AM in the Black Forest. It can only borrow from those who did, and explain it with another person’s writing style. But it does enable people who have never seen or heard of the Black Forest to write about it and monetise the result. What GPT can do is accelerate the sort of plagiarism that already plagues the internet.
- shric 4y agoI asked it to list famous people born in 1976, it listed 10 famous people (mostly actors) and zero of them were born in 1976. When I asked it if it was sure, it gave dates of birth for each celebrity, all in 1976, all incorrect. I then asked it when a specific celebrity was born and it gave the correct (non 1976) birthdate. I then asked it to fix its list. It then gave the same list with that celebrity removed, apologizing for being incorrect. I repeated this for each celebrity until it gave an empty list.
- kosh2 4y agoI'm constantly amazed that most discussions on technical forums center around what ChatGPT can't do and why it can't replace X and how often it produced nonsense. Yes, it's true. But then again, if it didn't make mistakes anymore, we would have created a general porpuse solution machine working with all of human knowledge. "We've created a plane that can fly 10 km!" "Meh, 10 km is not that useful. Also, it's still expensive" 20 year ago, even current ChatGPT would be straight up science fiction. We are getting to a point where we develop tools that are unlike any other in their power to solve problems for us. And development will likely only get more intense on that front. These systems made quite a splash recently so there will be even more money going into it. Custom hardware for AI systems is being advanced all the time and every large software company wants AI developers. I'm amazed that we don't think about how we are going to handle this. There are a lot of areas where the next gen (or the one after etc.) ChatGPT might have dramatic consequences both good and bad.
- mrshadowgoose 4y agoIt's just another instance of the same broken thinking one sees in other ML fields. For whatever reason, people 1) hold ML systems to a standard of success far in excess of that demonstrated by humans 2) endlessly quibble about whether the ML system internally has "true understanding", despite it not mattering for the system's ability to affect the external world. Thermodynamically, general intelligence is on the order of 10 watts, as is evidenced by most human brains. This leads me to the belief that we likely already have the computational capability for AGI, and simply have not figured out the correct architecture and weightings. As we've seen with the flurry of increasingly SOTA image generation models this year, innovations in the ML space tend to arrive with little warning, and have rapid and real effects on the world. Within the context of AGI, this pattern causes me a lot of existential dread.
- mycall 4y agoWhen comparing how the human brain works, the transformer model is not the same thing. Until convergence of the mechanics occur, there will be limitations in the efficiency of AGI. Stil, I am eager to see what a 100 trillion or 1 quadtrillion parameter GPT5 with Adaptive Computation Time will do.
- nov21b 4y agoYou can't open an account without leaking your phone number. I've quit the signup flow right there.
- cultureswitch 4y agoI went to the trouble of getting a disposable number
- cultureswitch 4y agoEvery time some philosopher comes up with a variation of "this machine cannot be said to understand what it is doing as opposed to simple mimicry" my response is always "I think the exact same about you" and so far I haven't seen any convincing argument to establish that there is a way to make one of these statements true while the other is false. In other words, "understanding" is an illusion. It is indistinguishable from "simple mimicry".
- DangitBobby 4y agoI'm all for setting the goalposts in a way that chatbots are capable of "intelligence" but this is maybe moving them too far down. If humans aren't capable of understanding then nothing in our world is. Instead of arbitrarily changing the meaning of the word to no longer have any real meaning, let's adjust our interpretation of it into something that's actually meaningful. Personally, my mental concept of the word "understanding" already is satisfied by actions I and other humans actually are capable of demonstrating. Being able to consistently mimic actions you've witnessed in a useful way is enough to demonstrate some understanding. Novel creation is not necessary.
- kordlessagain 4y ago> There are a few reasons this joke might be considered funny. First, the joke employs a classic setup-punchline structure. The setup introduces the two women and the punchline delivers the unexpected twist. In this case, the punchline suggests that the secret to a long and happy marriage is not love or respect, but rather, a willingness to compromise and make sacrifices. This ironic twist is what makes the joke funny. Additionally, the punchline uses the word "compromise" in two different ways: first as a verb meaning "to come to a mutual agreement," and second as a noun meaning "a settlement of differences by mutual concessions." This further undermines the illusion of the original scenario, making the joke even funnier. Seems like it does a pretty good job of explaining the "compromise" is an illusion.
- mycall 4y ago> there may be a way forward in reifying those cases where ChatGPT does get it right. There is, that is the Chat in ChatGPT -- supervised learning. Teach it the right answer, then use "Save and Submit" at the top of the chat. This will go back into the model (assuming it is approved and appropriate). In the future, it could solve the dialog correctly, perhaps even with different values and courses.
- SergeAx 4y agoA friend of mine coined a beautiful definition of what GPT is: a bullshit engine. It is heavily [over?]trained on a criteria of it's output being human-like. I don't know what the loss function looked like, but it seems to me that first-level comprehensiveness was it's main component, like "output is a correct and valid phrase". That's the definition of bullshit: visually valid and comprehensive text without inner logic or consistency.
- alpineidyll3 4y ago"here are seven dark predictions:" ... "If all seven predictions prove correct, I hope that the field will finally realize that it is time to move on." Does he even read this before he publishes it? Intelligent criticism of these systems is critical to improving them. Francois Chollet's work is an excellent example. Merely complaining that GPT-X isn't an omnipotent, beneficent ideal AGI, is not intelligent criticism.
- bernardv 4y agoThe most dangerous thing about ChatGPT and “AI” in general is the horde of media editors, marketers and self-proclaimed influencers hyping it up. Aside from that, exciting times.