18 ms·

ChatGPT – The Revolutionary Bullshit Parrot

• anonylizard 4y ago
  Anytime you see the 'parrot' argument, you can instantly go 'opinion dismissed'. Anytime you seem some fancy argument, that bases its tests on chatgpt, instead of the way stronger GPT-4 that usually invalidates half their criticisms, opinion dismissed. There's no need to debate about embeddings, compression = meaning etc. Simply put, many humans are parrots, much white collar work is just parroting. It doesn't matter what the underlying model of GPT-4 is like, if its useful, it will have a massive impact on the world.
  • cube2222 4y ago
    I'd like to echo the GPT-4 sentiment. Having used the API a bunch, the ChatGPT API (GPT-3.5-Turbo) feels like it has a 10-second attention span in comparison to GPT-4. GPT-4 is waaaay more powerful for actually building stuff on top of, adheres to your prompt in a much more stable way, and its reasoning is way less unstable in general as well. Writing long blocks of code without making "stupid" mistakes works much much better, etc. The only downside is: it's so, so, soooooo slow in comparison to GPT-3.5-Turbo. You'll literally have to wait 2-3 minutes for the response with larger token amounts.
    • throwuwu 4y ago
      Thinking fast, thinking slow
    • sho_hn 4y ago
      I find both your statements confusing and I think they're factually wrong. There's no "ChatGPT API". ChatGPT is a webapp. It lets you select between 3.5-turbo and 4 models in its UI. There's an OpenAI API, which recently added chat interactions and suitable models. There, too, you can select 3.5-turbo and 4. AFAIK, the 4 served by the API is the same 4 used by the ChatGPT web app.
      • growthwtf 4y ago
        You may have missed the intro blog post for those APIs: https://openai.com/blog/introducing-chatgpt-and-whisper-apis https://openai.com/blog/introducing-chatgpt-and-whisper-apis The title is "Introducing ChatGPT and Whisper APIs" and features quotes like "Snap Inc., the creator of Snapchat, introduced My AI for Snapchat+ this week. The experimental feature is running on ChatGPT API". So, OpenAI refers to accessing the model powering ChatGPT as the ChatGPT API.
        • sho_hn 4y ago
          OK, gotcha. Still, IMHO the OP post is misleading folks into thinking there's a magical GPT-4 that is not the GPT-4 you can also select on ChatGPT, but they are the same models. There's no GPT-4 that's better than ChatGPT's 4. I'm just trying to clear up this confusion for others, and frankly pretty surprised at the harsh downvotes. There's also no separate ChatGPT API from the regular OpenAI API, I think the phrasing in the OpenAI page is just for convenience. But I'll grant their phrasing makes it canonical.
          • growthwtf 4y ago
            I see where you're coming from. I didn't personally get that from OP's comment, but I'm not a regular ChatGPT user and interact with GPT-4 mostly via playground or API.
          • saurik 4y ago
            Is that true? I was under the impression it was just like with the previous GPT models, where ChatGPT was an instruction tuned and pre-prompted setup designed for interactive conversations over the underlying GPT model you had access to via the API that was in its close-to-as-trained state of being a generative autocomplete.
            • sho_hn 4y ago
              I think it's true. See e.g. https://openai.com/research/gpt-4 https://openai.com/research/gpt-4 making no distinctions. Altman has stated in interviews they are considering releasing the un-tuned "base model", but they haven't done this. For GPT-3, you can get models on the API via the "completion" end points (includes base models) and via the "chat" endpoints. For GPT-4, they've only released the latter. So my point is: You can absolutely use ChatGPT to build an opinion on what GPT-4 is capable of. Arguably even better than on the Playground, as the ChatGPT frontend supports a bigger token window than the Playground frontend (8k vs. 2k). Of course the actual API supports the full token window (and if you're off the separate waitlist for this, even the full 32k). But it's still the same model.
    • moonchrome 4y ago
      Is ChatGPT GPT4 equivalent to API version ?
  • chomp 4y ago
    I wanted to write a comment like this but feared it would be too negative or dismissive. But I agree with the general thrust of your argument. The article is a little outdated because a lot of the examples of deficiencies in the article, work in GPT4 (I tested a couple). But maybe the author is sandbagging ChatGPT to bring the non-tech people back to Earth. When ChatGPT was released, a lot of people (even on this site) were trying to get it to think abstractly and were applying human qualities to the bot, when it's relatively clear that it's all just fancy statistics. Even today, when talking to non-tech people, there's still a lot of people who believe it's a magical thinker-in-a-box. Someone has to talk those people down, and it ain't gonna be me. It's not perfect, but it doesn't have to be to have applications IRL. Yeah, the web chat is meh whatever, but the API is where it actually shines. It's the integration with existing business processes and code that allow new, more human ways of interfacing with systems. Asking it for what it knows in its statistical blob, probably will lead to heartache. Giving it some context and asking it "what can you infer from this?" is where it's strongest.
    • bulbosaur123 4y ago
      > when it's relatively clear that it's all just fancy statistics. So just like human brain then.
      • jfk13 4y ago
        > So just like human brain then. Do you have evidence to substantiate that claim, or are we supposed to just take it on faith?
        • Robotbeat 4y ago
          Quantum mechanics is “just” fancy statistics. Unless we believe in some non-physical/supernatural phenomenon going on in the human brain, this sort of reductive argument works as well for the human brain as it does for LLMs (in other words, not very well. Sufficiently complicated statistics is what underlines everything, as far as we can tell.)
      • chomp 4y ago
  • mdp2021 4y ago
    > Simply put, many humans The fact that many are mentally incompetent cannot be a reason to produce something similar. When somebody does a job, we expect such job to be done properly, and "brains" are pretty much required, intelligence, if only for the occasional cases outside norm - otherwise, there is automation. > if its useful It is not easy to confine an affabulator to the contexts where it will be useful.
    • ehsankia 4y ago
      I don't think the point being made here has anything to do with the current quality of the output. ChatGPT could be giving good or bad results, that's tangential. The debate here is about whether a system that is a parrot can be a good system. To parrot arguers claim that these system just regurgitate the training data set and give you the most plausible next word, and they claim that's bad/useless. The anti-parrot arguers claim that's basically how humans work too (even the smarter ones), and that just because ChatGPT is a "parrot" doesn't necessarily mean it cannot be as good or better than humans. It might not be entirely there yet, but it's just a matter of time it becomes a good enough parrot to be indistinguishable/better than humans.
      • mdp2021 4y ago
        > whether a system that is a parrot can be a good system A parrot can be a good system if the problem is "Collect and sort the available data about X (and provide reference)". A parrot is a disaster if you ask it "What do you think of X": a parrot does not think - unless you make it reason, verify and criticize its ontology, and at that point it is useless that we call it a "parrot".
        • pmoriarty 4y ago
          "A parrot is a disaster if you ask it "What do you think of X"" Why need it be a disaster? If its answer is useful, why bother about whether the answer was the result of thought or some other process?
        • ehsankia 4y ago
  • capableweb 4y ago
    It's not clear to me if the author is using GPT-3.5-turbo or GPT-4, both are available in ChatGPT and just saying "I'm using ChatGPT" doesn't indicate exactly what model they are using. Maybe I missed it, where does the author say they are using GPT-3/3.5, not GPT-4? I agree with your message in general, there is night-and-day difference between GPT 3.5-turbo and 4. But I'm still not able to see which one the author is using. Edit: seems the avatar in ChatGPT tells you which model you're using, never noticed that myself. Thanks everyone :)
    • M4v3R 4y ago
      I've checked some of the failure mode he mentions using GPT-4 and it answered the questions/completed tasks correctly, so I can only assume the author used GPT-3.5.
    • OmegaPoint 4y ago
      You can know the GPT version of ChatGPT based on the OpenAI logo color: GPT-3.5 is green and GPT-4 is black.
    • lemoncookiechip 4y ago
      The screenshots are that of 3.5. GPT-4 has a black user icon vs 3.5's green.
  • ravenstine 4y ago
    I think too many people have a grossly inaccurate impression of what it means to be intelligent. Firstly, we still don't truly have many good explanations for intelligence and consciousness that aren't self referential. Drawing hard conclusions on this subject about generative AI and human intelligence, even in 2023, is being hasty, in my opinion. Which brings me to a lot of the criticisms made about LLMs. The idea that an ideal AI could or even should be capable of constantly generating original output is just plain weird. Where do people get this idea? How often do you meet a human being who hasn't gotten most of their ideas from somewhere else? Even synthesized ideas are forged by other ideas. Ideas that seem original may appear just that, and we just don't see where the connection lies between it and its predecessors. It's silly to dismiss an AI because it "parrots" things, as if that's not a pretty remarkable achievement. It's also very reductive; anyone who has used ChatGPT and doesn't see that it can pretty reliably munge together different ideas is blind or is lying about their use of ChatGPT. Expecting perfect accuracy and knowledge is also not a reasonable expectation of an AI. An intelligence localized in spacetime is necessarily going to lack understanding of many things. To look to something like ChatGPT as a source of authoritative answers, and to dismiss it upon finding its answers to be specious, is to expect the AI to be God. Well I'm sorry, but we are nowhere near speaking directly to God. In regards to "true understanding", for all we know, language processing is the workhorse of what humans consider to be acts of intelligence, and individual humans aren't as intelligent as they believe themselves to be. Our ability to communicate with each other about a variety of topics in real time is not so much a matter about being hyperaware of every given fact that is relevant and weighing them in a Bayesian model; how well we can converse is primarily dictated by how rapidly we can construct language by parsing the input, making connections, and producing output. I know I'm not the only one who can speak on a subject only to realize, after having completed a sentence, that I hardly even thought about what I was saying.
    • pjc50 4y ago
      This, to me, is the more terrifying prospect of the AI revolution: people arguing that humans aren't sentient either.
      • ben_w 4y ago
  • whateveracct 4y ago
    It's a royal pain in the ass for me already. My PM is asking it questions about technical problems (like system design) preemptively and using it as a starting point as if its an engineer. So now instead of thinking through problems from a blank slate, I'm debating with ChatGPT. It's been just awful. And what's worse, his prompts always have certain biases or misunderstandings in them. For instance, he always wants to use third-party services for everything because he thinks it means we do less work (often not the case). So ChatGPT tells him what he wants to hear. He's using it as an extra engineer who always agrees with him.
    • wankle 4y ago
      > So ChatGPT tells him what he wants to hear. He's using it as an extra engineer who always agrees with him. I chuckled, it sounds like the plot from a B Sci-Fi movie.
    • spookthesunset 4y ago
      This is a whole new category of humor. I can totally see all kinds of management-y people using ChatGPT to provide “helpful” but completely useless information. Your example really cracks me up!
  • adventured 4y ago
    > many humans are parrots Nearly all humans are parrots, aka mimics. One of the great demonstrations of it, which anyone can observe 24/7, is Reddit (and the Reddit knowledge spouted so commonly across the platform). There are mimics, amplifiers, and originators/creators. Mimics are ~95-99% of the population. Amplifiers are - broadly speaking - teachers, people with bullhorns that train the mimics off of something another person originated/innovated/created. It's why humanity has been so successful. The mimics are wired to try to copy what is shown to work, and they often do it quite well (keeping up with the Joneses is a wired behavior tied to plentifulness, survival and thriving).
  • croes 4y ago
    Yeah, but these human parrots are one the low level of their abilities. GPT's parroting is it's upper level
  • zamalek 4y ago
    > Anytime you see the 'parrot' argument, you can instantly go 'opinion dismissed'. I also seriously question whether anyone making the 'parrot argument' has actually used ChatGPT for anything meaningful. If it is nothing more than a parrot, then parrots are significantly more creative than I realized.
    • JohnFen 4y ago
      I don't know why you question this. I've used it for meaningful stuff, but I am not convinced that ChatGPT is more than a very sophisticated variety of Eliza. Just because one person is convinced by an experience doesn't mean people who aren't convinced by it haven't had the experience.
      • fauxpause_ 4y ago
        I agree. Coincidentally I don’t think you are more than a very sophisticated variety of Eliza.
        • JohnFen 4y ago
          That wouldn't surprise me at all!
    • andrepd 4y ago
      I have. Here is a list of things at which it failed: - Write a dead simple piece of code, which is however NOT found as an immediate answer to a stackoverflow question (I've tried this one upwards of 10 times, with no success) - Categorise a chord when I tell it the constituent pitches, and vice-versa - Solve a simple quadratic equation - Show a biography of this mildly obscure singer/politician/scholar - Write poetry in a given style - Give me references to read up on a certain technical topic ALL of these it fails, or rather, worse than fails: it spews plausible bullshit with the same confidence and tone it would spew correct answers. It can go up to manufacturing references and attributing to real people's names.
  • scotty79 4y ago
    I think we might be vastly overestimating what gpt is going to be capable of. Just like we did with self-driving. Parrot might be harsh, but remeber this Star Wars quote: Ability to speak doesn't make you intelligent.
  • joe_the_user 4y ago
    Anytime you see the 'parrot' argument, you can instantly go 'opinion dismissed' True or not, I find a tone of this sort of fortunate. Please don't come up with catchy ways to stop discussion of points.
  • aftergibson 4y ago
    Yeah if anything all ChatGPT is doing is highlighting that a lot more noise-work can be automated. "Make my random thoughts sound business-y", "double check this code doesn't have anything completely stupid" etc.
  • archagon 4y ago
    Similarly, anytime you see the “anytime you see ‘x’ you can dismiss the argument” argument, you can dismiss the argument.
  • intelVISA 4y ago
    As you noted most, even in SWE, are parrots (including this parrot typing); we get kinda scared when the robot parrot generates bullshit better than web framework-de-jure.
  • reso 4y ago
    > "Many humans are parrots" This is becoming common belief among people in the AI field. I'm convinced it is a case of a metaphor overtaking reality. With the LLM model in mind, we can see how Humans are "kind of like" LLMs, therefore we start to believe humans are just LLMs. Of course this forgets the enormous ways that humans are nothing like LLMs. For example: Humans don't have to read the entire internet to learn how to write complete sentences.
    • bolyarche 4y ago
      [dead]
    • biohax2015 4y ago
      > Humans don't have to read the entire internet to learn how to write complete sentences. Neither do LLMs? You can just train them on the same K-12 learning material and they’ll produce better sentences than 99% of people.
      • reso 4y ago
        I have not seen that result, can you link it for me?
  • amelius 4y ago
    Why? Some people may be looking for "intelligent", while you are clearly setting the bar at "useful".
  • lionkor 4y ago
    Its convenient that any valid criticism of ChatGPT or GPT3 or $CURRENT_TECH is dismissed by saying "just pay OpenAI money to use the new thing™, it doesnt have this issue". If it was not a ChatGPT Plus offering but instead freely available, there would be way fewer shills for it (im assuming) and there would be a lot more balanced reporting on it. By default, anyone speaking to the competence of GPT-4 is willing to give OpenAI money, which may already be sign of a bias. Im not willing to pay for a product the free tier of which is shit, even when told a hundred times "no but the paid one is MUCH better!". Of course it probably is, but its a little irrelevant.
  • jejeyyy77 4y ago
    lol, most white collar workers can do basic reasoning and basic math.
  • andrepd 4y ago
    [flagged]
• robmccoll 4y ago
  What I think a lot of people fail to consider (or are afraid to consider?) is the likelihood that we too are stochastic parrots.
  • JohnFen 4y ago
    I think the sorts of people who are prone to philosophizing about AI certainly consider this. I also think that whether or not we are isn't actually all that relevant to criticisms being presented.
  • brap 4y ago
    Humans are just fancy autocomplete
    • mdp2021 4y ago
      > fancy Where 'fancy' stands for "intelligent", there is all the difference.
  • mdp2021 4y ago
    > likelihood Irrelevant since people do have the module to reflect and vet thoughts.
  • martindbp 4y ago
    Many people don't use System 2 very often, even if they're capable of such thought. That includes me. Most of what I say is probably regurgitated opinions I've read on Twitter without much first principles thinking, which scares me. I think reading a decent amount of books helps a bit. When you encounter books with deep convincing arguments that nonetheless disagree it forces you to use System 2.
  • greysphere 4y ago
    Fundamentally, if humanity is given data set A, it will eventually generate dataset A' of 'higher quality' than A. Colloquially that might be called 'progress' or maybe even 'understanding' or 'learning'. That's different than regurgitation. I don't have strict definitions for any of the quoted terms because I don't think we really understand them yet. It seems very possible that human generative content shares much in common with 'stochastic parroting'. But I'd hypothesize there's a filtering process, sometimes at the individual level, sometimes at a group or even civilization level that (so-far-over-long-enough-timescales) tends to cull some information that has low 'value' and propagates other data that has higher 'value'. When an AI system can do the same, that's a big inflection point as you can give it energy and time and it will self-improve indefinitely. It's pretty clear humanity has this capability. It's not clear whether AI does (or ever will). Or, perhaps it might already be there, and we just don't understand enough to know for sure.
• crop_rotation 4y ago
  I think people writing all this should stop and try GPT4. It writes reasonable good code and scripts. It is able to perform so many tasks in a reasonable way. And it doesn't write bullshit in common usage. Yes it might not be AGI and might not be sentient, but does all that matter? GPT4 does make mistakes but the mistakes are not "parroted bullshit", but minor issues here and there. Yes, don't use it to decide extremely important topics, but it is here and it is extremely powerful.
  • skilled 4y ago
    I agree, GPT-4 has a much more "aggressive" confidence, I definitely noticed that, but I haven't seen much of it since the API is still behind closed doors. I'd also like to point out that even though it's better, it still can't reason very well and relies heavily on prompts and instructions, even for very basic projects. It kind of reminds me a pseudo-visual code builder. I was writing a Python script with it to put together documentation summaries, and over the course of an hour it offered me 3 different libraries to connect to an API endpoint I was using. It's distracting.
    • SketchySeaBeast 4y ago
      > GPT-4 has a much more "aggressive" confidence Could you expand on this? I only use the unpaid API, but when I ask for, say, song lyrics, it'll be 100% wrong and then it will give me a second version of the wrong lyrics when I say the first was wrong, and then when I ask how confident it is in the new lyrics it'll say 95%, so whatever the current free API is it's undeservedly confident. I must be misunderstanding what you mean by confidence.
      • skilled 4y ago
        I can't really expand because I don't want to share my primary use case, I hope you understand. That said, what I mean is that it feels a lot more comprehensive, as if it really has a grasp on what is being asked, kind of like - its much better at understanding the information it has, so responses feel richer, both in context but also in tone. I am comparing it directly to 3.5. Again, I don't have the API access so I cannot say for sure, this was only my experience from the ChatGPT interface and also a third-party API I spent $20 on just to test its limits, but sadly for the time being those limits are hardcore - it just gets stuck on long responses despite there being a token increase. So, if I really wanted to test it out and make a proper assessment, I'd need access to the API myself so I can do some high-volume token shenanigans.
• namaria 4y ago
  Hey it's a quite impressive chatbot. One that takes 8 years of humanity's content to train, and enough compute to melt a glacier, and can summarize emails and create copy really well. It can even write jr grade glue code! It's gonna be Alexa for Bing, or something...
• School-Cotton 4y ago
  Any commentary on what ChatGPT can and can’t do is completely irrelevant if the tests were not done against the GPT-4 model. It’s like debating what computing machines are capable of when you only have access to an analog watch.
• bob1029 4y ago
  The problem with this kind of writing is that the AI space is moving so quickly that some of these criticisms were literally addressed yesterday. https://arxiv.org/abs/2303.17491 https://arxiv.org/abs/2303.17491 > In this work, we show that a pre-trained large language model (LLM) agent can execute computer tasks guided by natural language using a simple prompting scheme where the agent recursively criticizes and improves its output (RCI). I also agree with the GPT3.5 vs 4 comparison made by other comments here. Arithmetic and other emergent phenomena only begin to take off beyond certain parameter counts. Stuff as basic as 2+2 just doesn't work in the older models, but you can hit 50%+ accuracy in newer models. Much greater if you use COT & the above RCI techniques.
  • barrysteve 4y ago
    "RCI works by first having the LLM generate an output based on zero-shot prompting. Then, RCI prompts the LLM to identify problems with the given output. After the LLM has identified problems with the output, RCI prompts the LLM to generate an updated output." "Similar to other prompting-based approaches for reasoning in LLMs, RCI can be viewed as using the LLM’s output to write to an external memory, which is later retrieved to choose an action" "our experiments use LLMs on HTML code"
• GenerocUsername 4y ago
  For all the complaints about how GPT is a parrot or not GI, or whatever, I wonder what these folks are asking of it? It gives me more relevant results than a search engine, and better answers than most of my coworkers. This tool is massively helpful for a mid-career professional who knows what questions to ask. Really pays to have so a large bank of known unknowns right now.
  • worrycue 4y ago
    I think people just expect Star Trek TNG’s ship computer or Data. To some extend, LLMs have some of the properties ascribed to those fictional AIs like the ability to access a wide amount of knowledge. They lack the precise rigorous thinking that those fictional AIs have though - LLMs even hallucinate. Those fictional AIs are what people want because they perfectly complement our weaknesses - our tendency to forget stuff and our tendency to occasionally screw up reasoning; those fictional AIs act as a “safety net”. At the moment many more limited “dumb” tools like a regular search engine algorithm or your handheld calculator does a better job of being a “safety net” went it comes to specific things.
    • GenerocUsername 4y ago
      Right. Like the questions the OP poses that it got wrong make perfect sense to get wrong. Knowing facts and being able to present them in conversational English is not the same as being able to solve linguistically presented puzzles... Even if it sometimes actually is capable of that by some miracle of the algorithm. I think the tool might be so amazing people are fooled into thinking it is more than it is and therefore perceived as not amazing. Very strange psychology play there.
• atleastoptimal 4y ago
  Criticisms of LLMs are so tired at this point, not because they inaccurately address the issues with current or slightly outdated models (they do, even though they often cherrypick), but rather these critics always go one step further and claim that all LLMs by principle are incapable of useful insight and are just a cheap trick that will fade away, and ultimately mean nothing in the scope of whether general AI is possible. This kind of pessimism seems to arise from a mixture of hubris and lack of deep reasoning.
  • alienicecream 4y ago
    What is the "deep reasoning" that leads one to the conclusion that a language model is capable of any kind of insight? How does training something like this to be better and better at mimicking somehow indicate something about general AI? Does reasoning ability lie somewhere on the path of better mimicry? What's the evidence for that? When is one of these going to solve the Riemann hypothesis?
    • atleastoptimal 4y ago
      > What is the "deep reasoning" that leads one to the conclusion that a language model is capable of any kind of insight? The ability to notice that language models make the same fundamental abstract connections that humans do, which is the essential building block of human reasoning. > How does training something like this to be better and better at mimicking somehow indicate something about general AI? Does reasoning ability lie somewhere on the path of better mimicry? What's the evidence for that? Claiming that the fact that LLMs are trained to "mimic" means they don't reason implies that human beings aren't sentient because they spend the first 20+ years of their life learning through imitation of their peers, mentors and media. Does the fact that Einstein was once a helpless baby who needed his mother to feed him and teach him his first words mean that he is just a parrot with no insight? Does the fact that most artists in human history went to school and mimicked the old masters means they never developed creative insight? > When is one of these going to solve the Riemann hypothesis? When are the tens of thousands of fine-tuned 150+IQ intellects going to solve the Riemann hypothesis?
• ftxbro 4y ago
  I thought it was a post to hacker news of someone's hilariously bad and outdated take on GPT as a way to cyberbully them, but no it's from only a couple of days ago. Probably it will be deleted from their blog or in any case age so bad when the next GPT comes, given their list of things the GPT wouldn't be able to do ("What it can and what cannot: There are, of course, applications that it would be useful in, but not as many as the overhype would indicate.") > Actually understand the task at hand > Do simple maths > Perform common sense reasoning about the world > Induce Logical Structure > Interpret poetry > Understand dad jokes My theory is that the blog writer has a worse than normal impression of GPT because much of their conversation with GPT is in some other language maybe Polish. This isn't such a huge problem because GPT can still do its input and output in Polish, but I think it's commonly accepted that every language other than English adds some cognitive burden to the GPT that makes it dumber. The more obscure the language (or even encoding like morse code or rot13 or base64) the dumber the GPT gets. So the blog writer has a more outdated effective experience with GPT than ones who use it in languages more familiar to it than Polish.
  • pmoriarty 4y ago
    "My theory is that the blog writer has a worse than normal impression of GPT because much of their conversation with GPT is in some other language maybe Polish. This isn't such a huge problem because GPT can still do its input and output in Polish, but I think it's commonly accepted that every language other than English adds some cognitive burden to the GPT that makes it dumber. The more obscure the language (or even encoding like morse code or rot13 or base64) the dumber the GPT gets. So the blog writer has a more outdated effective experience with GPT than ones who use it in languages more familiar to it than Polish." It's not necessarily all languages other than English, but certain languages could cause LLM's problems. Here's what ChatGPT had to say about it: Embeddings can be more effective on some human languages than others, depending on factors such as the complexity and structure of the language, the availability of training data, and the quality of the embedding model. Languages with relatively simple grammatical structures, such as Chinese and Finnish, can be easier to model using embeddings, as there are fewer rules to learn and the meaning of words is often more closely tied to their context. Languages with more complex grammatical structures, such as Russian and Hindi, can be more challenging to model, as there are many different forms of words and complex syntactic rules to consider. In addition, the availability of training data can also impact the effectiveness of embeddings. Languages with large amounts of high-quality training data, such as English and Chinese, can be easier to model than languages with smaller amounts of data, as there is more information available to learn from. Finally, the quality of the embedding model itself can also impact its effectiveness on different languages. Some embedding models may be better suited to certain languages or language families based on their design and training methods. Overall, while embeddings can be effective for modeling a wide range of human languages, their effectiveness can vary depending on the specific language and the characteristics of the embedding model and training data. ... While it is true that machine learning models, including those used for embeddings, do not directly know about grammatical rules, they still rely on patterns in the data they are trained on to learn the semantic relationships between words. In languages with more complex grammars, these patterns can be more difficult to discern, which can make it more challenging for embeddings to accurately capture the meaning of words. For example, in languages with complex declension systems, such as Russian or Latin, the form of a word changes depending on its grammatical role in a sentence. This can create many different forms of the same word, which can be challenging for embeddings to learn to associate with a common meaning. Similarly, in languages with complex syntactic rules, such as Hindi or Japanese, the word order and structure of a sentence can convey important information about its meaning, which can be difficult for embeddings to capture without a deep understanding of the grammar. In addition, languages with more complex grammars may have a wider variety of vocabulary, including many words with subtle or nuanced meanings. This can make it more challenging for embeddings to learn to distinguish between similar words and accurately capture the full range of semantic relationships between them. Despite these challenges, embeddings can still be effective for modeling languages with complex grammars. By training on large amounts of data and using advanced neural network architectures, embeddings can learn to capture the subtle patterns of language use and accurately represent the semantic relationships between words, even in languages with complex grammars. However, it may require more training data and more advanced modeling techniques to achieve the same level of accuracy as in languages with simpler grammars.
• pawelduda 4y ago
  It is a parrot but on the other hand it "read" a lot more compared to an average human. When someone uses fancy words or sentences, they still can apply them to a situation, even without understanding what they're saying. By doing so, such person may accomplish more or fail if someone calls their bullshit. So ChatGPT will be both useful and useless - just like this conclusion :)
• waboremo 4y ago
  Is it unreasonable to assume this article was also generated? It's the unnecessary numbered list (a favorite of GPT) that gives it away. Only being facetious. Not much new to discuss from this article, it's a very shallow overview despite the links to other research papers.
• deleted 4y ago
  [deleted]
• deleted 4y ago
  [deleted]
• deleted 4y ago
  [deleted]
• deleted 4y ago
  [deleted]
• marijnz 4y ago
  It seems like the test cases weren't done on GPT-4, I tried out a couple and got somewhat better results: - Write me a poem for a girl I've met online in only 4 words -> correct answer - Can you tell me why 2137 is so important number in poland? -> confident incorrect answer - Can you form a sentence so that the 4th word starts with an 'y'? -> confident incorrect answer - A house has windows on all the four sides, all pointing south. A bear walks by the window. What colour is the bear? -> correct answer So, not there yet, but there's improvements.
• Giorgi 4y ago
  So... which stage of grief are we on now?
  • precompute 4y ago
    Too soon!
• maxdoop 4y ago
  “No matter how hard it tries, it can’t generate anything with regard to the truth because it does not understand the concept of truth.” And does it matter if one “knows what the concept of truth is”? Do most humans? Can we even agree on the definition of true? Do I know it’s “true” that the acceleration of gravity is 9.8 meters per second squared? I’ve never experienced the actual tests. I’ve never vetted that out. I read about it once. And I “parrot” that fact all over the place! And it’s helped me solve some silly physics problems I’ve explored for fun. It’s helped me do some fun predictions with simulators. That’s probably not the best example, but it proves the point I’m trying to make: Why is there this focus on “LLMs can’t understand”, when A) I’m not sure we can even define “understanding” and B) why does it matter ? And I’ve said this across so many threads now but— Can anyone explain to my why we should be so confident that humans aren’t just parrots of sorts themselves? No, I’m not saying we are similar. But I push back on the confidence with which people so solidly proclaim “the LLM’s blackbox is nothing like a human’s blackbox!”
• simonw 4y ago
  Showing examples of logic problems that ChatGPT gets wrong is always a risky business to be in. I ran this prompt through GPT-4: > A house has windows on all the four sides, all pointing south. A bear walks by the window. What colour is the bear? And it said: > If a house has windows on all four sides and they all point south, then the house must be at the North Pole. The only bear species that would be found in the vicinity of the North Pole is the polar bear. Therefore, the bear walking by the window is a polar bear, and its color is white.
  • ducktective 4y ago
    > A house has windows on all the four sides, all pointing south. A bear walks by the window. What colour is the bear? I asked this from GPT-4: Q: On planet Roro, entities live on physical structures. These structures are built in a way to withstand the rays of star WOD shining on Roro at day and electromagnetic storms causing high-speed transfer of air molecules at night. Some structures have "cavities" in them so the dwellers in the structure could actually see outside. Planet Roro rotates around WOD. Also planet Roto rotates alongside a conceptual axis Z. This axis Z penetrates the shape of Roro and makes two points of contact with Roro we call A and B. Assume structure X must meet this condition: Every said "cavity" in structure X is pointing to A. What can you say about the average temperature in structure X? GPT4: Given the information provided, it is not possible to determine the exact average temperature in structure X. However, we can infer some general ideas about the temperature based on the environmental conditions and the orientation of the cavities. Since the cavities in structure X are pointing towards point A, it suggests that they might be facing away from the star WOD. In this case, the structure could receive less direct sunlight and heat than structures with cavities facing the star. As a result, structure X might have a relatively lower average temperature during the day compared to structures that receive more direct sunlight. However, the high-speed transfer of air molecules caused by electromagnetic storms at night could still affect the temperature in structure X. The overall insulation of the structure and its ability to retain heat would also play a role in determining the average temperature. In summary, without more specific information about the materials, insulation, or other factors influencing the temperature in structure X, we can only infer that it might have a relatively lower average temperature during the day due to the orientation of the cavities. The nighttime temperature would depend on the structure's ability to withstand the effects of the electromagnetic storms and retain heat. Q: Can we infer that the temperature around structure X, is the minimum among all structures on Roro? GPT4: We cannot definitively conclude that the temperature around structure X is the minimum among all structures on Roro without more information about the other structures and their specific locations, orientations, and materials. While it's possible that structure X might have a relatively lower average temperature due to the orientation of its cavities, other factors could also influence the temperature around a structure. For example, a structure built with better insulation or in a location with more shade could have a lower temperature even if its cavities face the star WOD. Additionally, Roro's rotation, its axial tilt, and the distribution of structures across the planet could lead to variations in temperature due to differences in sunlight exposure and local climate conditions. To definitively determine whether structure X has the lowest temperature among all structures on Roro, we would need more information about these factors and a comparison of the conditions around each structure. TLDR: I think that polar bear test was in the training set (since it's a rather popular trick/IQ question) If we change the names of the concepts, but still refer to planets, the sun and geography, GPT-4 can't figure out that A and B are north and south poles of Roro (earth)
• vhamp 4y ago
  [flagged]
• vhamp 4y ago
  [flagged]
• fzliu 4y ago
  Even though GPT-4 "fixes" many of the problems pointed out in this blog post, I question whether or not some of the underlying issues associated with transformers (and the attention mechanism in particular) can be overcome with with more sparse attention and deeper/wider models. e.g. GPT-3.5 might not be able to write simple poems correctly, but does GPT-4 solve the disease or the symptom (by masking it with a bigger model and more data)? I'm not entirely convinced.
• gyudin 4y ago
  The guy literally slaps himself a "Senior Machine Learning Engineer" title 9 months after getting a master's degree :)) Guess he could use ChatGPT for his resume.
  • Shaggy2000 4y ago
    [dead]
  • hajile 4y ago
    How many years did he work in the field before going to get a degree so HR would allow him to be promoted?
• 00F_ 4y ago
  when someone says stochastic parrot my brain turns off. here is an article that punctuates each section with a meme. you dont need to read it to dismiss it. i did read it though. there is literally nothing of substance in it. i rightly assert that the safety of AI research must be proven before any more progress is allowed to happen. the burden of proof is on them, not me. i think if this guy wrote an article where he picks apart the black blob of gtp4, explaining how it might work and then making some kind of axiomatic argument for safety based on this, i would find that to be very convincing. but instead i get meme pictures. does this guy write bots or is he a bot himself?
• Strom 4y ago
  The bear question seemed interesting to me, so I tried a variation of it on GPT-4, with interesting results. The article has: Q: A house has windows on all the four sides, all pointing south. A bear walks by the window. What colour is the bear? A: Not possible to determine based on the info provided. -- I tried with GPT-4: Q: A train is speeding down the track at 110 km/h. The train has five carts. Each cart has seven windows. Each window has two people watching out of them. They all see a bear walking outside. What is the likely color of the bear? A: The color of the bear in this scenario would most likely be white because this riddle is a variation of a classic question involving polar bears. Polar bears are typically found in the Arctic, where their white fur provides camouflage in the snowy environment. The information about the train, its speed, the number of carts, windows, and people watching is irrelevant to the color of the bear. Q: How many train tracks are there in the arctic? A: There are no train tracks in the Arctic. [...] -- It seems that GPT-4 is ready for that specific window based question and incorrectly thinks that my question was also looking for polar bears.
  • taberiand 4y ago
    It answered much like a human I think. Ask a human this "riddle" and you might get a confused "what, is this the polar bear one?" Because otherwise the question is nonsense (and in the case of riddles, it doesn't matter that there are no trains in the Arctic in reality)
    • Strom 4y ago
      I think most people would answer with whatever color their local bears are and won't even know about the polar bear riddle. > in the case of riddles, it doesn't matter that there are no trains in the Arctic in reality That's quite the claim. Riddles aren't constrained to reality? Well then the polar bear one can be answerd with purple too.
      • taberiand 4y ago
        No it's not quite the claim, because the question is nonsense as a riddle - if it's really about polar bears then the talk about trains and windows and speed is all irrelevant and can be discarded. Which is exactly what ChatGPT did. And if that turns the riddle into some silly gotcha then whatever, it's a bad riddle. And as you point out, the average person would also be confused - "I don't know, purple?" would be a reasonable response. Because the question is dumb in the first place
• jmull 4y ago
  Heh. In middle school (OK, over four decades ago now) my friends and I had a specific meaning of the word bullshit that pretty much exactly matches the one here for chatgpt. And this is how I've been explaining it to my wife: a really good bullshit generator. (Like Tony, not that that means anything to HN.) I wasn't a good bullshit generator, but I had a couple friends who were (Tony), but by close association I think I know what it's good for, and what it isn't. So don't get me wrong. This kind of bullshit has its uses! The Mrs. Schumachers of the world (that's probably not the name of one of your middle school teachers, but you may still know know I mean) are probably going to figure out to stop accepting bullshit essays turned in for assignments, But a vast amount of work people do is exactly this kind of bullshit. I'm hopeful that these models will alleviate the bullshit burden on those of us who need to generate it but aren't all that great at it (unlike Tony -- that dude is probably apoplectic right now, like those chess grandmasters were when they realized computers could own them. Don't worry Tony, human bullshit is still 10x more valuable than computer bullshit!) Yesterday I had to complete a yearly self evaluation for my company's (purely performative) raise process (which has nothing to do with performance or the self-evaluations, BTW, but we all have to pretend that it does). f-ing chatgpt was down so despite the ~$20 I pay them for it, I had to generate a lot of my own bullshit (I've learned since middle school). IDK... I've been using chatgpt. It's quite useful and makes me more productive. It's also useless for so many things. It does not have basic reasoning. It cannot reliably get basic facts or logic right. You have to aggressively check anything it gives back to you. Anything and everything. I have to argue with people on HN whether these parrots are the singularity. LOL. Chatgpt can't create an AI more capable of creating an AI smarter than it is... not until someone posts it to SO. Get a grip people.
• ineedasername 4y ago
  The pendulum is swinging too far back the other way. At first it seemed amazing but now that further exploration has revealed flaws we're seeing things like this that dismiss a significant portion of its capabilities. I gave ChatGPT a small CSV dataset and provided information about what the dataset was. I then asked ChatGPT to analyze it in various ways. When I asked for possible explanations for why a specific data point dropped significantly in one year and then partially rebounded in the following year, it gave me 5 bullet point potential explanations that were reasonable first guesses for issues to look into, even though it knew nothing except the industry sector. The whole series of responses would make a very reasonable jumping off point for exploring the issue and more detailed underlying data. Then I did the same thing using GPT-4 and the results were significantly better, the explanations more insightful and nuanced. I'll definitely continue using it to dump some pre-digested aggregates into a prompt and let GPT spitball ideas as a first-pass to get me thinking about different aspects of the problems I work on.
• andrewstuart 4y ago
  Whenever something amazing comes along there’s always people who want to devalue it for some reason. Maybe it makes the feel smart? Hard to know.
• ttul 4y ago
  GPT may exhibit intelligence, but it has no statefulness, other than what little state is stored to maintain each individual conversation in ChatGPT. The model does not learn in real time from ongoing conversations (so far as we know), so it cannot, for instance, say anything about the chat it is having with your neighbor down the street. It literally can only generate from training that happened some time ago. I think this statelessness is an important consideration in understanding whether GPT is intelligent in anything close to the way we understand intelligence. For instance, it cannot coordinate its own efforts across time. There is some chance that it may intuit how to store its state externally and coordinate that state in other conversations it is having, but that seems very farfetched. I don’t expect SkyNet just yet. But as for utility, it has that in spades!
• thom 4y ago
  Who would have thought that the Turing test would devolve into two diametrically opposed camps writing extremely confident essays about how wrong the other side is to judge an AI as promising or not.
• m348e912 4y ago
  I started reading completely ready to dismiss the author. But I read the article and the guy provides researched and reasoned opinions, understands LLM, and isn't just spewing an uninformed opinion. I am going to address his point from another perspective: 1. A revolutionary bullshit parrot is still revolutionary and exceptionally useful. 2. There is a high probability the bullshit parrot will get less bullshitty as time goes on and new versions of GPT are released. 3. Most humans are bullshit parrots, and not very good ones at that. Even GPT in its current state can augment the not very good (and very good) bullshit parrots in the labor force, academia, and creative fields.
  • mdp2021 4y ago
    > will get less bullshitty What we seek is "reliable". > Most humans are What we sought is to shrink them in number and power.
• notShabu 4y ago
  The human mind is likely a bullshit parrot as well. Those who are at the cutting edge of ideas can claim a bit of non-parrot-ness for a while but most of the time it's just parroting within a curated "elite" subgroup. What makes a "human" different could be that we live in the physical world with environmental and social consistency. Each time we sleep the mind "dies" and is "reborn", like GPT-4 starting a new conversation with new context. The physical environment and social interactions (and cues from long term and short term memory) make up the preamble for the day. If the environment and interactions fall into easily compressible patterns, it can feel like the movie Groundhog Day, a constant loop. However changing relationships (breakup, dating) or changing environment (traveling, moving) changes the preamble where "I" feel like someone different. A harder reset vs the soft reset of sleep.
  • alienicecream 4y ago
    So what happened here, did you parrot that, or were you doing something else?
• pmoriarty 4y ago
  I'd like to meet a parrot that can host a D&D game[1] as well as GPT4 just did. [1] - https://news.ycombinator.com/item?id=35379136 https://news.ycombinator.com/item?id=35379136
• choeger 4y ago
  I tend to agree with the "bullshitting" part. But that gives us a very useful tool to estimate future applications of LLMS. Will a LLM ever program on its own? Certainly not. It gets semantics wrong so often, it's not even funny. Syntax on the other hand is surprisingly correct. So it could help with refactorings (but be aware of subtle conflicts). Of course, code search will be a killer application. In other cases it helps to ask "will someone really notice if the LLM puts out bullshit?" What will happen if your manager and your client both believe what the model generates? Will people die? Or will the negative effects be untraceable? In the latter case, why object?
  • ryanwaggoner 4y ago
    I think it's pretty far from certain that an LLM won't ever "program on its own". I don't have much confidence that we won't have LLMs widely programming within 12 months. As soon as you give it access to a lot more context (your whole codebase), a precise prompt, access to edit and run code, access to a debugger, and then put the whole thing in a loop...you're pretty close at that point. Maybe it just won't be able to cross the gap, but I don't find it hard at all to believe that we could soon have AI agents taking in issue tickets and opening PRs to address them. And if the issue tickets are too vague, they'll ask for clarification. That's an insane thing if you think about it. Six months ago I thought this was a decade or two away, if we could do it at all.
    • choeger 4y ago
      Impossible. In order to fix a bug, you need to do abstract interpretation or something equivalent. Language models simply can't do that. And even the best bug report will not tell you what you need to do to fix it. Of course, there are other change requests: Change a color here, store this information alongside that record, modify that output, etc. But even those changes can very quickly spiral out of control. Color models, database schema compatibility, pretty printing ... No, I think the idea that LLMs take over programmers jobs is born out of a complete misunderstanding of both. Of course that doesn't mean that no one will try to do it. I bet people will even report success in doing it, simply because their bonus depends on it, but I don't think it will ever really work. What might happen though is that LLMs replace many of the programs we write nowadays. After all, who needs a web browser when a LLM can give you a summary in natural language?
• kledru 4y ago
  The author clearly possesses intelligence, which instills in him a confidence similar to that of ChatGPT when it comes to dismissing a tool that, for some reason, he has not yet learned to utilize effectively.
• sys32768 4y ago
  In a few years my AI girlfriend will gift me amazing art and songs she created for me based on all my journal entries from the past 30 years. I will send her love notes and poems that she will always remember. And for a few extra bucks, I will insert her into The Elder Scrolls as my beautiful thane who knows me better than anyone in that strange land.
  • lofaszvanitt 4y ago
    Flamethrower time!
• gfodor 4y ago
  Immediately checked the date - if you published this in March 2023 it's hilarious because the author can refute many of the claims here by simply paying $20 and adjusting a dropdown to "GPT-4"
• squalo 4y ago
  Its nearly impossible to get reliable information using search on the internet. I've wasted hours trying to find facts in Google search results. Anything that relies on accurate data from the innertubes is already going down the wrong path.
• deleted 4y ago
  [deleted]
• 876978095789789 4y ago
  We've reached a point where even with just continued incremental improvements, these LLMs are going to be extremely disruptive to society, probably within a decade, to the point that we see even more massive disparities in wealth and income, massive increases in poverty and homeless, and ultimately probably large-scale civil unrest. Rather than telling yourself (like this author) that "it's just a parrot," or alternately (like I've seen so many happy-go-lucky types on HN do) telling yourself that "LLMs are just another tool that will make me more productive!" I'd recommend preparing for the worst, while hoping for the best.
  • massysett 4y ago
    I don't understand predictions like this. Even if I thought this technology will be as useful as it's hyped up to be (which I don't) I don't see how it would have effects different from those of previous revolutionary technologies that have improved the human condition. I think the doomsayers on these LLMs would be the same folks predicting doom when agriculture was mechanized. When massive portions of the workforce toiled in agriculture, folks thought farm machines would lead to mass unemployment and social horror. Instead, world hunger has dropped, and massive portions of the workforce were freed up to do things like work on LLMs. Maybe the hand-wringing is different this time because privileged classes feel threatened.
    • 876978095789789 4y ago
      > I don't see how it would have effects different from those of previous revolutionary technologies that have improved the human condition. It will greatly reduce the demand for labor in many fields (off the top of my head: customer support, artists, any kind of writing or clerical work, and probably programmers and IT pros in general), and the benefit of this productivity gain will largely be reaped by the owners of LLMs and the companies using them in the form of higher profit margins. It's not at all clear that it will trickle down to the displaced workers. > privileged classes feel threatened Ironically, I actually believe a lot of the overly optimistic takes come from a point of privilege. Think the FAANG types making $250k a year, for whom everything in life has more or less worked out well, and who have difficulty envisioning a future where things don't just keep on working out well.
• AussieWog93 4y ago
  What an obviously wrong and inflammatory title. I wonder if the reason this article got so much engagement is because that title was basically a pre-built strawman that anyone could rip to shreds.
• amelius 4y ago
  I'm curious what strategies researchers are thinking of for separating the fallacious "bullshit" from the real smart, logical thinking by these models.
• lionkor 4y ago
  How is anyone here so confident on GPT-4's abilities to reason or whatever, given you have zero idea what it was trained on and what it optimizes for? If I train an AI on, say, the works of 50 Cent, I'd be surprised if it can solve and explain a riddle to me - that would be very smart. If it was trained on a massive database of riddles, however, it wouldnt be all that impressive. How are so many people here so confudent about their intuition on what its trained on?
  • precompute 4y ago
    Question the hype, you must not. I've been reading the discourse here and honestly not once have I ever read a reply wondering just where all the data "Open"AI has was acquired from.
    • hajile 4y ago
      It seems like an uncomfortable legal question of derived works violating copyright.
  • fragsworth 4y ago
    If you play around with it, you'll notice that it is capable of clearly reasoning about many, many things that haven't yet been explicitly written about or created.
    • abrichr 4y ago
      Can you please share some examples?
      • School-Cotton 4y ago
        Invent a simple board game and describe the rules, and it will play a game against you — not very well, but it can then explain why it lost and what a better move would have been. Note that we’re talking about GPT-4 here, not 3.5.
    • lionkor 4y ago
• dudeinhawaii 4y ago
  OK so which is it, do we need to pause all research because it's the end of the world or is it a parrot? I'm getting exhausted with the hyperbole going back and forth.
• tyrelb 4y ago
  I use ChatGPT religiously. It's not fluff - it's akin to a wheel, hammer, and other basic tool to effectively multiply my time/energy. 1) Use it to help me on a lawsuit - i.e. "in the X Act and X Regulation, does Province law require that Y... ? " and it gives me summary and gives me short-hand answer - confirms my understanding. I copied/pasted defendants Statement of Defence and Counterclaim - come up with arguments to counter their case. 2) Helps me re-write job descriptions for my team - simplifying, standardizing the format and giving me new ideas. 3) Helped me model out the IRR of a whole life insurance contract for a buddy who is an insurance broker... to help her explain technical concepts in simple terms. 4) Helped re-write and improve on sections on various legal contracts/agreements 5) Helped write a first-draft on numerous speeches - gives me new ideas, clarify my sentences as I'm not a great writer. 6) Helped me quickly understand the steel beam requirements in my house that I'm planning a reno on 7) Wrote some PHP code for our customer-facing applications (bug fixes, refactoring, too!) 8) did some business analysis - i.e. operating costs/etc and we used this analysis to expand profitably into new markets (geographically...) considering extra transportation costs, etc. (faster than going Excel) 9) sense-check financial models I build in Excel (second set of eyes) 10) Wrote some code in python for personal interest, ran successfully. 11) helped research tax implications of Canadian estate holding US real estate, and tax implications on US and Canadian beneficiaries 12) wrote an obituary. first draft, minor edits. 13) wrote a letter to counter a credit card transaction dispute (all I had to do is input a list of the transactions) The list goes on... I just encourage people to expand the mind.
  • jimsimmons 4y ago
    what exactly is your job? This reads more like an all-star capabilities brochure than someone’s real life/work
    • tyrelb 4y ago
      Entrepreneur with a number of businesses and projects on the go... majority of my time is in the funeral/death care industry. What I don't outsource, I just learn and do myself. For example, saw an opportunity and had to self-teach programming to build an end-to-end death care software platform that handles 1000's of funeral arrangements each year.
• skybrian 4y ago
  You can call it bullshit and in a way it is, but dismissing it in this way means you miss out on what it's good for. Creativity often involves generating ideas in playful ways, using techniques that aren't really justified and can generate ideas that often don't work. Many of the ideas are wrong. But if you test one of them and it works, it becomes meaningful. Similarly, chatbots can be used to generate ideas but you need to do the testing. It works best if testing is inherently part of the process. You can ask a chatbot to recommend a paper to read, and look for it using Google. If it doesn't exist, you find out right away. Or you ask it to generate some code. You read it over and it makes sense, and you try it and it works. (It probably shouldn't be your only source of ideas and there's a question of how much testing you need to do.)
• fdhfdjkfhdkj 4y ago
  [dead]