13 ms·

I can't tell what is worse now: the sycophantic ChatGPT hype guys/gals who write articles "it's coming for all of our jerbs!", or articles like this one that de

I can't tell what is worse now: the sycophantic ChatGPT hype guys/gals who write articles "it's coming for all of our jerbs!", or articles like this one that deliberately misuse ChatGPT and then say "it's overhyped". They're both missing the point. Yes, ChatGPT can be tricked, confidently give wrong answers, but it is still ludicrously useful. It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field. Does this "person" make mistakes? Can't cite their sources? Yeah this definitely happens (especially the sources thing), but when you're trying to understand something new and complex and you can't get the "gist" of it, ChatGPT does surprisingly well. I've had it debug broken configs on my server and router (and explain to me why they were broken), help me practice a foreign language I've slowly been forgetting (hint: "I would like to practice $language, so let's have a conversation in $language where you only use very simple words." -> ChatGPT will obey), and help me understand how to use obscure software libraries that don't have much documentation online (e.g. Boost Yap, useful but with a dearth of blog / reddit posts about it). Does it sometimes goof up? Yep, but it is such an incredibly useful tool nonetheless for the messy process of learning something new.

karpierz 4y ago
I don't know that it's an "incredibly smart" anything; I think it'd be better described as incredibly well-read, with poor discretion. But there's definitely uses for a tool that produces a lot of boilerplate content that a human can then proofread (ex: writing cover letters, filling out unstructured paperwork, etc.) Edit: I personally underestimated the use-case, because the "more optimal" thing to do when auto-filling out paperwork would be to define a formal data-structure with annotations for each field, and then have a system read annotations and spit out autosuggestions. But this is a lot of overhead. If you can have a system that takes in arbitrary text, parses it, and fills it out with pretty good guesses, then that can be much more viable than the previous approach, as it doesn't require an overhaul of, say, the way the US bureaucracy generates paperwork. It's all on the consumer-end, the producer doesn't need to cooperate.
- joenot443 4y ago
  If you can think of a single individual who can demonstrate knowledge on the fly with the same accuracy and breadth as ChatGPT, I think most of us would agree in calling them “incredibly smart.”
  - HarHarVeryFunny 4y ago
    If ChatGPT was a person, you'd immediately realize something was very wrong with them. Some kind of idiot savant perhaps who'd memorized an encyclopedia but was confused by how it all fit together, or maybe someone hallucinating and delusional, only able to hold coherent thoughts together for brief stretches of time before babbling gibberish that made no sense.
  - freejazz 4y ago
    My gosh. Enough with the anthropomorphizing.
- VBprogrammer 4y ago
  I dunno, I recently learned about how the Toyota Synergy Drive system works in detail. Despite being the type of person who'd be interested in that kind of thing I thought it was not generally well understood, I've only come across one website which described it in detail. I asked ChatGPT about it and it game me a fairly surface level answer. I then asked it for more details and it went into detail far beyond what the average person would be able to tell you. I thought that was quite impressive for a fairly obscure bit of mechanical engineering.
  - karpierz 4y ago
    Out of curiosity, I'd have the following questions: 1. Would you be able to notice if it made something up, given that it's going beyond the detail that you know? 2. Did it have any details that aren't covered in: https://en.wikipedia.org/wiki/Hybrid_Synergy_Drive https://en.wikipedia.org/wiki/Hybrid_Synergy_Drive
    - 4y ago
endisneigh 4y ago
> It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field. Does this "person" make mistakes? Can't cite their sources? Yeah this definitely happens (especially the sources thing), but when you're trying to understand something new and complex and you can't get the "gist" of it, ChatGPT does surprisingly well. if you don't "understand something new", how would you know to trust what it says or not? the usefulness or not has to be based on something, not feelings.
- deleted 4y ago
  [deleted]
- jchw 4y ago
  That goes for pretty much any resource; there's a lot of botched or sub-optimal explanations of complex things out on the internet. But being able to have a conversation with someone who has a decent conceptual grasp on something is better than a perfect encyclopedic article alone, and ChatGPT is able to roughly do that for a lot of subjects. As for how you know it's accurate, you don't really have to know it's accurate most of the time, because if you're using it to try to learn something new it's probably going to become apparent quickly enough. Nobody is reading something and then going "Welp, now I'm done learning." As a software example, you can ask it about implementing the Fourier transform. If it says something wrong, you'll find out. The question then becomes how often it is inaccurate and how badly. But in that regard, as long as you don't treat it as an all-knowing oracle, it's at least as useful as having a friend that knows a lot of stuff you want to learn.
  - endisneigh 4y ago
    > That goes for pretty much any resource; there's a lot of botched or sub-optimal explanations of complex things out on the internet. But being able to have a conversation with someone who has a decent conceptual grasp on something is better than a perfect encyclopedic article alone, and ChatGPT is able to roughly do that for a lot of subjects. people keep saying this, but it isn't true. if you ignore sites like this one and reddit there are plenty of authoritative articles and explanations about things with provenance. and if they are wrong they'll update their information.
williamcotton 4y ago
All of this seems eerily similar to Wittgenstein's use theory of meaning: https://en.wikipedia.org/wiki/Use_theory_of_meaning https://en.wikipedia.org/wiki/Use_theory_of_meaning From wikipedia: The Investigations deals largely with the difficulties of language and meaning. Wittgenstein viewed the tools of language as being fundamentally simple and he believed that philosophers had obscured this simplicity by misusing language and by asking meaningless questions. He attempted in the Investigations to make things clear: "Der Fliege den Ausweg aus dem Fliegenglas zeigen"—to show the fly the way out of the fly bottle. "The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." - Edsger W. Dijkstra
deleted 4y ago
[deleted]
sampo 4y ago
> Can't cite their sources? Has there been research on neural nets who could cite their sources? In common cases, thousands or millions of items in the training set have contributed to the weights in the pathways that activate for a certain input. But maybe for some inputs, only a small number of items in the training set contributed strongly to the weights on the pathways the are decisive to the output? Could you design a data structure that would keep this information in the neural net?
- jesterman 4y ago
  perplexity.ai/ specifically uses web sources to construct natural language. I believe it's going to be in Bing soon?
  - mynameisvlad 4y ago
    It still has the exact same issues. I asked it to tell me about “Microsoft layoffs before 2022” and it confidently stated that Microsoft had no layoffs before 2022, with one of its sources even talking about the previous layoffs in 2009.
  - sampo 4y ago
    > perplexity.ai/ specifically uses web sources to construct natural language. I am going to guess that even though it uses the GPT language model, it works fundamentally differently than ChatGPT. My guess is, Perplexity AI is a traditional search engine, that first finds a list of links that match your questions. And then it feeds text from those links to the GPT language model to write a summary of the results. Whereas ChatGPT produces text based only on the trained deep neural net. Maybe I am guessing wrong how Perplexity AI works. Maybe it works in a different manner.
- etrautmann 4y ago
  I would imagine that citing sources would look more like finding sources that have a minimum distance to the output in a latent space representation? I don’t think it’ll make sense to track how each connection weight changed over time due to various inputs.
- 4y ago
thuuuomas 4y ago
“Sycophantic” to whom?
xiphias2 4y ago
I had problem with the citing as well (have to ask separate questions, then google it). It would be a huge improvement if it would be trained to provide references by default.
deleted 4y ago
[deleted]
random_upvoter 4y ago
I use ChatGPT not because it is so great at solving tricky technical question, but because Google has become so incredibly shitty at it. Google's response these days is ten YouTube videos and a random mix of web posts dating between 2005 and 2023 for a piece of software that I'm using the latest version of. I honestly don't even bother trying anymore. ChatGPT is a mixed bag. Sometimes it nails it, but I've also seen it hallucinate complete nonsense solutions, for instance in Davinci Fusion using a node type that doesn't even exist in the program. As for "incredibly intelligent philosopher", I don't even find it an interesting "person" to talk to. Its responses are pedestrian, pedantic and boring. A question like "if God is good then why evil in the world?" is met with a summary of the wikipedia article on the problem of evil. I can read a Wikipedia article, thank you.
Julesman 4y ago
Well said. Exactly. So much hyperbole! So much click bait. It makes me dizzy. I'm a front-end dev. I use it every day. I have CoPilot too. These things help me do my job. No question. They also help me learn better than just about any other tool. The big one for me is that they have made me more confident taking on bigger tasks. The impact is not trivial at all.
soperj 4y ago
> It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well This is the issue. It's more like the dumbest guy in the room confidently convincing the everyone else that they know what they're talking about.
- notahacker 4y ago
  Yup. As a pattern matching engine it's extraordinarily good. Ask it about the implications of some obscure theory and it will usually correctly interpret your question and return some sentences summarising the current state of knowledge on that theory. Ask it to explain it in simpler terms and it will simplify the sentence structure, swap some words and usually retain meaning. But then... ask it for some reading material on the same obscure theory and it'll use its knowledge of scientific papers on those theories and academic citation to return a well formatted list of papers with plausible titles and authors with the significant flaw that none of these papers exist. The dumbest guy in the room would struggle to make up plausible sounding scientific papers even if you gave them the reading list, but then the dumbest person in the room wouldn't be dumb enough to use the reading list to make up some nonexistent papers in the first place! (and ironically nor would the dumbest information retrieval algorithm in the room...)
  - deleted 4y ago
    [deleted]
- zdw 4y ago
  We've entered the Dunning–Kruger effect phase of the AI revolution.
- mokus 4y ago
  I think the bigger point is that it’s like a guy in the room. ChatGPT shows that the industry is graduating from Turing test territory to IQ test territory, and a lot of people are really hung up on the fact it’s failing the latter.
pdonis 4y ago
> It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field. No. You've fallen for exactly the con the article is describing. ChatGPT knows nothing. It is very good at generating text that seems plausible if you know nothing about the subject matter, but is not reliable as an actual source of information. If you happened to get information from it that turned out to be ok, you were lucky. Don't expect your luck to last.
- tester457 4y ago
  > If you happened to get information from it that turned out to be ok, you were lucky. Don't expect your luck to last. For creating bash scripts and english to code it remains remarkably consistent. No need to be a master of multiple languages syntaxes now.
  - kibwen 4y ago
    Of all the languages one could ask ChatGPT to generate, bash would be nearly the bottom of the list. The number of subtle footguns in bash are through the roof.
- msm_ 4y ago
  I think that's unnecessarily pessimistic view. I have recently started using ChatGPT for mid-hard questions (when I can't get an answer immediately using Google), mostly "how do I" questions. It lies of dreams up fake answers about half of the time, but this still means that 50% of time I get my solution without digging through obscure manuals. > It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field. I like to think about it like asking questions to a well read but dishonest student during an oral exam. They know some things, and try really hard to answer every question, even resort to making things up in hope of getting a passing grade.
  - jacobmischka 4y ago
    So it's a coinflip whether it's giving you correct information or something completely made up? And now you're not digging through the obscure manuals to actually verify? Seems objectively harmful.
rvz 4y ago
Well an AI that can't explain itself transparently makes it untrustworthy for more serious use and applications since ChatGPT it is being celebrated as a great sophist. There is nothing new in this, other than 'train more data on the internet'. The hype in ChatGPT reminds me of the short-lived hype of Clubhouse; the end result is the same: The bigger players will come in to surpass it, just like they did with GPT-3.
- vehemenz 4y ago
  I wonder. If the bigger players can make an AI good enough to be indistinguishable from human (but not quite), then wouldn't the AI slowly and incestuously train itself to become dumber, with nothing but AI-generated content left to "feed it?"
pjc50 4y ago
There's a significant epistemological problem: how do you know that what it's telling you is true? For things that you yourself can verify easily, this isn't too much of a problem. It gives you some boost tips, you see whether the program works or not. But as you come to rely on it, and it matters whether the answers are correct in important ways, what validation/verification/QA steps can you apply to ensure that you're getting the correct answer? I'm reminded of the FDIV bug https://en.wikipedia.org/wiki/Pentium_FDIV_bug https://en.wikipedia.org/wiki/Pentium_FDIV_bug , where people found that their CPU gave mathematically incorrect answers for certain inputs. I'm also thinking of a concept of "responsibility". If you ask a human for tips on, say, building a high power laser, or making explosives, they might decide whether they want to give you that information, and they might decide to give you some other information along with it ("wear goggles") for your own safety. Who is liable if an AI gives someone advice, they act on that advice, and serious injury or property damage results? If the AI (or its operator) has no responsibility at all, is that disclaimer adequately communicated to users?
- galleywest200 4y ago
  I use it like Wikipedia: a good jumping off point. Often times I will forget an exact thing I wish to find online so I will ask ChatGPT to list multiple things that could be what I just described. I then take this list to my search engine du jour.
  - bnralt 4y ago
    Good point about Wikipedia. In just a few years the image people had of Wikipedia changed from an untrustworthy joke (The Office had a joke about Michael Scott trusting Wikipedia) to one of the first places people went to get information. Before that, there was a lot of concern about blogs when they were the hot new thing. Here’s a New York Times article[1] from that period discussing the blogging controversy: > There are two obvious differences between bloggers and the traditional press: unlike bloggers, professional journalists have a) editors and b) the need to maintain a professional reputation so that sources will continue to talk to them. I've been a journalist for more than a decade, and on two occasions I asked acquaintances whether I could print information that they had told me in social situations. Both times, they made clear that if I published they would never speak to me again. Without a reputation for trustworthiness, neither friendship nor journalism can be sustained over time. The concerns weren’t incorrect - Wikipedia or Blogs are often unreliable. But the concerns overlooked how useful these things could be despite their problems. As an aside, it's interesting to see comments here bring up the potential unreliability of ChatGPT - and then say that people should go read Wikipedia instead. [1] https://www.nytimes.com/2004/12/19/magazine/your-blog-or-mine.html https://www.nytimes.com/2004/12/19/magazine/your-blog-or-min...
feifan 4y ago
A well-adjusted person should be able to signal some degree of confidence about what they're saying ("I know that…", "I think that…", "Maaaaaaybe…", etc). That's an anspect that ChatGPT doesn't have.
- tester457 4y ago
  For the api appending such language is a waste of tokens, therefore a waste of money. People should use it while knowing it hallucinates all the time.
  - kibwen 4y ago
    In other words, everyone should assume that it's lying at all times. Nobody should be using a product that both lies to you and then lies to you about the fact that it's lying to you.
    - Karunamon 4y ago
      Stating a falsehood without intent to deceive is just incorrectness, not a "lie". Let's dial the hyperbole back a few notches.
      - kibwen 4y ago
        If the people running the platform are advertising a product that produces truths while delivering falsehoods and pathetically refusing to label their falsehoods as falsehoods, that's a lie.
        Karunamon 4y ago
        There is a bit right on the chat box above every conversation that says, verbatim, "may occasionally generate incorrect information". Determining truth is not in the wheelhouse of a language learning model and it is not defective for not doing so in the same way a tablesaw is not defective for not driving nails. Adjust your expectations accordingly.
mrstone 4y ago
I agree. I regularly use ChatGPT to help reword paragraphs to be more succinct. I rarely ask it for actual information that it has to look up. At its core, it's an LLM which models how language is structured, and that's what it's good at.
- shagie 4y ago
  It has an impressive amount of understanding about language. There was a meme a bit ago about "Everyone on my floor is coding" and the different reaction of a software developer and a doctor. So I asked for jokes about a doctor and a software developer about the different meanings for the words they used. Write five jokes about a doctor and a programmer about the different meanings of words they use. To which GPT responded: 1. Doctor: I'm sending you a prescription. Programmer: Great! I'll take a look at it and see if I can debug it. 2. Doctor: I need you to take two aspirin. Programmer: Is that enough to solve the problem? 3. Doctor: I'm going to give you some instructions. Programmer: Do I need to compile them or can I just run them? 4. Doctor: I need you to take a break from work. Programmer: Does that mean I should pause the program or stop it completely? 5. Doctor: I need you to stay hydrated. Programmer: Does that mean I should store more data or drink more water? I will certainly admit that those jokes are awful, but the interesting part of it is that... it did it. I think that 3, 4 and 5 are the "best". I don't think it understands those things with the ability to reason about them, but that demonstrates more than I would expect from a Chinese room. Explain the joke "Doctor: I need you to stay hydrated. Programmer: Does that mean I should store more data or drink more water?" returns back: The joke is a play on the terms "hydrate" and "store data". In programming, "storing data" is the process of saving information to a computer's memory, while "hydrating" means drinking water to stay hydrated. The joke is implying that the programmer is more comfortable with programming than with taking care of their own health, so they interpret the doctor's advice to stay hydrated as a suggestion to store more data.
nathan_compton 4y ago
"It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field." I don't know how to tell you this without being a little rude, but if you think ChatGPT is incredibly smart or that it can tell you meaningful things about "any field," well, there is a difference in intelligence somewhere in the equation between you and ChatGPT, but its not the one you think.
b3morales 4y ago
> help me practice a foreign language I've slowly been forgetting So with the software stuff you can pretty easily verify the output: either the suggestion fixes the problem or it doesn't. But how can you trust it for something like this where you can't distinguish what's good and bad output? It could be leading you down the garden path with this language exercise. And it's not in the helpful thinking tool/bouncing ideas around category, either: there are rules of grammar that you don't know, and you don't know whether ChatGPT knows them either.
photochemsyn 4y ago
It is very useful but there's definitely a constant issue with quality-of-results, but I've found a pretty good workflow, involving jumping back and forth between search engine results and ChatGPT results. Fundamentally, it's: 1) Ask ChatGPT a series of questions, going from the general down to the more specific. For example, 'Give me a list of options for setting up a headless Raspberry Pi as a Nginx server from scratch using only command line tools'. ChatGPT will produce a decent list of options. 2) Run that list of options in truncated form, just the keywords, through Google or other search engine that allows for a 'verbatim' search. Set to 50 or more results per page to get a decent spread of results at a glance. Get the more specific information for various options. 3) Run that more narrowed information back through ChatGPT to see what it produces. At this point you might be pretty close to the correct specific information you need, but generally I run it through search again just to be sure. For example, this way I was able to quickly discover my headless RPi setup with the latest Raspberry Pi OS wasn't allowing me to ssh into it over wireless because the RPi engineers did away with default user/password settings for security reasons, so I had to manually include the 'userconf.txt' file in the boot directory on the SD card. Also it was very helpful with what needed to go into the wpa_supplicant.conf file. Trying to find stuff like that out by search alone, in the past, would have probably taken at least 10X as long. I wouldn't ever blindly trust what it suggests, however.
koprulusector 4y ago
What people also seem to forget (if they knew in the first place) is how bad GPT3 was compared to GPT4/ChatGPT. I subscribe to SubSimulatorGPT3 on Reddit and it generates tons of text that looks and feels like English, for example, but is devoid of any real meaning. Every sentence, every paragraph I read from GPT3 has me questioning whether or not I’ve had a stroke. Now, compare that to your comment, and it’s INSANE how much the capability (and the conversation) has shifted. GPT3 was what, three or four years ago? What will we be debating, what will the capabilities be, in another three or four years?
- chimprich 4y ago
  GPT4 hasn't been released yet. I think you're probably thinking of GPT3.5.
tiagod 4y ago
ChatGPT/GPT fails a lot when you probe its knowledge, but I find it works really well when you give it an explanation of the whole domain of the problem, then something to do with it. For example can give examples of some programming language, or an API, and natural language explaining what you want to do with it (or code in another language), and it will translate this with remarkable accuracy, 0 shot. We've never had anything that's even close to this capability.
osigurdson 4y ago
Agree. ChatGPT provides great results while doing quite a bit of dreaming at the same time. The role of the human is to separate dreams from reality.
brookst 4y ago
Thank you. Tons of noise because people are mistaking a statistical tool for a sentient consciousness. ChatGPT has no more interest in running a confidence game than it does in helping children learn critical reasoning. Which is to say none, zero, zilch in both cases. It has as much volition and desire as a calculator. It’s just a super impressive and useful calculator.
weinzierl 4y ago
> "It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field. Does this "person" make mistakes?" Does this person stubbornly insist on being right, even if they are wrong? Likely, that's just how humans usually behave - and ChatGPT does the same. At least that was my thinking when people complain about ChatGPT being a know-it-all bluffer.
- Karunamon 4y ago
  ChatGPT is pretty amenable to being corrected if it says something you know is crap. And then, in my experience, you usually get better results. That is not something I can say for most humans.
  - godelski 4y ago
    Only if you correct a wrong answer. If you "correct" a right answer ("I don't think that's right") then it gives an even worse answer. So be careful when correcting.
- godelski 4y ago
  > Does this person stubbornly insist on being right, even if they are wrong? Actually ChatGPT will almost always change tune if you tell it that it is wrong. That's because it is a stochastic parrot and that's an "unlikely" event if you actually got the answer right. But we'll see if this gets updated/broken too.
  - weinzierl 4y ago
    I've seen ChatGPT change tune but I've also seen it squirm plenty. Overall I think it behaves uncannily human in this regard.
evo_9 4y ago
Yeah, it's fascinating to watch, right? As others have pointed out — what did we expect of 'AI' created in our image? Aren't we all just bullshit artists of varying degree? HN is the perfect place to learn how much BS you are - I DARE any subject matter expdrts on here to post on any topic and I guarantee another expert will step in and tell you why you're wrong, or at best, nitpick you apart.
checkyoursudo 4y ago
> It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field. I have yet to see a single instance of this with any query that requires even a modicum of expertise. I have asked it a lot of questions about my field. I have asked it to summarize and paraphrase and ELI5 all kinds of shit. Everything it produces is either inane or a regurgitation of the input just with verbs and prepositions changed. I would like to see a nice example, input and output, of this. I really would.
- shawabawa3 4y ago
  First thing that came to mind: > Explain how git works in very simple english Git is a version control system for software development that helps track changes made to the source code over time. It allows multiple developers to collaborate and make changes to the same codebase without interfering with each other. When you start a project in Git, you create a "repository" that stores all the files and their changes. You can then "commit" changes to the repository, which is like saving a snapshot of the code at a certain point in time. You can also "push" the changes to a remote repository on a server, so that other people can access and work on the code. If someone else makes changes to the code, you can "pull" the updated version of the repository to your local machine and "merge" the changes with your own code. Git also allows you to create "branches" of the code, so that you can work on different features or fixes without affecting the main code. By using Git, you can easily revert back to previous versions of the code, compare changes between different versions, and track who made which changes
braingenious 4y ago
> It is basically like having an incredibly smart engineer/scientists/philosopher/etc It is a copy/paste machine that strings together words to make sentences that look like sentences it’s read. So it’s like an incredibly smart engineer/scientist/philosopher that does not and fundamentally cannot understand any of those concepts (or any concepts at all.) A parrot that can recite Macbeth’s monologue is not a s Shakespeare scholar.
- deleted 4y ago
  [deleted]
- thaw13579 4y ago
  I asked it to explain some very technical medical testing procedures to me "like I was five years old". It completely nailed it by truly reducing it to its essence and using appropriate metaphors that a child would understand (coloring books). Because of how niche the topic is, I'm fairly sure no such text exists in the training corpus. I'm not sure whether this constitutes "understanding", since that term usually applies to people's cognitive states; however, if a student provided this answer on an exam, they would certainly convince the examiner and pass the test!
guenthert 4y ago
> Does this "person" make mistakes? Can't cite their sources? Making citations up is not a "mistake", it's a lie. I have absolutely no use for a tool which lies to me, especially not when troubleshooting computer systems.
chx 4y ago
> It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field. No it can not explain anything. It's a (stochastic) parrot, nothing more. > Yep, but it is such an incredibly useful tool It is not. You are fooling yourself. You are building a castle on a swamp.
- vehemenz 4y ago
  ChatGPT tells me that Disney World's Magic Kingdom was built on a swamp. It's just a tool. You don't use a shovel to rake leaves, but that doesn't make a shovel any less useful.
Forgeties79 4y ago
>but it is still ludicrously useful. Wrote me a script to quickly convert SRT to VTT files as I edit and prep for uploading online. I haven't done anything akin to "coding" since I took a very basic class in college and messed around with HTML/CSS. Shaves several steps off of every video I do (I'm in a high volume world here).
Obscurity4340 4y ago
Its honestly astounding how coherent and dare, I say, eloquent it can be. From factual "Google"-type inquiries to jokes to whatever. I particularly enjoy the poetry it generates in defined formats like limericks, sonnets, etc. Yet to encounter any actual "bullshit" that it is frequently accused of (although I tend to ask closed-type questions or provide very specific parameters with which to constrain its output)
saiya-jin 4y ago
I can see tons of jobs disappearing even with current level. Web these days is filled with mediocre articles, and quite a few bureaucratic jobs can and will be replaced by some next iteration. Instead of army of mediocre journalists filling web portals, you just need 1 person to proof read it per business, maybe not even that. I can see the same for software dev in less than decade with current pace, 1 senior going over generated code and doing minor tweaks. Which is probably road to hell long term since how will juniors become seniors, but cost savings can be real and massive, drop in quality small if at all and it may be good enough for many. It has very good language skills, in my obscure and rather difficult native language which only cca 6 million people use it formulated perfect sentences consistently. I can easily imagine faking social, even love chats easily. This can help lonely or elderly people (with good speech recognition/generation, since language skill itself is there already). With image/video generation porn industry will be steamrolled, 24/7 streams of anything you can imagine, looking realistic enough, tailored just to you. Or just help with learning foreign languages, heck few months ago a guy was posting here on HN just a smart chat that could be used for training and got standing ovation also from me for that, this has it and everything else. I understand google's reaction, instead of browsing through paid entries it gives me answer directly, no need to click through search and then wikipedia and then parsing texts. At least in some cases. This could help google massively if they could somehow do it on their scale. People trying to find errors in it are missing the point now - its not about academic perfection, but often good-enough almost for free. Also, this is not final product, only 4th alpha/beta/whatever, previous ones were much worse. Imagine 10 or 20 years down the road...
autokad 4y ago
I agree, its incredibly useful if you know how to use it. I used it to make sentences more succinct for emails to managers (no sensitive data of course), I used it to turn my dreams into short stories, gather ideas for research, etc. People bashing on chatgpt just dont know how to use it. is there going to be an example in the world where it just plagiarizes bob dillan? sure ... so what? is there going to be instances where its not useful? duh. but if you dont like it, that's fine. not every tool is for everyone. I find it very useful for certain things, and I imagine its only going to improve from here.
belter 4y ago
Sometimes I think ChatGPT was just some twisted, treacherous, indirect plot, to find a way to finally discredit Jordan Peters.
freejazz 4y ago
>"It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field." LOL where do you get this notion? I haven't seen one chatGPT bot useful for law. They all have no idea what they are doing and are basically dangerous if you don't know better than the LLM. There is literally nothing chatGPT can do for me as an attorney that isn't already solved by using a template.
bitsnbytes 4y ago
FWIW I see it the same and have had the same experience. The tech is there now to be utilized as an assistant. However as the end user you better have enough knowledge to understand the solution or answer it spits at you to fact check it. The biggest danger I see is utilizing the tool for solutions or answers by novices that don't understand enough to grasp the solution and confirm if in fact the answer it gives is correct or appropriate. Simplistic example: A non techie manager trying to have it create sql queries to get the information they are asking for , is asking for trouble. However in the hands of a sql developer it could provide enough of the query build out to just require minimal tweaks.
jacobjjacob 4y ago
The latter group exists to counter the former. If folks weren’t making some of these over-the-top claims, then there wouldn’t be the claims to debunk.
choppaface 4y ago
What’s worse, too much hype, or the non-believers? Well the OpenAI hype squad is racking up personal fortunes from their work in a finding deal that is pretty clearly over-valued. Especially versus what Deepmind sold for [1]. While perhaps both sides are not being honest, YC’s own sama is clearly profiting excessively over the hubub. He’s clearly proving to be a symbol of the faction of YC that rakes in money through arbitrage not prolonged value. [1] https://techcrunch.com/2014/01/26/google-deepmind/ https://techcrunch.com/2014/01/26/google-deepmind/
danjoredd 4y ago
My problem is the type of person that proudly proclaims that all jobs can, should, and are going to be outdated by the use of AI. I have talked to people that believe that, and think that AI need to replace humanity for some reason. I don't understand them. AI is exciting, and it can be an useful tool. There are a lot of use cases for it, and in the end I think it will be a good thing. However, what's worse than the detractors are the people blindly hoping that AI will automate every single thing a human touches. It shouldn't be used in that way. It has to be a tool to assist humans, not take away our professions and reasons to get out of bed in the morning.
groby_b 4y ago
In all the fields I've tried (software/music/epidemiology/geosciences/mathematics) ChatGPT can mostly explain things at the lay level, but even there often gets things wrong that without more expert guidance, you will not notice. Experts in each of these areas were consistently better at giving a correct explanation. ChatGPT is great when occasionally being wrong doesn't matter. It's still pretty good when you can't be wrong, but you're an expert. But in any "no mistakes, I need to talk to an expert" situation, ChatGPT is setting you up for failure. That's the key part - the information is unreliable/untrustworthy. As long as that is not a problem, it's awesome. The problem is that it presents wrong data with absolute confidence in a very convincing voice. (Hence people calling it a con artist). It'll become a huge problem as it makes inroads in mis/disinfo campaigns. (The usual tech problem. Don't let large scale interests touch it, and you'll be OK)
deleted 4y ago
[deleted]
up2isomorphism 4y ago
None of the coding questions it answers is closed to usable, and i need to spend more time correcting it than just write it on my own. So it is just not “incredibly smart”.
deleted 4y ago
[deleted]

karpierz 4y ago

I don't know that it's an "incredibly smart" anything; I think it'd be better described as incredibly well-read, with poor discretion. But there's definitely uses for a tool that produces a lot of boilerplate content that a human can then proofread (ex: writing cover letters, filling out unstructured paperwork, etc.) Edit: I personally underestimated the use-case, because the "more optimal" thing to do when auto-filling out paperwork would be to define a formal data-structure with annotations for each field, and then have a system read annotations and spit out autosuggestions. But this is a lot of overhead. If you can have a system that takes in arbitrary text, parses it, and fills it out with pretty good guesses, then that can be much more viable than the previous approach, as it doesn't require an overhaul of, say, the way the US bureaucracy generates paperwork. It's all on the consumer-end, the producer doesn't need to cooperate.

joenot443 4y ago
If you can think of a single individual who can demonstrate knowledge on the fly with the same accuracy and breadth as ChatGPT, I think most of us would agree in calling them “incredibly smart.”
- HarHarVeryFunny 4y ago
  If ChatGPT was a person, you'd immediately realize something was very wrong with them. Some kind of idiot savant perhaps who'd memorized an encyclopedia but was confused by how it all fit together, or maybe someone hallucinating and delusional, only able to hold coherent thoughts together for brief stretches of time before babbling gibberish that made no sense.
- freejazz 4y ago
  My gosh. Enough with the anthropomorphizing.
VBprogrammer 4y ago
I dunno, I recently learned about how the Toyota Synergy Drive system works in detail. Despite being the type of person who'd be interested in that kind of thing I thought it was not generally well understood, I've only come across one website which described it in detail. I asked ChatGPT about it and it game me a fairly surface level answer. I then asked it for more details and it went into detail far beyond what the average person would be able to tell you. I thought that was quite impressive for a fairly obscure bit of mechanical engineering.
- karpierz 4y ago
  Out of curiosity, I'd have the following questions: 1. Would you be able to notice if it made something up, given that it's going beyond the detail that you know? 2. Did it have any details that aren't covered in: https://en.wikipedia.org/wiki/Hybrid_Synergy_Drive https://en.wikipedia.org/wiki/Hybrid_Synergy_Drive
  - 4y ago

endisneigh 4y ago

> It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field. Does this "person" make mistakes? Can't cite their sources? Yeah this definitely happens (especially the sources thing), but when you're trying to understand something new and complex and you can't get the "gist" of it, ChatGPT does surprisingly well. if you don't "understand something new", how would you know to trust what it says or not? the usefulness or not has to be based on something, not feelings.

deleted 4y ago
[deleted]
jchw 4y ago
That goes for pretty much any resource; there's a lot of botched or sub-optimal explanations of complex things out on the internet. But being able to have a conversation with someone who has a decent conceptual grasp on something is better than a perfect encyclopedic article alone, and ChatGPT is able to roughly do that for a lot of subjects. As for how you know it's accurate, you don't really have to know it's accurate most of the time, because if you're using it to try to learn something new it's probably going to become apparent quickly enough. Nobody is reading something and then going "Welp, now I'm done learning." As a software example, you can ask it about implementing the Fourier transform. If it says something wrong, you'll find out. The question then becomes how often it is inaccurate and how badly. But in that regard, as long as you don't treat it as an all-knowing oracle, it's at least as useful as having a friend that knows a lot of stuff you want to learn.
- endisneigh 4y ago
  > That goes for pretty much any resource; there's a lot of botched or sub-optimal explanations of complex things out on the internet. But being able to have a conversation with someone who has a decent conceptual grasp on something is better than a perfect encyclopedic article alone, and ChatGPT is able to roughly do that for a lot of subjects. people keep saying this, but it isn't true. if you ignore sites like this one and reddit there are plenty of authoritative articles and explanations about things with provenance. and if they are wrong they'll update their information.

williamcotton 4y ago

All of this seems eerily similar to Wittgenstein's use theory of meaning: https://en.wikipedia.org/wiki/Use_theory_of_meaning https://en.wikipedia.org/wiki/Use_theory_of_meaning From wikipedia: The Investigations deals largely with the difficulties of language and meaning. Wittgenstein viewed the tools of language as being fundamentally simple and he believed that philosophers had obscured this simplicity by misusing language and by asking meaningless questions. He attempted in the Investigations to make things clear: "Der Fliege den Ausweg aus dem Fliegenglas zeigen"—to show the fly the way out of the fly bottle. "The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." - Edsger W. Dijkstra

deleted 4y ago

[deleted]

sampo 4y ago

> Can't cite their sources? Has there been research on neural nets who could cite their sources? In common cases, thousands or millions of items in the training set have contributed to the weights in the pathways that activate for a certain input. But maybe for some inputs, only a small number of items in the training set contributed strongly to the weights on the pathways the are decisive to the output? Could you design a data structure that would keep this information in the neural net?

jesterman 4y ago
perplexity.ai/ specifically uses web sources to construct natural language. I believe it's going to be in Bing soon?
- mynameisvlad 4y ago
  It still has the exact same issues. I asked it to tell me about “Microsoft layoffs before 2022” and it confidently stated that Microsoft had no layoffs before 2022, with one of its sources even talking about the previous layoffs in 2009.
- sampo 4y ago
  > perplexity.ai/ specifically uses web sources to construct natural language. I am going to guess that even though it uses the GPT language model, it works fundamentally differently than ChatGPT. My guess is, Perplexity AI is a traditional search engine, that first finds a list of links that match your questions. And then it feeds text from those links to the GPT language model to write a summary of the results. Whereas ChatGPT produces text based only on the trained deep neural net. Maybe I am guessing wrong how Perplexity AI works. Maybe it works in a different manner.
etrautmann 4y ago
I would imagine that citing sources would look more like finding sources that have a minimum distance to the output in a latent space representation? I don’t think it’ll make sense to track how each connection weight changed over time due to various inputs.
4y ago

thuuuomas 4y ago

“Sycophantic” to whom?

xiphias2 4y ago

I had problem with the citing as well (have to ask separate questions, then google it). It would be a huge improvement if it would be trained to provide references by default.

deleted 4y ago

[deleted]

random_upvoter 4y ago

I use ChatGPT not because it is so great at solving tricky technical question, but because Google has become so incredibly shitty at it. Google's response these days is ten YouTube videos and a random mix of web posts dating between 2005 and 2023 for a piece of software that I'm using the latest version of. I honestly don't even bother trying anymore. ChatGPT is a mixed bag. Sometimes it nails it, but I've also seen it hallucinate complete nonsense solutions, for instance in Davinci Fusion using a node type that doesn't even exist in the program. As for "incredibly intelligent philosopher", I don't even find it an interesting "person" to talk to. Its responses are pedestrian, pedantic and boring. A question like "if God is good then why evil in the world?" is met with a summary of the wikipedia article on the problem of evil. I can read a Wikipedia article, thank you.

Julesman 4y ago

Well said. Exactly. So much hyperbole! So much click bait. It makes me dizzy. I'm a front-end dev. I use it every day. I have CoPilot too. These things help me do my job. No question. They also help me learn better than just about any other tool. The big one for me is that they have made me more confident taking on bigger tasks. The impact is not trivial at all.

soperj 4y ago

> It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well This is the issue. It's more like the dumbest guy in the room confidently convincing the everyone else that they know what they're talking about.

notahacker 4y ago
Yup. As a pattern matching engine it's extraordinarily good. Ask it about the implications of some obscure theory and it will usually correctly interpret your question and return some sentences summarising the current state of knowledge on that theory. Ask it to explain it in simpler terms and it will simplify the sentence structure, swap some words and usually retain meaning. But then... ask it for some reading material on the same obscure theory and it'll use its knowledge of scientific papers on those theories and academic citation to return a well formatted list of papers with plausible titles and authors with the significant flaw that none of these papers exist. The dumbest guy in the room would struggle to make up plausible sounding scientific papers even if you gave them the reading list, but then the dumbest person in the room wouldn't be dumb enough to use the reading list to make up some nonexistent papers in the first place! (and ironically nor would the dumbest information retrieval algorithm in the room...)
- deleted 4y ago
  [deleted]
zdw 4y ago
We've entered the Dunning–Kruger effect phase of the AI revolution.
mokus 4y ago
I think the bigger point is that it’s like a guy in the room. ChatGPT shows that the industry is graduating from Turing test territory to IQ test territory, and a lot of people are really hung up on the fact it’s failing the latter.

pdonis 4y ago

> It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field. No. You've fallen for exactly the con the article is describing. ChatGPT knows nothing. It is very good at generating text that seems plausible if you know nothing about the subject matter, but is not reliable as an actual source of information. If you happened to get information from it that turned out to be ok, you were lucky. Don't expect your luck to last.

tester457 4y ago
> If you happened to get information from it that turned out to be ok, you were lucky. Don't expect your luck to last. For creating bash scripts and english to code it remains remarkably consistent. No need to be a master of multiple languages syntaxes now.
- kibwen 4y ago
  Of all the languages one could ask ChatGPT to generate, bash would be nearly the bottom of the list. The number of subtle footguns in bash are through the roof.
msm_ 4y ago
I think that's unnecessarily pessimistic view. I have recently started using ChatGPT for mid-hard questions (when I can't get an answer immediately using Google), mostly "how do I" questions. It lies of dreams up fake answers about half of the time, but this still means that 50% of time I get my solution without digging through obscure manuals. > It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field. I like to think about it like asking questions to a well read but dishonest student during an oral exam. They know some things, and try really hard to answer every question, even resort to making things up in hope of getting a passing grade.
- jacobmischka 4y ago
  So it's a coinflip whether it's giving you correct information or something completely made up? And now you're not digging through the obscure manuals to actually verify? Seems objectively harmful.

rvz 4y ago

Well an AI that can't explain itself transparently makes it untrustworthy for more serious use and applications since ChatGPT it is being celebrated as a great sophist. There is nothing new in this, other than 'train more data on the internet'. The hype in ChatGPT reminds me of the short-lived hype of Clubhouse; the end result is the same: The bigger players will come in to surpass it, just like they did with GPT-3.

vehemenz 4y ago
I wonder. If the bigger players can make an AI good enough to be indistinguishable from human (but not quite), then wouldn't the AI slowly and incestuously train itself to become dumber, with nothing but AI-generated content left to "feed it?"

pjc50 4y ago

There's a significant epistemological problem: how do you know that what it's telling you is true? For things that you yourself can verify easily, this isn't too much of a problem. It gives you some boost tips, you see whether the program works or not. But as you come to rely on it, and it matters whether the answers are correct in important ways, what validation/verification/QA steps can you apply to ensure that you're getting the correct answer? I'm reminded of the FDIV bug https://en.wikipedia.org/wiki/Pentium_FDIV_bug https://en.wikipedia.org/wiki/Pentium_FDIV_bug , where people found that their CPU gave mathematically incorrect answers for certain inputs. I'm also thinking of a concept of "responsibility". If you ask a human for tips on, say, building a high power laser, or making explosives, they might decide whether they want to give you that information, and they might decide to give you some other information along with it ("wear goggles") for your own safety. Who is liable if an AI gives someone advice, they act on that advice, and serious injury or property damage results? If the AI (or its operator) has no responsibility at all, is that disclaimer adequately communicated to users?

galleywest200 4y ago
I use it like Wikipedia: a good jumping off point. Often times I will forget an exact thing I wish to find online so I will ask ChatGPT to list multiple things that could be what I just described. I then take this list to my search engine du jour.
- bnralt 4y ago
  Good point about Wikipedia. In just a few years the image people had of Wikipedia changed from an untrustworthy joke (The Office had a joke about Michael Scott trusting Wikipedia) to one of the first places people went to get information. Before that, there was a lot of concern about blogs when they were the hot new thing. Here’s a New York Times article[1] from that period discussing the blogging controversy: > There are two obvious differences between bloggers and the traditional press: unlike bloggers, professional journalists have a) editors and b) the need to maintain a professional reputation so that sources will continue to talk to them. I've been a journalist for more than a decade, and on two occasions I asked acquaintances whether I could print information that they had told me in social situations. Both times, they made clear that if I published they would never speak to me again. Without a reputation for trustworthiness, neither friendship nor journalism can be sustained over time. The concerns weren’t incorrect - Wikipedia or Blogs are often unreliable. But the concerns overlooked how useful these things could be despite their problems. As an aside, it's interesting to see comments here bring up the potential unreliability of ChatGPT - and then say that people should go read Wikipedia instead. [1] https://www.nytimes.com/2004/12/19/magazine/your-blog-or-mine.html https://www.nytimes.com/2004/12/19/magazine/your-blog-or-min...

feifan 4y ago

A well-adjusted person should be able to signal some degree of confidence about what they're saying ("I know that…", "I think that…", "Maaaaaaybe…", etc). That's an anspect that ChatGPT doesn't have.

tester457 4y ago
For the api appending such language is a waste of tokens, therefore a waste of money. People should use it while knowing it hallucinates all the time.
- kibwen 4y ago
  In other words, everyone should assume that it's lying at all times. Nobody should be using a product that both lies to you and then lies to you about the fact that it's lying to you.
  - Karunamon 4y ago
    Stating a falsehood without intent to deceive is just incorrectness, not a "lie". Let's dial the hyperbole back a few notches.
    - kibwen 4y ago
      If the people running the platform are advertising a product that produces truths while delivering falsehoods and pathetically refusing to label their falsehoods as falsehoods, that's a lie.
      - Karunamon 4y ago
        There is a bit right on the chat box above every conversation that says, verbatim, "may occasionally generate incorrect information". Determining truth is not in the wheelhouse of a language learning model and it is not defective for not doing so in the same way a tablesaw is not defective for not driving nails. Adjust your expectations accordingly.

mrstone 4y ago

I agree. I regularly use ChatGPT to help reword paragraphs to be more succinct. I rarely ask it for actual information that it has to look up. At its core, it's an LLM which models how language is structured, and that's what it's good at.

shagie 4y ago
It has an impressive amount of understanding about language. There was a meme a bit ago about "Everyone on my floor is coding" and the different reaction of a software developer and a doctor. So I asked for jokes about a doctor and a software developer about the different meanings for the words they used. Write five jokes about a doctor and a programmer about the different meanings of words they use. To which GPT responded: 1. Doctor: I'm sending you a prescription. Programmer: Great! I'll take a look at it and see if I can debug it. 2. Doctor: I need you to take two aspirin. Programmer: Is that enough to solve the problem? 3. Doctor: I'm going to give you some instructions. Programmer: Do I need to compile them or can I just run them? 4. Doctor: I need you to take a break from work. Programmer: Does that mean I should pause the program or stop it completely? 5. Doctor: I need you to stay hydrated. Programmer: Does that mean I should store more data or drink more water? I will certainly admit that those jokes are awful, but the interesting part of it is that... it did it. I think that 3, 4 and 5 are the "best". I don't think it understands those things with the ability to reason about them, but that demonstrates more than I would expect from a Chinese room. Explain the joke "Doctor: I need you to stay hydrated. Programmer: Does that mean I should store more data or drink more water?" returns back: The joke is a play on the terms "hydrate" and "store data". In programming, "storing data" is the process of saving information to a computer's memory, while "hydrating" means drinking water to stay hydrated. The joke is implying that the programmer is more comfortable with programming than with taking care of their own health, so they interpret the doctor's advice to stay hydrated as a suggestion to store more data.

nathan_compton 4y ago

"It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field." I don't know how to tell you this without being a little rude, but if you think ChatGPT is incredibly smart or that it can tell you meaningful things about "any field," well, there is a difference in intelligence somewhere in the equation between you and ChatGPT, but its not the one you think.

b3morales 4y ago

> help me practice a foreign language I've slowly been forgetting So with the software stuff you can pretty easily verify the output: either the suggestion fixes the problem or it doesn't. But how can you trust it for something like this where you can't distinguish what's good and bad output? It could be leading you down the garden path with this language exercise. And it's not in the helpful thinking tool/bouncing ideas around category, either: there are rules of grammar that you don't know, and you don't know whether ChatGPT knows them either.

photochemsyn 4y ago

It is very useful but there's definitely a constant issue with quality-of-results, but I've found a pretty good workflow, involving jumping back and forth between search engine results and ChatGPT results. Fundamentally, it's: 1) Ask ChatGPT a series of questions, going from the general down to the more specific. For example, 'Give me a list of options for setting up a headless Raspberry Pi as a Nginx server from scratch using only command line tools'. ChatGPT will produce a decent list of options. 2) Run that list of options in truncated form, just the keywords, through Google or other search engine that allows for a 'verbatim' search. Set to 50 or more results per page to get a decent spread of results at a glance. Get the more specific information for various options. 3) Run that more narrowed information back through ChatGPT to see what it produces. At this point you might be pretty close to the correct specific information you need, but generally I run it through search again just to be sure. For example, this way I was able to quickly discover my headless RPi setup with the latest Raspberry Pi OS wasn't allowing me to ssh into it over wireless because the RPi engineers did away with default user/password settings for security reasons, so I had to manually include the 'userconf.txt' file in the boot directory on the SD card. Also it was very helpful with what needed to go into the wpa_supplicant.conf file. Trying to find stuff like that out by search alone, in the past, would have probably taken at least 10X as long. I wouldn't ever blindly trust what it suggests, however.

koprulusector 4y ago

What people also seem to forget (if they knew in the first place) is how bad GPT3 was compared to GPT4/ChatGPT. I subscribe to SubSimulatorGPT3 on Reddit and it generates tons of text that looks and feels like English, for example, but is devoid of any real meaning. Every sentence, every paragraph I read from GPT3 has me questioning whether or not I’ve had a stroke. Now, compare that to your comment, and it’s INSANE how much the capability (and the conversation) has shifted. GPT3 was what, three or four years ago? What will we be debating, what will the capabilities be, in another three or four years?

chimprich 4y ago
GPT4 hasn't been released yet. I think you're probably thinking of GPT3.5.

tiagod 4y ago

ChatGPT/GPT fails a lot when you probe its knowledge, but I find it works really well when you give it an explanation of the whole domain of the problem, then something to do with it. For example can give examples of some programming language, or an API, and natural language explaining what you want to do with it (or code in another language), and it will translate this with remarkable accuracy, 0 shot. We've never had anything that's even close to this capability.

osigurdson 4y ago

Agree. ChatGPT provides great results while doing quite a bit of dreaming at the same time. The role of the human is to separate dreams from reality.

brookst 4y ago

Thank you. Tons of noise because people are mistaking a statistical tool for a sentient consciousness. ChatGPT has no more interest in running a confidence game than it does in helping children learn critical reasoning. Which is to say none, zero, zilch in both cases. It has as much volition and desire as a calculator. It’s just a super impressive and useful calculator.

weinzierl 4y ago

> "It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field. Does this "person" make mistakes?" Does this person stubbornly insist on being right, even if they are wrong? Likely, that's just how humans usually behave - and ChatGPT does the same. At least that was my thinking when people complain about ChatGPT being a know-it-all bluffer.

Karunamon 4y ago
ChatGPT is pretty amenable to being corrected if it says something you know is crap. And then, in my experience, you usually get better results. That is not something I can say for most humans.
- godelski 4y ago
  Only if you correct a wrong answer. If you "correct" a right answer ("I don't think that's right") then it gives an even worse answer. So be careful when correcting.
godelski 4y ago
> Does this person stubbornly insist on being right, even if they are wrong? Actually ChatGPT will almost always change tune if you tell it that it is wrong. That's because it is a stochastic parrot and that's an "unlikely" event if you actually got the answer right. But we'll see if this gets updated/broken too.
- weinzierl 4y ago
  I've seen ChatGPT change tune but I've also seen it squirm plenty. Overall I think it behaves uncannily human in this regard.

evo_9 4y ago

Yeah, it's fascinating to watch, right? As others have pointed out — what did we expect of 'AI' created in our image? Aren't we all just bullshit artists of varying degree? HN is the perfect place to learn how much BS you are - I DARE any subject matter expdrts on here to post on any topic and I guarantee another expert will step in and tell you why you're wrong, or at best, nitpick you apart.

checkyoursudo 4y ago

> It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field. I have yet to see a single instance of this with any query that requires even a modicum of expertise. I have asked it a lot of questions about my field. I have asked it to summarize and paraphrase and ELI5 all kinds of shit. Everything it produces is either inane or a regurgitation of the input just with verbs and prepositions changed. I would like to see a nice example, input and output, of this. I really would.

shawabawa3 4y ago
First thing that came to mind: > Explain how git works in very simple english Git is a version control system for software development that helps track changes made to the source code over time. It allows multiple developers to collaborate and make changes to the same codebase without interfering with each other. When you start a project in Git, you create a "repository" that stores all the files and their changes. You can then "commit" changes to the repository, which is like saving a snapshot of the code at a certain point in time. You can also "push" the changes to a remote repository on a server, so that other people can access and work on the code. If someone else makes changes to the code, you can "pull" the updated version of the repository to your local machine and "merge" the changes with your own code. Git also allows you to create "branches" of the code, so that you can work on different features or fixes without affecting the main code. By using Git, you can easily revert back to previous versions of the code, compare changes between different versions, and track who made which changes

braingenious 4y ago

> It is basically like having an incredibly smart engineer/scientists/philosopher/etc It is a copy/paste machine that strings together words to make sentences that look like sentences it’s read. So it’s like an incredibly smart engineer/scientist/philosopher that does not and fundamentally cannot understand any of those concepts (or any concepts at all.) A parrot that can recite Macbeth’s monologue is not a s Shakespeare scholar.

deleted 4y ago
[deleted]
thaw13579 4y ago
I asked it to explain some very technical medical testing procedures to me "like I was five years old". It completely nailed it by truly reducing it to its essence and using appropriate metaphors that a child would understand (coloring books). Because of how niche the topic is, I'm fairly sure no such text exists in the training corpus. I'm not sure whether this constitutes "understanding", since that term usually applies to people's cognitive states; however, if a student provided this answer on an exam, they would certainly convince the examiner and pass the test!

guenthert 4y ago

> Does this "person" make mistakes? Can't cite their sources? Making citations up is not a "mistake", it's a lie. I have absolutely no use for a tool which lies to me, especially not when troubleshooting computer systems.

chx 4y ago

> It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field. No it can not explain anything. It's a (stochastic) parrot, nothing more. > Yep, but it is such an incredibly useful tool It is not. You are fooling yourself. You are building a castle on a swamp.

vehemenz 4y ago
ChatGPT tells me that Disney World's Magic Kingdom was built on a swamp. It's just a tool. You don't use a shovel to rake leaves, but that doesn't make a shovel any less useful.

Forgeties79 4y ago

>but it is still ludicrously useful. Wrote me a script to quickly convert SRT to VTT files as I edit and prep for uploading online. I haven't done anything akin to "coding" since I took a very basic class in college and messed around with HTML/CSS. Shaves several steps off of every video I do (I'm in a high volume world here).

Obscurity4340 4y ago

Its honestly astounding how coherent and dare, I say, eloquent it can be. From factual "Google"-type inquiries to jokes to whatever. I particularly enjoy the poetry it generates in defined formats like limericks, sonnets, etc. Yet to encounter any actual "bullshit" that it is frequently accused of (although I tend to ask closed-type questions or provide very specific parameters with which to constrain its output)

saiya-jin 4y ago

I can see tons of jobs disappearing even with current level. Web these days is filled with mediocre articles, and quite a few bureaucratic jobs can and will be replaced by some next iteration. Instead of army of mediocre journalists filling web portals, you just need 1 person to proof read it per business, maybe not even that. I can see the same for software dev in less than decade with current pace, 1 senior going over generated code and doing minor tweaks. Which is probably road to hell long term since how will juniors become seniors, but cost savings can be real and massive, drop in quality small if at all and it may be good enough for many. It has very good language skills, in my obscure and rather difficult native language which only cca 6 million people use it formulated perfect sentences consistently. I can easily imagine faking social, even love chats easily. This can help lonely or elderly people (with good speech recognition/generation, since language skill itself is there already). With image/video generation porn industry will be steamrolled, 24/7 streams of anything you can imagine, looking realistic enough, tailored just to you. Or just help with learning foreign languages, heck few months ago a guy was posting here on HN just a smart chat that could be used for training and got standing ovation also from me for that, this has it and everything else. I understand google's reaction, instead of browsing through paid entries it gives me answer directly, no need to click through search and then wikipedia and then parsing texts. At least in some cases. This could help google massively if they could somehow do it on their scale. People trying to find errors in it are missing the point now - its not about academic perfection, but often good-enough almost for free. Also, this is not final product, only 4th alpha/beta/whatever, previous ones were much worse. Imagine 10 or 20 years down the road...

autokad 4y ago

I agree, its incredibly useful if you know how to use it. I used it to make sentences more succinct for emails to managers (no sensitive data of course), I used it to turn my dreams into short stories, gather ideas for research, etc. People bashing on chatgpt just dont know how to use it. is there going to be an example in the world where it just plagiarizes bob dillan? sure ... so what? is there going to be instances where its not useful? duh. but if you dont like it, that's fine. not every tool is for everyone. I find it very useful for certain things, and I imagine its only going to improve from here.

belter 4y ago

Sometimes I think ChatGPT was just some twisted, treacherous, indirect plot, to find a way to finally discredit Jordan Peters.

freejazz 4y ago

>"It is basically like having an incredibly smart engineer/scientists/philosopher/etc that can explain things quite well, but for pretty much every field." LOL where do you get this notion? I haven't seen one chatGPT bot useful for law. They all have no idea what they are doing and are basically dangerous if you don't know better than the LLM. There is literally nothing chatGPT can do for me as an attorney that isn't already solved by using a template.

bitsnbytes 4y ago

FWIW I see it the same and have had the same experience. The tech is there now to be utilized as an assistant. However as the end user you better have enough knowledge to understand the solution or answer it spits at you to fact check it. The biggest danger I see is utilizing the tool for solutions or answers by novices that don't understand enough to grasp the solution and confirm if in fact the answer it gives is correct or appropriate. Simplistic example: A non techie manager trying to have it create sql queries to get the information they are asking for , is asking for trouble. However in the hands of a sql developer it could provide enough of the query build out to just require minimal tweaks.

jacobjjacob 4y ago

The latter group exists to counter the former. If folks weren’t making some of these over-the-top claims, then there wouldn’t be the claims to debunk.

choppaface 4y ago

What’s worse, too much hype, or the non-believers? Well the OpenAI hype squad is racking up personal fortunes from their work in a finding deal that is pretty clearly over-valued. Especially versus what Deepmind sold for [1]. While perhaps both sides are not being honest, YC’s own sama is clearly profiting excessively over the hubub. He’s clearly proving to be a symbol of the faction of YC that rakes in money through arbitrage not prolonged value. [1] https://techcrunch.com/2014/01/26/google-deepmind/ https://techcrunch.com/2014/01/26/google-deepmind/

danjoredd 4y ago

My problem is the type of person that proudly proclaims that all jobs can, should, and are going to be outdated by the use of AI. I have talked to people that believe that, and think that AI need to replace humanity for some reason. I don't understand them. AI is exciting, and it can be an useful tool. There are a lot of use cases for it, and in the end I think it will be a good thing. However, what's worse than the detractors are the people blindly hoping that AI will automate every single thing a human touches. It shouldn't be used in that way. It has to be a tool to assist humans, not take away our professions and reasons to get out of bed in the morning.

groby_b 4y ago

In all the fields I've tried (software/music/epidemiology/geosciences/mathematics) ChatGPT can mostly explain things at the lay level, but even there often gets things wrong that without more expert guidance, you will not notice. Experts in each of these areas were consistently better at giving a correct explanation. ChatGPT is great when occasionally being wrong doesn't matter. It's still pretty good when you can't be wrong, but you're an expert. But in any "no mistakes, I need to talk to an expert" situation, ChatGPT is setting you up for failure. That's the key part - the information is unreliable/untrustworthy. As long as that is not a problem, it's awesome. The problem is that it presents wrong data with absolute confidence in a very convincing voice. (Hence people calling it a con artist). It'll become a huge problem as it makes inroads in mis/disinfo campaigns. (The usual tech problem. Don't let large scale interests touch it, and you'll be OK)

deleted 4y ago

[deleted]

up2isomorphism 4y ago

None of the coding questions it answers is closed to usable, and i need to spend more time correcting it than just write it on my own. So it is just not “incredibly smart”.

deleted 4y ago

[deleted]