13 ms·

Yeah, I read this sentiment all the time and here's what I always say – just don't use it. Leave it to the rest of us if it's so wrong / off / bad. BTW, have y

by evo_9 4y ago

Yeah, I read this sentiment all the time and here's what I always say – just don't use it. Leave it to the rest of us if it's so wrong / off / bad. BTW, have you considered maybe you aren't so good at using it? A friend has had very little luck with it, even said he's been 'arguing with it', which made me laugh. I've noticed that it's not obvious to most people that it's mostly about knowing the domain well enough to ask the right question(s). It's not magic, it won't think for you. Here's the thing… my experience is the opposite… but maybe I'm asking it the right questions. Maybe it's more about using it to reason through your problem in a dialog, and not just ask it something you can google/duckduckgo. It seems like a LOT of people think it's a replacement for Google/search engines – it's not, it's another tool to be used correctly. Here are some examples of successful uses for me: I carefully explained a complex work issue that involves multiple overlapping systems and our need to get off of one of them in the middle of this mess. My team has struggle for 8 months to come up with a plan. While in a meeting the other day I got into a conversation with ChatGPT about it, carefully explained all the details and then asked it to create a plan for us to get off the system while keeping everything up / running. It spit out a 2 page, 8 point plan that is nearly 100% correct. I showed it to my team, and we made a few minor changes, and then it was anointed 'the plan' and we're actually moving forward. THEN last night I got stuck on a funny syntax issue that googling could never find the answer. I got into a conversation with ChatGPT about it, and after it first gave me the wrong answer, I told it that I need this solution for the latest dontet library that follows the 'core' language syntax. It apologized! And then gave me the correct answer… My hunch is the people that are truly irked by this are too deep / close to the subject and because it doesn't match up with what they've worked on, studied, invested time, mental energy into, well then of course it's hot garbage and 'bad'.

CPLX 4y ago
I mean sure. In other news I asked it to make a list of all the dates in 2023 that were neither weekends nor US federal holidays and it left Christmas Day on the list.
- mashygpig 4y ago
  Yea, I think people hide “the magic smoke” by using complex queries and then filling in the gaps of chatGPT’s outputs with their own knowledge, which then makes them overvalue the output. Strip that away to simple examples like this and it becomes more clear what’s going on. (I think there IS a lot of value for them in their current state because they can jog your brain like this, just not to expect it to know how to do everything for you. Think of it as the most sophisticated rubber duck that we’ve made yet).
elorant 4y ago
I too have a very positive experience. I ask specific questions about algorithms and how technical projects work and I enjoy its answers. They won’t replace my need to visit a real search engine neither I take them at face value. But as a starting point for any research I think it’s an amazing tool. It’s also quite good for marketing stuff, like writing e-mails, cover letters, copy for your website, summarizing or classifying text, and all language related stuff. People think it’s Cortana from Halo and ask existential questions or they’re trying to get it to express feelings. I think the AI part on its presentation created too much expectations of what it can do.
Taurenking 4y ago
[dead]
youk 4y ago
Great write up. My experience is spot on with your examples. > I've noticed that it's not obvious to most people that it's mostly about knowing the domain well enough to ask the right question(s). It's not magic, it won't think for you. Absolutely right with the part of knowing the domain. I do not entertain or care about the AI fantasies because ChatGPT is extremely good at getting me other information. It saves me from opening a new tab, formulating my query and then hunting for the information. I can save that extra time for what latest / relevant information I should grab from Google. Google is still in my back pocket for the last mile verification and judgement. I am also skeptical of the information ChatGPT throws out (such as old links). Other than that, ChatGPT to me is as radical as putting the url and search bar into one input. I just move faster with the information.
YurgenJurgensen 4y ago
Don't like chlorofluorocarbons or tetraethyllead? Just don't use them.
timmytokyo 4y ago
I don't understand this take. These LLM-based AIs provide demonstrably incorrect answers to questions, they're being mass-marketed to the entire population, and the correct response to this state of affairs is "Don't use it if you don't know how"? As if that's going to stop millions of people from using it to unknowingly generate and propagate misinformation.
- roywiggins 4y ago
  Isn't that what people said about Google Search 20 years ago- that people won't know how to use it, that they will find junk information, etc. And they weren't entirely wrong, but it doesn't mean that web search isn't useful.
  - 10rm 4y ago
    That’s a good point. I don’t think anyone is denying that GPT will be useful though. I’m more worried that because of commercial reasons and public laziness / ignorance, it’s going to get shoehorned into use cases it’s not meant for and create a lot of misinformation. So a similar problem to search, but amplified
    - bun_at_work 4y ago
      There are some real concerns for a technology like ChatGPT or Bing's version or whatever AI. However, a lot of the criticisms are about the inaccuracy of the model's results. Saying "ChatGPT got this simple math wrong" isn't as useful or meaningful of a criticism when the product isn't being marketed as a calculator or some oracle of truth. It's being marketed as an LLM that you can chat with. If the majority of criticism was about how it could be abused to spread misinformation or enable manipulation of people at scale, or similar, the pushback on criticism would be less. It's nonsensical to say that ChatGPT doesn't have value because it gets things wrong. What makes much more sense is to say is that it could be leveraged to harm people, or manipulate them in ways they cannot prevent. Personally, it's more concerning that MS can embed high-value ad spots in responses through this integration, while farming very high-value data from the users, wrt advertising and digital surveillance.
wpietri 4y ago
> Yeah, I read this sentiment all the time and here's what I always say – just don't use it. Leave it to the rest of us if it's so wrong / off / bad. If it were only a matter of private, individual usage, I'd be fine with it. If that's all you're asking for, we can call it a deal. But it isn't, is it?
bambax 4y ago
> It seems like a LOT of people think it's a replacement for Google/search engines Well, that "lot" includes the highest levels of management from Microsoft and Google, so maybe the CAPS are justified. And the errors we're talking about here are errors produced by said management during demos of their own respective product. You would think they know how to use it "correctly".
- whimsicalism 4y ago
  I'm going to let you in on a secret: managers, even high-level ones, can be wrong - and indeed they frequently are.
  - bambax 4y ago
    Thanks for that unique insight. But the question is, are they wrong in that they don't know how to use / promote an otherwise good product, or are they wrong because they are choosing to put forward something that is completely ill-suited for the task?
    - krainboltgreene 4y ago
      Both.
10rm 4y ago
When did they say it’s garbage? They gave their opinions on its shortcomings and praised some of the things it excels at. You’re calling the critics too emotional but this reply is incredibly defensive. Your anecdotes are really cool and a great example of what GPT can do really well. But as a technical person, you’re much more aware of its limitations and what is and isn’t a good prompt for it. But as it is more and more marketed to the public, and with people already clamoring to replace traditional search engines with it, relying on the user to filter out disinformation well and not use it for prompts it struggles with isn’t good enough.
whimsicalism 4y ago
I feel similarly reading many critiques, but honestly the GP is one of the more measured ones that I've read - not sure that your comment is actually all that responsive or proportionate.
shrimpx 4y ago
"Just don't use it" is not salient advice for non-technical people who don't know how it works, and are misled by basically dishonest advertising and product packaging. But hopefully the market will speak, users at large will become educated about its limits via publicized blunders, and these products will be correctly delimited as "lies a lot but could be useful if you are able/willing to verify what it says."
- hughc 4y ago
  I think the original sentence was written more in of "Your loss is my gain" competitive advantage vein. The real trick is, as you say, to critically assess the output, and many people are incapable of that.
gptgpp 4y ago
You all say it's solving these amazing complex tasks for you, but then don't provide any details. Then "naysayers" like the linked article provide a whole document with images and appendixes showing it struggles with basic tasks... So show us. For the love of god all of us would very much LIKE this technology to be good at things! Whatever techniques you're using to get these fantastical results, why don't you share them? I can get it to provide snippets of code, CLI, toy functions that work. Beyond that, I am apparently an idiot compared to you AI-whisperers. Also... Whatever happened to "extraordinary claims require extraordinary proof?" An AI that creates a complex system, condensed into an actionable plan, that has stumped an entire team for 8 months is a (pardon the language) bat-shit insane claim. Things like this used to require proof to be taken seriously.
- oezi 4y ago
  My take: Because GPT is just stochasticly stringing words after each other, it is remarkedly good at producing text on par with other text available on the internet. So it can produce plans, strategies, itineraries and so on. The more abstract the better. The 8 point plan is likely great. It will much more likely fail on anything which involves precision/computation/logic. That's why it can come up with an generic strategy but fail to repeat unadjusted GAAP earnings.
  - harpiaharpyja 4y ago
    There was an article not too long ago, that I'm struggling to find, that did a great job of explaining why language models are much much better suited to reverse-engineering code than they are at forward-engineering it.
    - deleted 4y ago
      [deleted]
  - gptgpp 4y ago
    I agree it's pretty good at generalities, doesn't shit the bed quite so much. Yet to suggest a plan that an entire team of professionals, who have been working for 8 months could not figure out? It's certainly not that good, absent some amazing wizardry or some very silly professionals in a very squishy field. Yet I have no explanation for why someone would go on the internet and lie about something like that. There were comments a while back (less so now) of people making other claims like it was solving complex functions for them and writing sophisticated software. The entire thing baffles me. If I could get it to do that, I'd be showing you all of my marvelous works and bragging quite a bit as your newfound AI-whisperer. Hell, I'd get it to write a script for me to run that evangelized itself (edit: and me of course, as its chosen envoy to mankind) to the furthest corners of the internet!
joenot443 4y ago
I’d really love to hear more about your workplace use-case, what kind of systems are we talking about here? This is a way of using ChatGPT I haven’t really seen before, I’m really into it.
daveguy 4y ago
> My hunch is the people that are truly irked by this are too deep / close to the subject and because it doesn't match up with what they've worked on, studied, invested time, mental energy into, well then of course it's hot garbage and 'bad'. That's quite the straw man you've built. Recognizing the limitations of a technology is not the same as calling it hot garbage. As a language model it's amazing, but I agree with the GP. It's not intelligent. It's very good at responding to a series of tokens with its own series of tokens. That requires a degree of understanding of short scale context that we haven't had before in language models. It's an amazing breakthrough. But it's also like attributing the muscle memory of your hand to intelligence. It can solve lots of problems. It can come up with good configurations. It is not, on its own, intelligent.
acchow 4y ago
I imagine your first example includes private industry information that you are not allowed to divulge. But your latter example about syntax… mind sharing that ChatGPT conversation?
westoncb 4y ago
> Maybe it's more about using it to reason through your problem in a dialog, and not just ask it something you can google/duckduckgo. Your experience with it sounds very similar to my own. It exhibits something like on-demand precision; it's not a system with some fundamental limit to clarity (like Ted Chiang via his jpeg analogy, and others, have argued): it may say something fuzzy and approximate (or straight up wrong) to begin with but—assuming you haven't run into some corner where its knowledge just bottoms out—you can generally just tell it that it made a mistake or ask for it to elaborate/clarify etc., and it'll "zoom in" further and resolve fuzziness/incorrect approximation. There is a certain very powerful type of intelligence within it as well, but you've got to know what it's good at to use it well: from what I can tell it basically comes down to it being very good at identifying "structural similarity" between concepts (essentially the part of cognition which is rooted in analogy-making), allowing it to very effectively make connections between disparate subject matter. This is how it's able to effectively produce original work (though typically it will be directed there by a human): one of my favorite examples of this was someone asking it to write a Lisp program that implements "virtue ethics" (https://twitter.com/zetalyrae/status/1599167510099599360 https://twitter.com/zetalyrae/status/1599167510099599360). I've done a few experiments myself using it to formalize bizarre concepts from other domains and its ability to "reason" in both domains to make decisions about how to formalize, and then generating formalizations, is very impressive. It's not enough for me to say it is unqualifiedly "intelligent", but it imo its ability to do this kind of thing makes it clear why calling it a search engine, or something merely producing interpolated averages (a la Chiang), is so misleading.
pclmulqdq 4y ago
Just to flip this around for a second, with both of your examples, it sounds like you may have a problem with writer's block or analysis paralysis, and ChatGPT helped you overcome that simply due to the fact that it isn't afraid of what it doesn't know. If that helps you, go for it. On the other hand, it could also help you to just write a random plan or try a few random things when you get stuck, instead of trying to gaze deeply into the problem for it to reveal its secrets.
lossolo 4y ago
> THEN last night I got stuck on a funny syntax issue that googling could never find the answer. I got into a conversation with ChatGPT about it, and after it first gave me the wrong answer, I told it that I need this solution for the latest dontet library that follows the 'core' language syntax. It apologized! And then gave me the correct answer… Great that it worked for you, I had a similar problem (Google had no answer) but more complex than syntax issue, I'm also domain expert in what I was asking and chatGPT also gave me wrong answer the first time then apologized and gave me wrong answer again, I've explained what's wrong and it did it again and again.. Never providing correct answer so I just gave up and used human brain. Seems like your problem was in distribution.
sanderjd 4y ago
This doesn't seem like a response to your parent comment, which in no way suggested they were "irked" by this or consider it bad. It was an insightful comment contrasting strengths and weaknesses of these language models. It's a pretty weak rebuttal in my view to just say "there are no weaknesses, you're just doing it wrong!".
netheril96 4y ago
If only a small subset of people online are able to truly take advantage of ChatGPT, then I don't think Google is as threatened by it as many have portrayed.

by evo_9 4y ago

CPLX 4y ago

I mean sure. In other news I asked it to make a list of all the dates in 2023 that were neither weekends nor US federal holidays and it left Christmas Day on the list.

mashygpig 4y ago
Yea, I think people hide “the magic smoke” by using complex queries and then filling in the gaps of chatGPT’s outputs with their own knowledge, which then makes them overvalue the output. Strip that away to simple examples like this and it becomes more clear what’s going on. (I think there IS a lot of value for them in their current state because they can jog your brain like this, just not to expect it to know how to do everything for you. Think of it as the most sophisticated rubber duck that we’ve made yet).

elorant 4y ago

I too have a very positive experience. I ask specific questions about algorithms and how technical projects work and I enjoy its answers. They won’t replace my need to visit a real search engine neither I take them at face value. But as a starting point for any research I think it’s an amazing tool. It’s also quite good for marketing stuff, like writing e-mails, cover letters, copy for your website, summarizing or classifying text, and all language related stuff. People think it’s Cortana from Halo and ask existential questions or they’re trying to get it to express feelings. I think the AI part on its presentation created too much expectations of what it can do.

Taurenking 4y ago

[dead]

youk 4y ago

Great write up. My experience is spot on with your examples. > I've noticed that it's not obvious to most people that it's mostly about knowing the domain well enough to ask the right question(s). It's not magic, it won't think for you. Absolutely right with the part of knowing the domain. I do not entertain or care about the AI fantasies because ChatGPT is extremely good at getting me other information. It saves me from opening a new tab, formulating my query and then hunting for the information. I can save that extra time for what latest / relevant information I should grab from Google. Google is still in my back pocket for the last mile verification and judgement. I am also skeptical of the information ChatGPT throws out (such as old links). Other than that, ChatGPT to me is as radical as putting the url and search bar into one input. I just move faster with the information.

YurgenJurgensen 4y ago

Don't like chlorofluorocarbons or tetraethyllead? Just don't use them.

timmytokyo 4y ago

I don't understand this take. These LLM-based AIs provide demonstrably incorrect answers to questions, they're being mass-marketed to the entire population, and the correct response to this state of affairs is "Don't use it if you don't know how"? As if that's going to stop millions of people from using it to unknowingly generate and propagate misinformation.

roywiggins 4y ago
Isn't that what people said about Google Search 20 years ago- that people won't know how to use it, that they will find junk information, etc. And they weren't entirely wrong, but it doesn't mean that web search isn't useful.
- 10rm 4y ago
  That’s a good point. I don’t think anyone is denying that GPT will be useful though. I’m more worried that because of commercial reasons and public laziness / ignorance, it’s going to get shoehorned into use cases it’s not meant for and create a lot of misinformation. So a similar problem to search, but amplified
  - bun_at_work 4y ago
    There are some real concerns for a technology like ChatGPT or Bing's version or whatever AI. However, a lot of the criticisms are about the inaccuracy of the model's results. Saying "ChatGPT got this simple math wrong" isn't as useful or meaningful of a criticism when the product isn't being marketed as a calculator or some oracle of truth. It's being marketed as an LLM that you can chat with. If the majority of criticism was about how it could be abused to spread misinformation or enable manipulation of people at scale, or similar, the pushback on criticism would be less. It's nonsensical to say that ChatGPT doesn't have value because it gets things wrong. What makes much more sense is to say is that it could be leveraged to harm people, or manipulate them in ways they cannot prevent. Personally, it's more concerning that MS can embed high-value ad spots in responses through this integration, while farming very high-value data from the users, wrt advertising and digital surveillance.

wpietri 4y ago

> Yeah, I read this sentiment all the time and here's what I always say – just don't use it. Leave it to the rest of us if it's so wrong / off / bad. If it were only a matter of private, individual usage, I'd be fine with it. If that's all you're asking for, we can call it a deal. But it isn't, is it?

bambax 4y ago

> It seems like a LOT of people think it's a replacement for Google/search engines Well, that "lot" includes the highest levels of management from Microsoft and Google, so maybe the CAPS are justified. And the errors we're talking about here are errors produced by said management during demos of their own respective product. You would think they know how to use it "correctly".

whimsicalism 4y ago
I'm going to let you in on a secret: managers, even high-level ones, can be wrong - and indeed they frequently are.
- bambax 4y ago
  Thanks for that unique insight. But the question is, are they wrong in that they don't know how to use / promote an otherwise good product, or are they wrong because they are choosing to put forward something that is completely ill-suited for the task?
  - krainboltgreene 4y ago
    Both.

10rm 4y ago

When did they say it’s garbage? They gave their opinions on its shortcomings and praised some of the things it excels at. You’re calling the critics too emotional but this reply is incredibly defensive. Your anecdotes are really cool and a great example of what GPT can do really well. But as a technical person, you’re much more aware of its limitations and what is and isn’t a good prompt for it. But as it is more and more marketed to the public, and with people already clamoring to replace traditional search engines with it, relying on the user to filter out disinformation well and not use it for prompts it struggles with isn’t good enough.

whimsicalism 4y ago

I feel similarly reading many critiques, but honestly the GP is one of the more measured ones that I've read - not sure that your comment is actually all that responsive or proportionate.

shrimpx 4y ago

"Just don't use it" is not salient advice for non-technical people who don't know how it works, and are misled by basically dishonest advertising and product packaging. But hopefully the market will speak, users at large will become educated about its limits via publicized blunders, and these products will be correctly delimited as "lies a lot but could be useful if you are able/willing to verify what it says."

hughc 4y ago
I think the original sentence was written more in of "Your loss is my gain" competitive advantage vein. The real trick is, as you say, to critically assess the output, and many people are incapable of that.

gptgpp 4y ago

You all say it's solving these amazing complex tasks for you, but then don't provide any details. Then "naysayers" like the linked article provide a whole document with images and appendixes showing it struggles with basic tasks... So show us. For the love of god all of us would very much LIKE this technology to be good at things! Whatever techniques you're using to get these fantastical results, why don't you share them? I can get it to provide snippets of code, CLI, toy functions that work. Beyond that, I am apparently an idiot compared to you AI-whisperers. Also... Whatever happened to "extraordinary claims require extraordinary proof?" An AI that creates a complex system, condensed into an actionable plan, that has stumped an entire team for 8 months is a (pardon the language) bat-shit insane claim. Things like this used to require proof to be taken seriously.

oezi 4y ago
My take: Because GPT is just stochasticly stringing words after each other, it is remarkedly good at producing text on par with other text available on the internet. So it can produce plans, strategies, itineraries and so on. The more abstract the better. The 8 point plan is likely great. It will much more likely fail on anything which involves precision/computation/logic. That's why it can come up with an generic strategy but fail to repeat unadjusted GAAP earnings.
- harpiaharpyja 4y ago
  There was an article not too long ago, that I'm struggling to find, that did a great job of explaining why language models are much much better suited to reverse-engineering code than they are at forward-engineering it.
  - deleted 4y ago
    [deleted]
- gptgpp 4y ago
  I agree it's pretty good at generalities, doesn't shit the bed quite so much. Yet to suggest a plan that an entire team of professionals, who have been working for 8 months could not figure out? It's certainly not that good, absent some amazing wizardry or some very silly professionals in a very squishy field. Yet I have no explanation for why someone would go on the internet and lie about something like that. There were comments a while back (less so now) of people making other claims like it was solving complex functions for them and writing sophisticated software. The entire thing baffles me. If I could get it to do that, I'd be showing you all of my marvelous works and bragging quite a bit as your newfound AI-whisperer. Hell, I'd get it to write a script for me to run that evangelized itself (edit: and me of course, as its chosen envoy to mankind) to the furthest corners of the internet!

joenot443 4y ago

I’d really love to hear more about your workplace use-case, what kind of systems are we talking about here? This is a way of using ChatGPT I haven’t really seen before, I’m really into it.

daveguy 4y ago

> My hunch is the people that are truly irked by this are too deep / close to the subject and because it doesn't match up with what they've worked on, studied, invested time, mental energy into, well then of course it's hot garbage and 'bad'. That's quite the straw man you've built. Recognizing the limitations of a technology is not the same as calling it hot garbage. As a language model it's amazing, but I agree with the GP. It's not intelligent. It's very good at responding to a series of tokens with its own series of tokens. That requires a degree of understanding of short scale context that we haven't had before in language models. It's an amazing breakthrough. But it's also like attributing the muscle memory of your hand to intelligence. It can solve lots of problems. It can come up with good configurations. It is not, on its own, intelligent.

acchow 4y ago

I imagine your first example includes private industry information that you are not allowed to divulge. But your latter example about syntax… mind sharing that ChatGPT conversation?

westoncb 4y ago

> Maybe it's more about using it to reason through your problem in a dialog, and not just ask it something you can google/duckduckgo. Your experience with it sounds very similar to my own. It exhibits something like on-demand precision; it's not a system with some fundamental limit to clarity (like Ted Chiang via his jpeg analogy, and others, have argued): it may say something fuzzy and approximate (or straight up wrong) to begin with but—assuming you haven't run into some corner where its knowledge just bottoms out—you can generally just tell it that it made a mistake or ask for it to elaborate/clarify etc., and it'll "zoom in" further and resolve fuzziness/incorrect approximation. There is a certain very powerful type of intelligence within it as well, but you've got to know what it's good at to use it well: from what I can tell it basically comes down to it being very good at identifying "structural similarity" between concepts (essentially the part of cognition which is rooted in analogy-making), allowing it to very effectively make connections between disparate subject matter. This is how it's able to effectively produce original work (though typically it will be directed there by a human): one of my favorite examples of this was someone asking it to write a Lisp program that implements "virtue ethics" (https://twitter.com/zetalyrae/status/1599167510099599360 https://twitter.com/zetalyrae/status/1599167510099599360). I've done a few experiments myself using it to formalize bizarre concepts from other domains and its ability to "reason" in both domains to make decisions about how to formalize, and then generating formalizations, is very impressive. It's not enough for me to say it is unqualifiedly "intelligent", but it imo its ability to do this kind of thing makes it clear why calling it a search engine, or something merely producing interpolated averages (a la Chiang), is so misleading.

pclmulqdq 4y ago

Just to flip this around for a second, with both of your examples, it sounds like you may have a problem with writer's block or analysis paralysis, and ChatGPT helped you overcome that simply due to the fact that it isn't afraid of what it doesn't know. If that helps you, go for it. On the other hand, it could also help you to just write a random plan or try a few random things when you get stuck, instead of trying to gaze deeply into the problem for it to reveal its secrets.

lossolo 4y ago

> THEN last night I got stuck on a funny syntax issue that googling could never find the answer. I got into a conversation with ChatGPT about it, and after it first gave me the wrong answer, I told it that I need this solution for the latest dontet library that follows the 'core' language syntax. It apologized! And then gave me the correct answer… Great that it worked for you, I had a similar problem (Google had no answer) but more complex than syntax issue, I'm also domain expert in what I was asking and chatGPT also gave me wrong answer the first time then apologized and gave me wrong answer again, I've explained what's wrong and it did it again and again.. Never providing correct answer so I just gave up and used human brain. Seems like your problem was in distribution.

sanderjd 4y ago

This doesn't seem like a response to your parent comment, which in no way suggested they were "irked" by this or consider it bad. It was an insightful comment contrasting strengths and weaknesses of these language models. It's a pretty weak rebuttal in my view to just say "there are no weaknesses, you're just doing it wrong!".

netheril96 4y ago

If only a small subset of people online are able to truly take advantage of ChatGPT, then I don't think Google is as threatened by it as many have portrayed.