8 ms·

I still don't understand how you can talk to something that doesn't provide factual information and just take it at face value? The other day I asked it about

I still don't understand how you can talk to something that doesn't provide factual information and just take it at face value? The other day I asked it about the place I live and it made up nonsense, I was trying to get it to help me with an essay and it was just wrong, it was telling me things about this region that weren't real. Do we just drive through a town, ask for a made up history about it and just be satisfied with whatever is provided?

skepticATX 3y ago
What LLMs have made me realize more than anything is that we just don't care that much the information we receive being completely factual. I have tried to use it many times to learn a topic, and my experience has been that it is either frustratingly vague or incorrect. It's not a tool that I can completely add to my workflow until it is reliable, but I seem to be the odd one out.
- bamboozled 3y ago
  > What LLMs have made me realize more than anything is that we just don't care that much the information we receive being completely factual. I find this highly concerning but I feel similar. Even "smart people" I work with seem to have gulped down the LLM cool aid because it's convenient and it's "cool". Sometimes I honestly think: "just surrender to it all, believe in all the machine tells you unquestionably, forget the fact checking, it feels good to be ignorant... it will be fine...". I just can't do it though.
  - steve_adams_86 3y ago
    I just verify the information I need. I find it useful as a sort of search engine for solutions. Like, how could I use generators as hierarchical state machines? Are there other approaches that would work? What are some issues with these solutions? Etc. By the end I have enough information to begin searching the web for comparisons, other solutions, and so on. The benefit is that I got a quick look at various solutions and quickly satisfied a curiosity, and decided if I’m interested in the concept or not. Without AI, I might just leave the idea alone or spend too much time figuring it out. Or perhaps never quite figure out the terms of what I’m trying to discover, as it’s good at connecting dots when you have an idea with some missing pieces. I wouldn’t use it for a conversation about things as others are describing. I need a way to verify its output at any time. I find that idea bizarre. Just chatting with a hallucinating machine. Yet I still find it useful as a sort of “idea machine”.
    - bamboozled 3y ago
      I think this is a fine use case though because you're doing your due diligence. The problems arise when you don't do this. I think even if an AGI was created, and humans survived this event. I'd still have trouble trusting it. The quote "trust but verify" is everything to me.
  - thfuran 3y ago
    Ignorance is Strength
  - huijzer 3y ago
    > just surrender to it all, believe in all the machine tells you unquestionably, forget the fact checking, it feels good to be ignorant... it will be fine... It's the same issue with Google Search, any web page, or, heck, any book. Fact checking gets you only so far. You need critical thinking. It's okay to "learn" wrong facts from time to time as long as you are willing to be critical and throw the ideas away if they turn out to be wrong. I think this Popperian view is much more useful than living with the idea that you can only accept information that is provably true. Life is too short to verify every fact. Most things outside programming are not even verifiable anyway. By the time that Steve Jobs would have "verified" that the iPhone was certainly a good idea to pursue, Apple might have been bankrupt. Or in the old days, by the time you have verified that there is a tiger in the bush, it has already eaten you.
    - cmiles74 3y ago
      There's a lot of truth in this comment and a lot that I wholeheartedly agree with. When I spend time on something that turns out to be incorrect, I would prefer it to be because of choice I made instead of some random choice made by an LLM. Maybe the author is someone I'm interested in, maybe there's value in understanding other sides of the issue, etc. When I learn something erroneous from an LLM, all I know is that the LLM told me.
    - DharmaPolice 3y ago
      The issue is far more serious with ChatGPT/similar models because things that are laughably untrue are delivered exactly the same as something that's solidly true. When doing a normal search I can make some assessment on the quality of the source and the likelihood the source is wrong. People should be able "throw the ideas away if they turn out to be wrong" but the problem is these ideas unconsciously or not help build your model of the world. Once you find out something isn't true it's hard to unpick your mental model of the world.
      - huijzer 3y ago
        > Once you find out something isn't true it's hard to unpick your mental model of the world. Intuitively, I would think the same, but a book about education research that I read and my own experience taught me that new information is surprisingly easy to unlearn. It’s probably because new information sits at the edges of your neural networks and do not yet provide a foundation for other knowledge. This will only happen if the knowledge stands the test of time (which is exactly how it should be according to Popper). If a counterexample is found, then the information can easily be discarded since it’s not foundational anyway and the brain learns the counterexample too (the brain is very good in remembering surprising things).
  - flkenosad 3y ago
    The smart people I've seen using ChatGPT always double check the facts it gives. However, the truth is that RLHF works well to extinguish these lies over time. As more people use the platform and give feedback, the thing gets better. And now, I find it to be pretty darn accurate.
    - macNchz 3y ago
      I see this conversation pretty frequently and I think the root of it lies in the fact that we have mental heuristics for determining whether we need to fact check another human because they are a bullshitter, an idiot, a charlatan etc, but most people haven’t really developed this sense for AIs. I think the current state of AI trustworthiness (“very impressive and often accurate but occasionally extremely wrong”) triggers similar mental pathways to interacting with a true sociopath or pathological liar for the first time in real life, which can be intensely disorienting and cause one to question their trust in everyone else, as they try to comprehend this type of person.
    - TerrifiedMouse 3y ago
      > The smart people I've seen using ChatGPT always double check the facts it gives. I don't like being told lies in the first place and having to unlearn it. It doesn't help that I might as well have just gone straight to the "verification" instead.
    - FooBarWidget 3y ago
      I don't know. The other day I was asking about a biology topic and it straight up gave me a self-contradicting chemical reaction process description. It kept doing that after I pointed out the contradiction. Eventually I got out of this hallucination loop by resetting the conversation and asking again. It's smart but can also be very dumb.
  - deleted 3y ago
    [deleted]
- uoaei 3y ago
  I think this post-factual attitude is stronger and more common in some cultures than others. I'm afraid to say but given my extensive travels it appears American culture (and its derivatives in other countries) seems to be spearheading this shift.
  - bamboozled 3y ago
    Warning, my opinion ahead: I think it's because Americans, more than nearly all other cultures, love convenience. It's why the love for driving is so strong in the US. Don't walk or ride, drive. Once I was walking back from the grocer in Florida with 4 shopping bags, and people pulled over and asked if my car had broken down and if I needed a ride, people were stunned...I was walking for exercise and for the environment...and I was stunned. More evidence of this trend can be seen in the products and marketing being produced: Do you need to write a wedding speech? Click here. Do you need to go get something from the store? get your fat ass in the car and drive, better yet, get a car that drives for you? Better than this, we'll deliver it with a drone...don't move a muscle. Don't want to do your homework? Here... Want to produce art? Please enter your prompt... Want to lose weight? We have a drug for that... Want to be the authority on some topic? We'll generate the facts you need.
    - uoaei 3y ago
      I've also identified convenience as a core factor. Another dynamic at play is this: As convenience in a domain becomes ubiquitous or at least expected among consumers, they quickly readjust their evaluation of "having time for X" around the new expectation of the convenient service, treating all alternatives as positive opportunity cost. This would explain a lot of those folks who are upset when it's suggested that they don't need Amazon, Instacart, etc. in their lives if they are to do something about their contributions to mass labor exploitation. Of course these conveniences quickly become ubiquitous in large economies with a glut of disposable income, which encourages VCs to dump money into these enterprises so they're first to market, and also to encourage the public to believe that the future is already here and there's no reason to worry about backsliding or sustainability of the business model. Yet in every single case we see prices eventually rise, laborers squeezed, etc. A critical mass of people haven't yet acknowledged this inevitability, in no small part due to this fixation on convenience at the expense of more objective, reasoned understandings (read: post-truth mindset).
- pbhjpbhj 3y ago
  ChatGPT 3.5 is terrible on technical subjects IME. Phind is best for me rn. Hugging Chat (Llama) works quite well too. They're only good on universal truths. An amalgam of laws from around the globe doesn't tell me what the law is in my country, for example.
- TerrifiedMouse 3y ago
  > It's not a tool that I can completely add to my workflow until it is reliable, but I seem to be the odd one out. This. I hate being told the wrong information because I will have to unlearn the wrong information. I would rather have been told nothing.
xnorswap 3y ago
> how you can talk to something that doesn't provide factual information and just take it at face value Like talking to most people you mean?
- philipwhiuk 3y ago
  When OpenAI buys me a drink at the bar in exchange for the rubbish it produces, I might have a more favourable view.
  - ilaksh 3y ago
    As soon as they release the API, we can build an AI "bartender". Combine the voice output and input with NeRF talking heads such as from Diarupt or https://github.com/harlanhong/awesome-talking-head-generation https://github.com/harlanhong/awesome-talking-head-generatio.... You will now be able to feed it images and responses of the customers. Give it a function to call complementaryDrink(customerId) Combine it with a simple vending machine style robot or something more complex that can mix drinks. I'm not actually in a hurry to try to replace bartenders. Just saying these types of things immediately become more feasible. You can also see the possibilities of the speech input and output for "virtual girlfriends". I assume someone at OpenAI must have been tempted to train a model on Scarlett Johansson's voice.
- lostcolony 3y ago
  Hopefully people know not to ask others for factual information (unless it's an area they're actually well educated/knowledgeable in), but for opinions and subjective viewpoints. "How's your day going", "How are you feeling", "What did you think of X", etc, not "So what was the deal with the Hundred Year's War?" or whatever. If people are treating LLMs like a random stranger and only making small talk, fair enough, but more often they're treating it like an inerrable font of knowledge, and that's concerning.
  - TeMPOraL 3y ago
    > If people are treating LLMs like a random stranger and only making small talk, fair enough, but more often they're treating it like an inerrable font of knowledge, and that's concerning. That's on them. I mean, people need to figure out that LLMs aren't random strangers, they're unfiltered inner voices of random strangers, spouting the first reaction they have to what you say to them. Anyway, there is a middle ground. I like to ask GPT-4 questions within my area of expertise, because I'm able to instantly and instinctively - read: effortlessly - judge how much to trust any given reply. It's very useful this way, because rating an answer in your own field takes much less work than coming up with it on your own.
whack 3y ago
Joe Rogan has made tons of money off talking without providing factual information. Hollywood has also made tons of money off movies "inspired by real events" that hallucinate key facts relevant to the movie's plot and characters. There's a huge market for infotainment that is "inspired by facts" but doesn't even try to be accurate.
- layer8 3y ago
  If that’s your benchmark, I don’t want your AI.
- bilsbie 3y ago
  Wait until you learn about the mainstream media.
  - bamboozled 3y ago
    For a certain demographic and generation, Joe Rogan is the mainstream media.
    - deleted 3y ago
      [deleted]
  - flangola7 3y ago
    Rogan is literally the largest podcast on the Spotify. It's the definition of mainstream.
- absrec 3y ago
  You listen to Joe Rogan with the idea that this is a normal dude talking not an expert beyond martial arts and comedy. A person who uses ChatGPT must have the understanding that it's not like Google search. The layman, however, has no idea that ChatGPT can give coherent incorrect information and treats the information as true. Most people won't use it for infotainment and OpenAI will try its best to downplay the hallucination as fine print if it goes fully mainstream like google search.
nh23423fefe 3y ago
Because it doesn't always make up stuff. Because I'm a human and can ask for more information. I don't want an encyclopedia on a podcast. I want to "talk" to someone about stuff. Not have an enumerated list of truths firehosed at me.
mmahemoff 3y ago
I'm curious if you're using GPT-4 ($)? I find a lot of the criticisms about hallucination come from users who aren't, and my experience with GPT-4 is it's far less likely to make stuff up. Does it know all the answers, certainly not, but it's self-aware enough to say sorry I don't know instead of making a wild guess.
- elicksaur 3y ago
  Why would anyone pay for something if the free trial doesn’t work? “Hey, you know how we gave you a product that doesn’t quit work as you expect and is super frustrating? Just pay us money, and we’ll give you the same product, but it just works. Just trust us!”
  - sacred_numbers 3y ago
    GPT-4 is not the same product. I know it seems like it due to the way they position 3.5 and 4 on the same page, but they are really quite separate things. When I signed up for ChatGPT plus I didn't even bother using 3.5 because I knew it would be inferior. I still have only used it a handful of times. GPT-4 is just so much farther ahead that using 3.5 is just a waste of time.
    - elicksaur 3y ago
      Would you mind sharing some threads where you thought ChatGPT was useful? These discussions always feel like I’m living on a different planet with a different implementation of large language models than others who claim they’re great. The problems I run into seem to stem from the fundamental nature of this class of products.
      - michaelt 3y ago
        The usefulness of ChatGPT is a bit situational, in my experience. But in the right situations it can be pretty powerful. Take a look at https://chat.openai.com/share/41bdb053-facd-448b-b446-1ba1f12b2f92 https://chat.openai.com/share/41bdb053-facd-448b-b446-1ba1f1... for example.
bilsbie 3y ago
> I still don't understand how you can talk to something that doesn't provide factual information and just take it at face value? All human interactions from all of history called and they …
londons_explore 3y ago
Pay for the Plus version. Then it makes stuff up far less frequently. If the next version has the same step up in performance, I will no longer consider inaccuracy an issue - even the best books have mistakes in them, they just need to be infrequent enough.
- nerdbert 3y ago
  > Pay for the Plus version. > Then it makes stuff up far less frequently. Now there's a business model for a ChatGPT-like service. $1/month: Almost always wrong $10/month: 50/50 chance of being right or wrong $100/month: right 95% of the time
  - TeMPOraL 3y ago
    You make it sound like business shenanigans, but the truth is, it's a natural fit for now, as performance of LLMs improves with their size, but costs of training (up-front investment) and inference (marginal, per-query) also go up.
  - md5wasp 3y ago
    Pay for the $1/month version and invert the responses; now you have the $100/month one for cheap :D
jocaal 3y ago
> ...talk to something that doesn't provide factual information and... Ah yes, I dont understand how to talk to people either!
- bamboozled 3y ago
  I always thought a better future would be full of more and more distilled, accurate, useful knowledge and truthful people to promote that. Comments like yours make me think that no one cares about this...and judging by a lot of the other comments, I guess they don't. Probably going to be people, wading through a sea of AI generated shit, and the individual is supposed to just forever "apply critical thinking" to it all. Even a call from ones spouse could be fake, and you'll just have to apply critical thinking or whatever to workout if you were scammed or not.
  - astrange 3y ago
    There aren't any real world sources of truth you can avoid applying critical thinking to. Much published research is false, and when it isn't, you need to know when it's expired or what context it's valid in.
    - ChatGTP 3y ago
      But do we need 9999999x the amount of information to critically be thinking about, is this going to be helpful ?
School-Cotton 3y ago
Are you using 3.5 or 4?
rickspencer3 3y ago
In my experience, LLVMs are not about being provided facts. They are about synthesizing new content and insights based on the model and inputs. Rather than asking it about facts, I find it useful to derive new insights. For example: "Tell me 5 topics about databases that might make it to the front page of hacker news." It can generate an interesting list. That is much more like the example they provided in the article, synthesizing a bed time story is not factual. Also, "write me some python code to do x" where x is based on libraries that were well documented before 2022 also has similarly creative results in my experience.
NikolaNovak 3y ago
This is a fairly perpetual discussion, but I'll go for another round: I feel like using LLM today is like using search 15 years ago - you get a feel for getting results you want. I'd never use chatGPT for anything that's even remotely obscure, controversial, or niche. But through all my double-checking, I've had phenomenal success rate in getting useful, readable, valid responses to well-covered / documented topics such as introductory french, introductory music theory, well-covered & non-controversial history and science. I'd love to see the example you experienced; if I ask chatGPT "tell me about Toronto, Canada", my expectation would be to get high accuracy. If I asked it "Was Hum, Croatia, part of the Istrian liberation movement in the seventies", I'd have far less confidence - it's a leading question, on a less covered topic, introducing inaccuracies in the prompt. My point is - for a 3 hour drive to cottage, I'm OK with something that's only 95% accurate on easy topics! I'd get no better from my spouse or best friend if they made it on the same drive :). My life will not depend on it, I'll have an educationally good time and miles will pass faster :). (also, these conversations always seem to end in suffocatingly self-righteous "I don't know how others can live in this post-fact free world of ignorance", but that has a LOT of assumptions and, ironically, non-factual bias in it as well)
- doug_durham 3y ago
  Exactly this! This is my experience also. Your point about "well covered & non-controversial" is spot on. I know not to expect great results when asking about topics that have very little coverage. To be honest I wouldn't expect to go to an arbitrary human and get solid answers on a little covered topic, unless that person just happened to be topic expert. There is so much value in having the basics to intermediate levels of topics covered in a reliable way. That's where most of commercial activity occurs.
  - jabradoodle 3y ago
    I think a key difference is that humans very rarely sound convincing talking about subjects they have no clue about. I've seen the hallucination rate of LLMs improve significantly, if you stick to well covered topics they probably do quite well. The issue is they often have no tells when making things up.
deleted 3y ago
[deleted]
FooBarWidget 3y ago
A human driving buddy can make up a lot of stuff too. Have an interesting conversation but don't take it too seriously. If you're really researching something serious then take a mental note to double check things later, pretend as if you're talking to a semi-reliable human who knows a lot but occasionally makes mistakes.

skepticATX 3y ago

What LLMs have made me realize more than anything is that we just don't care that much the information we receive being completely factual. I have tried to use it many times to learn a topic, and my experience has been that it is either frustratingly vague or incorrect. It's not a tool that I can completely add to my workflow until it is reliable, but I seem to be the odd one out.

bamboozled 3y ago
> What LLMs have made me realize more than anything is that we just don't care that much the information we receive being completely factual. I find this highly concerning but I feel similar. Even "smart people" I work with seem to have gulped down the LLM cool aid because it's convenient and it's "cool". Sometimes I honestly think: "just surrender to it all, believe in all the machine tells you unquestionably, forget the fact checking, it feels good to be ignorant... it will be fine...". I just can't do it though.
- steve_adams_86 3y ago
  I just verify the information I need. I find it useful as a sort of search engine for solutions. Like, how could I use generators as hierarchical state machines? Are there other approaches that would work? What are some issues with these solutions? Etc. By the end I have enough information to begin searching the web for comparisons, other solutions, and so on. The benefit is that I got a quick look at various solutions and quickly satisfied a curiosity, and decided if I’m interested in the concept or not. Without AI, I might just leave the idea alone or spend too much time figuring it out. Or perhaps never quite figure out the terms of what I’m trying to discover, as it’s good at connecting dots when you have an idea with some missing pieces. I wouldn’t use it for a conversation about things as others are describing. I need a way to verify its output at any time. I find that idea bizarre. Just chatting with a hallucinating machine. Yet I still find it useful as a sort of “idea machine”.
  - bamboozled 3y ago
    I think this is a fine use case though because you're doing your due diligence. The problems arise when you don't do this. I think even if an AGI was created, and humans survived this event. I'd still have trouble trusting it. The quote "trust but verify" is everything to me.
- thfuran 3y ago
  Ignorance is Strength
- huijzer 3y ago
  > just surrender to it all, believe in all the machine tells you unquestionably, forget the fact checking, it feels good to be ignorant... it will be fine... It's the same issue with Google Search, any web page, or, heck, any book. Fact checking gets you only so far. You need critical thinking. It's okay to "learn" wrong facts from time to time as long as you are willing to be critical and throw the ideas away if they turn out to be wrong. I think this Popperian view is much more useful than living with the idea that you can only accept information that is provably true. Life is too short to verify every fact. Most things outside programming are not even verifiable anyway. By the time that Steve Jobs would have "verified" that the iPhone was certainly a good idea to pursue, Apple might have been bankrupt. Or in the old days, by the time you have verified that there is a tiger in the bush, it has already eaten you.
  - cmiles74 3y ago
    There's a lot of truth in this comment and a lot that I wholeheartedly agree with. When I spend time on something that turns out to be incorrect, I would prefer it to be because of choice I made instead of some random choice made by an LLM. Maybe the author is someone I'm interested in, maybe there's value in understanding other sides of the issue, etc. When I learn something erroneous from an LLM, all I know is that the LLM told me.
  - DharmaPolice 3y ago
    The issue is far more serious with ChatGPT/similar models because things that are laughably untrue are delivered exactly the same as something that's solidly true. When doing a normal search I can make some assessment on the quality of the source and the likelihood the source is wrong. People should be able "throw the ideas away if they turn out to be wrong" but the problem is these ideas unconsciously or not help build your model of the world. Once you find out something isn't true it's hard to unpick your mental model of the world.
    - huijzer 3y ago
      > Once you find out something isn't true it's hard to unpick your mental model of the world. Intuitively, I would think the same, but a book about education research that I read and my own experience taught me that new information is surprisingly easy to unlearn. It’s probably because new information sits at the edges of your neural networks and do not yet provide a foundation for other knowledge. This will only happen if the knowledge stands the test of time (which is exactly how it should be according to Popper). If a counterexample is found, then the information can easily be discarded since it’s not foundational anyway and the brain learns the counterexample too (the brain is very good in remembering surprising things).
- flkenosad 3y ago
  The smart people I've seen using ChatGPT always double check the facts it gives. However, the truth is that RLHF works well to extinguish these lies over time. As more people use the platform and give feedback, the thing gets better. And now, I find it to be pretty darn accurate.
  - macNchz 3y ago
    I see this conversation pretty frequently and I think the root of it lies in the fact that we have mental heuristics for determining whether we need to fact check another human because they are a bullshitter, an idiot, a charlatan etc, but most people haven’t really developed this sense for AIs. I think the current state of AI trustworthiness (“very impressive and often accurate but occasionally extremely wrong”) triggers similar mental pathways to interacting with a true sociopath or pathological liar for the first time in real life, which can be intensely disorienting and cause one to question their trust in everyone else, as they try to comprehend this type of person.
  - TerrifiedMouse 3y ago
    > The smart people I've seen using ChatGPT always double check the facts it gives. I don't like being told lies in the first place and having to unlearn it. It doesn't help that I might as well have just gone straight to the "verification" instead.
  - FooBarWidget 3y ago
    I don't know. The other day I was asking about a biology topic and it straight up gave me a self-contradicting chemical reaction process description. It kept doing that after I pointed out the contradiction. Eventually I got out of this hallucination loop by resetting the conversation and asking again. It's smart but can also be very dumb.
- deleted 3y ago
  [deleted]
uoaei 3y ago
I think this post-factual attitude is stronger and more common in some cultures than others. I'm afraid to say but given my extensive travels it appears American culture (and its derivatives in other countries) seems to be spearheading this shift.
- bamboozled 3y ago
  Warning, my opinion ahead: I think it's because Americans, more than nearly all other cultures, love convenience. It's why the love for driving is so strong in the US. Don't walk or ride, drive. Once I was walking back from the grocer in Florida with 4 shopping bags, and people pulled over and asked if my car had broken down and if I needed a ride, people were stunned...I was walking for exercise and for the environment...and I was stunned. More evidence of this trend can be seen in the products and marketing being produced: Do you need to write a wedding speech? Click here. Do you need to go get something from the store? get your fat ass in the car and drive, better yet, get a car that drives for you? Better than this, we'll deliver it with a drone...don't move a muscle. Don't want to do your homework? Here... Want to produce art? Please enter your prompt... Want to lose weight? We have a drug for that... Want to be the authority on some topic? We'll generate the facts you need.
  - uoaei 3y ago
    I've also identified convenience as a core factor. Another dynamic at play is this: As convenience in a domain becomes ubiquitous or at least expected among consumers, they quickly readjust their evaluation of "having time for X" around the new expectation of the convenient service, treating all alternatives as positive opportunity cost. This would explain a lot of those folks who are upset when it's suggested that they don't need Amazon, Instacart, etc. in their lives if they are to do something about their contributions to mass labor exploitation. Of course these conveniences quickly become ubiquitous in large economies with a glut of disposable income, which encourages VCs to dump money into these enterprises so they're first to market, and also to encourage the public to believe that the future is already here and there's no reason to worry about backsliding or sustainability of the business model. Yet in every single case we see prices eventually rise, laborers squeezed, etc. A critical mass of people haven't yet acknowledged this inevitability, in no small part due to this fixation on convenience at the expense of more objective, reasoned understandings (read: post-truth mindset).
pbhjpbhj 3y ago
ChatGPT 3.5 is terrible on technical subjects IME. Phind is best for me rn. Hugging Chat (Llama) works quite well too. They're only good on universal truths. An amalgam of laws from around the globe doesn't tell me what the law is in my country, for example.
TerrifiedMouse 3y ago
> It's not a tool that I can completely add to my workflow until it is reliable, but I seem to be the odd one out. This. I hate being told the wrong information because I will have to unlearn the wrong information. I would rather have been told nothing.

xnorswap 3y ago

> how you can talk to something that doesn't provide factual information and just take it at face value Like talking to most people you mean?

philipwhiuk 3y ago
When OpenAI buys me a drink at the bar in exchange for the rubbish it produces, I might have a more favourable view.
- ilaksh 3y ago
  As soon as they release the API, we can build an AI "bartender". Combine the voice output and input with NeRF talking heads such as from Diarupt or https://github.com/harlanhong/awesome-talking-head-generation https://github.com/harlanhong/awesome-talking-head-generatio.... You will now be able to feed it images and responses of the customers. Give it a function to call complementaryDrink(customerId) Combine it with a simple vending machine style robot or something more complex that can mix drinks. I'm not actually in a hurry to try to replace bartenders. Just saying these types of things immediately become more feasible. You can also see the possibilities of the speech input and output for "virtual girlfriends". I assume someone at OpenAI must have been tempted to train a model on Scarlett Johansson's voice.
lostcolony 3y ago
Hopefully people know not to ask others for factual information (unless it's an area they're actually well educated/knowledgeable in), but for opinions and subjective viewpoints. "How's your day going", "How are you feeling", "What did you think of X", etc, not "So what was the deal with the Hundred Year's War?" or whatever. If people are treating LLMs like a random stranger and only making small talk, fair enough, but more often they're treating it like an inerrable font of knowledge, and that's concerning.
- TeMPOraL 3y ago
  > If people are treating LLMs like a random stranger and only making small talk, fair enough, but more often they're treating it like an inerrable font of knowledge, and that's concerning. That's on them. I mean, people need to figure out that LLMs aren't random strangers, they're unfiltered inner voices of random strangers, spouting the first reaction they have to what you say to them. Anyway, there is a middle ground. I like to ask GPT-4 questions within my area of expertise, because I'm able to instantly and instinctively - read: effortlessly - judge how much to trust any given reply. It's very useful this way, because rating an answer in your own field takes much less work than coming up with it on your own.

whack 3y ago

Joe Rogan has made tons of money off talking without providing factual information. Hollywood has also made tons of money off movies "inspired by real events" that hallucinate key facts relevant to the movie's plot and characters. There's a huge market for infotainment that is "inspired by facts" but doesn't even try to be accurate.

layer8 3y ago
If that’s your benchmark, I don’t want your AI.
bilsbie 3y ago
Wait until you learn about the mainstream media.
- bamboozled 3y ago
  For a certain demographic and generation, Joe Rogan is the mainstream media.
  - deleted 3y ago
    [deleted]
- flangola7 3y ago
  Rogan is literally the largest podcast on the Spotify. It's the definition of mainstream.
absrec 3y ago
You listen to Joe Rogan with the idea that this is a normal dude talking not an expert beyond martial arts and comedy. A person who uses ChatGPT must have the understanding that it's not like Google search. The layman, however, has no idea that ChatGPT can give coherent incorrect information and treats the information as true. Most people won't use it for infotainment and OpenAI will try its best to downplay the hallucination as fine print if it goes fully mainstream like google search.

nh23423fefe 3y ago

Because it doesn't always make up stuff. Because I'm a human and can ask for more information. I don't want an encyclopedia on a podcast. I want to "talk" to someone about stuff. Not have an enumerated list of truths firehosed at me.

mmahemoff 3y ago

I'm curious if you're using GPT-4 ($)? I find a lot of the criticisms about hallucination come from users who aren't, and my experience with GPT-4 is it's far less likely to make stuff up. Does it know all the answers, certainly not, but it's self-aware enough to say sorry I don't know instead of making a wild guess.

elicksaur 3y ago
Why would anyone pay for something if the free trial doesn’t work? “Hey, you know how we gave you a product that doesn’t quit work as you expect and is super frustrating? Just pay us money, and we’ll give you the same product, but it just works. Just trust us!”
- sacred_numbers 3y ago
  GPT-4 is not the same product. I know it seems like it due to the way they position 3.5 and 4 on the same page, but they are really quite separate things. When I signed up for ChatGPT plus I didn't even bother using 3.5 because I knew it would be inferior. I still have only used it a handful of times. GPT-4 is just so much farther ahead that using 3.5 is just a waste of time.
  - elicksaur 3y ago
    Would you mind sharing some threads where you thought ChatGPT was useful? These discussions always feel like I’m living on a different planet with a different implementation of large language models than others who claim they’re great. The problems I run into seem to stem from the fundamental nature of this class of products.
    - michaelt 3y ago
      The usefulness of ChatGPT is a bit situational, in my experience. But in the right situations it can be pretty powerful. Take a look at https://chat.openai.com/share/41bdb053-facd-448b-b446-1ba1f12b2f92 https://chat.openai.com/share/41bdb053-facd-448b-b446-1ba1f1... for example.

bilsbie 3y ago

> I still don't understand how you can talk to something that doesn't provide factual information and just take it at face value? All human interactions from all of history called and they …

londons_explore 3y ago

Pay for the Plus version. Then it makes stuff up far less frequently. If the next version has the same step up in performance, I will no longer consider inaccuracy an issue - even the best books have mistakes in them, they just need to be infrequent enough.

nerdbert 3y ago
> Pay for the Plus version. > Then it makes stuff up far less frequently. Now there's a business model for a ChatGPT-like service. $1/month: Almost always wrong $10/month: 50/50 chance of being right or wrong $100/month: right 95% of the time
- TeMPOraL 3y ago
  You make it sound like business shenanigans, but the truth is, it's a natural fit for now, as performance of LLMs improves with their size, but costs of training (up-front investment) and inference (marginal, per-query) also go up.
- md5wasp 3y ago
  Pay for the $1/month version and invert the responses; now you have the $100/month one for cheap :D

jocaal 3y ago

> ...talk to something that doesn't provide factual information and... Ah yes, I dont understand how to talk to people either!

bamboozled 3y ago
I always thought a better future would be full of more and more distilled, accurate, useful knowledge and truthful people to promote that. Comments like yours make me think that no one cares about this...and judging by a lot of the other comments, I guess they don't. Probably going to be people, wading through a sea of AI generated shit, and the individual is supposed to just forever "apply critical thinking" to it all. Even a call from ones spouse could be fake, and you'll just have to apply critical thinking or whatever to workout if you were scammed or not.
- astrange 3y ago
  There aren't any real world sources of truth you can avoid applying critical thinking to. Much published research is false, and when it isn't, you need to know when it's expired or what context it's valid in.
  - ChatGTP 3y ago
    But do we need 9999999x the amount of information to critically be thinking about, is this going to be helpful ?

School-Cotton 3y ago

Are you using 3.5 or 4?

rickspencer3 3y ago

In my experience, LLVMs are not about being provided facts. They are about synthesizing new content and insights based on the model and inputs. Rather than asking it about facts, I find it useful to derive new insights. For example: "Tell me 5 topics about databases that might make it to the front page of hacker news." It can generate an interesting list. That is much more like the example they provided in the article, synthesizing a bed time story is not factual. Also, "write me some python code to do x" where x is based on libraries that were well documented before 2022 also has similarly creative results in my experience.

NikolaNovak 3y ago

This is a fairly perpetual discussion, but I'll go for another round: I feel like using LLM today is like using search 15 years ago - you get a feel for getting results you want. I'd never use chatGPT for anything that's even remotely obscure, controversial, or niche. But through all my double-checking, I've had phenomenal success rate in getting useful, readable, valid responses to well-covered / documented topics such as introductory french, introductory music theory, well-covered & non-controversial history and science. I'd love to see the example you experienced; if I ask chatGPT "tell me about Toronto, Canada", my expectation would be to get high accuracy. If I asked it "Was Hum, Croatia, part of the Istrian liberation movement in the seventies", I'd have far less confidence - it's a leading question, on a less covered topic, introducing inaccuracies in the prompt. My point is - for a 3 hour drive to cottage, I'm OK with something that's only 95% accurate on easy topics! I'd get no better from my spouse or best friend if they made it on the same drive :). My life will not depend on it, I'll have an educationally good time and miles will pass faster :). (also, these conversations always seem to end in suffocatingly self-righteous "I don't know how others can live in this post-fact free world of ignorance", but that has a LOT of assumptions and, ironically, non-factual bias in it as well)

doug_durham 3y ago
Exactly this! This is my experience also. Your point about "well covered & non-controversial" is spot on. I know not to expect great results when asking about topics that have very little coverage. To be honest I wouldn't expect to go to an arbitrary human and get solid answers on a little covered topic, unless that person just happened to be topic expert. There is so much value in having the basics to intermediate levels of topics covered in a reliable way. That's where most of commercial activity occurs.
- jabradoodle 3y ago
  I think a key difference is that humans very rarely sound convincing talking about subjects they have no clue about. I've seen the hallucination rate of LLMs improve significantly, if you stick to well covered topics they probably do quite well. The issue is they often have no tells when making things up.

deleted 3y ago

[deleted]

FooBarWidget 3y ago

A human driving buddy can make up a lot of stuff too. Have an interesting conversation but don't take it too seriously. If you're really researching something serious then take a mental note to double check things later, pretend as if you're talking to a semi-reliable human who knows a lot but occasionally makes mistakes.