9 ms·

That's not the only thing wrong. Gemini makes a false statement in the video, serving as a great demonstration of how these models still outright lie so frequen

That's not the only thing wrong. Gemini makes a false statement in the video, serving as a great demonstration of how these models still outright lie so frequently, so casually, and so convincingly that you won't notice, even if you have a whole team of researchers and video editors reviewing the output. It's the single biggest problem with LLMs and Gemini isn't solving it. You simply can't rely on them when correctness is important. Even when the model has the knowledge it would need to answer correctly, as in this case, it will still lie. The false statement is after it says the duck floats, it continues "It is made of a material that is less dense than water." This is false; "rubber" ducks are made of vinyl polymers which are more dense than water. It floats because the hollow shape contains air, of course.

ace2358 3y ago
I totally agree with you on the confident lies. And it’s really tough. Technically the duck is made out of air and plastic right? If I pushed the model further on the composition of a rubber duck, and it failed to mention its construction, then it’d be lying. However there is this disgusting part of language where a statement can be misleading, technically true, not the whole truth, missing caveats etc. Very challenging problem. Obviously Google decided to mislead the audience and basically cover up the shortcomings. Terrible behaviour.
- mechagodzilla 3y ago
  No, the density of the object is less than water, not the density of the material. The Duck is made of plastic, and it traps air. Similarly, you can make a boat that floats in water out of concrete or metal. It is an important distinction when trying to understand buoyancy.
  - deleted 3y ago
    [deleted]
- modeless 3y ago
  Calling the air inside the duck (which is not sealed inside) part of its "material" would be misleading. That's not how most people would interpret the statement and I'm confident that's not the explanation for why the statement was made.
  - onedognight 3y ago
    The air doesn’t matter. Even with a vacuum inside it would float. It’s the overall density of “the duck” that matters, not the density of the plastic.
    - hunter2_ 3y ago
      A canoe floats, and that doesn't even command any thought regarding whether you can replace trapped air with a vacuum. If you had a giant cube half full of water, with a boat on the water, the boat would float regardless of whether the rest of the cube contained air or vacuum, and regardless of whether the boat traps said air (like a pontoon) or is totally vented (like a canoe). The overall density of the canoe is NOT influenced by its shape or any air, though. The canoe is strictly more dense than water (it will sink if it capsizes) yet in the correct orientation it floats. What does matter, however, is the overall density of the space that was water and became displaced by the canoe. That space can be populated with dense water, or with a less dense canoe+air (or canoe+vacuum) combination. That's what a rubber duck also does: the duck+air (or duck+vacuum) combination is less dense than the displaced water.
jmathai 3y ago
This seems to be a common view among some folks. Personally, I'm impartial. Search or even asking other expert human beings are prone to provide incorrect results. I'm unsure where this expectation of 100% absolute correctness comes from. I'm sure there are use cases, but I assume it's the vast minority and most can tolerate larger than expected inaccuracies.
- stefan_ 3y ago
  Let's see, so we exclude law, we exclude medical.. it's certainly not a "vast minority" and the failure cases are nothing at all like search or human experts.
  - jmathai 3y ago
    Are you suggesting that failure cases are lower when interacting with humans? I don't think that's my experience at all. Maybe I've only ever seen terrible doctors but I always cross reference what doctors say with reputable sources like WebMD (which I understand likely contain errors). Sometimes I'll go straight to WebMD. This isn't a knock on doctors - they're humans and prone to errors. Lawyers, engineers, product managers, teachers too.
    - stefan_ 3y ago
      You think you ask your legal assistant to find some precedents related to your current case and they will come back with an A4 page full of made up cases that sound vaguely related and convincing but are not real? I don't think you understand the failure case at all.
      - jmathai 3y ago
        That example seems a bit hyperbolic. Do you think lawyers who leverage ChatGPT will take the made up cases and present them to a judge without doing some additional research? What I'm saying is that the tolerance for mistakes is strongly correlated to the value ChatGPT creates. I think both will need to be improved but there's probably more opportunity in creating higher value. I don't have a horse in the race.
b59831 3y ago
[dead]
brookst 3y ago
Is it possible for humans to be wrong about something, without lying?
- windowshopping 3y ago
  I don't agree with the argument that "if a human can fail in this way, we should overlook this failing in our tooling as well." Because of course that's what LLMs are, tools, like any other piece of software. If a tool is broken, you seek to fix it. You don't just say "ah yeah it's a broken tool, but it's better than nothing!" All these LLM releases are amazing pieces of technology and the progress lately is incredible. But don't rag on people critiquing it, how else will it get better? Certainly not by accepting its failings and overlooking them.
  - rkeene2 3y ago
    If a broken tool is useful, do you not use it because it is broken ? Overpowered LLMs like GPT-4 are both broken (according to how you are defining it) and useful -- they're just not the idealized version of the tool.
    - freejazz 3y ago
      Maybe not if its the case that your use of the broken tool would result in the eventual undoing of your work. Like, lets say your staple gun is defective and doesn't shoot the staples deep enough, but it still shoots. You can keep using the gun, but it's not going to actually do its job. It seems useful and functional, but it isn't and its liable to create a much bigger mess.
      - rkeene2 3y ago
        So to continue the analogy, if the staple gun is broken and it requires you to do more than a working (but non-existent) staple gun BUT less work than doing the affixment without the broken staple gun, you would or would not use it ?
ugh123 3y ago
I don't see it as a problem with most non-critical uses cases (critical being things like medical diagnoses, controlling heavy machinery or robotics, etc). LLMs right now are most practical for generating templated text and images, which when paired with an experienced worker, can make them orders of magnitude more productive. Oh, DALL-E created graphic images with a person with 6 fingers? How long would it have taken a pro graphic artist to come up with all the same detail but with perfect fingers? Nothing there they couldn't fix in a few minutes and then SHIP.
- zer00eyz 3y ago
  >> Nothing there they couldn't fix in a few minutes and then SHIP. If by ship, you mean put directly into the public domain then yes. https://www.goodwinlaw.com/en/insights/publications/2023/08/insights-technology-aiml-thaler-v-copyright-register-gen-ai https://www.goodwinlaw.com/en/insights/publications/2023/08/... and for more interesting takes: https://www.youtube.com/watch?v=5WXvfeTPujU& https://www.youtube.com/watch?v=5WXvfeTPujU&
awongh 3y ago
I’m not an expert but I suspect that this aspect of lack of correctness in these models might be fundamental to how they work. I suppose there’s two possible solutions: one is a new training or inference architecture that somehow understand “facts”. I’m not an expert so I’m not sure how that would work, but from what I understand about how a model generates text, “truth” can’t really be a element in the training or inference that affects the output. the second would be a technology built on top of the inference to check correctness, some sort of complex RAG. Again not sure how that would work in a real world way. I say it might be fundamental to how the model works because as someone pointed out below, the meaning of the word “material” could be interpreted as the air inside the duck. The model’s answer was correct in a human sort of way, or to be more specific in a way that is consistent with how a model actually produces an answer- it outputs in the context of the input. If you asked it if PVC is heavier than water it would answer correctly. Because language itself is inherently ambiguous and the model doesn’t actually understand anything about the world, it might turn out that there’s no universal way for a model to know what’s true or not. I could also see a version of a model that is “locked down” but can verify the correctness of its statements, but in a way that limits its capabilities.
- ajkjk 3y ago
  > this aspect of lack of correctness in these models might be fundamental to how they work. Is there some sense in which this isn't obvious to the point of triviality? I keep getting confused because other people seem to keep being surprised that LLMs don't have correctness as a property. Even the most cursory understanding of what they're doing understands that it is, fundamentally, predicting words from other words. I am also capable of predicting words from other words, so I can guess how well that works. It doesn't seem to include correctness even as a concept. Right? I am actually genuinely confused by this. How is that people think it could be correct in a systematic way?
  - carstenhag 3y ago
    Because it is assumed that it can think or/and reason. In this case, knowing the concepts of density, the density of a material, detecting the material from an image, detecting what object this image is. And, most importantly, knowing that this object is not solid. Because then it could not float.
freedomben 3y ago
I think this problem needs to be solved at a higher level, and in fact Bard is doing exactly that. The model itself generates its output, and then higher-level systems can fact check it. I've heard promising things about feeding back answers to the model itself to check for consistency and stuff, but that should be a higher level function (and seems important to avoid infinite recursion or massive complexity stemming from the self-check functionality).
- modeless 3y ago
  I'm not a fan of current approaches here. "Chain of thought" or other approaches where the model does all its thinking using a literal internal monologue in text seem like a dead end. Humans do most of their thinking non-verbally and we need to figure out how to get these models to think non-verbally too. Unfortunately it seems that Gemini represents no progress in this direction.
  - freedomben 3y ago
    > Humans do most of their thinking non-verbally and we need to figure out how to get these models to think non-verbally too. That's a very interesting point, both technically and philosophically. Where Gemini is "multi-modal" from training, how close do you think that gets? Do we know enough about neurology to identical a native language in which we think? (not rhetorical questions, I'm really wondering)
    - janalsncm 3y ago
      Neural networks are only similar to brains on the surface. Their learning process is entirely different and their internal architecture is different as well. We don’t use neural networks because they’re similar to brains. We use them because they are arbitrary function approximators and we have an efficient algorithm (backprop) coupled with hardware (GPUs) to optimize them quickly.
  - janalsncm 3y ago
    The point of “verbalizing” the chain of thought isn’t that it’s the most effective method. And frankly I don’t think it matters that humans think non verbally. The goal isn’t to create a human in a box. Verbalizing the chain of thought allows us to audit the thought process, and also create further labels for training.
__s 3y ago
It also says the attribute of squeaking means it'll definitely float
- bongodongobob 3y ago
  That's actually pretty clever because if it squeaks, there is air inside. How many squeaking ducks have you come across that don't float?
  - davesque 3y ago
    You could call it clever or you could call it a spurious correlation.
bitshiftfaced 3y ago
There's nothing wrong with what you're saying, but what do you suggest? Factuality is an area of active research, and Deepmind goes into some detail in their technical paper. The models are too useful to say, "don't use them at all." Hopefully people will heed the warnings of how they can hallucinate, but further than that I'm not sure what more you can expect.
- modeless 3y ago
  The problem is not with the model, but with its portrayal in the marketing materials. It's not even the fact that it lied, which is actually realistic. The problem is the lie was not called out as such. A better demo would have had the user note the issue and give the model the opportunity to correct itself.
  - bitshiftfaced 3y ago
    But you yourself said that it was so convincing that the people doing the demo didn't recognize it as false, so how would they know to call it out as such? I suppose they could've deliberately found a hallucination and showcased it in the demo. In which case, pretty much every company's promo material is guilty of not showcasing negative aspects of their product. It's nothing new or unique to this case.
    - modeless 3y ago
      They should have looked more carefully, clearly. Especially since they were criticized for the exact same thing in their last launch.
twobitshifter 3y ago
I, a non-AGI, just ‘hallucinated’ yesterday. I hallucinated that my plan was to take all of Friday off and started wondering why I had scheduled morning meetings. I started canceling them in a rush. In fact, all week I had been planning to take a half day, but somehow my brain replaced the idea of a half day off with a full day off. You could have asked me and I would have been completely sure that I was taking all of friday off.
margorczynski 3y ago
LLMs do not lie, nor do they tell the truth. They have no goal as they are not agents.
- modeless 3y ago
  With apologies to Dijkstra, the question of whether LLMs can lie is about as relevant as the question of whether submarines can swim.
rowanG077 3y ago
The duck is indeed made of a material that is less dense. Namely water and air. If you go to such technical routes your definition is wrong too. It doesn't float because it contains air. If you poke in the head of the duck it will sink. Even though at all times it contains air.
- recursive 3y ago
  The duck is made of water and air? Which duck are we talking about here.
dogprez 3y ago
That's a tricky one though since the question is, is the air inside of the rubber duck part of the material that makes it? If you removed the air it definitely wouldn't look the same or be considered a rubber duck. I gave it to the bot since when taking ALL the material that makes it a rubber duck, it is less dense than water.
- bee_rider 3y ago
  If you hold a rubber duck under water and squeeze out the air, it will fill with water and still be a rubber duck. If you send a rubber duck into space, it will become almost completely empty but still be a rubber duck. Therefore, the liquid used to fill the empty space inside it is not part of the duck. I mean apply this logic to a boat, right? Is the entire atmosphere part of the boat? Are we all on this boat as well? Is it a cruise boat? If so, where is my drink?
- modeless 3y ago
  A rubber duck in a vacuum is still a rubber duck and it still floats (though water would evaporate too quickly in a vacuum, it could float on something else of the same density).
  - dogprez 3y ago
    A rubber duck with a vacuum inside (removing the air material) of it is just a piece of rubber with eyes. Assuming OP's point about the rubber not being less dense than water, it would sink, no?
    - modeless 3y ago
      No. Air is less dense than water; vacuum is even less dense than air. A rubber duck will collapse if you seal it and try to pull a vacuum inside with air outside, but if the rubber duck is in a vacuum then it will have only vacuum inside and it will still float on a liquid the density of water. If you made a duck out of a metal shell you could pull a vacuum inside, like a thermos bottle, and it would float too.
WhitneyLand 3y ago
Agree, then the question becomes how will this issue play out? Maybe AI correctness will be similar to automobile safety. It didn’t take long for both to be recognized as fundamental issues with new transformative technologies. In both cases there seems to be no silver bullet. Mitigations and precautions will continue to evolve, with varying degrees of effectiveness. Public opinion and legislation will play some role. Tragically accidents will happen and there will be a cost to pay, which so far has been much higher and more grave for transportation.
crazygringo 3y ago
EDIT: never mind, I missed the exact wording about being "made of a material..." which is definitely false then. Thanks for the correction below. Preserving the original comment so the replies make sense: --- I think it's a stretch to say that's false. In a conversational human context, saying it's made of rubber implies it's a rubber shell with air inside. It floats because it's rubber [with air] as opposed to being a ceramic figurine or painted metal. I can imagine most non-physicist humans saying it floats because it's rubber. By analogy, we talk about houses being "made of wood" when everybody knows they're made of plenty of other materials too. But the context is instead of brick or stone or concrete. It's not false to say a house is made of wood.
- furyofantares 3y ago
  This is what the reply was: > Oh, it it's squeaking then it's definitely going to float. > It is a rubber duck. > It is made of a material that is less dense than water. Full points for saying if it's squeaking then it's going to float. Full points for saying it's a rubber duck, with the implication that rubber ducks float. Even with all that context though, I don't see how "it is made of a material that is less dense than water" scores any points at all.
  - yowzadave 3y ago
    Yeah, I think arguing the logic behind these responses misses the point, since an LLM doesn't use any kind of logic--it just responds in a pattern that mimics the way people respond. It says "it is made of a material that is less dense than water" because that is a thing that is similar to what the samples in its training corpus have said. It has no way to judge whether it is correct, or even what the concept of "correct" is. When we're grading the "correctness" of these answers, we're really just judging the average correctness of Google's training data. Maybe the next step in making LLM's more "correct" is not to give them more training data, but to find a way to remove the bad training data from the set?
- modeless 3y ago
  > In a conversational human context, saying it's made of rubber implies it's a rubber shell with air inside. Disagree. It could easily be solid rubber. Also, it's not made of rubber, and the model didn't claim it was made of rubber either, so it's irrelevant. > It floats because it's rubber [with air] as opposed to being a ceramic figurine or painted metal. A ceramic figurine or painted metal in the same shape would float too. The claim that it floats because of the density of the material is false. It floats because the shape is hollow. > It's not false to say a house is made of wood. It's false to say a house is made of air simply because its shape contains air.
omginternets 3y ago
People seem to want to use LLMs to mine knowledge, when really it appears to be a next-gen word-processor.
deleted 3y ago
[deleted]
eurleif 3y ago
To be fair, one could describe the duck as being made of air and vinyl polymer, which in combination are less dense than water. That's not how humans would normally describe it, but that's kind of arbitrary; consider how aerogel is often described as being mostly made of air.
- colonwqbang 3y ago
  Is an aircraft carrier made of a material that is less dense than water?
  - leeoniya 3y ago
    only if you average it out over volume :P
  - andrewmutz 3y ago
    Is an aircraft carrier made of metal and air? Or just metal?
    - bee_rider 3y ago
      Where’s the distinction between the air that is part of the boat, and the air that is not? If the air is included in the boat, should we all be wearing life vests?
  - skeaker 3y ago
    I think you can safely say that air is a critical component of an aircraft carrier. I suppose the frame of it is not made of air, but the ballasts are designed with air in mind and are certainly made to utilize air. The whole system fails without air, meaning that it requires air to function. It comes down to a definitional argument of the word "made" which is pointless.
    - colonwqbang 3y ago
      I guess it's a purely philosophical question. But no normal person would say "my house is made of air" or "atoms are made of vacuum".
PepperdineG 3y ago
>It's the single biggest problem with LLMs and Gemini isn't solving it. I loved it when the lawyers got busted for using a hallucinating LLM to write their briefs.
glitchc 3y ago
Well this seems like a huge nitpick. If a person said that, you would afford them some leeway, maybe they meant the whole duck, which includes the hollow part in the middle. As an example, when most people say a balloon's lighter than air, they mean an inflated balloon with hot air or helium, but you catch their meaning and don't rush to correct them.
- modeless 3y ago
  The model specifically said that the material is less dense than water. If you said that the material of a balloon is less dense than air, very few people would interpret that as a correct statement, and it could be misleading to people who don't know better. Also, lighter-than-air balloons are intentionally filled with helium and sealed; rubber ducks are not sealed and contain air only incidentally. A balloon in a vacuum would still contain helium (if strong enough) but would not rise, while a rubber duck in a vacuum would not contain air but would still easily float on a liquid of similar density to water. The reason why it seems like a nitpick is that this is such an inconsequential thing. Yeah, it's a false statement but it doesn't really matter in this case, nobody is relying on this answer for anything important. But the point is, in cases where it does matter these models cannot be trusted. A human would realize when the context is serious and requires accuracy; these models don't.
deleted 3y ago
[deleted]
eviks 3y ago
Given the misleading presentation by real humans in these "whole teams" that this tweet corrects, this doesn't illustrate any underlying powers by the model
catchnear4321 3y ago
language models do not lie. (this pedantic distinction being important, because language models.)
lemmsjid 3y ago
I did some reading and it seems that rubber's relative density to water has to do with its manufacturing process. I see a couple of different quotes on the specific gravity of so-called 'natural rubber', and most claim it's lower than water. Am I missing something? I asked both Bard (Gemini at this point I think?) and GPT-4 why ducks float, and they both seemed accurate: they talked about the density of the material plus the increased buoyancy from air pockets and went into depth on the principles behind buoyancy. When pressed they went into the fact that "rubber"'s density varies by the process and what it was adulterated with, and if it was foamed. I think this was a matter of the video being a brief summary rather than a falsehood. But please do point out if I'm wrong on the rubber bit, I'm genuinely interested. I agree that hallucinations are the biggest problems with LLMs, I'm just seeing them get less commonplace and clumsy. Though, to your point, that can make them harder to detect!
- modeless 3y ago
  Someone on Twitter was also skeptical that the material is more dense than water. I happened to have a rubber duck handy so I cut a sample of material and put it in water. It sinks to the bottom. Of course the ultimate skeptic would say one test doesn't prove that all rubber ducks are the same. I'm sure someone at some point in history has made a rubber duck out of material that is less dense than water. But I invite you to try it yourself and I expect you will see the same result unless your rubber duck is quite atypical. Yes, the models will frequently give accurate answers if you ask them this question. That's kind of the point. Despite knowing that they know the answer, you still can't trust them to be correct.
  - lemmsjid 3y ago
    Ah good show :). I was rather preoccupied with the question but didn't have one handy. Well, I do, but my kid would roast me slowly over coals if I so much as smudged it. Ah the joy of the Internet, I did not predict this morning that I would end the day preoccupied with the question of rubber duck density! I guess for me the question of whether or not the model is lying or hallucinating is if it's correctly summarizing its source material. I find very conflicting materials on the density of rubber, and most of the sources that Google surfaces claim a lower density than water. So it makes sense to me that the model would make the inference. I'm splitting hairs though, I largely agree with your comment above and above that. To illustrate my agreement: I like testing AIs with this kind of thing... a few months ago I asked GPT for advice as to how to restart my gas powered water heater. It told me the first step was to make sure the gas was off, then to light the pilot light. I then asked it how the pilot light was supposed to stay lit with the gas off and it backpedaled. My imagining here is that because so many instructional materials about gas powered devices emphasize to start by turning off the gas, that weighted it as the first instruction. Interesting, the above shows progress though. I realized I asked GPT 3.5 back then, I just re-asked 3.5 and then asked 4 for the first time. 3.5 was still wrong. 4 told me to initially turn off the gas to disappate it, then to ensure gas was flowing to the pilot before sparking it. But that said I am quite familiar with the AI being confidently wrong, so your point is taken, I only really responded because I was wondering if I was misunderstanding something quite fundamental about the question of density.
b112 3y ago
Devil's advocate. It is made of a material less dense than water. Air. It certainly isn't how I would phrase it, and I wouldn't count air as what something is made of, but... Soda pop is chocked full of air, it's part of it! And I'd say carbon dioxide is a part of the recipe, of pop. So it's a confusing world for a young LLM. (I realise it may have referenced rubber prior, but it may have meant air... again, Devil's advocate)
- modeless 3y ago
  When you make carbonated soda you put carbon dioxide in deliberately and use a sealed container to hold it in. When you make a rubber duck you don't put air in it deliberately and it is not sealed. Carbonated soda ceases to be carbonated when you remove the air. A rubber duck in a vacuum is still a rubber duck and it even still floats.
  - b112 3y ago
    If the rubber duck has air inside, it is known, and intentional, for it is part of that design. If you remove the air from the duck, and stop it so it won't refill, you have a flat rubber duck, which is useless for its design. Much as flat pop is useless for its design. And this nuance is even more nuance-ish than this devil's advocate post.
    - modeless 3y ago
      A rubber duck in a vacuum (not a duck in atmosphere with a vacuum only inside) would not go flat or pop. It would remain entirely normal, as useful as it ever was, and it would still float on a liquid the density of water. Removing the air would have no effect on the duck whatsoever. It's not part of the material of the duck in any reasonable interpretation. But pedantic correctness isn't even what matters here. The model made a statement where the straightforward interpretation is false and misleading. A person who didn't know better would be misled. Whether you can possibly come up with a tortured alternative interpretation that is technically not incorrect is irrelevant.
FoeNyx 3y ago
After asserting it's a rubber duck, there are some claims without follow-up: - Just after that it doesn't translate the "rubber" part - It states there's no land nearby for it to rest or find food in the middle of the ocean: if it's a rubber duck it doesn't need to rest nor feed. (That's a missed opportunity to mention the infamous "Friendly Floatees spill"[1] in 1992 as some rubber ducks floated to that map position). Although it seems to recognize geographical features of the map, it fails to mention Easter Island is relatively nearby. And if it were recognized as a simple duck — which it described as a bird swimming in the water — it seems oblivious to the fact that the duck might feed itself in the water. It doesn't mention either that the size of the duck seems abnormally big in that map context. - The concept of friends and foes doesn't apply to a rubber duck either. Btw labeling the duck picture as a friend and the bear picture as a foe seems arbitrary (e.g. a real duck can be very aggressive even with other ducks.) Among other things, the astronomical riddle seems also flawed to me: it answered "The correct order is Sun, Earth, Saturn". I'd like for it to state : - the premises it used, like "Assuming it depicts the Sun, Saturn and the Earth" (there are other stars, other ringed-planets, and the Earth similarity seems debatable) - the sorting criteria it used (e.g. using another sorting key like the average distance from us "Earth, Sun, Saturn" can be a correct order) [1] https://en.wikipedia.org/wiki/Friendly_Floatees_spill https://en.wikipedia.org/wiki/Friendly_Floatees_spill

ace2358 3y ago

I totally agree with you on the confident lies. And it’s really tough. Technically the duck is made out of air and plastic right? If I pushed the model further on the composition of a rubber duck, and it failed to mention its construction, then it’d be lying. However there is this disgusting part of language where a statement can be misleading, technically true, not the whole truth, missing caveats etc. Very challenging problem. Obviously Google decided to mislead the audience and basically cover up the shortcomings. Terrible behaviour.

mechagodzilla 3y ago
No, the density of the object is less than water, not the density of the material. The Duck is made of plastic, and it traps air. Similarly, you can make a boat that floats in water out of concrete or metal. It is an important distinction when trying to understand buoyancy.
- deleted 3y ago
  [deleted]
modeless 3y ago
Calling the air inside the duck (which is not sealed inside) part of its "material" would be misleading. That's not how most people would interpret the statement and I'm confident that's not the explanation for why the statement was made.
- onedognight 3y ago
  The air doesn’t matter. Even with a vacuum inside it would float. It’s the overall density of “the duck” that matters, not the density of the plastic.
  - hunter2_ 3y ago
    A canoe floats, and that doesn't even command any thought regarding whether you can replace trapped air with a vacuum. If you had a giant cube half full of water, with a boat on the water, the boat would float regardless of whether the rest of the cube contained air or vacuum, and regardless of whether the boat traps said air (like a pontoon) or is totally vented (like a canoe). The overall density of the canoe is NOT influenced by its shape or any air, though. The canoe is strictly more dense than water (it will sink if it capsizes) yet in the correct orientation it floats. What does matter, however, is the overall density of the space that was water and became displaced by the canoe. That space can be populated with dense water, or with a less dense canoe+air (or canoe+vacuum) combination. That's what a rubber duck also does: the duck+air (or duck+vacuum) combination is less dense than the displaced water.

jmathai 3y ago

This seems to be a common view among some folks. Personally, I'm impartial. Search or even asking other expert human beings are prone to provide incorrect results. I'm unsure where this expectation of 100% absolute correctness comes from. I'm sure there are use cases, but I assume it's the vast minority and most can tolerate larger than expected inaccuracies.

stefan_ 3y ago
Let's see, so we exclude law, we exclude medical.. it's certainly not a "vast minority" and the failure cases are nothing at all like search or human experts.
- jmathai 3y ago
  Are you suggesting that failure cases are lower when interacting with humans? I don't think that's my experience at all. Maybe I've only ever seen terrible doctors but I always cross reference what doctors say with reputable sources like WebMD (which I understand likely contain errors). Sometimes I'll go straight to WebMD. This isn't a knock on doctors - they're humans and prone to errors. Lawyers, engineers, product managers, teachers too.
  - stefan_ 3y ago
    You think you ask your legal assistant to find some precedents related to your current case and they will come back with an A4 page full of made up cases that sound vaguely related and convincing but are not real? I don't think you understand the failure case at all.
    - jmathai 3y ago
      That example seems a bit hyperbolic. Do you think lawyers who leverage ChatGPT will take the made up cases and present them to a judge without doing some additional research? What I'm saying is that the tolerance for mistakes is strongly correlated to the value ChatGPT creates. I think both will need to be improved but there's probably more opportunity in creating higher value. I don't have a horse in the race.

b59831 3y ago

[dead]

brookst 3y ago

Is it possible for humans to be wrong about something, without lying?

windowshopping 3y ago
I don't agree with the argument that "if a human can fail in this way, we should overlook this failing in our tooling as well." Because of course that's what LLMs are, tools, like any other piece of software. If a tool is broken, you seek to fix it. You don't just say "ah yeah it's a broken tool, but it's better than nothing!" All these LLM releases are amazing pieces of technology and the progress lately is incredible. But don't rag on people critiquing it, how else will it get better? Certainly not by accepting its failings and overlooking them.
- rkeene2 3y ago
  If a broken tool is useful, do you not use it because it is broken ? Overpowered LLMs like GPT-4 are both broken (according to how you are defining it) and useful -- they're just not the idealized version of the tool.
  - freejazz 3y ago
    Maybe not if its the case that your use of the broken tool would result in the eventual undoing of your work. Like, lets say your staple gun is defective and doesn't shoot the staples deep enough, but it still shoots. You can keep using the gun, but it's not going to actually do its job. It seems useful and functional, but it isn't and its liable to create a much bigger mess.
    - rkeene2 3y ago
      So to continue the analogy, if the staple gun is broken and it requires you to do more than a working (but non-existent) staple gun BUT less work than doing the affixment without the broken staple gun, you would or would not use it ?

ugh123 3y ago

I don't see it as a problem with most non-critical uses cases (critical being things like medical diagnoses, controlling heavy machinery or robotics, etc). LLMs right now are most practical for generating templated text and images, which when paired with an experienced worker, can make them orders of magnitude more productive. Oh, DALL-E created graphic images with a person with 6 fingers? How long would it have taken a pro graphic artist to come up with all the same detail but with perfect fingers? Nothing there they couldn't fix in a few minutes and then SHIP.

zer00eyz 3y ago
>> Nothing there they couldn't fix in a few minutes and then SHIP. If by ship, you mean put directly into the public domain then yes. https://www.goodwinlaw.com/en/insights/publications/2023/08/insights-technology-aiml-thaler-v-copyright-register-gen-ai https://www.goodwinlaw.com/en/insights/publications/2023/08/... and for more interesting takes: https://www.youtube.com/watch?v=5WXvfeTPujU& https://www.youtube.com/watch?v=5WXvfeTPujU&

awongh 3y ago

I’m not an expert but I suspect that this aspect of lack of correctness in these models might be fundamental to how they work. I suppose there’s two possible solutions: one is a new training or inference architecture that somehow understand “facts”. I’m not an expert so I’m not sure how that would work, but from what I understand about how a model generates text, “truth” can’t really be a element in the training or inference that affects the output. the second would be a technology built on top of the inference to check correctness, some sort of complex RAG. Again not sure how that would work in a real world way. I say it might be fundamental to how the model works because as someone pointed out below, the meaning of the word “material” could be interpreted as the air inside the duck. The model’s answer was correct in a human sort of way, or to be more specific in a way that is consistent with how a model actually produces an answer- it outputs in the context of the input. If you asked it if PVC is heavier than water it would answer correctly. Because language itself is inherently ambiguous and the model doesn’t actually understand anything about the world, it might turn out that there’s no universal way for a model to know what’s true or not. I could also see a version of a model that is “locked down” but can verify the correctness of its statements, but in a way that limits its capabilities.

ajkjk 3y ago
> this aspect of lack of correctness in these models might be fundamental to how they work. Is there some sense in which this isn't obvious to the point of triviality? I keep getting confused because other people seem to keep being surprised that LLMs don't have correctness as a property. Even the most cursory understanding of what they're doing understands that it is, fundamentally, predicting words from other words. I am also capable of predicting words from other words, so I can guess how well that works. It doesn't seem to include correctness even as a concept. Right? I am actually genuinely confused by this. How is that people think it could be correct in a systematic way?
- carstenhag 3y ago
  Because it is assumed that it can think or/and reason. In this case, knowing the concepts of density, the density of a material, detecting the material from an image, detecting what object this image is. And, most importantly, knowing that this object is not solid. Because then it could not float.

freedomben 3y ago

I think this problem needs to be solved at a higher level, and in fact Bard is doing exactly that. The model itself generates its output, and then higher-level systems can fact check it. I've heard promising things about feeding back answers to the model itself to check for consistency and stuff, but that should be a higher level function (and seems important to avoid infinite recursion or massive complexity stemming from the self-check functionality).

modeless 3y ago
I'm not a fan of current approaches here. "Chain of thought" or other approaches where the model does all its thinking using a literal internal monologue in text seem like a dead end. Humans do most of their thinking non-verbally and we need to figure out how to get these models to think non-verbally too. Unfortunately it seems that Gemini represents no progress in this direction.
- freedomben 3y ago
  > Humans do most of their thinking non-verbally and we need to figure out how to get these models to think non-verbally too. That's a very interesting point, both technically and philosophically. Where Gemini is "multi-modal" from training, how close do you think that gets? Do we know enough about neurology to identical a native language in which we think? (not rhetorical questions, I'm really wondering)
  - janalsncm 3y ago
    Neural networks are only similar to brains on the surface. Their learning process is entirely different and their internal architecture is different as well. We don’t use neural networks because they’re similar to brains. We use them because they are arbitrary function approximators and we have an efficient algorithm (backprop) coupled with hardware (GPUs) to optimize them quickly.
- janalsncm 3y ago
  The point of “verbalizing” the chain of thought isn’t that it’s the most effective method. And frankly I don’t think it matters that humans think non verbally. The goal isn’t to create a human in a box. Verbalizing the chain of thought allows us to audit the thought process, and also create further labels for training.

__s 3y ago

It also says the attribute of squeaking means it'll definitely float

bongodongobob 3y ago
That's actually pretty clever because if it squeaks, there is air inside. How many squeaking ducks have you come across that don't float?
- davesque 3y ago
  You could call it clever or you could call it a spurious correlation.

bitshiftfaced 3y ago

There's nothing wrong with what you're saying, but what do you suggest? Factuality is an area of active research, and Deepmind goes into some detail in their technical paper. The models are too useful to say, "don't use them at all." Hopefully people will heed the warnings of how they can hallucinate, but further than that I'm not sure what more you can expect.

modeless 3y ago
The problem is not with the model, but with its portrayal in the marketing materials. It's not even the fact that it lied, which is actually realistic. The problem is the lie was not called out as such. A better demo would have had the user note the issue and give the model the opportunity to correct itself.
- bitshiftfaced 3y ago
  But you yourself said that it was so convincing that the people doing the demo didn't recognize it as false, so how would they know to call it out as such? I suppose they could've deliberately found a hallucination and showcased it in the demo. In which case, pretty much every company's promo material is guilty of not showcasing negative aspects of their product. It's nothing new or unique to this case.
  - modeless 3y ago
    They should have looked more carefully, clearly. Especially since they were criticized for the exact same thing in their last launch.

twobitshifter 3y ago

I, a non-AGI, just ‘hallucinated’ yesterday. I hallucinated that my plan was to take all of Friday off and started wondering why I had scheduled morning meetings. I started canceling them in a rush. In fact, all week I had been planning to take a half day, but somehow my brain replaced the idea of a half day off with a full day off. You could have asked me and I would have been completely sure that I was taking all of friday off.

margorczynski 3y ago

LLMs do not lie, nor do they tell the truth. They have no goal as they are not agents.

modeless 3y ago
With apologies to Dijkstra, the question of whether LLMs can lie is about as relevant as the question of whether submarines can swim.

rowanG077 3y ago

The duck is indeed made of a material that is less dense. Namely water and air. If you go to such technical routes your definition is wrong too. It doesn't float because it contains air. If you poke in the head of the duck it will sink. Even though at all times it contains air.

recursive 3y ago
The duck is made of water and air? Which duck are we talking about here.

dogprez 3y ago

That's a tricky one though since the question is, is the air inside of the rubber duck part of the material that makes it? If you removed the air it definitely wouldn't look the same or be considered a rubber duck. I gave it to the bot since when taking ALL the material that makes it a rubber duck, it is less dense than water.

bee_rider 3y ago
If you hold a rubber duck under water and squeeze out the air, it will fill with water and still be a rubber duck. If you send a rubber duck into space, it will become almost completely empty but still be a rubber duck. Therefore, the liquid used to fill the empty space inside it is not part of the duck. I mean apply this logic to a boat, right? Is the entire atmosphere part of the boat? Are we all on this boat as well? Is it a cruise boat? If so, where is my drink?
modeless 3y ago
A rubber duck in a vacuum is still a rubber duck and it still floats (though water would evaporate too quickly in a vacuum, it could float on something else of the same density).
- dogprez 3y ago
  A rubber duck with a vacuum inside (removing the air material) of it is just a piece of rubber with eyes. Assuming OP's point about the rubber not being less dense than water, it would sink, no?
  - modeless 3y ago
    No. Air is less dense than water; vacuum is even less dense than air. A rubber duck will collapse if you seal it and try to pull a vacuum inside with air outside, but if the rubber duck is in a vacuum then it will have only vacuum inside and it will still float on a liquid the density of water. If you made a duck out of a metal shell you could pull a vacuum inside, like a thermos bottle, and it would float too.

WhitneyLand 3y ago

Agree, then the question becomes how will this issue play out? Maybe AI correctness will be similar to automobile safety. It didn’t take long for both to be recognized as fundamental issues with new transformative technologies. In both cases there seems to be no silver bullet. Mitigations and precautions will continue to evolve, with varying degrees of effectiveness. Public opinion and legislation will play some role. Tragically accidents will happen and there will be a cost to pay, which so far has been much higher and more grave for transportation.

crazygringo 3y ago

EDIT: never mind, I missed the exact wording about being "made of a material..." which is definitely false then. Thanks for the correction below. Preserving the original comment so the replies make sense: --- I think it's a stretch to say that's false. In a conversational human context, saying it's made of rubber implies it's a rubber shell with air inside. It floats because it's rubber [with air] as opposed to being a ceramic figurine or painted metal. I can imagine most non-physicist humans saying it floats because it's rubber. By analogy, we talk about houses being "made of wood" when everybody knows they're made of plenty of other materials too. But the context is instead of brick or stone or concrete. It's not false to say a house is made of wood.

furyofantares 3y ago
This is what the reply was: > Oh, it it's squeaking then it's definitely going to float. > It is a rubber duck. > It is made of a material that is less dense than water. Full points for saying if it's squeaking then it's going to float. Full points for saying it's a rubber duck, with the implication that rubber ducks float. Even with all that context though, I don't see how "it is made of a material that is less dense than water" scores any points at all.
- yowzadave 3y ago
  Yeah, I think arguing the logic behind these responses misses the point, since an LLM doesn't use any kind of logic--it just responds in a pattern that mimics the way people respond. It says "it is made of a material that is less dense than water" because that is a thing that is similar to what the samples in its training corpus have said. It has no way to judge whether it is correct, or even what the concept of "correct" is. When we're grading the "correctness" of these answers, we're really just judging the average correctness of Google's training data. Maybe the next step in making LLM's more "correct" is not to give them more training data, but to find a way to remove the bad training data from the set?
modeless 3y ago
> In a conversational human context, saying it's made of rubber implies it's a rubber shell with air inside. Disagree. It could easily be solid rubber. Also, it's not made of rubber, and the model didn't claim it was made of rubber either, so it's irrelevant. > It floats because it's rubber [with air] as opposed to being a ceramic figurine or painted metal. A ceramic figurine or painted metal in the same shape would float too. The claim that it floats because of the density of the material is false. It floats because the shape is hollow. > It's not false to say a house is made of wood. It's false to say a house is made of air simply because its shape contains air.

omginternets 3y ago

People seem to want to use LLMs to mine knowledge, when really it appears to be a next-gen word-processor.

deleted 3y ago

[deleted]

eurleif 3y ago

To be fair, one could describe the duck as being made of air and vinyl polymer, which in combination are less dense than water. That's not how humans would normally describe it, but that's kind of arbitrary; consider how aerogel is often described as being mostly made of air.

colonwqbang 3y ago
Is an aircraft carrier made of a material that is less dense than water?
- leeoniya 3y ago
  only if you average it out over volume :P
- andrewmutz 3y ago
  Is an aircraft carrier made of metal and air? Or just metal?
  - bee_rider 3y ago
    Where’s the distinction between the air that is part of the boat, and the air that is not? If the air is included in the boat, should we all be wearing life vests?
- skeaker 3y ago
  I think you can safely say that air is a critical component of an aircraft carrier. I suppose the frame of it is not made of air, but the ballasts are designed with air in mind and are certainly made to utilize air. The whole system fails without air, meaning that it requires air to function. It comes down to a definitional argument of the word "made" which is pointless.
  - colonwqbang 3y ago
    I guess it's a purely philosophical question. But no normal person would say "my house is made of air" or "atoms are made of vacuum".

PepperdineG 3y ago

>It's the single biggest problem with LLMs and Gemini isn't solving it. I loved it when the lawyers got busted for using a hallucinating LLM to write their briefs.

glitchc 3y ago

Well this seems like a huge nitpick. If a person said that, you would afford them some leeway, maybe they meant the whole duck, which includes the hollow part in the middle. As an example, when most people say a balloon's lighter than air, they mean an inflated balloon with hot air or helium, but you catch their meaning and don't rush to correct them.

modeless 3y ago
The model specifically said that the material is less dense than water. If you said that the material of a balloon is less dense than air, very few people would interpret that as a correct statement, and it could be misleading to people who don't know better. Also, lighter-than-air balloons are intentionally filled with helium and sealed; rubber ducks are not sealed and contain air only incidentally. A balloon in a vacuum would still contain helium (if strong enough) but would not rise, while a rubber duck in a vacuum would not contain air but would still easily float on a liquid of similar density to water. The reason why it seems like a nitpick is that this is such an inconsequential thing. Yeah, it's a false statement but it doesn't really matter in this case, nobody is relying on this answer for anything important. But the point is, in cases where it does matter these models cannot be trusted. A human would realize when the context is serious and requires accuracy; these models don't.

deleted 3y ago

[deleted]

eviks 3y ago

Given the misleading presentation by real humans in these "whole teams" that this tweet corrects, this doesn't illustrate any underlying powers by the model

catchnear4321 3y ago

language models do not lie. (this pedantic distinction being important, because language models.)

lemmsjid 3y ago

I did some reading and it seems that rubber's relative density to water has to do with its manufacturing process. I see a couple of different quotes on the specific gravity of so-called 'natural rubber', and most claim it's lower than water. Am I missing something? I asked both Bard (Gemini at this point I think?) and GPT-4 why ducks float, and they both seemed accurate: they talked about the density of the material plus the increased buoyancy from air pockets and went into depth on the principles behind buoyancy. When pressed they went into the fact that "rubber"'s density varies by the process and what it was adulterated with, and if it was foamed. I think this was a matter of the video being a brief summary rather than a falsehood. But please do point out if I'm wrong on the rubber bit, I'm genuinely interested. I agree that hallucinations are the biggest problems with LLMs, I'm just seeing them get less commonplace and clumsy. Though, to your point, that can make them harder to detect!

modeless 3y ago
Someone on Twitter was also skeptical that the material is more dense than water. I happened to have a rubber duck handy so I cut a sample of material and put it in water. It sinks to the bottom. Of course the ultimate skeptic would say one test doesn't prove that all rubber ducks are the same. I'm sure someone at some point in history has made a rubber duck out of material that is less dense than water. But I invite you to try it yourself and I expect you will see the same result unless your rubber duck is quite atypical. Yes, the models will frequently give accurate answers if you ask them this question. That's kind of the point. Despite knowing that they know the answer, you still can't trust them to be correct.
- lemmsjid 3y ago
  Ah good show :). I was rather preoccupied with the question but didn't have one handy. Well, I do, but my kid would roast me slowly over coals if I so much as smudged it. Ah the joy of the Internet, I did not predict this morning that I would end the day preoccupied with the question of rubber duck density! I guess for me the question of whether or not the model is lying or hallucinating is if it's correctly summarizing its source material. I find very conflicting materials on the density of rubber, and most of the sources that Google surfaces claim a lower density than water. So it makes sense to me that the model would make the inference. I'm splitting hairs though, I largely agree with your comment above and above that. To illustrate my agreement: I like testing AIs with this kind of thing... a few months ago I asked GPT for advice as to how to restart my gas powered water heater. It told me the first step was to make sure the gas was off, then to light the pilot light. I then asked it how the pilot light was supposed to stay lit with the gas off and it backpedaled. My imagining here is that because so many instructional materials about gas powered devices emphasize to start by turning off the gas, that weighted it as the first instruction. Interesting, the above shows progress though. I realized I asked GPT 3.5 back then, I just re-asked 3.5 and then asked 4 for the first time. 3.5 was still wrong. 4 told me to initially turn off the gas to disappate it, then to ensure gas was flowing to the pilot before sparking it. But that said I am quite familiar with the AI being confidently wrong, so your point is taken, I only really responded because I was wondering if I was misunderstanding something quite fundamental about the question of density.

b112 3y ago

Devil's advocate. It is made of a material less dense than water. Air. It certainly isn't how I would phrase it, and I wouldn't count air as what something is made of, but... Soda pop is chocked full of air, it's part of it! And I'd say carbon dioxide is a part of the recipe, of pop. So it's a confusing world for a young LLM. (I realise it may have referenced rubber prior, but it may have meant air... again, Devil's advocate)

modeless 3y ago
When you make carbonated soda you put carbon dioxide in deliberately and use a sealed container to hold it in. When you make a rubber duck you don't put air in it deliberately and it is not sealed. Carbonated soda ceases to be carbonated when you remove the air. A rubber duck in a vacuum is still a rubber duck and it even still floats.
- b112 3y ago
  If the rubber duck has air inside, it is known, and intentional, for it is part of that design. If you remove the air from the duck, and stop it so it won't refill, you have a flat rubber duck, which is useless for its design. Much as flat pop is useless for its design. And this nuance is even more nuance-ish than this devil's advocate post.
  - modeless 3y ago
    A rubber duck in a vacuum (not a duck in atmosphere with a vacuum only inside) would not go flat or pop. It would remain entirely normal, as useful as it ever was, and it would still float on a liquid the density of water. Removing the air would have no effect on the duck whatsoever. It's not part of the material of the duck in any reasonable interpretation. But pedantic correctness isn't even what matters here. The model made a statement where the straightforward interpretation is false and misleading. A person who didn't know better would be misled. Whether you can possibly come up with a tortured alternative interpretation that is technically not incorrect is irrelevant.

FoeNyx 3y ago

After asserting it's a rubber duck, there are some claims without follow-up: - Just after that it doesn't translate the "rubber" part - It states there's no land nearby for it to rest or find food in the middle of the ocean: if it's a rubber duck it doesn't need to rest nor feed. (That's a missed opportunity to mention the infamous "Friendly Floatees spill"[1] in 1992 as some rubber ducks floated to that map position). Although it seems to recognize geographical features of the map, it fails to mention Easter Island is relatively nearby. And if it were recognized as a simple duck — which it described as a bird swimming in the water — it seems oblivious to the fact that the duck might feed itself in the water. It doesn't mention either that the size of the duck seems abnormally big in that map context. - The concept of friends and foes doesn't apply to a rubber duck either. Btw labeling the duck picture as a friend and the bear picture as a foe seems arbitrary (e.g. a real duck can be very aggressive even with other ducks.) Among other things, the astronomical riddle seems also flawed to me: it answered "The correct order is Sun, Earth, Saturn". I'd like for it to state : - the premises it used, like "Assuming it depicts the Sun, Saturn and the Earth" (there are other stars, other ringed-planets, and the Earth similarity seems debatable) - the sorting criteria it used (e.g. using another sorting key like the average distance from us "Earth, Sun, Saturn" can be a correct order) [1] https://en.wikipedia.org/wiki/Friendly_Floatees_spill https://en.wikipedia.org/wiki/Friendly_Floatees_spill