4 ms·

Yeah Google Bard seems to make up stuff way more than chatgpt and gives just wrong code. I asked for numpy code to fit a polyline rejecting outliers and it just

Yeah Google Bard seems to make up stuff way more than chatgpt and gives just wrong code. I asked for numpy code to fit a polyline rejecting outliers and it just made up an argument "robust" for polyfit. Chatgpt on the other hand generated a function that calculated standard deviation and removed points outside two sigma.

sedatk 3y ago
Google Bard shamelessly tells my World War II stories[1]. Bing Chat refuses to answer such a question or says “no information” depending on its creativity level. [1] https://twitter.com/esesci/status/1669066574366646274 https://twitter.com/esesci/status/1669066574366646274
- eyegor 3y ago
  Although that's garbage for factual tasks, the creative writing behavior is a perfect fit for fleshing out dnd characters for example. On the other hand, Google is a search/ad company... I'd think they'd be the most interested in integrating search/facts into an llm. Hallucinating details about your customers is a good way to cause problems (some lawsuits already filed).
  - TeMPOraL 3y ago
    > On the other hand, Google is a search/ad company... I'd think they'd be the most interested in integrating search/facts into an llm. At its core, an advertisement is made of two components: the subject - a store, a brand, a product, etc. - with which the victim is supposed to develop positive associations, ideally strong enough to motivate a purchase and/or advertising the subject to their acquaintances, and the message, which is meant to create those associations. Only the first part, the subject, has to be factual. The message does not, and in fact it usually isn't - manipulative bullshit performs much, much better, and generally the optimum for advertising seems to be asymptotically close to the line past which it would be legally fraud. The only factual part, the subject, needs accurate handling, and is best suited for classical database systems - which is exactly how Google, and everyone else, is handling it. The message part - that's a good match to LLMs, which excel at producing convincingly sounding bullshit. For advertising, it seems what you need is to crank up the hallucinations a bit, but have some plausible deniability built into the whole system, so that when the LLM hallucinates in too obvious a way, no one can actually be held responsible.
- sofixa 3y ago
  Wow that's a lot of creativity - not only did it make up a bunch of information, that information doesn't even make sense within the well known and understood historical context of WW2 (Turkey was neutral so there is no scenario under which a Turkish soldier is "sent to the Eastern Front to fight the Soviet Union").
nonrepeating 3y ago
Sorry, I’m sure this was annoying at the time, but making up that argument is hilarious in retrospect, no?
deleted 3y ago
[deleted]
sakesun 3y ago
ํYes. Google Bard often make up imaginary syntax and commands.
ec109685 3y ago
Here’s an example where Bard is making up studies to support RFK’s thesis that drinking water causes transgenderism: https://twitter.com/EbenThurston/status/1670619027608092674?s=20 https://twitter.com/EbenThurston/status/1670619027608092674?... It’s flat out lying, yet folks are using its output in arguments.
- logicchains 3y ago
  Here's a real study: https://pubmed.ncbi.nlm.nih.gov/29314190/ https://pubmed.ncbi.nlm.nih.gov/29314190/ . It found when frogs were exposed to atrazine (a herbicide) in wastewater runoff, "female minnows were defeminized, whereas male frogs were feminized". Similarly https://www.pnas.org/doi/10.1073/pnas.0909519107 https://www.pnas.org/doi/10.1073/pnas.0909519107 found that "Atrazine-exposed males were both demasculinized (chemically castrated) and completely feminized as adults. Ten percent of the exposed genetic males developed into functional females that copulated with unexposed males and produced viable eggs." So Atrazine can cause frogs to change their gender, and contaminates drinking waters in part of the US (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6164008/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6164008/), so it's not beyond the realm of possibility that it also influences gender expression in humans to some degree.
  - mfjordvald 3y ago
    Egg temperature plays a definitive role in the sex of turtles, but no one would claim it in the realm of possibility for that to affect humans. It's always good to be aware of how things affect human hormones and absolutely worth study and regulation, but it's a huge jump from frogs are affected to humans are affected. I know right wing people are world class long jumpers so this kind of "it could be!" really isn't helping towards a reasonable debate.
- rsynnott 3y ago
  Hadn't realised he was on the General Ripper bandwagon.

sedatk 3y ago

Google Bard shamelessly tells my World War II stories[1]. Bing Chat refuses to answer such a question or says “no information” depending on its creativity level. [1] https://twitter.com/esesci/status/1669066574366646274 https://twitter.com/esesci/status/1669066574366646274

eyegor 3y ago
Although that's garbage for factual tasks, the creative writing behavior is a perfect fit for fleshing out dnd characters for example. On the other hand, Google is a search/ad company... I'd think they'd be the most interested in integrating search/facts into an llm. Hallucinating details about your customers is a good way to cause problems (some lawsuits already filed).
- TeMPOraL 3y ago
  > On the other hand, Google is a search/ad company... I'd think they'd be the most interested in integrating search/facts into an llm. At its core, an advertisement is made of two components: the subject - a store, a brand, a product, etc. - with which the victim is supposed to develop positive associations, ideally strong enough to motivate a purchase and/or advertising the subject to their acquaintances, and the message, which is meant to create those associations. Only the first part, the subject, has to be factual. The message does not, and in fact it usually isn't - manipulative bullshit performs much, much better, and generally the optimum for advertising seems to be asymptotically close to the line past which it would be legally fraud. The only factual part, the subject, needs accurate handling, and is best suited for classical database systems - which is exactly how Google, and everyone else, is handling it. The message part - that's a good match to LLMs, which excel at producing convincingly sounding bullshit. For advertising, it seems what you need is to crank up the hallucinations a bit, but have some plausible deniability built into the whole system, so that when the LLM hallucinates in too obvious a way, no one can actually be held responsible.
sofixa 3y ago
Wow that's a lot of creativity - not only did it make up a bunch of information, that information doesn't even make sense within the well known and understood historical context of WW2 (Turkey was neutral so there is no scenario under which a Turkish soldier is "sent to the Eastern Front to fight the Soviet Union").

nonrepeating 3y ago

Sorry, I’m sure this was annoying at the time, but making up that argument is hilarious in retrospect, no?

deleted 3y ago

[deleted]

sakesun 3y ago

ํYes. Google Bard often make up imaginary syntax and commands.

ec109685 3y ago

Here’s an example where Bard is making up studies to support RFK’s thesis that drinking water causes transgenderism: https://twitter.com/EbenThurston/status/1670619027608092674?s=20 https://twitter.com/EbenThurston/status/1670619027608092674?... It’s flat out lying, yet folks are using its output in arguments.

logicchains 3y ago
Here's a real study: https://pubmed.ncbi.nlm.nih.gov/29314190/ https://pubmed.ncbi.nlm.nih.gov/29314190/ . It found when frogs were exposed to atrazine (a herbicide) in wastewater runoff, "female minnows were defeminized, whereas male frogs were feminized". Similarly https://www.pnas.org/doi/10.1073/pnas.0909519107 https://www.pnas.org/doi/10.1073/pnas.0909519107 found that "Atrazine-exposed males were both demasculinized (chemically castrated) and completely feminized as adults. Ten percent of the exposed genetic males developed into functional females that copulated with unexposed males and produced viable eggs." So Atrazine can cause frogs to change their gender, and contaminates drinking waters in part of the US (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6164008/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6164008/), so it's not beyond the realm of possibility that it also influences gender expression in humans to some degree.
- mfjordvald 3y ago
  Egg temperature plays a definitive role in the sex of turtles, but no one would claim it in the realm of possibility for that to affect humans. It's always good to be aware of how things affect human hormones and absolutely worth study and regulation, but it's a huge jump from frogs are affected to humans are affected. I know right wing people are world class long jumpers so this kind of "it could be!" really isn't helping towards a reasonable debate.
rsynnott 3y ago
Hadn't realised he was on the General Ripper bandwagon.