6 ms·

Google shares lose $100B after Bard makes an error during a demo

47 points by GalenErso 4y ago

ddmma 4y ago
[flagged]
chrisbolt 4y ago
https://news.ycombinator.com/item?id=34711244 https://news.ycombinator.com/item?id=34711244
badRNG 4y ago
> a user asks Bard: “What new discoveries from the James Webb Space Telescope can I tell my 9 year old about?” Bard responds with a series of bullet points, including one that reads: “JWST took the very first pictures of a planet outside of our own solar system.” > the first image showing an exoplanet...was actually taken nearly two decades ago, in 2004. Is this the kind of error we might expect ChatGPT to make? Or is this the first indication that "Bard" is an inferior AI to ChatGPT? For that big a hit on stock price, I'd think investors are viewing this as being the latter. Interested in views here.
- jljljl 4y ago
  When I ask ChatGPT the same question, it doesn't make the mistake of claiming it took the first picture of an exoplanet. It's answers change over time though, so it's hard to say Interestingly, If I ask it directly whether the JWST photographed an exoplanet, it says this: > As of my knowledge cutoff in 2021, the James Webb Space Telescope has not yet been launched, so it has not yet taken any photos. However, once it is operational, it is expected to be capable of capturing images of exoplanets and studying their atmospheres. The James Webb Space Telescope is equipped with cutting-edge technology that will allow it to study distant celestial objects in greater detail than any previous telescope. This includes the ability to capture images of exoplanets, which are planets that orbit around stars other than our sun. The data gathered by the James Webb Space Telescope will help us better understand the conditions on these distant worlds and whether they might be capable of supporting life.
RugnirViking 4y ago
The bbc article on this mentioned that a poorly recieved meeting with investors around AI strategy also happened today which can't have helped
jay888 4y ago
I don't think the drop is related to Bard making an error. It is due to the rise of chat based search (as shown in Bing demo by Microsoft). In the chat based search, there will be less ads being displayed. Ad revenue forms the bulk of Google revenue.
- lm28469 4y ago
  > there will be less ads being displayed The top minds of the world are working on delivering you ads, I wouldn't be so sure
- 83 4y ago
  Chat based just means the ads will be more insidious. User: Bing, how do I make a cake? Bing: Start with 2 cups flour - be sure to use King Arthur premium baking flour for a proper batter - then get 1 cup sugar...
  - buffet_overflow 4y ago
    I didn’t think about that angle for advertising, but it’s somewhat terrifying. Imagine being able to pay more to have your “company friendly” dataset prioritized in upcoming LLM training sets.
  - bioemerl 4y ago
    Man, that's already how a decent percent of recipes work. They come off cans and exist in part to sell products.
    - 83 4y ago
      While somewhat true, the capability is there to make this much more pervasive and subtle. Any recipe mentioning tequila can instantly become <insert brand> tequila. Also don't limit your imagination to recipes. People googling car check engine codes will be recommended <paid advertiser> parts from <paid advertiser> websites which can helpfully be installed by <paid service center> nearby. I fear searching is going to get much uglier/noisier than it already is.
agentofoblivion 4y ago
Anyone that has paid any attention to ChatGPT, or LLMs in general, knows that this is absolutely par for the course. There’s no reliability that you get correct answers with current training methods. They are optimized to create plausible (from a word-statistics perspective), human sounding sentences. They’re also overfit on an incredible amount of data, which is why they are so large—-they memorize web scale data. But we can’t tell the difference between a correctly memorized fact and an incorrectly conjured one. Until someone figures out, and implements, a good way to wire these up to an external fact data store, I don’t see a way around it. But I also don’t think that’s a very hard problem to solve and will likely result in far smaller models. I suspect companies launch these products knowing these deficiencies because right now it’s about market share and UX, and the technical ML problems will be confidently solved. Especially if you already own a search engine.
- penjelly 4y ago
  i wonder if gpt4 and hooking into bing search will change this for microsoft
lvl102 4y ago
They need a management shakeup. Pichai is never going to lead that company with breakthroughs. I know it’s not entirely his fault but Google has been a shit show since he took over. Compare that to what Nadella accomplished in Microsoft. It boggles my mind how Nadella is not hailed as the CEO of the past decade (and this is coming from someone with disdain for MS products). Google needs to change right now before they end up as Intel or Yahoo.
- Hnaomyiph 4y ago
  I think it’s mainly because Pichai has the title of CEO and the real CEO is their CFO, Ruth Porat.
- tapoxi 4y ago
  I think for me, this was most evident in the Stadia disaster. Bad tech (requires a Linux port), bad and unclear business model (Stadia Pro vs regular, buying games, Founders Edition hardware), required a weird custom Chromecast, and Sundar basically kicked off the GDC presentation with a "I only play one videogame ha ha" which they didn't even support.
  - lvl102 4y ago
    Nadella is a true software engineer. Sundar is a management consultant. Google needs a shake up and I am sure big investors are pissed.
  - pjmlp 4y ago
    Google has no clue about game development, really none. Compare the tooling in the box for Android with iOS, for example. The AGDK took 10 years to come to life, and is still a kind of basics. If you have a look at game development talks from Google IO, or the few times Google was present at GDC (like that talk), you can watch how the large majority are about PlayStore tooling, all kinds of analytics and player engagement. Technical talks like Sony, Nintendo, Microsoft or even Apple happen to do, are almost inexistent. With such background, their attention span, and then asking everyone to rewrite their games, Stadia was a disaster waiting to happen from the day it was announced.
dqpb 4y ago
1. They named it Bard 2. They didn't actually release it 3. The one thing they showed contained an error Surely this is the strong response to OpenAI that investors were waiting for
im3w1l 4y ago
So I think I know how Bard got it wrong "JWST took the very first pictures of a planet outside of our own solar system" this is wrong as the first pictures of an exoplanet were taken 2004. However, if you search for its statement you get an official Nasa release. "NASA’s Webb Confirms Its First Exoplanet Lee esta historia en español aquí. Researchers confirmed an exoplanet, a planet that orbits another star, using NASA’s James Webb Space Telescope for the first time. Formally classified as LHS 475 b, the planet is almost exactly the same size as our own, clocking in at 99% of Earth’s diameter. The research team is led by Kevin Stevenson and Jacob Lustig-Yaeger, both of the Johns Hopkins University Applied Physics Laboratory in Laurel, Maryland." So there is a confusion about "First pictures ever", and "First for JWST". With some goodwill you could maybe fit both those meanings on Bards statement, but given the context this is probably not what the questioner is after.
michpoch 4y ago
The stock went up before the demo. It was a bit underwhelming so the price is back were it was before the presentation was announced.
deleted 4y ago
[deleted]
penjelly 4y ago
whenever you see a headline "X event wipes Y value" from a stock, its usually safe to assume that no, that isnt the reason it happened. Single events rarely control stock prices in my exp. There are always external pressures that matter more, but get ignored.
- AussieWog93 4y ago
  My thought exactly. Sounded too similar to that time when Ronaldo rejected a bottle of Coke at a press event, and billions were wiped off Coca Cola's market cap. Turns out they'd released a negative financial report that day, and the fall in value had actually started before the press event.
sorokod 4y ago
Perhaps the markets are spooked by Google's inability to produce a demo without emberassing themselves.
- fatjokes 4y ago
  I agree with this. All investors may see is a panicking company and a leader who can't inspire or control. Get rid of Sundar. He's a peacetime CEO or a COO.

47 points by GalenErso 4y ago

ddmma 4y ago

[flagged]

chrisbolt 4y ago

https://news.ycombinator.com/item?id=34711244 https://news.ycombinator.com/item?id=34711244

badRNG 4y ago

> a user asks Bard: “What new discoveries from the James Webb Space Telescope can I tell my 9 year old about?” Bard responds with a series of bullet points, including one that reads: “JWST took the very first pictures of a planet outside of our own solar system.” > the first image showing an exoplanet...was actually taken nearly two decades ago, in 2004. Is this the kind of error we might expect ChatGPT to make? Or is this the first indication that "Bard" is an inferior AI to ChatGPT? For that big a hit on stock price, I'd think investors are viewing this as being the latter. Interested in views here.

jljljl 4y ago
When I ask ChatGPT the same question, it doesn't make the mistake of claiming it took the first picture of an exoplanet. It's answers change over time though, so it's hard to say Interestingly, If I ask it directly whether the JWST photographed an exoplanet, it says this: > As of my knowledge cutoff in 2021, the James Webb Space Telescope has not yet been launched, so it has not yet taken any photos. However, once it is operational, it is expected to be capable of capturing images of exoplanets and studying their atmospheres. The James Webb Space Telescope is equipped with cutting-edge technology that will allow it to study distant celestial objects in greater detail than any previous telescope. This includes the ability to capture images of exoplanets, which are planets that orbit around stars other than our sun. The data gathered by the James Webb Space Telescope will help us better understand the conditions on these distant worlds and whether they might be capable of supporting life.

RugnirViking 4y ago

The bbc article on this mentioned that a poorly recieved meeting with investors around AI strategy also happened today which can't have helped

jay888 4y ago

I don't think the drop is related to Bard making an error. It is due to the rise of chat based search (as shown in Bing demo by Microsoft). In the chat based search, there will be less ads being displayed. Ad revenue forms the bulk of Google revenue.

lm28469 4y ago
> there will be less ads being displayed The top minds of the world are working on delivering you ads, I wouldn't be so sure
83 4y ago
Chat based just means the ads will be more insidious. User: Bing, how do I make a cake? Bing: Start with 2 cups flour - be sure to use King Arthur premium baking flour for a proper batter - then get 1 cup sugar...
- buffet_overflow 4y ago
  I didn’t think about that angle for advertising, but it’s somewhat terrifying. Imagine being able to pay more to have your “company friendly” dataset prioritized in upcoming LLM training sets.
- bioemerl 4y ago
  Man, that's already how a decent percent of recipes work. They come off cans and exist in part to sell products.
  - 83 4y ago
    While somewhat true, the capability is there to make this much more pervasive and subtle. Any recipe mentioning tequila can instantly become <insert brand> tequila. Also don't limit your imagination to recipes. People googling car check engine codes will be recommended <paid advertiser> parts from <paid advertiser> websites which can helpfully be installed by <paid service center> nearby. I fear searching is going to get much uglier/noisier than it already is.

agentofoblivion 4y ago

Anyone that has paid any attention to ChatGPT, or LLMs in general, knows that this is absolutely par for the course. There’s no reliability that you get correct answers with current training methods. They are optimized to create plausible (from a word-statistics perspective), human sounding sentences. They’re also overfit on an incredible amount of data, which is why they are so large—-they memorize web scale data. But we can’t tell the difference between a correctly memorized fact and an incorrectly conjured one. Until someone figures out, and implements, a good way to wire these up to an external fact data store, I don’t see a way around it. But I also don’t think that’s a very hard problem to solve and will likely result in far smaller models. I suspect companies launch these products knowing these deficiencies because right now it’s about market share and UX, and the technical ML problems will be confidently solved. Especially if you already own a search engine.

penjelly 4y ago
i wonder if gpt4 and hooking into bing search will change this for microsoft

lvl102 4y ago

They need a management shakeup. Pichai is never going to lead that company with breakthroughs. I know it’s not entirely his fault but Google has been a shit show since he took over. Compare that to what Nadella accomplished in Microsoft. It boggles my mind how Nadella is not hailed as the CEO of the past decade (and this is coming from someone with disdain for MS products). Google needs to change right now before they end up as Intel or Yahoo.

Hnaomyiph 4y ago
I think it’s mainly because Pichai has the title of CEO and the real CEO is their CFO, Ruth Porat.
tapoxi 4y ago
I think for me, this was most evident in the Stadia disaster. Bad tech (requires a Linux port), bad and unclear business model (Stadia Pro vs regular, buying games, Founders Edition hardware), required a weird custom Chromecast, and Sundar basically kicked off the GDC presentation with a "I only play one videogame ha ha" which they didn't even support.
- lvl102 4y ago
  Nadella is a true software engineer. Sundar is a management consultant. Google needs a shake up and I am sure big investors are pissed.
- pjmlp 4y ago
  Google has no clue about game development, really none. Compare the tooling in the box for Android with iOS, for example. The AGDK took 10 years to come to life, and is still a kind of basics. If you have a look at game development talks from Google IO, or the few times Google was present at GDC (like that talk), you can watch how the large majority are about PlayStore tooling, all kinds of analytics and player engagement. Technical talks like Sony, Nintendo, Microsoft or even Apple happen to do, are almost inexistent. With such background, their attention span, and then asking everyone to rewrite their games, Stadia was a disaster waiting to happen from the day it was announced.

dqpb 4y ago

1. They named it Bard 2. They didn't actually release it 3. The one thing they showed contained an error Surely this is the strong response to OpenAI that investors were waiting for

im3w1l 4y ago

So I think I know how Bard got it wrong "JWST took the very first pictures of a planet outside of our own solar system" this is wrong as the first pictures of an exoplanet were taken 2004. However, if you search for its statement you get an official Nasa release. "NASA’s Webb Confirms Its First Exoplanet Lee esta historia en español aquí. Researchers confirmed an exoplanet, a planet that orbits another star, using NASA’s James Webb Space Telescope for the first time. Formally classified as LHS 475 b, the planet is almost exactly the same size as our own, clocking in at 99% of Earth’s diameter. The research team is led by Kevin Stevenson and Jacob Lustig-Yaeger, both of the Johns Hopkins University Applied Physics Laboratory in Laurel, Maryland." So there is a confusion about "First pictures ever", and "First for JWST". With some goodwill you could maybe fit both those meanings on Bards statement, but given the context this is probably not what the questioner is after.

michpoch 4y ago

The stock went up before the demo. It was a bit underwhelming so the price is back were it was before the presentation was announced.

deleted 4y ago

[deleted]

penjelly 4y ago

whenever you see a headline "X event wipes Y value" from a stock, its usually safe to assume that no, that isnt the reason it happened. Single events rarely control stock prices in my exp. There are always external pressures that matter more, but get ignored.

AussieWog93 4y ago
My thought exactly. Sounded too similar to that time when Ronaldo rejected a bottle of Coke at a press event, and billions were wiped off Coca Cola's market cap. Turns out they'd released a negative financial report that day, and the fall in value had actually started before the press event.

sorokod 4y ago

Perhaps the markets are spooked by Google's inability to produce a demo without emberassing themselves.

fatjokes 4y ago
I agree with this. All investors may see is a panicking company and a leader who can't inspire or control. Get rid of Sundar. He's a peacetime CEO or a COO.