5 ms·

(Disclaimer: Google employee, I don’t work on Bard, views my own, etc.) I don’t think either Bing Chat or Bard is ready to be released widely. You need a certa

(Disclaimer: Google employee, I don’t work on Bard, views my own, etc.) I don’t think either Bing Chat or Bard is ready to be released widely. You need a certain mindset to be able to wring value out of them, and most normal users will not think this way. That being said, it’s probably fine to release these tools to enthusiastic early adopters who understand how to use them.

system16 4y ago
This is what I find the most surprising. I was under the impression that ChatGPT was still "research" and wouldn't be ready for prime time for a while. These rushed products that don't live anywhere near up to the hype seem like they will sour public opinion on it.
- kweingar 4y ago
  OpenAI seems to have been the forcing function. Investors tried out GPT-3/ChatGPT and saw dollar signs.
- SilverBirch 4y ago
  From some of the reporting it sounds like AI was in a bit of a cold war that accidentally turned hot. OpenAI got it into their head that someone else was going to release a chat bot imminently, so they panicked and raced to release their own, which is why it's GPT3 based, doesn't have up to date info and has all sorts of other problems. OpenAI panicked and rushed something out and now Google is doing the same. All of these companies know the tech isn't ready, but they couldn't afford to be left behind when someone moved. It's funny, because this happens quite often in tech. Remember when Tesla announced self-driving? Suddenly Intel were doing deals with MobilEye and BMW, GM did Cruise etc. etc. and then 5-10 years later nothing has been delivered? They all jumped the gun and couldn't afford ot be left behind.
  - Ambolia 4y ago
    I haven't used it, but I know plenty of people who today use ChatGPT for work or for school every day. If you are ok with a higher degree of inaccuracy in exchange for minimizing your efford by a lot, ChatGPT is a game changer for many people.
  - codekansas 4y ago
    Some important points about large language models verses self-driving cars: 1. The former is entirely in software and has the favorable economics that brings with it 2. Language models don't need to be that good to provide value, whereas self-driving cars basically need to be solved before they become significantly useful 3. There's a cottage industry of secondary startups being built around using language models for various applications, whereas no such thing exists for self-driving (probably because of 2)
    - paganel 4y ago
      > The former is entirely in software and has the favorable economics that brings with it You still need data labelling, which is not done by software and which is quite difficult to scale up and to carry out in a consistent manner. I know that most probably there are self-learning LLM models that would partly alleviate that issue, but (absent AGI) I don't think that they'll really solve the problem going forward.
  - amf12 4y ago
    I think this was Google. Google showed that it's working on LLM chatbots in previous I/O. Then, the "sentient thing" about LaMDA blew up in June 2022. It's quite possible OpenAI thought Google would release it soon and they panicked. And now, Google who didn't have any intention of releasing LaMDA, panicked..
faizshah 4y ago
Yea, I think CoPilot style tools that generate the prompts for you and pipeline the models are going to see wider adoption in the long run. I think this raw chat UX where you are directly interacting with the model is more of a demo of where we are at with the tech and RLHF rather than something that will have wide applications in the long run. There’s lots of papers on hybrid information retrieval with LLMs that is probably the right strategy for Google rather than a chat bot (which they tried with Google Assistant which was fairly accurate but not very useful). The main issue with these models right now is they are horrible at recalling facts from their training set but incredible at pulling facts from their prompt. Google has the best IR and LLM researchers in the world, they just need a bolder strategy rather than this FOMO strategy.
blihp 4y ago
I think Google internally has a different perception as to the quality of their services vs. the general public and especially tech enthusiasts. Your comment about 'a certain mindset' reminds me that I still[1] have to do exactly that when using Google to try to find anything. Also, whatever happened to just slapping 'beta' on the product and calling it good to roll out? I'm only half kidding as the issue wasn't that products were labeled 'beta', it's that they seemingly never came out of beta. [1] Actually even more so today than pre-ML Google since the services now are constantly (incorrectly) deciding that what I'm asking for isn't actually what I want.
- Insanity 4y ago
  Somewhat related. But the way "the average person" enters queries is not how engineers would enter it. The average person already uses somewhat natural language queries that Google answers. While I (and probably others in the tech field) would go for the keywords. e.g, my wife and family members would enter: * "Distance from Toronto to vancouver" * "How old is the universe" * "What is the capital of Belgium" I'd enter: * "Toronto vancouver distance" * "Universe age" * "Capital Belgium" etc.. you get the point. So for those, I think the ChatGPT approach of "ask question in natural language, get natural language response" is actually quite attractive. Provided it's not outright hallucinating the answer
charles_f 4y ago
> I don’t think either Bing Chat or Bard is ready to be released widely. And yet it will be, because everyone is wowed so we need something out to capture those wowbucks. There are meetings all over the place to figure out "how to best leverage that technology in our product". While there is some value to raise GPT3 as a technology available in your toolbox, these are largely "let's find a problem that this can fix" meetings, and you can bet your sweet ass we'll see GPT in all sorts of stupid maladapted use cases very soon.
ChickenNugger 4y ago
The conclusion (gross oversimplification) I came to yesterday is that these ML chatbots are basically just sophisticated madlibs. It's really misleading to even describe them as AI as they're not intelligences, but just rehashing things previously written by other humans (and I guess now other ML bots). They're not going to spit out anything completely new. That requires intelligence.
- berniedurfee 4y ago
  This is my take as well. Interesting technology, but useful?

system16 4y ago

This is what I find the most surprising. I was under the impression that ChatGPT was still "research" and wouldn't be ready for prime time for a while. These rushed products that don't live anywhere near up to the hype seem like they will sour public opinion on it.

kweingar 4y ago
OpenAI seems to have been the forcing function. Investors tried out GPT-3/ChatGPT and saw dollar signs.
SilverBirch 4y ago
From some of the reporting it sounds like AI was in a bit of a cold war that accidentally turned hot. OpenAI got it into their head that someone else was going to release a chat bot imminently, so they panicked and raced to release their own, which is why it's GPT3 based, doesn't have up to date info and has all sorts of other problems. OpenAI panicked and rushed something out and now Google is doing the same. All of these companies know the tech isn't ready, but they couldn't afford to be left behind when someone moved. It's funny, because this happens quite often in tech. Remember when Tesla announced self-driving? Suddenly Intel were doing deals with MobilEye and BMW, GM did Cruise etc. etc. and then 5-10 years later nothing has been delivered? They all jumped the gun and couldn't afford ot be left behind.
- Ambolia 4y ago
  I haven't used it, but I know plenty of people who today use ChatGPT for work or for school every day. If you are ok with a higher degree of inaccuracy in exchange for minimizing your efford by a lot, ChatGPT is a game changer for many people.
- codekansas 4y ago
  Some important points about large language models verses self-driving cars: 1. The former is entirely in software and has the favorable economics that brings with it 2. Language models don't need to be that good to provide value, whereas self-driving cars basically need to be solved before they become significantly useful 3. There's a cottage industry of secondary startups being built around using language models for various applications, whereas no such thing exists for self-driving (probably because of 2)
  - paganel 4y ago
    > The former is entirely in software and has the favorable economics that brings with it You still need data labelling, which is not done by software and which is quite difficult to scale up and to carry out in a consistent manner. I know that most probably there are self-learning LLM models that would partly alleviate that issue, but (absent AGI) I don't think that they'll really solve the problem going forward.
- amf12 4y ago
  I think this was Google. Google showed that it's working on LLM chatbots in previous I/O. Then, the "sentient thing" about LaMDA blew up in June 2022. It's quite possible OpenAI thought Google would release it soon and they panicked. And now, Google who didn't have any intention of releasing LaMDA, panicked..

faizshah 4y ago

Yea, I think CoPilot style tools that generate the prompts for you and pipeline the models are going to see wider adoption in the long run. I think this raw chat UX where you are directly interacting with the model is more of a demo of where we are at with the tech and RLHF rather than something that will have wide applications in the long run. There’s lots of papers on hybrid information retrieval with LLMs that is probably the right strategy for Google rather than a chat bot (which they tried with Google Assistant which was fairly accurate but not very useful). The main issue with these models right now is they are horrible at recalling facts from their training set but incredible at pulling facts from their prompt. Google has the best IR and LLM researchers in the world, they just need a bolder strategy rather than this FOMO strategy.

blihp 4y ago

I think Google internally has a different perception as to the quality of their services vs. the general public and especially tech enthusiasts. Your comment about 'a certain mindset' reminds me that I still[1] have to do exactly that when using Google to try to find anything. Also, whatever happened to just slapping 'beta' on the product and calling it good to roll out? I'm only half kidding as the issue wasn't that products were labeled 'beta', it's that they seemingly never came out of beta. [1] Actually even more so today than pre-ML Google since the services now are constantly (incorrectly) deciding that what I'm asking for isn't actually what I want.

Insanity 4y ago
Somewhat related. But the way "the average person" enters queries is not how engineers would enter it. The average person already uses somewhat natural language queries that Google answers. While I (and probably others in the tech field) would go for the keywords. e.g, my wife and family members would enter: * "Distance from Toronto to vancouver" * "How old is the universe" * "What is the capital of Belgium" I'd enter: * "Toronto vancouver distance" * "Universe age" * "Capital Belgium" etc.. you get the point. So for those, I think the ChatGPT approach of "ask question in natural language, get natural language response" is actually quite attractive. Provided it's not outright hallucinating the answer

charles_f 4y ago

> I don’t think either Bing Chat or Bard is ready to be released widely. And yet it will be, because everyone is wowed so we need something out to capture those wowbucks. There are meetings all over the place to figure out "how to best leverage that technology in our product". While there is some value to raise GPT3 as a technology available in your toolbox, these are largely "let's find a problem that this can fix" meetings, and you can bet your sweet ass we'll see GPT in all sorts of stupid maladapted use cases very soon.

ChickenNugger 4y ago

The conclusion (gross oversimplification) I came to yesterday is that these ML chatbots are basically just sophisticated madlibs. It's really misleading to even describe them as AI as they're not intelligences, but just rehashing things previously written by other humans (and I guess now other ML bots). They're not going to spit out anything completely new. That requires intelligence.

berniedurfee 4y ago
This is my take as well. Interesting technology, but useful?