4 ms·

LLM is not only a service, it shook the fundation of Google's vision. https://www.google.com/search/howsearchworks/our-approach/ https://www.google.com/search/

by est 4y ago

LLM is not only a service, it shook the fundation of Google's vision. https://www.google.com/search/howsearchworks/our-approach/ https://www.google.com/search/howsearchworks/our-approach/ > Google’s mission is to organize the world's information yeah but you only organized the info, but GPT-3/ChatGPT was the first of is kind to actually "understand" the information.

gooseus 4y ago
I'd say that LLMs are a different way of organizing the world's information through a kind of lossy compression into enormous probability distributions that can approximate the original information with enough fidelity to be useful for a lot of low-stakes applications.
- donkeyd 4y ago
  That was a lot of information compressed into a single sentence, but I think you hit the nail on its head. Most things I read about ChatGPT makes me think that most people don't understand what an LLM really is though and make assumptions on what it can do and what it 'understands'.
  - jacquesm 4y ago
    That's because people look at what it can do rather than what it really is. Maybe what it really is doesn't really matter all that much except to 'AI purists'. But to the rest of the world if a company markets something as 'AI' they will treat it as such, including calibration of their response to the effects of that technology independent of how it works under the hood. Whether your job gets obsoleted by LLM or AGI doesn't really matter to people who couldn't tell the difference between the two and it shouldn't matter to them. Their voice deserves to be heard in this whole debate but the tech world has decided they don't really matter.
    - wrs 4y ago
      It’s worse: they look at what they anthropomorphically think it’s doing. Like looking at an animal picture classifier work and thinking it can see animals, or a watching a self-driving car and thinking it has some kind of simulation of a human driver in it. I think the “AI purists” need to make a lot more noise about the fact that nobody knows what LLMs are doing.
      - daniel_reetz 4y ago
        There's a term for this anthropomorphic error in the neurosciences/psychology - a homonculus. Like your theory implies there's a little man in there doing the work. (homonculus also has other meanings in neuro, including cortical mappings, just sayin)
  - _0ffh 4y ago
    When I was doing my master's thesis, my industrial supervisor opined that a neural network "is really just a glorified lookup table". I think, given the succinctness of the characterisation, it's not too far off the mark.
  - pharmakom 4y ago
    Yeah this is a great description. People are saying it's a "lossy jpeg" in a way that is disparaging and minimises how awesome LLMs actually are! They are a form of text compression that leverages the semantics of the text! Amazing!
- jacquesm 4y ago
  And can approximate combinations of that information (hopefully with enough fidelity) that weren't present in the original training body. This is where the value lies, if all it did was spit back out what went in it really would be just a compression algorithm.
oh_sigh 4y ago
Besides for the fact that current LLMs don't understand anything, being first isn't part of that statement. Google also wasn't first to have a search engine, or a video sharing site, or an ad exchange, or an email service.
xnx 4y ago
You left out the other 2/3 of the mission statement: "...and make it universally accessible and useful."
- est 4y ago
  yeah the second half gets interesting. Making it accessible is not enough, LLMs can intepret it in different ways.
MAGZine 4y ago
I'm not sure if that's the case or not. In 2017 I could ask Google which episode of the Futurama was the one where the save the planet from a trash ball, and it could answer the question in natural language. I suspect that Google X has had these, and other capabilities for a while even if they've never been shipped in any specific product. Or how about Duplex, where Google could call a restaurant to make a reservation, including negotiating times? That was also like 2017-2018. These are only the public, polished, productized ideas. Google has bard, a response to chat gpt, probably only because they didn't realize that people actually wanted a chatbot. I think it was a product failure. How quickly they responded with something similar I think shows you this tech exists inside Google already. Remember, OpenAI was created in part to help make AI accessible to people outside of Google/Microsoft. Who knows what's going on behind closed doors.
shostack 4y ago
I don't know that it ruined it. However it creates a challenge for search which makes previous iterations of SEO spam look like child's play. So the signal to noise ratio is going to get massively worse. And if they don't give strong signal to users and continue with an experience increasingly suffocated by ads, users will go wherever the AI prompt is embedded with the least friction in their UI with the minimum acceptable results. Which makes Microsoft relevant again because they own the lower level UI experience on a staggering percentage of screens. I'd expect nothing but aggressively shoving clippy 2.0 into every surface Microsoft owns. And I expect they will be very light with the ads because they have the additional revenue streams to play a longer game on this if it hurts Google's market share. Google doesn't really have the same luxury.

by est 4y ago

gooseus 4y ago

I'd say that LLMs are a different way of organizing the world's information through a kind of lossy compression into enormous probability distributions that can approximate the original information with enough fidelity to be useful for a lot of low-stakes applications.

donkeyd 4y ago
That was a lot of information compressed into a single sentence, but I think you hit the nail on its head. Most things I read about ChatGPT makes me think that most people don't understand what an LLM really is though and make assumptions on what it can do and what it 'understands'.
- jacquesm 4y ago
  That's because people look at what it can do rather than what it really is. Maybe what it really is doesn't really matter all that much except to 'AI purists'. But to the rest of the world if a company markets something as 'AI' they will treat it as such, including calibration of their response to the effects of that technology independent of how it works under the hood. Whether your job gets obsoleted by LLM or AGI doesn't really matter to people who couldn't tell the difference between the two and it shouldn't matter to them. Their voice deserves to be heard in this whole debate but the tech world has decided they don't really matter.
  - wrs 4y ago
    It’s worse: they look at what they anthropomorphically think it’s doing. Like looking at an animal picture classifier work and thinking it can see animals, or a watching a self-driving car and thinking it has some kind of simulation of a human driver in it. I think the “AI purists” need to make a lot more noise about the fact that nobody knows what LLMs are doing.
    - daniel_reetz 4y ago
      There's a term for this anthropomorphic error in the neurosciences/psychology - a homonculus. Like your theory implies there's a little man in there doing the work. (homonculus also has other meanings in neuro, including cortical mappings, just sayin)
- _0ffh 4y ago
  When I was doing my master's thesis, my industrial supervisor opined that a neural network "is really just a glorified lookup table". I think, given the succinctness of the characterisation, it's not too far off the mark.
- pharmakom 4y ago
  Yeah this is a great description. People are saying it's a "lossy jpeg" in a way that is disparaging and minimises how awesome LLMs actually are! They are a form of text compression that leverages the semantics of the text! Amazing!
jacquesm 4y ago
And can approximate combinations of that information (hopefully with enough fidelity) that weren't present in the original training body. This is where the value lies, if all it did was spit back out what went in it really would be just a compression algorithm.

oh_sigh 4y ago

Besides for the fact that current LLMs don't understand anything, being first isn't part of that statement. Google also wasn't first to have a search engine, or a video sharing site, or an ad exchange, or an email service.

xnx 4y ago

You left out the other 2/3 of the mission statement: "...and make it universally accessible and useful."

est 4y ago
yeah the second half gets interesting. Making it accessible is not enough, LLMs can intepret it in different ways.

MAGZine 4y ago

I'm not sure if that's the case or not. In 2017 I could ask Google which episode of the Futurama was the one where the save the planet from a trash ball, and it could answer the question in natural language. I suspect that Google X has had these, and other capabilities for a while even if they've never been shipped in any specific product. Or how about Duplex, where Google could call a restaurant to make a reservation, including negotiating times? That was also like 2017-2018. These are only the public, polished, productized ideas. Google has bard, a response to chat gpt, probably only because they didn't realize that people actually wanted a chatbot. I think it was a product failure. How quickly they responded with something similar I think shows you this tech exists inside Google already. Remember, OpenAI was created in part to help make AI accessible to people outside of Google/Microsoft. Who knows what's going on behind closed doors.

shostack 4y ago

I don't know that it ruined it. However it creates a challenge for search which makes previous iterations of SEO spam look like child's play. So the signal to noise ratio is going to get massively worse. And if they don't give strong signal to users and continue with an experience increasingly suffocated by ads, users will go wherever the AI prompt is embedded with the least friction in their UI with the minimum acceptable results. Which makes Microsoft relevant again because they own the lower level UI experience on a staggering percentage of screens. I'd expect nothing but aggressively shoving clippy 2.0 into every surface Microsoft owns. And I expect they will be very light with the ads because they have the additional revenue streams to play a longer game on this if it hurts Google's market share. Google doesn't really have the same luxury.