6 ms·
What does a leaked Google memo reveal about the future of AI?
- deleted 4y ago[deleted]
- janalsncm 4y agoI don’t know. If you define “AI” as having a great chatbot then probably no company has a moat anymore. But there are a lot more types of tasks which have nothing to do with that. Robotics, for one. Self-driving cars is another example. “AI” in the abstract is a meaninglessly vague concept but for all intents and purposes Google still has advantages over others in important areas. Their perpetual disadvantage of course is comically poor product vision, which puts any of their actual advantages in jeopardy.
- CharlesW 4y ago> But there are a lot more types of tasks which have nothing to do with that. I've found the Gartner AI Hype Cycle to be a useful tool for making this point with people who are under the impression that "AI" is mostly generative AI, and LLMs specifically: https://www.gartner.com/en/articles/what-s-new-in-artificial-intelligence-from-the-2022-gartner-hype-cycle https://www.gartner.com/en/articles/what-s-new-in-artificial... In particular, the chart conveys this really quickly.
- deleted 4y ago[deleted]
- gumby 4y agoWas it leaked or “leaked”? Google has a good, two part strategy available. First is to neutralize the threat: support the open source efforts covertly and/or overtly, as those efforts undercut the revenue of OpenAI and other potential competitors for the base technology. This is what they did with Android: at a minimum all they had to do was ensure that Apple did not become a monopoly, which they accomplished (the secondary effect was to cut the underbrush of any other competitors). A name for this is to “commoditize your competition”. The second half is to use their internal experience with LLMs and their other ecosystem to use supercharge their strengths. Side inventions can either be expanded in the hope of a (desperately needed, unrelated to LLMs) new revenue stream or given away to starve potential new entrants. The big guys have good experience in weaponizing open source. But google has become flabby and seemingly lost any culture of execution so I don’t know if they can pull it off. But the path is there.
- sitkack 4y agoThe doc is too short and not puffy to be a normal google doc. There is a high likelihood this was placed to proactively shake off any regulation or scruitiny while they get the deck chairs in order. They might not appear to have a moat, but they definitely have an advantage that they can keep for 5+ years. They also have a "copy of the internet" that no one else does.
- akiselev 4y agoWhat are the chances that their copy is more useful than CommonCrawl? It’s been the target of SEO for decades - are they even capable of separating the good content from the crap anymore? Google may already be terminal
- sitkack 4y agoData pollution isn't the problem it is made out to be. You can even use a model trained on garbage data to relabel the original set and retrain. Google of all companies is sitting on exaflops of TPU capacity, they can retrain models larger than anyone else, faster than anyone else.
- visarga 4y agoI agree, but it would probably require to use LLMs to filter or augment the raw text. They can afford it.
- chaxor 4y agoTheir data may not be that much better; however Deepmind is still one of the best institutions that study AI fundamentals. OpenAI works hard and focused towards products, but the academic research is far better from Deepmind. So since Google hasn't been trying to put any AI into products really at all for the past 10 years, they're going to have to push a little bit to make the public facing interfaces and put polish on things, but the underlying models they have made are far better than OpenAI (just not the ones made publicly accessible to anyone).
- siliconc0w 4y agoOn the low end, google can sprinkle in well integrated cost-optimized 'pretty good' LLMs into all their products which already have billions of DAU. It's hard for OpenAI to compete here, their models have to be significantly better to encourage people to copy and paste into ChatGPT and yet still cost effective. The high end is more interesting in that I'm not sure Google has a response to Gpt4. Specialized Palm2 models are allegedly SOTA in certain verticals but they're not generally available so it's hard to say. It's possible Google could offer a beefy model to Google One subscribers or something but they seem to be focused on the low end for now at least in the B2C arena with specialized models for B2B. I honestly don't think OSS is a big factor business-wise but it's nice to see and certainly enables more participation in the ecosystem.
- chaxor 4y agoThis idea that ChatGPT is better than the models that Google has is false; but I understand the perception. Deepmind has far better models than OpenAI, and the models OpenAI have made are based on the research done at Deepmind. For example, ChatGPT was not the first model trained by RLHF - that was Sparrow by Deepmind. The difference is that Google has been very disinterested in making products out of these systems, and didn't spend any effort to make them publicly accessible. Deepmind is still the leader in AI, and they have much better models - it's just going to take a while for people who don't study ML to realize this because Google hasn't released any of their good models for public use yet (and yeah, I'm not counting Bard, etc because the real model was probably put together by one guy in an afternoon at Google, and the rest of the time working was marketing).
- visarga 4y ago> I honestly don't think OSS is a big factor business-wise Lots of companies are evaluating fine-tuning strategies for <60B LLMs. Such models can be efficient, private, cheaper and adapted to the use case. They may not be generalists like GPT-4, but you need them for scaling up usage on high volume calls on the same task if you don't want to have to pay a fortune to OpenAI, or for processing confidential information such as in medical, B2B and other cases. Not to mention the fear that OpenAI might snoop on their private data and use it in their corpus.
- Animats 4y agoWasn't this note from someone at Google discussed on HN last week, before The Economist picked it up? It is a huge issue for Google. Not because they're behind, but because chat engines don't seem to map well to an ad-supported model. Unless they talk like the Youtubers who start with "But first, let me tell you about our sponsor, FBX Crypto". They do map well to a surveillance model. That's already getting regulator attention. Look what happened to TikTok, which is really good at focusing on a user's interests. Chat engines focus like that. They have to, to work well.
- flutas 4y agoYep. Related: Google “We have no moat, and neither does OpenAI” - https://news.ycombinator.com/item?id=35813322 https://news.ycombinator.com/item?id=35813322 (May 2023)
- numbchuckskills 4y ago[dead]
- mdp2021 4y ago> AI burst into the public consciousness with the launch in late 2022 Because 70 years of named work based on centuries of interest and efforts can remain unseen if one really tries. /R > But easier access to AI also means bad actors will be able to fine-tune systems for nefarious purposes, such as generating disinformation. It means Western attempts to prevent hostile regimes from gaining access to powerful AI technology will fail. And it makes AI harder to regulate, because the genie is out of the bottle And the fact that after centuries of steep increase of technical possibilities in spite of low spiritual growth, with clear consequences every time met with bewilderment - you know, e.g., WWI and the surprise (historically noted) post-facto after the heightened possible devastation -, should not suggest that what are to be tackled are the root issues? The «genie is out of the bottle» at least after the industrial revolution.
- sitkack 4y agoYou are just starting a flame war.
- mdp2021 4y agoOr we could opt for a bet that this public is selected enough to possibly avoid the «flame war» and allow some "normal" liberty and maybe instead risk some constructive exchange, even productive.
- scrollaway 4y ago> Because 70 years of named work based on centuries of interest and efforts can remain unseen if one really tries. You’re really trying to be uncharitable eh? We’ve been talking about time travel for millennia. But I can guarantee you that if tomorrow we figure out a way for most people to actually reliably perform it and play with it, it will be defined as “bursting into the public consciousness”.
- AnonymousPlanet 4y agoI must have overlooked all those conferences on time travel that are on the same level as, e.g., AAAI. But yes, public perception is a different thing. Most people are totally baffled when I tell them that the first spam email was sent in 1978, because for a lot of people the internet (and with it email) popped into existence somewhere around 2004.
- UncleMeat 4y agoAll the presentation of this has been weird. "Leaked Google memo" is true in a sense, but the doc was written by one person who isn't the primary leading force within relevant orgs at Google. The headline is more accurately described as "what does the opinion of one Googler reveal..."
- dehrmann 4y agoAt the same time, I'm not sure I'd give any more weight to something written by "leading forces within relevant orgs" when these are the same people who missed ChatGPT and are playing catch-up.
- FooBarBizBazz 4y ago"Leaks" can be press releases. "Oops, you weren't supposed to see that" can make the message seem more credible. Is this Google preemptively fending off accusations of monopoly in an important space? On the other hand, if Google is perceived as weak, then its share price will be negatively affected. So maybe not. Maybe it's just what some guy wrote. But then how did the leak happen? Somebody chose to publicize this particular bit of text. Why?
- UncleMeat 4y ago> "Leaks" can be press releases. It'd be weird not to integrate the comments first if this was being done by PR. > But then how did the leak happen? Somebody thought the doc was interesting and shared it with a news outlet. I've read 100+ docs that read like this. Some sort of position paper designed to spark discussion, much of which happens in the comments alongside the doc. All of them are just the opinion of a person, not the company.
- mrtranscendence 4y agoWell, how do you tell the difference between a truly leaked internal document that reflects the opinion of one person, and a "leaked" document that just so happens to put forth a position that's beneficial to Google given its current standing in the LLM space? I think some skepticism is healthy here, as long as we understand that no one knows for sure.
- warrenmiller 4y agohttps://archive.is/pK4nv https://archive.is/pK4nv