5 ms·

We've got to stop calling this a "Google Memo." That's a false narrative. It's just a random doc written by one of 140000+ employees. > Google has been contact

We've got to stop calling this a "Google Memo." That's a false narrative. It's just a random doc written by one of 140000+ employees. > Google has been contacted for comment but it is understood that the document is not an official company memo. [1] There are moats to products, but less so to pure language models trained on the same web-scale scraped data that many share. Not all data is readily available to language models, and integration can be difficult. A company that specializes in say AI for trash sorting likely still has a moat. Microsoft integrating AI into Windows still has a moat (for Windows). GPT-4 is ~200 Elo better than the next best semi-public Vicuna-13B in Chatbot Arena [2]. That is a non-zero moat - perhaps due to hosting larger models, training data, licensing, output postprocessing, etc. [1] https://www.theguardian.com/technology/2023/may/05/google-engineer-open-source-technology-ai-openai-chatgpt https://www.theguardian.com/technology/2023/may/05/google-en... [2] https://lmsys.org/blog/2023-05-25-leaderboard/ https://lmsys.org/blog/2023-05-25-leaderboard/

dragonwriter 3y ago
> GPT-4 is ~200 Elo better than the next best semi-public Vicuna-13B in Chatbot Arena [2]. That is a non-zero moat Its a non-zero advantage. A moat is something that inhibits someone from closing an advantage. (Also, its odd that the biggest models, outside of the big vendor centralized ones, they are testing are 13B-14B when 30B-ish and 65B-ish versions exist.)
- throwaway55905 3y ago
  Perhaps it's not a moat. However, if the advantage is due to things like inference infrastructure to support a massive model, that isn't easy to duplicate. I would also say that the quality of these smaller models are good, but we also may not be measuring them correctly. Recent papers suggest that these smaller LMs dont fully capture ChatGPT quality in ways that may not have appeared with crowd worker ratings [1]. It's easy to have your inputs be inside a happy distribution for a paper but fail in the real world in ways that GPT-4 doesnt. Lmsys would love to compare with bigger models but have limited resources. Contributions are welcome [2] [1] https://arxiv.org/abs/2305.15717 https://arxiv.org/abs/2305.15717 [2] https://lmsys.org/blog/2023-05-25-leaderboard/#next-steps https://lmsys.org/blog/2023-05-25-leaderboard/#next-steps
  - bradleyjg 3y ago
    OpenAI doesn’t make their own hardware, even in the fabless sense of “make.” So in what way is inference infrastructure a moat?
- kordlessagain 3y ago
  The moat (or the water/crocodiles in it) is the content that the company gathers in relation to the offering that is being defended. Microsoft has Github, which is a source of code that the model can operate upon, as well as the interactions/queries with the users. OpenAI is playing around with sharing content because of this. They want to build a moat and will use us to do it. If someone just has an approach to solving the problem, i.e. code that does this that and the other, then there is no moat.
hiddencost 3y ago
(250,000 employees. Just about half of staff is TVC)
- avarun 3y ago
  for others, TVC apparently means "Temporary, Vendor, or Contractor", aka non-FTE
  - zubairshaik 3y ago
    for others, FTE apparently means "Full-Time Employee", aka non-TVC
    - abwizz 3y ago
      was just gonna ask, thanks
Jenk 3y ago
> We've got to stop calling this a "Google Memo." That's a false narrative. It's just a random doc written by one of 140000+ employees. So it's a.. memorandum?
- flokie 3y ago
  yes but "{person name} memo" is not the same as "Google Memo" (published and endorsed by the company)
  - chii 3y ago
    > (published and endorsed by the company) this is not implied by the name "Google memo".
    - numbchuckskills 3y ago
      [dead]
kodah 3y ago
It is a memo written by a Googler on internal systems that makes it a Google memo. Companies almost never officially sponsor internal memos escaping without PR and legal having a crack at the content. What I'm really curious about is why you think this isn't a Google Memo, and why you think that's a false narrative.
- rippercushions 3y ago
  There is a huge difference between a leadership-endorsed strategy memo like the Nokia/Elop "Burning Platform" memo (1), and a memo by a random engineer like Steve Yegge's Platform rant (2). Random engineer memos can certainly still be influential, but they do not dictate company direction. (1) https://www.engadget.com/2011-02-08-nokia-ceo-stephen-elop-rallies-troops-in-brutally-honest-burnin.html https://www.engadget.com/2011-02-08-nokia-ceo-stephen-elop-r... (2) https://gist.github.com/chitchcock/1281611 https://gist.github.com/chitchcock/1281611
- seattle_spring 3y ago
  Anywhere I've worked, junior engineers have put together ridiculous memorandums, especially when they're on the way out. That does not mean that the opinions of the author align with the company's.
- danpalmer 3y ago
  To me, "memo" implies it being for the business. If this was written by leadership or intended to be some sort of instruction to others, then I think it would be more reasonable to call it that. But it wasn't. It was an opinion written down by an individual, for no other reason than to share their personal opinion. It was closer to an HN comment, not a document necessary as part of business operations.
JoeAltmaier 3y ago
Not random. A 'researcher at Google'.
moneywoes 3y ago
> A company that specializes in say AI for trash sorting likely still has a moat Is the most the trash data

dragonwriter 3y ago

> GPT-4 is ~200 Elo better than the next best semi-public Vicuna-13B in Chatbot Arena [2]. That is a non-zero moat Its a non-zero advantage. A moat is something that inhibits someone from closing an advantage. (Also, its odd that the biggest models, outside of the big vendor centralized ones, they are testing are 13B-14B when 30B-ish and 65B-ish versions exist.)

throwaway55905 3y ago
Perhaps it's not a moat. However, if the advantage is due to things like inference infrastructure to support a massive model, that isn't easy to duplicate. I would also say that the quality of these smaller models are good, but we also may not be measuring them correctly. Recent papers suggest that these smaller LMs dont fully capture ChatGPT quality in ways that may not have appeared with crowd worker ratings [1]. It's easy to have your inputs be inside a happy distribution for a paper but fail in the real world in ways that GPT-4 doesnt. Lmsys would love to compare with bigger models but have limited resources. Contributions are welcome [2] [1] https://arxiv.org/abs/2305.15717 https://arxiv.org/abs/2305.15717 [2] https://lmsys.org/blog/2023-05-25-leaderboard/#next-steps https://lmsys.org/blog/2023-05-25-leaderboard/#next-steps
- bradleyjg 3y ago
  OpenAI doesn’t make their own hardware, even in the fabless sense of “make.” So in what way is inference infrastructure a moat?
kordlessagain 3y ago
The moat (or the water/crocodiles in it) is the content that the company gathers in relation to the offering that is being defended. Microsoft has Github, which is a source of code that the model can operate upon, as well as the interactions/queries with the users. OpenAI is playing around with sharing content because of this. They want to build a moat and will use us to do it. If someone just has an approach to solving the problem, i.e. code that does this that and the other, then there is no moat.

hiddencost 3y ago

(250,000 employees. Just about half of staff is TVC)

avarun 3y ago
for others, TVC apparently means "Temporary, Vendor, or Contractor", aka non-FTE
- zubairshaik 3y ago
  for others, FTE apparently means "Full-Time Employee", aka non-TVC
  - abwizz 3y ago
    was just gonna ask, thanks

Jenk 3y ago

> We've got to stop calling this a "Google Memo." That's a false narrative. It's just a random doc written by one of 140000+ employees. So it's a.. memorandum?

flokie 3y ago
yes but "{person name} memo" is not the same as "Google Memo" (published and endorsed by the company)
- chii 3y ago
  > (published and endorsed by the company) this is not implied by the name "Google memo".
  - numbchuckskills 3y ago
    [dead]

kodah 3y ago

It is a memo written by a Googler on internal systems that makes it a Google memo. Companies almost never officially sponsor internal memos escaping without PR and legal having a crack at the content. What I'm really curious about is why you think this isn't a Google Memo, and why you think that's a false narrative.

rippercushions 3y ago
There is a huge difference between a leadership-endorsed strategy memo like the Nokia/Elop "Burning Platform" memo (1), and a memo by a random engineer like Steve Yegge's Platform rant (2). Random engineer memos can certainly still be influential, but they do not dictate company direction. (1) https://www.engadget.com/2011-02-08-nokia-ceo-stephen-elop-rallies-troops-in-brutally-honest-burnin.html https://www.engadget.com/2011-02-08-nokia-ceo-stephen-elop-r... (2) https://gist.github.com/chitchcock/1281611 https://gist.github.com/chitchcock/1281611
seattle_spring 3y ago
Anywhere I've worked, junior engineers have put together ridiculous memorandums, especially when they're on the way out. That does not mean that the opinions of the author align with the company's.
danpalmer 3y ago
To me, "memo" implies it being for the business. If this was written by leadership or intended to be some sort of instruction to others, then I think it would be more reasonable to call it that. But it wasn't. It was an opinion written down by an individual, for no other reason than to share their personal opinion. It was closer to an HN comment, not a document necessary as part of business operations.

JoeAltmaier 3y ago

Not random. A 'researcher at Google'.

moneywoes 3y ago

> A company that specializes in say AI for trash sorting likely still has a moat Is the most the trash data