4 ms·

What motivates the commercial AI companies to share their research results and know-how? Why did Google published the Transformer architecture instead of keepi

What motivates the commercial AI companies to share their research results and know-how? Why did Google published the Transformer architecture instead of keeping it to themselves? I understand that people may want to do good things for humanity, facilitate progress, etc. But if an action goes against commercial interest, how can the company management take it and not get objections from shareholders? Or there is a commercial logic that motivates sharing of information and intellectual property? What logic is that?

varelse 2y ago
[dead]
lofaszvanitt 2y ago
The more people copy your outdated thing, the better for you, because they always gonna lag behind you.
bcoughlan 2y ago
I would guess it comes down to that the best researchers in the world want their work out in the open
- behnamoh 2y ago
  Ilya doesn't. He's a strong proponent of closed source and censorship.
  - esperent 2y ago
    He is just one person. He happens to be the most famous scientist working on this field at the moment it became a gold rush, but it's work built on the shoulders of those who came before, whose discoveries are just as important.
  - Zambyte 2y ago
    Let's hope Totally Safe Intelligence doesn't end up having the same relationship with their name as Totally Open AI.
  - deleted 2y ago
    [deleted]
nodja 2y ago
My understanding is that frontier researchers will work for companies that will let them publish papers and discuss them with their peers. When you're an engineer at the tier of these AI researchers, winning an extra 100k/year on top of you current 500k (numbers out of my ass) is not worth it vs getting name recognition. Being known as one of the authors that made the transformer for example will enable you work with other bright minded individuals and create even better things. So essentially these commercial companies have "we'll let you publish papers when you work for us" as a perk.
- htrp 2y ago
  > When you're an engineer at the tier of these AI researchers, winning an extra 100k/year on top of you current 500k (numbers out of my ass) is not worth it vs getting name recognition. Being known as one of the authors that made the transformer for example will enable you work with other bright minded individuals and create even better things. Also, instead of an extra 100k a year, you get to raise a billion dollars in VC funds for your next company
  - make3 2y ago
    & then sell it back to Google, in this case https://www.axios.com/2024/08/05/google-characterai-venture-capital https://www.axios.com/2024/08/05/google-characterai-venture-...
- deleted 2y ago
  [deleted]
Der_Einzige 2y ago
The ACL, NeurIPS, ICLR and the rest of AI professional organizations are why this happens. Forced open sourcing of everything. No pay to access. It’s the ideal open academic environment for rapid innovation. We must jealously defend our current system, as it will soon come under attack by those who get angry about democratization of the means of computation. Also, lots of copyright abolitionists in AI. Many people who work in the space delight in the idea of making information, especially their own, free. The ghost of Aaron Swartz runs through every researcher in this space.
poin3tkn2 2y ago
[dead]
xwolfi 2y ago
Well Deepseek's survival also depends on the giant amount of hype they can generate, and they won't get more investor money just by having done a one-hit wonder. Becoming deeply integrated in the AI ecosystem with various tools and innovative discoveries will most like be more beneficial than protecting the secrets of their first success.
- WiSaGaN 2y ago
  Deepseek doesn't need hype to survive. They are bankrolled by their now billionaire founder.
Kholin 2y ago
This may be related to Google's business model. Google's main businesses - search engine and advertising - both rely on an open web ecosystem. Therefore, Google has long maintained a friendly attitude toward open source and the open web, such as with Chromium, Noto fonts, Go, Flutter, and others. By providing infrastructure tools that benefit the open web, Google extends the reach of its searchable content and advertising. When the entire Web ecosystem benefits, Google ultimately benefits as well. This model also aligns with the philosophy of the open source community, where everyone is a beneficiary and naturally becomes a contributor.
larodi 2y ago
Indeed, is there a chance Google did not evaluate properly what the transformer will eventually be used for/become. It was created for translation as an improvement on seq2seq, right? Which was for translation, not for thinking, and to a certain extent... still is about translation, and are not other emergent capabilities actually a side-effect, only observed later when parameter size grew?
anon373839 2y ago
> Or there is a commercial logic that motivates sharing of information and intellectual property? What logic is that? There absolutely is a sound commercial justification to share research: long-term growth through advancement of the field. (Deep learning would never have made the progress it has without open research!) If this seems quaint, it’s because we’re too accustomed to short-term, transactional, Wall Street thinking.
- esperent 2y ago
  It's not so much that it seems quaint, it's that we are accustomed to short-term, transactional, Wall Street thinking from companies like Google. For very good reason, because that's exactly how they behave in all other areas. The question remains, why do they appear altruistic when it comes to sharing papers? I find it hard to believe that it's actual altruism. It's far more likely that it's transactional behavior that just appears altruistic from the outside.
  - andai 2y ago
    Everyone benefits from the gains, everyone gets more customers and more investment.
  - avodonosov 2y ago
    People may be altruistic, but in a company setting they may have no possibility for altruism. CEO decisions influence property of others (the shareholders) so he can not freely pursue altruistic goals. I heard that Dodge v. Ford Motor Co. was an important precedent in the US. https://en.m.wikipedia.org/wiki/Dodge_v._Ford_Motor_Co https://en.m.wikipedia.org/wiki/Dodge_v._Ford_Motor_Co.
    - owisd 2y ago
      These days the courts give wide latitude to companies to offer virtually any plausible reason why superficially altruistic acts are in fact good long term for shareholder value. Anyone wanting to do what Ford did just needs to keep their mouth shut about the real reasons.
0x008 2y ago
All of the major labs have one thing in common: they have nearly unlimited data and money, but what they don’t have unlimited is talent and ideas. It’s just a way of progressing without having to „hire every idea“.
HH_GU 2y ago
Just as the company's name DEEPSEEK, it's commercial company and invest their based on AI, but the company's founder has more targets which are more common for human. Money is number for them, they want to do more, especially for DEEPSEEK.
runeks 2y ago
> Why did Google published the Transformer architecture instead of keeping it to themselves? Because they make their money from advertisements. Not their AI models. Same for Meta. Compare that to e.g. OpenAI who's trying to make money from their AI models, and are thus underbid by Google and Meta.
choonway 2y ago
If you don't allow them to publish research work, your greatest talents will leave. I used to work in such a restrictive environment. Nobody worth their salt stayed long.
bobxmax 2y ago
It's worth noting that, while a noteworthy paper, nobody really expected the Transformer at the time to be the breakthrough it eventually became.
- nialv7 2y ago
  and back then in 2017, AI hasn't really been productized yet, people behind Transformer were researchers, publishing their results were the norm.
- avodonosov 2y ago
  Transformer is just an example. We observe a constant stream of information shared by companies, even now, when "AI" is booming.
timClicks 2y ago
There are a few commercially valid strategies. 1. Goodwill and mindshare. If you're known as "the best" or "the most innovative", then you'll attract customers. 2. Talent acquisition. Smart people like working with smart people. 3. Becoming the standard. If your technology becomes widely adopted, and you've been using it the longest, then you're suddenly be the best placed in your industry to make use of the technology while everyone retools. 4. Deception. Sometimes you publish work that's "old" internally but is still state of the art. This provides your competition with a false sense of where your research actually is. 5. Freeride on others' work. Maybe experimenting with extending an idea is too expensive/risky to fund internally? Perhaps a wave of startups will try. Acquire one of them that actually makes it work. 6. Undercut the market leader. If your industry has a clear market leader, the others can use open source to cooperate to erode that leadership position.
buyucu 2y ago
Deepseek is not a commercial AI company. They are the hobby of a hedge fund, something they do on the side for fun and glory.
victorbjorklund 2y ago
If google never published it (and we pretend like it would not have leaked) then we would never have the LLM:s we have today (including Googles). Everyone would loose.

varelse 2y ago

[dead]

lofaszvanitt 2y ago

The more people copy your outdated thing, the better for you, because they always gonna lag behind you.

bcoughlan 2y ago

I would guess it comes down to that the best researchers in the world want their work out in the open

behnamoh 2y ago
Ilya doesn't. He's a strong proponent of closed source and censorship.
- esperent 2y ago
  He is just one person. He happens to be the most famous scientist working on this field at the moment it became a gold rush, but it's work built on the shoulders of those who came before, whose discoveries are just as important.
- Zambyte 2y ago
  Let's hope Totally Safe Intelligence doesn't end up having the same relationship with their name as Totally Open AI.
- deleted 2y ago
  [deleted]

nodja 2y ago

My understanding is that frontier researchers will work for companies that will let them publish papers and discuss them with their peers. When you're an engineer at the tier of these AI researchers, winning an extra 100k/year on top of you current 500k (numbers out of my ass) is not worth it vs getting name recognition. Being known as one of the authors that made the transformer for example will enable you work with other bright minded individuals and create even better things. So essentially these commercial companies have "we'll let you publish papers when you work for us" as a perk.

htrp 2y ago
> When you're an engineer at the tier of these AI researchers, winning an extra 100k/year on top of you current 500k (numbers out of my ass) is not worth it vs getting name recognition. Being known as one of the authors that made the transformer for example will enable you work with other bright minded individuals and create even better things. Also, instead of an extra 100k a year, you get to raise a billion dollars in VC funds for your next company
- make3 2y ago
  & then sell it back to Google, in this case https://www.axios.com/2024/08/05/google-characterai-venture-capital https://www.axios.com/2024/08/05/google-characterai-venture-...
deleted 2y ago
[deleted]

Der_Einzige 2y ago

The ACL, NeurIPS, ICLR and the rest of AI professional organizations are why this happens. Forced open sourcing of everything. No pay to access. It’s the ideal open academic environment for rapid innovation. We must jealously defend our current system, as it will soon come under attack by those who get angry about democratization of the means of computation. Also, lots of copyright abolitionists in AI. Many people who work in the space delight in the idea of making information, especially their own, free. The ghost of Aaron Swartz runs through every researcher in this space.

poin3tkn2 2y ago

[dead]

xwolfi 2y ago

Well Deepseek's survival also depends on the giant amount of hype they can generate, and they won't get more investor money just by having done a one-hit wonder. Becoming deeply integrated in the AI ecosystem with various tools and innovative discoveries will most like be more beneficial than protecting the secrets of their first success.

WiSaGaN 2y ago
Deepseek doesn't need hype to survive. They are bankrolled by their now billionaire founder.

Kholin 2y ago

This may be related to Google's business model. Google's main businesses - search engine and advertising - both rely on an open web ecosystem. Therefore, Google has long maintained a friendly attitude toward open source and the open web, such as with Chromium, Noto fonts, Go, Flutter, and others. By providing infrastructure tools that benefit the open web, Google extends the reach of its searchable content and advertising. When the entire Web ecosystem benefits, Google ultimately benefits as well. This model also aligns with the philosophy of the open source community, where everyone is a beneficiary and naturally becomes a contributor.

larodi 2y ago

Indeed, is there a chance Google did not evaluate properly what the transformer will eventually be used for/become. It was created for translation as an improvement on seq2seq, right? Which was for translation, not for thinking, and to a certain extent... still is about translation, and are not other emergent capabilities actually a side-effect, only observed later when parameter size grew?

anon373839 2y ago

> Or there is a commercial logic that motivates sharing of information and intellectual property? What logic is that? There absolutely is a sound commercial justification to share research: long-term growth through advancement of the field. (Deep learning would never have made the progress it has without open research!) If this seems quaint, it’s because we’re too accustomed to short-term, transactional, Wall Street thinking.

esperent 2y ago
It's not so much that it seems quaint, it's that we are accustomed to short-term, transactional, Wall Street thinking from companies like Google. For very good reason, because that's exactly how they behave in all other areas. The question remains, why do they appear altruistic when it comes to sharing papers? I find it hard to believe that it's actual altruism. It's far more likely that it's transactional behavior that just appears altruistic from the outside.
- andai 2y ago
  Everyone benefits from the gains, everyone gets more customers and more investment.
- avodonosov 2y ago
  People may be altruistic, but in a company setting they may have no possibility for altruism. CEO decisions influence property of others (the shareholders) so he can not freely pursue altruistic goals. I heard that Dodge v. Ford Motor Co. was an important precedent in the US. https://en.m.wikipedia.org/wiki/Dodge_v._Ford_Motor_Co https://en.m.wikipedia.org/wiki/Dodge_v._Ford_Motor_Co.
  - owisd 2y ago
    These days the courts give wide latitude to companies to offer virtually any plausible reason why superficially altruistic acts are in fact good long term for shareholder value. Anyone wanting to do what Ford did just needs to keep their mouth shut about the real reasons.

0x008 2y ago

All of the major labs have one thing in common: they have nearly unlimited data and money, but what they don’t have unlimited is talent and ideas. It’s just a way of progressing without having to „hire every idea“.

HH_GU 2y ago

Just as the company's name DEEPSEEK, it's commercial company and invest their based on AI, but the company's founder has more targets which are more common for human. Money is number for them, they want to do more, especially for DEEPSEEK.

runeks 2y ago

> Why did Google published the Transformer architecture instead of keeping it to themselves? Because they make their money from advertisements. Not their AI models. Same for Meta. Compare that to e.g. OpenAI who's trying to make money from their AI models, and are thus underbid by Google and Meta.

choonway 2y ago

If you don't allow them to publish research work, your greatest talents will leave. I used to work in such a restrictive environment. Nobody worth their salt stayed long.

bobxmax 2y ago

It's worth noting that, while a noteworthy paper, nobody really expected the Transformer at the time to be the breakthrough it eventually became.

nialv7 2y ago
and back then in 2017, AI hasn't really been productized yet, people behind Transformer were researchers, publishing their results were the norm.
avodonosov 2y ago
Transformer is just an example. We observe a constant stream of information shared by companies, even now, when "AI" is booming.

timClicks 2y ago

There are a few commercially valid strategies. 1. Goodwill and mindshare. If you're known as "the best" or "the most innovative", then you'll attract customers. 2. Talent acquisition. Smart people like working with smart people. 3. Becoming the standard. If your technology becomes widely adopted, and you've been using it the longest, then you're suddenly be the best placed in your industry to make use of the technology while everyone retools. 4. Deception. Sometimes you publish work that's "old" internally but is still state of the art. This provides your competition with a false sense of where your research actually is. 5. Freeride on others' work. Maybe experimenting with extending an idea is too expensive/risky to fund internally? Perhaps a wave of startups will try. Acquire one of them that actually makes it work. 6. Undercut the market leader. If your industry has a clear market leader, the others can use open source to cooperate to erode that leadership position.

buyucu 2y ago

Deepseek is not a commercial AI company. They are the hobby of a hedge fund, something they do on the side for fun and glory.

victorbjorklund 2y ago

If google never published it (and we pretend like it would not have leaked) then we would never have the LLM:s we have today (including Googles). Everyone would loose.