4 ms·
> But.. somehow, they still account for only a small fraction of papers. They are only going to publish results which have no effect on their competitive advan
by DrFalkyn 3y ago
> But.. somehow, they still account for only a small fraction of papers.
They are only going to publish results which have no effect on their competitive advantage as a business ... which could be a small fraction of their results.
- light_hue_1 3y agoEh. I don't buy that. Business is built on engineering. That's what they spend the vast majority of their resources on. No one has any secret sauce science at the moment. And if they did their engineers would be bragging about it in private, bringing it to new companies when they move, and we would be seeing it in products or features. Heck they would be telling us when they apply to grad school.
- Jensson 3y agoImagine if Google didn't publish the transformers paper, Google does not just do engineering they also do research. If they find something similar today it wont go into the public since OpenAI set a precedent of not saying what they do.
- light_hue_1 3y agoIf Google didn't publish the Transformers paper their engineers would have given talks about it, would have used those ideas to apply to grad school, would have switched companies and built a similar model that's just different enough not to technically be the same, and we would have reverse engineered it immediately. Simply knowing a few keywords would be enough for anyone competent to reproduce all of the work. The space of possible models is huge. And we don't know in which direction to go to build better models. But knowing the vague general direction is all it takes. Any researcher can follow along given that knowledge. That's true of a lot of science not just ML. No one has any secret sauce. Not even OpenAI.