5 ms·

Does companies publishing papers about internal tools serve any purpose other than PR?

by Veuxdo 3y ago

Keyframe 3y ago
Google ones seemed to have sparked few revolutions on their own.
williamstein 3y ago
This paper says: “Customized for the organization: CodeCompose is fine-tuned on Meta’s internal code repository, and is thus able to handle Meta-specific languages such as Hack and Flow.” If you work at an org that might want to build their own LLM trained on their own internal code base, then the lessons of this paper would be of value to you.
- Veuxdo 3y ago
  Makes sense. For Meta, though, by publishing papers like this are they hoping for something else other than PR? My only other guess would be attracting talent.
  - bee_rider 3y ago
    Lots of companies have research departments and release papers; the people working in these departments have some academic roots at least. The incentives for releasing papers are: * To raise your profile and reputation generally * The specific publish or perish incentives in academia * Because you really think you’ve done something interesting and novel, and want to share it with the world Only the middle one is removed when going to industry.
chromatin 3y ago
There are many scholars working in industry (from software engineering to biotech) who believe in the ideals of information sharing and publication of research.
- deleted 3y ago
  [deleted]
lmeyerov 3y ago
It was a useful read for me, esp seeing numbers on fine-tuning & use. We are piloting a DB analyst tool where users can use natural language to do DB queries, generate AI analyses, make interactive GPU charts, etc, so many nearby questions we think about a lot. Previously as a PhD publishing on program synthesis, most of our writeups were much smaller scale wrt live user evals, so all combiner... super cool to see. For FB... It probably helps keep the team valued internally + helps with retention & recruiting. For PhD trained types, this kind of paper is almost table stakes. Less obvious... FB has been laying off teams like this despite productivity ROI intuitions, so if I was there, I'd be careful to quantify current + future ROI - I'm sure there are key #'s not being shared.
IshKebab 3y ago
Of course it does. They give lots of details about the tools and their use which is obviously helpful to anyone wanting to do something similar. A great example is https://www.uber.com/blog/research/keeping-master-green-at-scale/ https://www.uber.com/blog/research/keeping-master-green-at-s... I think maybe Gitlab Merge Trains predate it, but it was definitely influential.

Does companies publishing papers about internal tools serve any purpose other than PR?

by Veuxdo 3y ago

Does companies publishing papers about internal tools serve any purpose other than PR?

Keyframe 3y ago

Google ones seemed to have sparked few revolutions on their own.

williamstein 3y ago

This paper says: “Customized for the organization: CodeCompose is fine-tuned on Meta’s internal code repository, and is thus able to handle Meta-specific languages such as Hack and Flow.” If you work at an org that might want to build their own LLM trained on their own internal code base, then the lessons of this paper would be of value to you.

Veuxdo 3y ago
Makes sense. For Meta, though, by publishing papers like this are they hoping for something else other than PR? My only other guess would be attracting talent.
- bee_rider 3y ago
  Lots of companies have research departments and release papers; the people working in these departments have some academic roots at least. The incentives for releasing papers are: * To raise your profile and reputation generally * The specific publish or perish incentives in academia * Because you really think you’ve done something interesting and novel, and want to share it with the world Only the middle one is removed when going to industry.

chromatin 3y ago

There are many scholars working in industry (from software engineering to biotech) who believe in the ideals of information sharing and publication of research.

deleted 3y ago
[deleted]

lmeyerov 3y ago

It was a useful read for me, esp seeing numbers on fine-tuning & use. We are piloting a DB analyst tool where users can use natural language to do DB queries, generate AI analyses, make interactive GPU charts, etc, so many nearby questions we think about a lot. Previously as a PhD publishing on program synthesis, most of our writeups were much smaller scale wrt live user evals, so all combiner... super cool to see. For FB... It probably helps keep the team valued internally + helps with retention & recruiting. For PhD trained types, this kind of paper is almost table stakes. Less obvious... FB has been laying off teams like this despite productivity ROI intuitions, so if I was there, I'd be careful to quantify current + future ROI - I'm sure there are key #'s not being shared.

IshKebab 3y ago

Of course it does. They give lots of details about the tools and their use which is obviously helpful to anyone wanting to do something similar. A great example is https://www.uber.com/blog/research/keeping-master-green-at-scale/ https://www.uber.com/blog/research/keeping-master-green-at-s... I think maybe Gitlab Merge Trains predate it, but it was definitely influential.