4 ms·

Can anyone here explain to me how a monorepo like Google or Facebook handles security? If I pull the repo - I have the entire contents of Google or Facebook? I

Can anyone here explain to me how a monorepo like Google or Facebook handles security? If I pull the repo - I have the entire contents of Google or Facebook? Is that right? Surely that lacks the normal security measures around what must be highly sensitive information, so there must be more to it than I know of?

madhadron 8y ago
Sensitive information like user data or certificates doesn't live in a source repository. It lives in databases and systems for managing certificates. Treating straight source code as sensitive information is security by obscurity.
- vageli 8y ago
  > Treating straight source code as sensitive information is security by obscurity. For many companies, their source code is a large part of their business.
  - deleted 8y ago
    [deleted]
joshuamorton 8y ago
(there's an acm paper about Google's repo that dives deeper into this). First thing, you can't just "pull the whole repo" at Google or fb scale. It doesn't fit on a single hard drive. This means the enitre repo is normally accessed via networked means. As a result, builds can also be done over the network transparently. So building and testing is done as a different user. That user can have different privileges than the individual requesting the build. So there is a way to hide source code so that only the output artifacts (compiled binaries) can be accessed. But I think the other part of that is that that's normally a tiny minority of code. The other option is of course to live outside the monorepo, as some projects do.
- ddulay 8y ago
  Link to the paper "Why Google Stores Billions of Lines of Code in a Single Repository" https://cacm.acm.org/magazines/2016/7/204032-why-google-stores-billions-of-lines-of-code-in-a-single-repository/pdf https://cacm.acm.org/magazines/2016/7/204032-why-google-stor... Also, "Software Engineering at Google" https://arxiv.org/pdf/1702.01715.pdf https://arxiv.org/pdf/1702.01715.pdf
malkia 8y ago
(Worked at google for 2-3 years, under google3 depot) - AFAIK, only few hundreth files are not visible to employees, and certain folks (contractors) maybe limited there too. You can actually compile, and run in dev/staging certain things, inspect code, click on function, and see callsites, even "debug" from the browser (debugging, is pretty much, if your binary have stepped through a bookmark, then it'll tell you, and you may print out locals from the scope - like vars, etc. - hard to explain in short). Then "checking out", to put in p4/svn - is not really like that - but you can find pretty much videos, docs explaining it. It's more like - you create a "workspace", "client" (piper has history from p4, so some terms are similar to perforce), and in this client - you "view" the entire depot, and changes you've made are "overlayed". Then you can submit these in a CL (much like perforce). There is also git (git5) and hg mode, but I've never used them. I've used the "CitC" one (client in the cloud), and was great as I was able to edit, and later edit, even build,... even deploy all from the browser (though prefer real IDE there - like Eclipse/IntelliJ/CLion).
- tome 8y ago
  FTI: Apparently "google3" is the Google monorepo https://news.ycombinator.com/item?id=17123620 https://news.ycombinator.com/item?id=17123620
ec109685 8y ago
You aren’t allowed to clone google’s repo to a laptop.
deleted 8y ago
[deleted]

madhadron 8y ago

Sensitive information like user data or certificates doesn't live in a source repository. It lives in databases and systems for managing certificates. Treating straight source code as sensitive information is security by obscurity.

vageli 8y ago
> Treating straight source code as sensitive information is security by obscurity. For many companies, their source code is a large part of their business.
- deleted 8y ago
  [deleted]

joshuamorton 8y ago

(there's an acm paper about Google's repo that dives deeper into this). First thing, you can't just "pull the whole repo" at Google or fb scale. It doesn't fit on a single hard drive. This means the enitre repo is normally accessed via networked means. As a result, builds can also be done over the network transparently. So building and testing is done as a different user. That user can have different privileges than the individual requesting the build. So there is a way to hide source code so that only the output artifacts (compiled binaries) can be accessed. But I think the other part of that is that that's normally a tiny minority of code. The other option is of course to live outside the monorepo, as some projects do.

ddulay 8y ago
Link to the paper "Why Google Stores Billions of Lines of Code in a Single Repository" https://cacm.acm.org/magazines/2016/7/204032-why-google-stores-billions-of-lines-of-code-in-a-single-repository/pdf https://cacm.acm.org/magazines/2016/7/204032-why-google-stor... Also, "Software Engineering at Google" https://arxiv.org/pdf/1702.01715.pdf https://arxiv.org/pdf/1702.01715.pdf

malkia 8y ago

(Worked at google for 2-3 years, under google3 depot) - AFAIK, only few hundreth files are not visible to employees, and certain folks (contractors) maybe limited there too. You can actually compile, and run in dev/staging certain things, inspect code, click on function, and see callsites, even "debug" from the browser (debugging, is pretty much, if your binary have stepped through a bookmark, then it'll tell you, and you may print out locals from the scope - like vars, etc. - hard to explain in short). Then "checking out", to put in p4/svn - is not really like that - but you can find pretty much videos, docs explaining it. It's more like - you create a "workspace", "client" (piper has history from p4, so some terms are similar to perforce), and in this client - you "view" the entire depot, and changes you've made are "overlayed". Then you can submit these in a CL (much like perforce). There is also git (git5) and hg mode, but I've never used them. I've used the "CitC" one (client in the cloud), and was great as I was able to edit, and later edit, even build,... even deploy all from the browser (though prefer real IDE there - like Eclipse/IntelliJ/CLion).

tome 8y ago
FTI: Apparently "google3" is the Google monorepo https://news.ycombinator.com/item?id=17123620 https://news.ycombinator.com/item?id=17123620

ec109685 8y ago

You aren’t allowed to clone google’s repo to a laptop.

deleted 8y ago

[deleted]