4 ms·

- I'm not sold on Bazel (I'm more of a Nix person), but hermetic builds are obviously a good idea - You don't need the added complexity of having to manage too

- I'm not sold on Bazel (I'm more of a Nix person), but hermetic builds are obviously a good idea - You don't need the added complexity of having to manage tooling to track dependencies across multiple repos. A single repo is just fine. You'll obviously want to split off repositories when you want to contribute stuff as open source, and in other cases http://danluu.com/monorepo/ http://danluu.com/monorepo/ - what you describe as "Google-style canary" is not actually Google-style or recommended by Google See the SRE workbook: https://landing.google.com/sre/workbook/chapters/canarying-releases/#use-a-gradual-canary-for-better-metric-selection https://landing.google.com/sre/workbook/chapters/canarying-r... (obviously, Google is big, so different teams might do things differently)

mlthoughts2018 8y ago
Whether using a monorepo or not, you should be creating versioned artifacts that are stored in a separate artifact repository, and different sections of your codebase (whether sharing the same repo or not) should express dependency via consumption of versioned artifacts from the artifact repository. As a result, sometimes you can even require _more_ tooling to adequately handle dependency in a monorepo vs multiple repos. When you consider that a default natural constraint of all software development is that it must adapt and cope with changing tools, usage demands, etc., then it quickly becomes important to allow any given project to use arbitrarily unique build tooling, deployment tooling, libraries, languages, databases, etc. Trying to shoehorn them all into a mandated set of monorepo tooling just doesn’t work, once again meaning you often need much more tooling to reach minimum required levels of flexibility with monorepos as compared with polyrepos.
- joshuamorton 8y ago
  There's no reason to rely on versioned artifacts as dependencies if each binary bundles all deps. This is what bazel does. In practice your second complaint about tooling isn't particularly true. Android apps, webservers, and language runtimes live in the same repo without issue.
  - mlthoughts2018 8y ago
    You’re making egregious assumptions that you are only using languages that compile to executable binaries and that the way the dependencies interact with ambient details of the runtime environment aren’t problematic. This is exactly why the bazel-style build model fails so badly. In the companies where I’ve worked with this pattern, you end up with behemoth binaries, like a giant jar for all possible things you may do that interact with HDFS, for example. Somebody comes along and has a business case where they need a few sub-slices of that jar to function inside a special environment, say like a Jupyter-Scala notebook running inside a custom container that something like Databricks can’t currently support, and suddenly you’re doing insane dependency surgery, mangling build commands to build different versions of the jar for the different use cases. So drastically much better to publish the jar / binary / package / container / whatever to an internal registry, then express the dependency on the specific artifact that’s needed. This is genuinely a line of questioning I ask in order to reject job candidates or to reject potential employers. Your suggested way is just far too dysfunctional. > “Android apps, webservers, and language runtimes live in the same repo without issue” I don’t buy it. “Without issue” is entirely subjective here, and it’s clear from comments you make about Google and your own preferences regarding monorepos that you are at best unlikely to be sympathetic to the kinds of issues people may find to be unacceptably frustrating and choose to view them as non-issues, to falsely believe they would manifest with at least as much headache in a polyrepo situation, or to believe that even if they are sincere downsides of a monorepo that still they don’t rise to a level of importance to see them as distinguishing failure cases. Companies like Google exhibit deep-seated dogmatism about such things, and given this, veteran employees in such places are almost by definition produced by a selection bias effect that reinforces the dogma: anyone with valid dissatisfaction with the monorepo steamroller most likely would have left or declined the job in the first place, just reinforcing the monoculture.
    - joshuamorton 8y ago
      I work mainly in Python. So no I'm not at all making that assumption. If you need slices of that jar, you never depend on the jar artifact. You depend on the source files/ targets you need. It's not hard or tricky. You should only ever produce executable binaries. Library targets or jars that are there as deps should not ever be collected as artifacts. Trying to store intermediate artifacts externally is bad. Don't do it. If you try to use a system in exactly the way it says not to, you'll have problems. That's your fault though.

mlthoughts2018 8y ago

Whether using a monorepo or not, you should be creating versioned artifacts that are stored in a separate artifact repository, and different sections of your codebase (whether sharing the same repo or not) should express dependency via consumption of versioned artifacts from the artifact repository. As a result, sometimes you can even require _more_ tooling to adequately handle dependency in a monorepo vs multiple repos. When you consider that a default natural constraint of all software development is that it must adapt and cope with changing tools, usage demands, etc., then it quickly becomes important to allow any given project to use arbitrarily unique build tooling, deployment tooling, libraries, languages, databases, etc. Trying to shoehorn them all into a mandated set of monorepo tooling just doesn’t work, once again meaning you often need much more tooling to reach minimum required levels of flexibility with monorepos as compared with polyrepos.

joshuamorton 8y ago
There's no reason to rely on versioned artifacts as dependencies if each binary bundles all deps. This is what bazel does. In practice your second complaint about tooling isn't particularly true. Android apps, webservers, and language runtimes live in the same repo without issue.
- mlthoughts2018 8y ago
  You’re making egregious assumptions that you are only using languages that compile to executable binaries and that the way the dependencies interact with ambient details of the runtime environment aren’t problematic. This is exactly why the bazel-style build model fails so badly. In the companies where I’ve worked with this pattern, you end up with behemoth binaries, like a giant jar for all possible things you may do that interact with HDFS, for example. Somebody comes along and has a business case where they need a few sub-slices of that jar to function inside a special environment, say like a Jupyter-Scala notebook running inside a custom container that something like Databricks can’t currently support, and suddenly you’re doing insane dependency surgery, mangling build commands to build different versions of the jar for the different use cases. So drastically much better to publish the jar / binary / package / container / whatever to an internal registry, then express the dependency on the specific artifact that’s needed. This is genuinely a line of questioning I ask in order to reject job candidates or to reject potential employers. Your suggested way is just far too dysfunctional. > “Android apps, webservers, and language runtimes live in the same repo without issue” I don’t buy it. “Without issue” is entirely subjective here, and it’s clear from comments you make about Google and your own preferences regarding monorepos that you are at best unlikely to be sympathetic to the kinds of issues people may find to be unacceptably frustrating and choose to view them as non-issues, to falsely believe they would manifest with at least as much headache in a polyrepo situation, or to believe that even if they are sincere downsides of a monorepo that still they don’t rise to a level of importance to see them as distinguishing failure cases. Companies like Google exhibit deep-seated dogmatism about such things, and given this, veteran employees in such places are almost by definition produced by a selection bias effect that reinforces the dogma: anyone with valid dissatisfaction with the monorepo steamroller most likely would have left or declined the job in the first place, just reinforcing the monoculture.
  - joshuamorton 8y ago
    I work mainly in Python. So no I'm not at all making that assumption. If you need slices of that jar, you never depend on the jar artifact. You depend on the source files/ targets you need. It's not hard or tricky. You should only ever produce executable binaries. Library targets or jars that are there as deps should not ever be collected as artifacts. Trying to store intermediate artifacts externally is bad. Don't do it. If you try to use a system in exactly the way it says not to, you'll have problems. That's your fault though.