4 ms·

They made a bad design decision 10 years ago, have been fighting the fallout for years, and will be doing so forever and ever because things will only ever grow

They made a bad design decision 10 years ago, have been fighting the fallout for years, and will be doing so forever and ever because things will only ever grow. They wrote a blog post on how clever they think all their workarounds are, at least one of which involves sparse-checkout -- which is perilously close to chopping up your monorepo into several, while still pretending monorepo is fine. I feel like somebody's job and/or ego is heavily invested in keeping things as they are, even if it demonstrably does not scale to their needs, and the solution is blindingly obvious to even a casual observer. That is institutional insanity.

andi999 4y ago
What would be a better structure?
- roflulz 4y ago
  monorepo until it becomes too big -> then, splitting it into 2-3 repos, until each one also gets too big to manage...
  - masklinn 4y ago
    The problem is not the repository, it’s having the translation data in the repo. We have the same issue at $dayjob, the repo is quite large and 80% of it is translation data. Even if they compress ridiculously well (99% last I checked) the number of translation files and the number of exports makes them the vast majority of the cost. However that structure remains a convenient nuisance, and more importantly removing them would really only be useful if we rewrote the entire repository, which breaks all working copies. There’s been a task in a wishlist for years now, but the business incentive just isn’t there. Exit: actually the translation files are 80% of the working copy, they’re closer to 90% of the repo, and on the far side.
    - andi999 4y ago
      But isnt it sort of correct to have translation data, which is text data, and there might be bug fixes, in the repo?
      - masklinn 4y ago
        Which is why it commonly is there, but at the same time the translation data is kinda independent: you can run the software with no or part of the translations, as far as the software is concerned it’s usually close to configuration data or assets. Not to mention transgressions often have their own lifecycle e.g. is common to find translatable strings which are untranslated or incorrectly translated, and want to ship updates independently from the software’s. As such keeping the translations outside the source is also perfectly defensible.
      - klysm 4y ago
        Yes
    - MisterTea 4y ago
      > The problem is not the repository, it’s having the translation data in the repo. Does this data change as often as the code does? If not then get it out of the repo.
      - andi999 4y ago
        What if it changes more often than the code? Throw out the code?
        pessimizer 4y ago
        There's no material difference between taking the translations out of the repo and taking the code out of the repo. "She shouldn't divorce him, he should divorce her!"
- carapace 4y ago
  In a word, Sqlite. You can always put your files in a file. ("Yo dawg, I heard you like files so I...") Depending on your use cases a zip file can work, e.g. Python packages can be imported from within a zip file, and the standard lib is distributed that way.
victor9000 4y ago
Yeah, this sounds more like a pile of technical debt attached to a competent business.
pech0rin 4y ago
So every company ever? I love these comments like hindsight is not 20/20 and it would have been so simple for you to have made all of the right engineering decisions as an armchair CTO. Give me a break. Git repos get out of control, bad decisions are made, this is an interesting solution and write up. So over the whole “just make perfect decisions all the time and you would never need X comments”. Guess what? Everybody makes mistakes
- the_biot 4y ago
  This isn't just a matter of hindsight. They don't even have that, as demonstrated by the fact they're putting ever more work into their workaround instead of fixing the glaringly obvious problem: it's a monorepo with half a million files growing ever bigger. Certainly I wouldn't have decided to put all that junk into a single git repo from the start, since I'm not an idiot. But, even if I did, or I inherited something like that, I would fix it, not double down as they're doing.
__alexs 4y ago
Yet another example of Additive Bias: https://brainlenses.substack.com/p/additive-bias https://brainlenses.substack.com/p/additive-bias
deleted 4y ago
[deleted]
sangjinyu35 4y ago

andi999 4y ago

What would be a better structure?

roflulz 4y ago
monorepo until it becomes too big -> then, splitting it into 2-3 repos, until each one also gets too big to manage...
- masklinn 4y ago
  The problem is not the repository, it’s having the translation data in the repo. We have the same issue at $dayjob, the repo is quite large and 80% of it is translation data. Even if they compress ridiculously well (99% last I checked) the number of translation files and the number of exports makes them the vast majority of the cost. However that structure remains a convenient nuisance, and more importantly removing them would really only be useful if we rewrote the entire repository, which breaks all working copies. There’s been a task in a wishlist for years now, but the business incentive just isn’t there. Exit: actually the translation files are 80% of the working copy, they’re closer to 90% of the repo, and on the far side.
  - andi999 4y ago
    But isnt it sort of correct to have translation data, which is text data, and there might be bug fixes, in the repo?
    - masklinn 4y ago
      Which is why it commonly is there, but at the same time the translation data is kinda independent: you can run the software with no or part of the translations, as far as the software is concerned it’s usually close to configuration data or assets. Not to mention transgressions often have their own lifecycle e.g. is common to find translatable strings which are untranslated or incorrectly translated, and want to ship updates independently from the software’s. As such keeping the translations outside the source is also perfectly defensible.
    - klysm 4y ago
      Yes
  - MisterTea 4y ago
    > The problem is not the repository, it’s having the translation data in the repo. Does this data change as often as the code does? If not then get it out of the repo.
    - andi999 4y ago
      What if it changes more often than the code? Throw out the code?
      - pessimizer 4y ago
        There's no material difference between taking the translations out of the repo and taking the code out of the repo. "She shouldn't divorce him, he should divorce her!"
carapace 4y ago
In a word, Sqlite. You can always put your files in a file. ("Yo dawg, I heard you like files so I...") Depending on your use cases a zip file can work, e.g. Python packages can be imported from within a zip file, and the standard lib is distributed that way.

victor9000 4y ago

Yeah, this sounds more like a pile of technical debt attached to a competent business.

pech0rin 4y ago

So every company ever? I love these comments like hindsight is not 20/20 and it would have been so simple for you to have made all of the right engineering decisions as an armchair CTO. Give me a break. Git repos get out of control, bad decisions are made, this is an interesting solution and write up. So over the whole “just make perfect decisions all the time and you would never need X comments”. Guess what? Everybody makes mistakes

the_biot 4y ago
This isn't just a matter of hindsight. They don't even have that, as demonstrated by the fact they're putting ever more work into their workaround instead of fixing the glaringly obvious problem: it's a monorepo with half a million files growing ever bigger. Certainly I wouldn't have decided to put all that junk into a single git repo from the start, since I'm not an idiot. But, even if I did, or I inherited something like that, I would fix it, not double down as they're doing.

__alexs 4y ago

Yet another example of Additive Bias: https://brainlenses.substack.com/p/additive-bias https://brainlenses.substack.com/p/additive-bias

deleted 4y ago

[deleted]

sangjinyu35 4y ago