4 ms·

If the developer can have a local copy of the monorepo it's not a "large" codebase.

by tex0 5mo ago

digitaltrees 5mo ago
If your codebase can’t fit on a single developer dev machine it’s too big.
- ramraj07 5mo ago
  You mean like Teslas multi terabyte repo is not normal?
  - rtpg 5mo ago
    I think it's obvious that multi terabyte repos are not the norm.
  - digitaltrees 5mo ago
    It couldn’t be broken in to domain specific components? Listen, I am a rails developer, so a monolith doesn’t scare me, and yet, there are limits. Why does it need to be a multi terabyte monolith?
  - mrweasel 5mo ago
    How did they even manage to generate a terabyte sized repo, that's crazy. Do they have something written up on how it's structured and why they'd even go that route?
    - 1718627440 5mo ago
      A terabyte is ~220 thousand books (1000 pages, 50 rows, 100 columns) uncompressed. VCS generally store objects compressed.
- nfg 5mo ago
  Ever work on a AAA game?
  - Mashimo 5mo ago
    That probably mostly assets, no?
    - maccard 5mo ago
      My last project was about 400Gb, and probably 2M lines of C++. The days size is mostly assets but there’s still a lot of code.
      - djeastm 5mo ago
        >2M lines of C++ Even with a git history, wouldn't that only be maybe a few hundred MB?
        maccard 5mo ago
        Dunno, we use perforce. The 499GB is assets.
    - qznc 5mo ago
      Probably, but you want to version control assets too. People usually mention git-lfs at this point, but that is always annoying to use in practice. There is also shallow-clones and sparse-checkouts, but these only mitigate the problem as there is no way around cloning at least one revision completely with git.
  - j_w 5mo ago
    Even a large AAA game should be able to be cloned to a machine. You don't need to clone history, just use --depth to specify the number of commits you want. Obviously a 20 year git repo with all commits is going to be massive, but you don't need that locally. Also, it seems like it would be reasonable for a AAA game to version control assets separately from code.
aulin 5mo ago
If you can't clone it it's not a repo
regnerba 5mo ago
Disagree, but also what do you classify as local storage? Does the repo “size” include all projects or just one? What about multiple branches? How much capacity is local storage? A stock Unreal Engine project is several hundred gigs, consists of multiple solutions, multiple languages, and I would classify as large personally. Without some kind of indexing it’s very awkward to work with and very slow. To work with LLMs and Unreal projects we create a local index, that index file alone is 46GB. Without distributed compilers and caches it can take multiple hours to compile the main solution per platform (usually PC, Linux, Xbox, PlayStation, Switch, and sometimes mobile). So the codebase easily fits on local storage so long as you don’t count assets (those are several TB) and extra so for source assets (10s of TB), and that’s per stream per large project. Anyways, point is I disagree and think Unreal Engine is an example of large codebase that fits locally.

If the developer can have a local copy of the monorepo it's not a "large" codebase.

by tex0 5mo ago

If the developer can have a local copy of the monorepo it's not a "large" codebase.

digitaltrees 5mo ago

If your codebase can’t fit on a single developer dev machine it’s too big.

ramraj07 5mo ago
You mean like Teslas multi terabyte repo is not normal?
- rtpg 5mo ago
  I think it's obvious that multi terabyte repos are not the norm.
- digitaltrees 5mo ago
  It couldn’t be broken in to domain specific components? Listen, I am a rails developer, so a monolith doesn’t scare me, and yet, there are limits. Why does it need to be a multi terabyte monolith?
- mrweasel 5mo ago
  How did they even manage to generate a terabyte sized repo, that's crazy. Do they have something written up on how it's structured and why they'd even go that route?
  - 1718627440 5mo ago
    A terabyte is ~220 thousand books (1000 pages, 50 rows, 100 columns) uncompressed. VCS generally store objects compressed.
nfg 5mo ago
Ever work on a AAA game?
- Mashimo 5mo ago
  That probably mostly assets, no?
  - maccard 5mo ago
    My last project was about 400Gb, and probably 2M lines of C++. The days size is mostly assets but there’s still a lot of code.
    - djeastm 5mo ago
      >2M lines of C++ Even with a git history, wouldn't that only be maybe a few hundred MB?
      - maccard 5mo ago
        Dunno, we use perforce. The 499GB is assets.
  - qznc 5mo ago
    Probably, but you want to version control assets too. People usually mention git-lfs at this point, but that is always annoying to use in practice. There is also shallow-clones and sparse-checkouts, but these only mitigate the problem as there is no way around cloning at least one revision completely with git.
- j_w 5mo ago
  Even a large AAA game should be able to be cloned to a machine. You don't need to clone history, just use --depth to specify the number of commits you want. Obviously a 20 year git repo with all commits is going to be massive, but you don't need that locally. Also, it seems like it would be reasonable for a AAA game to version control assets separately from code.

aulin 5mo ago

If you can't clone it it's not a repo

regnerba 5mo ago

Disagree, but also what do you classify as local storage? Does the repo “size” include all projects or just one? What about multiple branches? How much capacity is local storage? A stock Unreal Engine project is several hundred gigs, consists of multiple solutions, multiple languages, and I would classify as large personally. Without some kind of indexing it’s very awkward to work with and very slow. To work with LLMs and Unreal projects we create a local index, that index file alone is 46GB. Without distributed compilers and caches it can take multiple hours to compile the main solution per platform (usually PC, Linux, Xbox, PlayStation, Switch, and sometimes mobile). So the codebase easily fits on local storage so long as you don’t count assets (those are several TB) and extra so for source assets (10s of TB), and that’s per stream per large project. Anyways, point is I disagree and think Unreal Engine is an example of large codebase that fits locally.