4 ms·

Git LFS has file locking, and no VCS can provide you with the tools for diffing binary assets. I don't see any meaningful difference between Perforce, Diversion

Git LFS has file locking, and no VCS can provide you with the tools for diffing binary assets. I don't see any meaningful difference between Perforce, Diversion or Lore and git + LFS + file locking. Unless there's a meaningful performance impact for large projects (I only work on small / medium projects), the capabilities are the same. However, I get excellent git support for code in any editor, as opposed to Diversion or Lore which have none.

regnerba 4mo ago
Git LFS for example does not support file chunking. So a single byte change on a large (100s of gigs) file means downloading the whole file again. Lore does chunking of binary files which means faster downloads and better de-dupping on the backend.
- regnerba 4mo ago
  Permissions is another thing. Git permissions are done one a per repo basis. https://epicgames.github.io/lore/explanation/system-design/#251-lore-vs-git https://epicgames.github.io/lore/explanation/system-design/#...
- danudey 4mo ago
  Also, Lore seems to support checking out only the assets you actually need (on-demand hydration and sparse checkouts), meaning that a level designer can check out just the level that they're working on without having to manually configure a git sparse-checkout (and then not being able to see any of the non-checked-out files). If this supports dynamic hydration of files, either as they're accessed (like Dropbox with offline files) or by somehow knowing which files need which other files (building a dependency graph) then it could be a massive win both for speed and efficiency of downloads but also for conserving disk space on developer machines. And since it has API bindings, it's possible that's something that could be built into IDE plugins, so that your editor (Godot, Unity, etc) can know which assets need which other assets and automatically trigger hydration, including when you e.g. try to use a new model/texture/etc in a scene that hasn't used it yet.
throw2ih020 4mo ago
I haven't made games for a long time so I can't speak for my experience, only my friends. From what I understand (1) Perforce has decent integrations with the game engine editors my friends work with, so editor support is no factor for them and (2) it has better delta support for the file types they work with - I believe Git LFS mostly uses a generic xdelta diff which is kind of mediocre at everything versus Perforce can understand different file types and be extended to support custom types.
lentil_soup 4mo ago
don't think it supports branches it's also tough when you have 1TB of data, over 1mm files and you might want to lock hundreds files in one go
- danudey 4mo ago
  I mean, Git LFS 'supports branches' in that the LFS content identifiers are checked into git as files and Git operated normally; LFS is just a way to replace those content identifiers with the actual content, and then vice-versa when you commit. I think branching is the one thing that didn't get more complicated with LFS.
  - lentil_soup 4mo ago
    No, I meant the locking. Being able to branch a lock so you can edit an asset in a feature branch and then discard it (or force a merge)
zipy124 4mo ago
Git LFS breaks so often that it can't be seen as a serious professional tool tbh. I've had nothing but trouble with it. If it wants to be serious it needs to be built into Git, not added as some after thought.
- danudey 4mo ago
  > Git LFS breaks so often that it can't be seen as a serious professional tool tbh. I've had nothing but trouble with it. If it wants to be serious it needs to be built into Git, not added as some after thought. At Fortinet we migrated our SVN repositories to git and ran into a ton of issues; developers over the past ten years had done tons of little mistakes that added up, like accidentally checking an entire Windows virtual machine into the repo. In SVN they deleted it and no one ended up caring, but in Git of course it became part of the repo history. I did a huge amount of work for the migration, 99% of which was analyzing each repo to find out what files/file extensions were overly large, and then either: 1. Filtering them out of the git history completely during import 2. Converting them to LFS objects after the import The LFS process was certainly better than the other alternatives, which were 'check everything into the git history' or 'remove all the un-diffable binary files and hope that they weren't needed for anything', but it was still not ideal. Every developer (out of thousands, across multiple countries, timezones, and native languages) had to set their system up properly; if you missed a command, or if you reinstalled your OS and forgot to set up one of the aliases or hooks, then you would end up checking binary blobs into git rather than LFS, or checking out LFS idents rather than the actual files they needed. We also had the issue of developers fetching code over SSH but LFS files over HTTPS, which would be fine except that we wanted to prevent access to HTTPS from most subnets, so while the developers could use SSH to clone or pull using their 2FA token their client would then make an HTTP request that wouldn't work unless they were on the version control VPN, which.... blah blah blah. So yeah, it worked better than the alternative, but it did not work _well_ a lot of the time.
flohofwoe 4mo ago
I guess you never worked with anything but git? The devil is in the details, and those details generally suck more in git (or generally: distributed version control systems) than in traditional centralized VCS's. Also git-lfs is a crutch that breaks more often than it works :/ (I agree though that for small game projects, git is mostly 'good enough', even without lfs).
Tiktaalik 4mo ago
There's also the tooling. Game teams have artists and designers where baroque command line incantations are headwinds to their workflow pace. For the longest time Git tools were really poor. In recent years there's a few ok ones, like Git Fork, though I wouldn't know if those tools scale to the level of a AAA team size repo and not fall over.
- Scramblejams 4mo ago
  Maybe more than a headwind. From my time in AAA here are two things I learned that were true in that environment: 1. Artists hate Perforce. 2. You will never get them to try anything else. I'm exaggerating, but not by that much. I lost count of the number of artists who are deeply uncomfortable with technology and just manage to learn the bare minimum to do their job, and then will take no more. So besides everything else Lore needs to nail to be acceptable, they need to make it easy for artists to switch. Maybe when UnrealGameSync grows enough knobs and switches to make it unnecessary for an artist to ever touch P4V, Epic can roll Lore into UGS as an unobtrusive option. And if by then there's good support in Unity, in JetBrains, in Maya, etc., then maybe they'll have something.

regnerba 4mo ago

Git LFS for example does not support file chunking. So a single byte change on a large (100s of gigs) file means downloading the whole file again. Lore does chunking of binary files which means faster downloads and better de-dupping on the backend.

regnerba 4mo ago
Permissions is another thing. Git permissions are done one a per repo basis. https://epicgames.github.io/lore/explanation/system-design/#251-lore-vs-git https://epicgames.github.io/lore/explanation/system-design/#...
danudey 4mo ago
Also, Lore seems to support checking out only the assets you actually need (on-demand hydration and sparse checkouts), meaning that a level designer can check out just the level that they're working on without having to manually configure a git sparse-checkout (and then not being able to see any of the non-checked-out files). If this supports dynamic hydration of files, either as they're accessed (like Dropbox with offline files) or by somehow knowing which files need which other files (building a dependency graph) then it could be a massive win both for speed and efficiency of downloads but also for conserving disk space on developer machines. And since it has API bindings, it's possible that's something that could be built into IDE plugins, so that your editor (Godot, Unity, etc) can know which assets need which other assets and automatically trigger hydration, including when you e.g. try to use a new model/texture/etc in a scene that hasn't used it yet.

throw2ih020 4mo ago

I haven't made games for a long time so I can't speak for my experience, only my friends. From what I understand (1) Perforce has decent integrations with the game engine editors my friends work with, so editor support is no factor for them and (2) it has better delta support for the file types they work with - I believe Git LFS mostly uses a generic xdelta diff which is kind of mediocre at everything versus Perforce can understand different file types and be extended to support custom types.

lentil_soup 4mo ago

don't think it supports branches it's also tough when you have 1TB of data, over 1mm files and you might want to lock hundreds files in one go

danudey 4mo ago
I mean, Git LFS 'supports branches' in that the LFS content identifiers are checked into git as files and Git operated normally; LFS is just a way to replace those content identifiers with the actual content, and then vice-versa when you commit. I think branching is the one thing that didn't get more complicated with LFS.
- lentil_soup 4mo ago
  No, I meant the locking. Being able to branch a lock so you can edit an asset in a feature branch and then discard it (or force a merge)

zipy124 4mo ago

Git LFS breaks so often that it can't be seen as a serious professional tool tbh. I've had nothing but trouble with it. If it wants to be serious it needs to be built into Git, not added as some after thought.

danudey 4mo ago
> Git LFS breaks so often that it can't be seen as a serious professional tool tbh. I've had nothing but trouble with it. If it wants to be serious it needs to be built into Git, not added as some after thought. At Fortinet we migrated our SVN repositories to git and ran into a ton of issues; developers over the past ten years had done tons of little mistakes that added up, like accidentally checking an entire Windows virtual machine into the repo. In SVN they deleted it and no one ended up caring, but in Git of course it became part of the repo history. I did a huge amount of work for the migration, 99% of which was analyzing each repo to find out what files/file extensions were overly large, and then either: 1. Filtering them out of the git history completely during import 2. Converting them to LFS objects after the import The LFS process was certainly better than the other alternatives, which were 'check everything into the git history' or 'remove all the un-diffable binary files and hope that they weren't needed for anything', but it was still not ideal. Every developer (out of thousands, across multiple countries, timezones, and native languages) had to set their system up properly; if you missed a command, or if you reinstalled your OS and forgot to set up one of the aliases or hooks, then you would end up checking binary blobs into git rather than LFS, or checking out LFS idents rather than the actual files they needed. We also had the issue of developers fetching code over SSH but LFS files over HTTPS, which would be fine except that we wanted to prevent access to HTTPS from most subnets, so while the developers could use SSH to clone or pull using their 2FA token their client would then make an HTTP request that wouldn't work unless they were on the version control VPN, which.... blah blah blah. So yeah, it worked better than the alternative, but it did not work _well_ a lot of the time.

flohofwoe 4mo ago

I guess you never worked with anything but git? The devil is in the details, and those details generally suck more in git (or generally: distributed version control systems) than in traditional centralized VCS's. Also git-lfs is a crutch that breaks more often than it works :/ (I agree though that for small game projects, git is mostly 'good enough', even without lfs).

Tiktaalik 4mo ago

There's also the tooling. Game teams have artists and designers where baroque command line incantations are headwinds to their workflow pace. For the longest time Git tools were really poor. In recent years there's a few ok ones, like Git Fork, though I wouldn't know if those tools scale to the level of a AAA team size repo and not fall over.

Scramblejams 4mo ago
Maybe more than a headwind. From my time in AAA here are two things I learned that were true in that environment: 1. Artists hate Perforce. 2. You will never get them to try anything else. I'm exaggerating, but not by that much. I lost count of the number of artists who are deeply uncomfortable with technology and just manage to learn the bare minimum to do their job, and then will take no more. So besides everything else Lore needs to nail to be acceptable, they need to make it easy for artists to switch. Maybe when UnrealGameSync grows enough knobs and switches to make it unnecessary for an artist to ever touch P4V, Epic can roll Lore into UGS as an unobtrusive option. And if by then there's good support in Unity, in JetBrains, in Maya, etc., then maybe they'll have something.