4 ms·

Wow I'm surprised game development is still like that. I haven't done game development in almost 9 years, but the "Perforce saturating the LAN" problem sounds

by chubot 13y ago

Wow I'm surprised game development is still like that. I haven't done game development in almost 9 years, but the "Perforce saturating the LAN" problem sounds very familiar! So I'm actually working on a (binary) software distribution system, and I think it would work very well for this use case. This problem can be solved with software -- it doesn't require an upgrade to fiber. The (straightforward) idea is to use content-addressed storage and differential compression, pretty much like Git does. Then you can actually sync data from your neighbor's machine, in a BitTorrent-ish fashion (possibly getting tiny amounts of metadata from a central server for consistency). Perforce definitely has problems with spurious lock contention, and just a few team members syncing from the same machine can easily clog its pipes if you're not careful. I'm not up to date, but I'm pretty sure you can do better than Perforce's delta compression. You can probably do it with a single generic algorithm, but one way to really improve would be to use file-type specific compression, e.g. bsdiff or Chrome's Courgette for executables, and use other heuristics for game data, raw audio, video, etc. You actually want to avoid single-file compression in the repository so you can take advantage of the differential compression. I can understand that the dynamics of game development teams means that this will never get written in house (although perhaps in the years since I've left, people started placing more emphasis on tools). But for all the entrepreneurs out there, I wouldn't dismiss the market of selling tools to game developers. Perforce is a tiny company that I know made an absolute killing in that market, and I'm somewhat surprised that after a decade they're still the state of the art.

MaulingMonkey 13y ago
> Wow I'm surprised game development is still like that. Some of the problems are fundamental. We're forced to use certain compressed and signed packaging schemes which aren't sanely delta compressable. We distribute in self contained blobs because not everyone has older versions lying around, or will not have a worthwhile pipe when they get to where they need to install things and will further sneakernet on their end. Even if we did assume sane deltas are possible, simple information density puts a lower bound on patch size based on how much content changed, and even that theoretical lower bound can frequently be too much and take too long for relatively "beefy" pipes. Heck, even with fiber, people will still be saving these things out to USB drives for good reason. > Then you can actually sync data from your neighbor's machine, in a BitTorrent-ish fashion (possibly getting tiny amounts of metadata from a central server for consistency). Perforce definitely has problems with spurious lock contention, and just a few team members syncing from the same machine can easily clog its pipes if you're not careful. That helps if the bottleneck is the server. Our perforce machines are beefy enough that our bottlenecks are frequently the client and the LAN pipe, neither of which will be helped by BitTorrent style networking. Work from home would be 100% pipe bottlenecked, as the work pipe is able to saturate it quite easily. FTP is still bottlenecked by either the developer's or the publisher's pipe, and wouldn't be helped by p2p either. > I'm not up to date, but I'm pretty sure you can do better than Perforce's delta compression. There's always room for incremental improvements... but not enough to beat out fiber. > You actually want to avoid single-file compression in the repository so you can take advantage of the differential compression. Not always possible due to format requirements. Windows 8 .appx packages, for example, are basically self contained signed .zips containing an application and all it's resources. Want to provide three builds? You just compressed and signed three different entire copies of all your built resources. For bonus points, those were .zip ed again. Insanity! But software cannot magically solve the social problems that let such designs reach the marketplace. And since things break unless you use the exact "correct" certificate for signing, any script that generated the .appx from one set of resources would require distributing the actual private key which is a non starter...

by chubot 13y ago

MaulingMonkey 13y ago

> Wow I'm surprised game development is still like that. Some of the problems are fundamental. We're forced to use certain compressed and signed packaging schemes which aren't sanely delta compressable. We distribute in self contained blobs because not everyone has older versions lying around, or will not have a worthwhile pipe when they get to where they need to install things and will further sneakernet on their end. Even if we did assume sane deltas are possible, simple information density puts a lower bound on patch size based on how much content changed, and even that theoretical lower bound can frequently be too much and take too long for relatively "beefy" pipes. Heck, even with fiber, people will still be saving these things out to USB drives for good reason. > Then you can actually sync data from your neighbor's machine, in a BitTorrent-ish fashion (possibly getting tiny amounts of metadata from a central server for consistency). Perforce definitely has problems with spurious lock contention, and just a few team members syncing from the same machine can easily clog its pipes if you're not careful. That helps if the bottleneck is the server. Our perforce machines are beefy enough that our bottlenecks are frequently the client and the LAN pipe, neither of which will be helped by BitTorrent style networking. Work from home would be 100% pipe bottlenecked, as the work pipe is able to saturate it quite easily. FTP is still bottlenecked by either the developer's or the publisher's pipe, and wouldn't be helped by p2p either. > I'm not up to date, but I'm pretty sure you can do better than Perforce's delta compression. There's always room for incremental improvements... but not enough to beat out fiber. > You actually want to avoid single-file compression in the repository so you can take advantage of the differential compression. Not always possible due to format requirements. Windows 8 .appx packages, for example, are basically self contained signed .zips containing an application and all it's resources. Want to provide three builds? You just compressed and signed three different entire copies of all your built resources. For bonus points, those were .zip ed again. Insanity! But software cannot magically solve the social problems that let such designs reach the marketplace. And since things break unless you use the exact "correct" certificate for signing, any script that generated the .appx from one set of resources would require distributing the actual private key which is a non starter...