23 ms·
Helldivers 2 on-disk size 85% reduction
- habbekrats 10mo agoit seems wild the state of games and development today... imagine 131GB out of 154GB of data was not needed....
- wvbdmp 10mo agoThe whole world took a wrong turn when we moved away from physical media.
- breve 10mo agoHard drives and optical discs are the reason they duplicated the data. The duplicated the data to reduce load times.
- habbekrats 10mo agodo they even sell disc of these game?...
- jsheard 10mo agoThey do, but it's irrelevant to performance nowadays since you're required to install all of the disc data to the SSD before you can play. The PS3/360 generation was the last time you could play games directly from a disc (and even then some games had an install process).
- Cthulhu_ 10mo agoI believe even then it was already "most", at least for the PS3; that was the era where always-online devices became the norm, where game developers were more eager to release patches after release, etc.
- tetris11 10mo agoIn terms of ownership, yes absolutely. In terms of read/write speeds to physical media, the switch to an SSD has been unsung gamechanger. That being said, cartridges were fast. The move away from cartridges was a wrong turn
- BizarroLand 10mo agoI hate it when you buy a physical game, insert the disk, and immediately have to download the game in order to play the game because the disk only contains a launcher and a key. Insanity of the worst kind.
- crest 10mo agoOr the launch day patch is >80% the size of the game, but I don't want to go back to game design limited by optical media access speeds.
- hbn 10mo agoNintendo is pretty good for putting a solid 1.0 version of their games on the cartridges on release. But on the other hand, the Switch cartridges use NAND memory which means if you aren't popping them into a system to refresh the charge every once in a while, your physical cartridge might not last as long as they keep the servers online so you could download a digital purchase. I've kinda given up on physical games at this point. I held on for a long time, but the experience is just so bad now. They use the cheapest, flimsiest, most fragile plastic in the cases. You don't get a nice instruction manual anymore. And honestly, keeping a micro SD card in your system that can hold a handful of games is more convenient than having to haul around a bunch of cartridges that can be lost. I take solace in knowing that if I do still have a working Switch in 20 years and lose access to games I bought a long time ago, hopefully the hackers/pirates will have a method for me to play them again.
- wtallis 10mo ago> the Switch cartridges use NAND memory which means if you aren't popping them into a system to refresh the charge every once in a while, your physical cartridge might not last as long You've been paying attention to the wrong sources for information about NAND flash. A new Switch cartridge will have many years of reliable data retention, even just sitting on a shelf. Data retention only starts to become a concern for SSDs that have used up most of their write endurance; a Switch cartridge is mostly treated as ROM and only written to once.
- jayd16 10mo agoThe (de)-optimization exists, essentially, because of physical media.
- maccard 10mo agoThis isn't unique to games, and it's not just "today". Go back a decade [0] find people making similar observations about one of the largest tech companies on the planet. [0] https://news.ycombinator.com/item?id=10066338 https://news.ycombinator.com/item?id=10066338
- lynnharry 10mo ago> FB App is 114MB in size, but loading this page in Chrome will use a good 450MB, idk how they managed that. This reminds me of the old days when I check who's using my PC memory every now and then.
- Cthulhu_ 10mo agoAnd that's consumer apps, having only glimpsed in the world of back-end / cloud shenanigans, there's heaps of data being generated and stored in datacenters. Useful data? Dunno, how useful are all access logs ever? But it's stored because it's possible, easy, and cheap. Unlike older games, where developers would hide unused blocks of empty data for some last-minute emergency cramming if they needed it.
- high_na_euv 10mo agoIt was needed. Just the trade off wasnt worth it.
- Zambyte 10mo agoIt was wanted and intentionally selected, but it wasn't needed.
- Tepix 10mo agoI'd argue it was incompetence.
- red-iron-pine 10mo agoit wasn't needed -- need means "must have" they're a fantastically popular franchise with a ton of money... and did it without the optimizations. if they never did these optimizations they'd still have a hugely popular, industry leading game minor tweaks to weapon damage will do more to harm their bottom line compared to any backend optimization
- andrewstuart 10mo agoHow is there so much duplication?
- jy14898 10mo agoThe post stated that it was believed duplication improved loading times on computers with HDDs rather than SSDs
- dontlaugh 10mo agoWhich is true. It’s an old technique going back to CD games consoles, to avoid seeks.
- SergeAx 10mo agoIs it really possible to control file locations on HDD via Windows NTFS API?
- dontlaugh 10mo agoNo, not at all. But by putting every asset a level (for example) needs in the same file, you can pretty much guarantee you can read it all sequentially without additional seeks. That does force you to duplicate some assets a lot. It's also more important the slower your seeks are. This technique is perfect for disc media, since it has a fixed physical size (so wasting space on it is irrelevant) and slow seeks.
- viraptor 10mo ago> by putting every asset a level (for example) needs in the same file, you can pretty much guarantee you can read it all sequentially I'd love to see it analysed. Specifically, the average number of nonseq jumps vs overall size of the level. I'm sure you could avoid jumps within megabytes. But if someone ever got closer to filling up the disk in the past, the chances of contiguous gigabytes are much lower. This paper effectively says that if you have long files, there's almost guaranteed gaps https://dfrws.org/wp-content/uploads/2021/01/2021_APAC_paper-a_contemporary_investigation_of_ntfs_file_fragmentation.pdf https://dfrws.org/wp-content/uploads/2021/01/2021_APAC_paper... so at that point, you may be better off preallocating the individual does where eating the cost of switching between them.
- Calzifer 10mo agoI was curious if they optimized the download. Did it download the 'optimized' ~150 GB and wasting a lot of time there or did it download the ~20 GB unique data and duplicated as part of the installation. I still don't know but found instead an interesting reddit post were users found and analyzed this "waste of space" three month ago. https://www.reddit.com/r/Helldivers/comments/1mw3qcx/why_the_game_is_130_gb_install/ https://www.reddit.com/r/Helldivers/comments/1mw3qcx/why_the... PS: just found it. According to this Steam discussion it does not download the duplicate data and back then it only blew up to ~70 GB. https://steamcommunity.com/app/553850/discussions/0/4372501943310519926/ https://steamcommunity.com/app/553850/discussions/0/43725019...
- SergeAx 10mo agoThey downloaded 43 GB instead of 152 GB, according to SteamDB: https://steamdb.info/app/553850/depots/ https://steamdb.info/app/553850/depots/ Now it is 20 GB => 21 GB.
- maccard 10mo agoSteam breaks your content into 1MB Chunks and compresses/dedupes them [0] [0] https://partner.steamgames.com/doc/sdk/uploading#AppStructureBestPractices https://partner.steamgames.com/doc/sdk/uploading#AppStructur...
- tetris11 10mo agoI wonder if a certain Amelie-clad repacker noticed the same reduction in their release of the same game.
- squigz 10mo agoFitgirl and Anna (of Anna's Archive) are modern day heroes.
- debugnik 10mo agoHelldivers 2 is an always-online game, you won't find it cracked.
- NullPrefix 10mo agosuch games can only get private servers
- debugnik 10mo agoYes, but those are rarely a thing for most live service games. Unless someone is working on a reimplementation of the entire server side, there's no point in offering or downloading pirate copies.
- simplyinfinity 10mo agoThere is - albeit a dwindling - community that does reimplement entire backends for mmo games. Look up the ragezone forums. I grew up around Mu Online private servers. And I'm sure in time a private server for HD 2 will appear if arrowhead don't release one themselves :)
- deleted 10mo ago[deleted]
- rawling 10mo agohttps://news.ycombinator.com/item?id=46134178 https://news.ycombinator.com/item?id=46134178 282 comments
- HelloUsername 10mo agohttps://news.ycombinator.com/item?id=46123009 https://news.ycombinator.com/item?id=46123009
- _aavaa_ 10mo agoMy takeaway is that it seems like they did NO benchmarking of their own before choosing to do all that duplication. They only talk about performance tradeoff now that they are removing it. Wild
- maccard 10mo agoI've been involved in decisions like this that seem stupid and obvious. There's a million different things that could/should be fixed, and unless you're monitoring this proactively you're unlikely to know it hsould be changed. I'm not an arrowhead employee, but my guess is at some point in the past, they benchmarked it, got a result, and went with it. And that's about all there is to it.
- Xelbair 10mo ago>These loading time projections were based on industry data - comparing the loading times between SSD and HDD users where data duplication was and was not used. In the worst cases, a 5x difference was reported between instances that used duplication and those that did not. We were being very conservative and doubled that projection again to account for unknown unknowns. >We now know that, contrary to most games, the majority of the loading time in HELLDIVERS 2 is due to level-generation rather than asset loading. This level generation happens in parallel with loading assets from the disk and so is the main determining factor of the loading time. We now know that this is true even for users with mechanical HDDs. they did absolutely zero benchmarking beforehand, just went with industry haresay, and decided to double it just in case.
- everdrive 10mo agoI love Helldivers 2, but from what I can tell it's a bunch of enthusiasts using a relatively broken engine to try to do cool stuff. It almost reminds me of the first pokemon game. I'll bet there's all sorts of stuff they get wrong from a strictly technical standpoint. I love the game so much I see this more as a charming quirk than I do something which really deserves criticism. The team never really expected their game to be as popular as it's become, and I think we're still inheriting flaws from the surprise interest in the game. (some of this plays out in the tug of war between the dev team's hopes for a realistic grunt fantasy vs. and the player base's horde power fantasy.)
- rincebrain 10mo agoA lot of things suddenly made sense when I learned their prior work was Magicka.
- brainzap 10mo agooh no
- jfindper 10mo agoI never played Magicka, but the reviews seem fine (76% GameRankings, 74/100 Metacritic, 8/10 EuroGamer, etc.) Was it a bad game? Or jankey? What parts of Helldivers are "making sense" now?
- darthcircuit 10mo agoNot op, but magicka is a pretty fun game. You cast spells in a similar way as calling in strategems in hd2. The spell system was super neat. There’s several different elements (fire, air, water, earth, electricity, ice, ands maybe something else. It’s been a while since I played). Each element can be used on its own or is combinable. Different combinations would cast different spells. Fire+water makes steam for instance. Ice + air is a focused blizzard, etc. there’s hundreds to learn and that’s your main weapon in the game. There’s even a spell you can cast that will randomly kick someone you’re playing with out of the game. It’s great fun with friends, but can be annoying to play sometimes. If you try it, go with kb/m. It supports controller, but is way more difficult to build the spells.
- dnrvs 10mo agotoo many arm chair game devs who think they know better in this thread
- lordnikon001 10mo agoI think what irks people is the number one rule of optimization is to always measure You never assume something is an optimization or needed and never do hypothetical optimizations I can see why it would happen in this case though, gamedev is chaotic and you're often really pressed for time
- forrestthewoods 10mo agoWebDevs who have build systems that take ten minutes and download tens of megabytes of JS and have hundreds of milliseconds of lag are sooooooooooooo not allowed to complain about game devs ever.
- Dylan16807 10mo agoOh, at first I thought you were talking about websites doing that and I was going to say "sure, those people can't complain, but the rest of us can". Then I realized you said build systems and eh, whatever. It's not good for build systems to be bloated, but it matters a lot less than the end product being bloated. And you seem to be complaining about the people that are dealing with these build systems themselves, not inflicting them on other people? Why don't they get to complain?
- forrestthewoods 10mo agoDownload bloat is net less impactful than build time bloat imho. Game download and install size bloat is bad. But is a mostly one time cost. Build time bloat doesn’t directly impact users, but iteration time is GodKing so bad build times indirectly hurt consumers. But that’s all beside the point. What I was really doing was criticizing the <waves hands wildly> HN commenters. HN posters are mostly webdevs because most modern programmers are webdevs. And while I won’t say the file bloat here wasn’t silly, I wonder stand for game dev slander from devs that commit faaaaaaaaaaaaaar greater sins.
- renewiltord 10mo agoPretty cool. I think it’s completely normal to be under a crunch and just go with some standard practices under normal conditions. Cool that they went back and sorted it out afterwards! I’ve got to say. I do find it somewhat unusual that despite the fact that every HN engineer has John Carmack level focus on craftsmanship, about 1/100k here produce that kind of outcome. I don’t get it. All of you guys are good at pointing out how to do good engineering. Why don’t you make good things?
- mort96 10mo agoThe negativity towards this is wild. A company followed relatively widely accepted industry practice (lots and lots of other games also have huge sizes on disk for the exact same reason), then eventually they decided to do their own independent testing to check whether said common practice actually makes things better or not in their case, found that it didn't, so they reversed it. In addition, they wrote up some nice technical articles on the topic, helping to change the old accepted industry wisdom. This seems great to me. Am I crazy? This feels like it should be Hacker News's bread and butter, articles about "we moved away from Kubernetes/microservices/node.js/serverless/React because we did our own investigation and found that the upsides aren't worth the downsides" tend to do really well here. How is this received so differently?
- scsh 10mo agoIt's because shitting on game devs is the trendy thing these days, even among more technically inclined crowds unfortunately. It seems like there's a general unwillingness to accept that game development is hard and you can't just wave the magic "optimize" wand at everything when your large project is also a world of edge cases. But it seems like it should be that simple according to all the armchair game devs on the internet.
- taeric 10mo agoThere has long been a trend that "software engineers" and "computer scientists" both have been rather uninterested in learning the strategies that gaming developers use. Really, the different factions in software development are a fascinating topic to explore. Add embedded to the discussion, and you could probably start fights in ways that flat out don't make sense.
- buildbot 10mo agoThe level of work that goes into even “small” games is pretty incredible. When I was a grad student another student was doing their (thesis based, research focused) masters while working at EA on a streetfighter(?) game. The game programming was actually just as research focused and involved as the actual research. They were trying to figure out how to get the lowest latency and consistency for impact sounds.
- ok_coo 10mo agoI'm glad they've been able to do this, looks like a huge improvement for HD2 on PC. I've been on PS5 since launch and aside from Baldur's Gate 3, it's been the best game this gen IMO. The negativity I see towards the game (especially on Youtube) is weird. Some of the critiques seem legit but a lot of feels like rage bait, which appears to be a lot of YT videos around gaming lately. Anyway, a big improvement for a great game. Seems like less of an incentive now to uninstall if you only play now and then. iO
- doener 10mo ago[dupe] https://news.ycombinator.com/item?id=46134178 https://news.ycombinator.com/item?id=46134178
- sergiotapia 10mo agoIf this article was exciting for you, I also highly recommend this one. A random dude fixed a bug in GTA 5 that was the root cause of it loading insanely slowly since the game came out! https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times-by-70/ https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times...
- oceansky 10mo agoThe write up of how Windows 11 24H2 broke GTA San Andreas was excellent. https://cookieplmonster.github.io/2025/04/23/gta-san-andreas-win11-24h2-bug/ https://cookieplmonster.github.io/2025/04/23/gta-san-andreas...
- roflchoppa 10mo agoDo HDDs make up that much of the gaming market segment?
- aidenn0 10mo ago1/9th of Helldivers 2 players, per TFA.
- forrestthewoods 10mo agoMoral of the Story: don’t roll out a fix like this all at once. Do it over 6 months over several patches. Keep finding “new improvements”. Just don’t get caught at the end!
- tlonny 10mo agoIf this is somewhat common for games, could one create a virtual fs with FUSE that dedupes using via content-defined chunking and install games there? I feel like writes would probably be quite painful, but with game assets are essentially write-once read-forever so not the end of the world? As an aside, its messed up that people with expensive SSDs are unnecessarily paying this storage tax. Just feels lazy...