4 ms·

This feels wrong. In that, I'd be surprised if they are not using compression. I think folks are just drastically underestimating how many resources are under

by taeric 4y ago

This feels wrong. In that, I'd be surprised if they are not using compression. I think folks are just drastically underestimating how many resources are under discussion. And how big they are. Heck, just the geometry is probably far larger than folks anticipate.

sudosysgen 4y ago
It's true. I've seen tons of games use basically uncompressed audio, badly compressed textures, etc... People often repack video games, making programs to compress and decompress them so they are easier to pirate. It's not uncommon to see a 2-3x reduction in size, for bit-identical results.
- sroussey 4y ago
  Textures are not compressed for size on disk, they are compressed to the format that can be directly loaded into the GPU.
  - sudosysgen 4y ago
    Yes, that's the problem. You can compress them for size at download time, and recompress them to a GPU-optimal format at the time of installation. You may even be able to do it at runtime with modern hardware, shortening loadtimes and sometimes even improving performance indirectly.
    - coffeebeqn 4y ago
      Then you have to have the game reprocess all the assets on the users machine which could easily take an hour
      - sudosysgen 4y ago
        Essentially any modern image compression format is going to be limited, or close to, by storage speed (for decompression). So unless the user has insanely fast internet (and the servers are also able to saturate their connection), it will be faster in the end.
      - xboxnolifes 4y ago
        With the size of games nowadays, that's likely a net time save for many people from the discounted download time.
        jon-wood 4y ago
        Gamers being gamers though they’re not going to say “well, we on reflection I’ve not had to wait so long”, they’re going to slam you with one star reviews for the fact there’s an hour long loading screen on first start.
        xboxnolifes 4y ago
        The recompression/decompression logic could (and I would argue, should) be packaged separately for distribution platform instead of being embedded as part of the game's logic. In the case of using Steam, it would be no different than how it shows separation of time spent downloading vs time spent installing. This allows it to be ran automatically after download without having the user need to launch the game.
    - lpghatguy 4y ago
      You can combine the approaches! KTX2, for example, supports "supercompressing" GPU block compressed formats with LZ4 or Zstandard. The project I have been working on ships BCn textures compressed with zstd and it has been pretty dang good.
  - terafo 4y ago
    You can add lossless compression, like Kraken, on top of that. PS5 has hardware accelerated Kraken decompression. And you can encode textures for GPU in a way that is better compressable with lossless algorithms.
    - midnightclubbed 4y ago
      Hardware accelerated is still not zero cost. Plus if you are publishing for the other console and/or PC there is a question as to whether you want to do the extra work for one platform.
      - maccard 4y ago
        This is where using something like unreal shines. You get the correct default for all the platforms.
- taeric 4y ago
  That is a little surprising to me. Especially if we are talking about games that are delivered online. Thinking on it, I'm actually only surprised for online delivered games. I don't see any incentive to reduce size on delivered disks. I'd expect that they use a compression that is good for transport, at transport time. I'm not too shocked that they have the assets in a way that is easy to use at runtime.
  - coffeebeqn 4y ago
    Games are duct taped together at last minute on impossible deadlines pretty much every time.
    - bombcar 4y ago
      This is exactly it - and once released there is rarely the motivation to go back and optimize side things. The only game I know that has done anything like that is Factorio, or the open source reimplementations of various engines. The crackers who remove copy protection often repackage (or someone else does) to greatly reduce size; since they’re in there poking around the code they can see what isn’t used. This isn’t new, either, ancient console games have unused assets and even entire levels in the code but disabled.
marcellus23 4y ago
Maybe not for that particular game, but a lot of games have been shown to use uncompressed assets. It's not just speculation.
- taeric 4y ago
  Even for transferring the game to local? I can see some benefits to having uncompressed assets in that it would help with asset loading. I'd still be surprised to not have any form of compression at all happening. I'd also be curious to know just how much the assets could be compressed.
- forrestthewoods 4y ago
  > It's not just speculation. So it’s exactly speculation. A lot of games have done a lot of things for a lot of reasons over the decades. Why are new games so large? Good question. This entire thread is speculative.
  - marcellus23 4y ago
    Um, no. It's not speculation. Many games have been shown to use uncompressed assets, that's an objective fact. The thread might be speculating about _why_ they do that, or whether this particular game does that, but that's a non sequitur, that isn't what I was talking about.
    - forrestthewoods 4y ago
      Your statement that some games in the past have used uncompressed assets is a non-sequitur. Has any game in the past 5 years that has an install size >100gb used uncompressed assets? I would be very very surprised! The likelihood of this being the case on PS5 or XSX hardware is approximately zero. Yes I’m familiar with the infamous uncompressed audio in Titanfall.
      - terafo 4y ago
        Hitman would be very notable example. Fortnite almost got to 100 gigs when they enabled some kind of asset compression.
xaduha 4y ago
> This feels wrong. In that, I'd be surprised if they are not using compression. Check this out, you can sort by the number of submissions to filter out indie games https://docs.google.com/spreadsheets/d/14CVXd6PTIYE9XlNpRsxJUGaoUzhC5titIC1rzQHI4yI/edit?usp=sharing https://docs.google.com/spreadsheets/d/14CVXd6PTIYE9XlNpRsxJ... from https://github.com/IridiumIO/CompactGUI https://github.com/IridiumIO/CompactGUI
- taeric 4y ago
  Thinking more on it, I am not too shocked that they would use uncompressed assets "at rest" on the machine. This is especially true for games mailed out on disks or cartridges. I'm still a little surprised, so the links are neat to read on. I am kind of curious on why they wouldn't use compression in transport, as it were. I'm still mostly assuming that they do.
  - MetaWhirledPeas 4y ago
    Yeah I don't understand why they don't just decompress the audio into RAM as the map loads. Or even just to some swap space.
- kevingadd 4y ago
  This list appears to be largely indie games, many with sizes under a gigabyte. I'm not sure why at-rest compression would be a high priority for a small team when their game fits on a $5 thumbdrive and will be compressed during the install process by Steam or Itch.
  - xaduha 4y ago
    You can sort by the number of submissions to see more popular games.
uoaei 4y ago
The geometry would be hard to compress, but I suspect there's some clever tricks with some kind of fractal compression[1] that could drastically reduce texture sizes for camo, foliage, etc. without any effect on aesthetics. Encoding is the hard part, but only need be done once when building the final release. [1] https://en.wikipedia.org/wiki/Fractal_compression https://en.wikipedia.org/wiki/Fractal_compression
- gatane 4y ago
  Wait, are there more papers on this? Seems interesting.
  - zimpenfish 4y ago
    When I was messing around with this (~1995?), there was a Yuval Fisher paper floating around which did a good job of explaining it. Although it looks like it now costs ~$40 to get that paper or ~$100 for his book. Helpful!
- zimpenfish 4y ago
  > there's some clever tricks with some kind of fractal compression It's a shame that IFS hamstrung the adoption of fractal compression with their expensive license fees - wouldn't necessarily have been The Future Of Images but at least we'd have had the option.

by taeric 4y ago

sudosysgen 4y ago

It's true. I've seen tons of games use basically uncompressed audio, badly compressed textures, etc... People often repack video games, making programs to compress and decompress them so they are easier to pirate. It's not uncommon to see a 2-3x reduction in size, for bit-identical results.

sroussey 4y ago
Textures are not compressed for size on disk, they are compressed to the format that can be directly loaded into the GPU.
- sudosysgen 4y ago
  Yes, that's the problem. You can compress them for size at download time, and recompress them to a GPU-optimal format at the time of installation. You may even be able to do it at runtime with modern hardware, shortening loadtimes and sometimes even improving performance indirectly.
  - coffeebeqn 4y ago
    Then you have to have the game reprocess all the assets on the users machine which could easily take an hour
    - sudosysgen 4y ago
      Essentially any modern image compression format is going to be limited, or close to, by storage speed (for decompression). So unless the user has insanely fast internet (and the servers are also able to saturate their connection), it will be faster in the end.
    - xboxnolifes 4y ago
      With the size of games nowadays, that's likely a net time save for many people from the discounted download time.
      - jon-wood 4y ago
        Gamers being gamers though they’re not going to say “well, we on reflection I’ve not had to wait so long”, they’re going to slam you with one star reviews for the fact there’s an hour long loading screen on first start.
        xboxnolifes 4y ago
        The recompression/decompression logic could (and I would argue, should) be packaged separately for distribution platform instead of being embedded as part of the game's logic. In the case of using Steam, it would be no different than how it shows separation of time spent downloading vs time spent installing. This allows it to be ran automatically after download without having the user need to launch the game.
  - lpghatguy 4y ago
    You can combine the approaches! KTX2, for example, supports "supercompressing" GPU block compressed formats with LZ4 or Zstandard. The project I have been working on ships BCn textures compressed with zstd and it has been pretty dang good.
- terafo 4y ago
  You can add lossless compression, like Kraken, on top of that. PS5 has hardware accelerated Kraken decompression. And you can encode textures for GPU in a way that is better compressable with lossless algorithms.
  - midnightclubbed 4y ago
    Hardware accelerated is still not zero cost. Plus if you are publishing for the other console and/or PC there is a question as to whether you want to do the extra work for one platform.
    - maccard 4y ago
      This is where using something like unreal shines. You get the correct default for all the platforms.
taeric 4y ago
That is a little surprising to me. Especially if we are talking about games that are delivered online. Thinking on it, I'm actually only surprised for online delivered games. I don't see any incentive to reduce size on delivered disks. I'd expect that they use a compression that is good for transport, at transport time. I'm not too shocked that they have the assets in a way that is easy to use at runtime.
- coffeebeqn 4y ago
  Games are duct taped together at last minute on impossible deadlines pretty much every time.
  - bombcar 4y ago
    This is exactly it - and once released there is rarely the motivation to go back and optimize side things. The only game I know that has done anything like that is Factorio, or the open source reimplementations of various engines. The crackers who remove copy protection often repackage (or someone else does) to greatly reduce size; since they’re in there poking around the code they can see what isn’t used. This isn’t new, either, ancient console games have unused assets and even entire levels in the code but disabled.

marcellus23 4y ago

Maybe not for that particular game, but a lot of games have been shown to use uncompressed assets. It's not just speculation.

taeric 4y ago
Even for transferring the game to local? I can see some benefits to having uncompressed assets in that it would help with asset loading. I'd still be surprised to not have any form of compression at all happening. I'd also be curious to know just how much the assets could be compressed.
forrestthewoods 4y ago
> It's not just speculation. So it’s exactly speculation. A lot of games have done a lot of things for a lot of reasons over the decades. Why are new games so large? Good question. This entire thread is speculative.
- marcellus23 4y ago
  Um, no. It's not speculation. Many games have been shown to use uncompressed assets, that's an objective fact. The thread might be speculating about _why_ they do that, or whether this particular game does that, but that's a non sequitur, that isn't what I was talking about.
  - forrestthewoods 4y ago
    Your statement that some games in the past have used uncompressed assets is a non-sequitur. Has any game in the past 5 years that has an install size >100gb used uncompressed assets? I would be very very surprised! The likelihood of this being the case on PS5 or XSX hardware is approximately zero. Yes I’m familiar with the infamous uncompressed audio in Titanfall.
    - terafo 4y ago
      Hitman would be very notable example. Fortnite almost got to 100 gigs when they enabled some kind of asset compression.

xaduha 4y ago

> This feels wrong. In that, I'd be surprised if they are not using compression. Check this out, you can sort by the number of submissions to filter out indie games https://docs.google.com/spreadsheets/d/14CVXd6PTIYE9XlNpRsxJUGaoUzhC5titIC1rzQHI4yI/edit?usp=sharing https://docs.google.com/spreadsheets/d/14CVXd6PTIYE9XlNpRsxJ... from https://github.com/IridiumIO/CompactGUI https://github.com/IridiumIO/CompactGUI

taeric 4y ago
Thinking more on it, I am not too shocked that they would use uncompressed assets "at rest" on the machine. This is especially true for games mailed out on disks or cartridges. I'm still a little surprised, so the links are neat to read on. I am kind of curious on why they wouldn't use compression in transport, as it were. I'm still mostly assuming that they do.
- MetaWhirledPeas 4y ago
  Yeah I don't understand why they don't just decompress the audio into RAM as the map loads. Or even just to some swap space.
kevingadd 4y ago
This list appears to be largely indie games, many with sizes under a gigabyte. I'm not sure why at-rest compression would be a high priority for a small team when their game fits on a $5 thumbdrive and will be compressed during the install process by Steam or Itch.
- xaduha 4y ago
  You can sort by the number of submissions to see more popular games.

uoaei 4y ago

The geometry would be hard to compress, but I suspect there's some clever tricks with some kind of fractal compression[1] that could drastically reduce texture sizes for camo, foliage, etc. without any effect on aesthetics. Encoding is the hard part, but only need be done once when building the final release. [1] https://en.wikipedia.org/wiki/Fractal_compression https://en.wikipedia.org/wiki/Fractal_compression

gatane 4y ago
Wait, are there more papers on this? Seems interesting.
- zimpenfish 4y ago
  When I was messing around with this (~1995?), there was a Yuval Fisher paper floating around which did a good job of explaining it. Although it looks like it now costs ~$40 to get that paper or ~$100 for his book. Helpful!
zimpenfish 4y ago
> there's some clever tricks with some kind of fractal compression It's a shame that IFS hamstrung the adoption of fractal compression with their expensive license fees - wouldn't necessarily have been The Future Of Images but at least we'd have had the option.