4 ms·

This 12GB update managed to trigger the bizarre Steam behavior on my Linux desktop where the game patching process pegs all cores to 100% and thrashes the disk

This 12GB update managed to trigger the bizarre Steam behavior on my Linux desktop where the game patching process pegs all cores to 100% and thrashes the disk so hard the system eventually stops allowing eg. launching new processes (though the system isn't frozen stiff like running out of RAM - switching Niri desktops is fine, but launching eg. htop hangs forever, and eventually browsers stop responding). After walking away for two hours and coming back to the system still in this state, I gave up and hard-rebooted with the power button. But if you survive the 12GB update process, I'm sure this is great news :) Maybe I'll finally have to make some time to play this game - bought it two years ago, but never ended up making time for it, despite having played Cyberpunk 2077 a time and a half, and most of Factorio: Space Age, since then.

skavi 1y ago
A bit of a tangent, but I’ve seen these issues mentioned before and to me it’s always felt like more the OS’s fault than Steam’s. Like shouldn’t Steam be free to express full utilization of the available resources? And isn’t it the OS’s job to manage QoS? What am I missing here?
- colechristensen 1y ago
  Systems tend to not have particularly strong guardrails against pathological access patterns which aren't trying to use 100% but a large multiple of that or are abusing some subsystem or another. The application is almost always also unresponsive. Putting up those guardrails temporarily hides big problems more often than it avoids needing to have them solved.
  - cyberpunk 1y ago
    BEAM handles this very gracefully. Shame preemptive scheduling isn't more common..
kaztal 1y ago
Update was intense for me too. 12 gb with hotfixes, downloaded after kids had gone to bed. It took about 30 minutes to apply. That was about the allotted time for me.
mattmanser 1y ago
I get that on windows when there's no enough space on my disk to install a whole other copy of the game being patched. So for BG3, if you don't have 150Gb free on your disk, steam will download it on a different disk and then transfer it over, thrashing you disk. It's bizarre, incredibly annoying, behaviour and I wish it would just ask so I'd know that was about to happen and just clean up some space. Or refuse the upgrade. But steam want to force upgrades on users before you can play anything, which for single player games is incredibly frustrating. I get why they do it, but it's another one of those things where you feel like you aren't in control of the thing you paid a lot of money for.
- OtomotO 1y ago
  Can you no longer disable updates on a per game basis? You could do that in the past and I did occasionally for single player games because my internet connection wasn't the best and I did not want to waste the little time I could allocate for gaming.
whs 1y ago
I had that with MHW and I nailed it down to shader (fossilize-replay - https://github.com/ValveSoftware/Fossilize https://github.com/ValveSoftware/Fossilize). From my guess, Steam support Vulkan shader pre-compilation so that you don't have to wait in game (like the infamous 10 min Monster Hunter Wilds startup delay). They also seems to also be able to download the compilation cache from Steam if someone already have done the process on the same GPU + driver version. Since fewer Windows games use Vulkan this feature is often not used, but on Linux most games will run on Vulkan (esp. Proton games with dxvk) you may experience the process more often.
- pretzel5297 1y ago
  Background shader pre-compilation does not use all cores by default and the only way to change that is to manually edit a file. So unless you're consciously changing it, you won't have this problem. It'll only use all cores when you launch the game. I have been having the issue with the system hanging up when steam is doing big writes. I had assumed it was due to something wrong with my drive and was contemplating reformatting it.
- voodooEntity 1y ago
  I have massive doubts about the "They also seems to also be able to download the compilation cache from Steam if someone already have done the process on the same GPU + driver version." This would imply that if I already calculated the shaders for the current game state than i could reuse them and not have to go through the whole compilation step (if no changes happen inbetween). Matter of fact, i have to recompile the shaders on every game start for every game, even if i restart the game just x times in a row. For context: using linux/debian and basically running everything on vulcan
  - izacus 1y ago
    What are your doubts exactly? Shader precompilation is a standard thing to do now - consoles mostly ship precompiled shaders for their GPU + driver combo, Steam Deck will also download precompiled shader for its Linux + AMD + driver version combo. The infrastructure for that Steam side is there and is in active use.
baq 1y ago
The linux kernel’s handling of IO under memory pressure is abysmal. I have to tune dirty ratios and write back ages and swap and whatnot just to get the system to not hard lock when running multiple node microservices in stages which run fine, just slower, when starting them all at once on a MacBook. Disclaimer: I don’t even like macOS.
- hackernudes 1y ago
  I've had similar problems but no amount of tweaking vm and vfs cache settings helped. Swap or not, both 32gb and 128gb of ram. Manually reclaiming memory would un-lock the system (/sys/fs/cgroup/memory.reclaim). I wrote a user space memory reclaimer and have not got a lockup since. https://gist.github.com/EBADBEEF/f168458028f684a91148f4d3e791ba84 https://gist.github.com/EBADBEEF/f168458028f684a91148f4d3e79...
  - deleted 1y ago
    [deleted]
  - baq 1y ago
    I don't know how to give more visibility to this hack, but it deserves it. Bookmarking and deploying on my boxen.
seviu 1y ago
I had to free 100GB so that it has enough disk space. I am amazed this game is even playable on the steam deck. Was trying to find an excuse to play it after cyberpunk. I guess this one it is…
mhitza 1y ago
Are you using full disk encryption (LUKS) without enabling the Cloudflare contributed flags? Because that's the most common syndrome of high IO causing high CPU usage until lockup.
KolibriFly 1y ago
If you survived the warzone of your desktop’s update process, BG3 is absolutely worth diving into
johnbellone 1y ago
That pausing issue is plaguing me with several other titles. I think it tracks with background downloading of game updates, but haven’t had enough hours to entirely confirm it. What I did notice is that after installing Decky there are background jobs from some of the plugins that run native Linux updates (flatpak) and snapshotting.

skavi 1y ago

A bit of a tangent, but I’ve seen these issues mentioned before and to me it’s always felt like more the OS’s fault than Steam’s. Like shouldn’t Steam be free to express full utilization of the available resources? And isn’t it the OS’s job to manage QoS? What am I missing here?

colechristensen 1y ago
Systems tend to not have particularly strong guardrails against pathological access patterns which aren't trying to use 100% but a large multiple of that or are abusing some subsystem or another. The application is almost always also unresponsive. Putting up those guardrails temporarily hides big problems more often than it avoids needing to have them solved.
- cyberpunk 1y ago
  BEAM handles this very gracefully. Shame preemptive scheduling isn't more common..

kaztal 1y ago

Update was intense for me too. 12 gb with hotfixes, downloaded after kids had gone to bed. It took about 30 minutes to apply. That was about the allotted time for me.

mattmanser 1y ago

I get that on windows when there's no enough space on my disk to install a whole other copy of the game being patched. So for BG3, if you don't have 150Gb free on your disk, steam will download it on a different disk and then transfer it over, thrashing you disk. It's bizarre, incredibly annoying, behaviour and I wish it would just ask so I'd know that was about to happen and just clean up some space. Or refuse the upgrade. But steam want to force upgrades on users before you can play anything, which for single player games is incredibly frustrating. I get why they do it, but it's another one of those things where you feel like you aren't in control of the thing you paid a lot of money for.

OtomotO 1y ago
Can you no longer disable updates on a per game basis? You could do that in the past and I did occasionally for single player games because my internet connection wasn't the best and I did not want to waste the little time I could allocate for gaming.

whs 1y ago

I had that with MHW and I nailed it down to shader (fossilize-replay - https://github.com/ValveSoftware/Fossilize https://github.com/ValveSoftware/Fossilize). From my guess, Steam support Vulkan shader pre-compilation so that you don't have to wait in game (like the infamous 10 min Monster Hunter Wilds startup delay). They also seems to also be able to download the compilation cache from Steam if someone already have done the process on the same GPU + driver version. Since fewer Windows games use Vulkan this feature is often not used, but on Linux most games will run on Vulkan (esp. Proton games with dxvk) you may experience the process more often.

pretzel5297 1y ago
Background shader pre-compilation does not use all cores by default and the only way to change that is to manually edit a file. So unless you're consciously changing it, you won't have this problem. It'll only use all cores when you launch the game. I have been having the issue with the system hanging up when steam is doing big writes. I had assumed it was due to something wrong with my drive and was contemplating reformatting it.
voodooEntity 1y ago
I have massive doubts about the "They also seems to also be able to download the compilation cache from Steam if someone already have done the process on the same GPU + driver version." This would imply that if I already calculated the shaders for the current game state than i could reuse them and not have to go through the whole compilation step (if no changes happen inbetween). Matter of fact, i have to recompile the shaders on every game start for every game, even if i restart the game just x times in a row. For context: using linux/debian and basically running everything on vulcan
- izacus 1y ago
  What are your doubts exactly? Shader precompilation is a standard thing to do now - consoles mostly ship precompiled shaders for their GPU + driver combo, Steam Deck will also download precompiled shader for its Linux + AMD + driver version combo. The infrastructure for that Steam side is there and is in active use.

baq 1y ago

The linux kernel’s handling of IO under memory pressure is abysmal. I have to tune dirty ratios and write back ages and swap and whatnot just to get the system to not hard lock when running multiple node microservices in stages which run fine, just slower, when starting them all at once on a MacBook. Disclaimer: I don’t even like macOS.

hackernudes 1y ago
I've had similar problems but no amount of tweaking vm and vfs cache settings helped. Swap or not, both 32gb and 128gb of ram. Manually reclaiming memory would un-lock the system (/sys/fs/cgroup/memory.reclaim). I wrote a user space memory reclaimer and have not got a lockup since. https://gist.github.com/EBADBEEF/f168458028f684a91148f4d3e791ba84 https://gist.github.com/EBADBEEF/f168458028f684a91148f4d3e79...
- deleted 1y ago
  [deleted]
- baq 1y ago
  I don't know how to give more visibility to this hack, but it deserves it. Bookmarking and deploying on my boxen.

seviu 1y ago

I had to free 100GB so that it has enough disk space. I am amazed this game is even playable on the steam deck. Was trying to find an excuse to play it after cyberpunk. I guess this one it is…

mhitza 1y ago

Are you using full disk encryption (LUKS) without enabling the Cloudflare contributed flags? Because that's the most common syndrome of high IO causing high CPU usage until lockup.

KolibriFly 1y ago

If you survived the warzone of your desktop’s update process, BG3 is absolutely worth diving into

johnbellone 1y ago

That pausing issue is plaguing me with several other titles. I think it tracks with background downloading of game updates, but haven’t had enough hours to entirely confirm it. What I did notice is that after installing Decky there are background jobs from some of the plugins that run native Linux updates (flatpak) and snapshotting.