6 ms·

Factorio – Statistics improvements, Linux adventures

141 points by ibobev 3y ago

milliams 3y ago
When they say they removing their Allegro dependency "eliminated 123,024 lines of code from the game" I assume this is because they'd vendored in the code?
- xboxnolifes 3y ago
  Vendored or not, it's code that ends up as part of the game.
  - db48x 3y ago
    But they probably wouldn’t have bothered to count the lines of code in the library if they hadn’t checked a copy of it into their source code.
- Arnavion 3y ago
  Yes. I believe they also vendor SDL, because they've mentioned in the past making local changes to SDL for their needs. There's also a comment in the Reddit discussion by the author of the Linux about deleting a large number of lines of code when he removed their vendored FreeType, so it's likely the case for Allegro too. https://old.reddit.com/r/factorio/comments/1cdifrh/friday_facts_408_statistics_improvements_linux/l1d48sa/?context=3 https://old.reddit.com/r/factorio/comments/1cdifrh/friday_fa...
- raiguard 3y ago
  Yes, we vendor and statically link as much as possible to avoid dependency hell across distros and platforms.
hoten 3y ago
I'm a heavy user of and minor contributor to the Allegro library. We thought Factorio had got rid of Allegro long ago, but apparently some vestigial component remained. So long!
- philsnow 3y ago
  Allegro has been around for decades. I still remember using it with DJGPP in the mid-90s, which is at least four forevers ago. One thing I remember making was a fire/heat visualization, not optimized at all but that didn’t matter much because I was targeting 320x240x8bpp.
- raiguard 3y ago
  Thanks for making such a great library! It served us well for many years. Edit: My brain skipped over the "user of and minor" part of the sentence. Still, thanks for contributing to Allegro!
lupusreal 3y ago
If they drop X11 support that will finally be the end of my cracktorio addiction.
- db48x 3y ago
  Same here.
- jprete 3y ago
  I don't see anything to indicate that they're dropping X11 support? It sounds like they're dropping dependencies that would require X11 to be installed.
  - db48x 3y ago
    No, it doesn’t sound like they’ll be dropping X11 support any time soon. At least, not before the 2.0 release. It is nevertheless one of those worrisome things.
    - bombcar 3y ago
      They’re dropping a hard dependency on X11 and instead using a library that interfaces with X11 or Wayland for it. It still runs fine on X11.
    - Arnavion 3y ago
      There is a comment in the forum discussion by the author of the Linux section that X11 support is not going away, simply because SDL supports it. https://forums.factorio.com/viewtopic.php?p=609071#p609071 https://forums.factorio.com/viewtopic.php?p=609071#p609071
- earthling8118 3y ago
  It's time to make WaylandX, for running Wayland applications on X. Well, I happily use Wayland so I won't use it, but maybe you would.
  - diath 3y ago
SCUSKU 3y ago
Will never cease to be amazed by the performance that Factorio gets.
nubinetwork 3y ago
> The game has decorations now, but the theme doesn't match. Thanks GNOME! Yeah good luck with that one :)
IAmLiterallyAB 3y ago
Mandatory CSD was always a bad idea imo and this is yet another reason
- lpribis 3y ago
  Sorry, what is CSD? I can't find any reference to this acronym.
  - runiq 3y ago
    Client-side decorations: The title bar, the min-max-close buttons, and the window border.
replyifuagree 3y ago
I covet that guys job big time. However, I'm actually too busy playing factorio to do his job so there's that.
KeplerBoy 3y ago
Using fork to save the game state in the background is really clever. love it.
- bno1 3y ago
  I wonder how they deal with multi-threading. Forking a multi-threaded process is very error-prone (e.g. allocator locks can remain orphaned).
  - xboxnolifes 3y ago
    Is that relevant if the forked process only has to save the game-state to disk and then die? I'd think reading the game-state wouldn't require some mutex lock.
  - gizmo686 3y ago
    The good thing about this usecase is that they do not need to solve the general problem of forking a multithreades program. The child process is doing a very specific job that amounts to serializing the contents of memory and writing it to a file. Once that is done is can simply exit, and all the orphans are cleaned up. While this is admittadly more complicated, it is the same general idea behind fork+exec. Sure, your serialization logic cannot call malloc, but how often do you really need dynamic memory allocation. And if you really do need to, you can use a simple mmap backed bump allocator (or any other custom allocator you want) For datastructure locks protecting gamestate; just wait until the gamestate is in a consistent state before forking. Of course, if you try reusing existing code, or forget about the limitations while writing new code; it would be easy to introduce non-deterministic bugs.
    - nh2 3y ago
      It still seems super error prone: It is very difficult to write general purpose C++ code that does not call malloc somehere. Something fails and you want to display a message `"Error: " + reason`? Bam, malloc right here, serialisation process may hang forever in a malloc lock. You quit the game, the parent process exits, and the serialisation process gets reparented to init, invisibly using up your RAM until you reboot. fork()+exec() works in C because C has no invisible memory allocation, and even there you'd usually try to not call any function in between the two to be very sure. Using fork() without being 100% sure there can be no malloc usually means inviting years of rare, hard-to-reproduce random weird bug reports. Beyond that, as the post mentions, fork() needs "requires a significant amount of RAM to work" if many pages are touched due to copy-on-write, and copy-on-write also slows down the main game. It seems much safer to use a thread for saving the game state.
doesnt_know 3y ago
> Most desktop environments will allow windows to supply their own decorations if they wish but will provide a default implementation on the server side as an alternative. GNOME, in their infinite wisdom, have decided that all clients must provide their own decorations, and if a client does not, they will simply be missing I'm sure there was/is a good reason why this design decision was made, but it certainly seems a bit odd. Imagine if every game had to write decorations for every platform and desktop environment...
- Arnavion 3y ago
  Their reason was that applications want to control everything about how they look, so they don't want compositor-provided decorations that clash with the rest of the application window. Whether that's a *good* reason... Well lots of electrons have been used up in that discussion already. https://gitlab.gnome.org/GNOME/mutter/-/issues/217 https://gitlab.gnome.org/GNOME/mutter/-/issues/217 https://gitlab.gnome.org/GNOME/mutter/-/issues/1143 https://gitlab.gnome.org/GNOME/mutter/-/issues/1143 ... and probably many other places.
  - naikrovek 3y ago
    Ok, so make it optional to use custom window chrome/decorations, rather than make it mandatory. GNOME devs are very arrogant. “Do it our way or go away.” They make it very hard to hold onto any hope that there will ever be a reasonable, cohesive, Linux desktop experience.
    - mksybr 3y ago
      KDE is reasonable and cohesive
      - failbuffer 3y ago
        I really like the hearty amount of configurability they make available thru the control panels. They're not afraid of giving you a lot of options, yet it's organized well, easy to use, and the defaults are sensible.
lucb1e 3y ago
Is this about the upcoming expansion or the base game? I am not reading FFF to avoid spoilers and instead get the full wow effect the first time I play the 2.0 Edit: expansion I guess, since the last 3 releases don't mention statistics (https://forums.factorio.com/viewforum.php?f=3 https://forums.factorio.com/viewforum.php?f=3)
- reaperman 3y ago
  Many of the changes affect the base game as well. Accumulator charge graphs were very overdue.
- raiguard 3y ago
  Everything I talked about here will apply to Factorio 2.0 as a whole, whether or not you have the expansion DLC.
- positr0n 3y ago
  Every FFF since they started doing them again has been about the expansion or base game updates that won't be released until the expansion, so I would continue to not read them :) The way they are doing it is a bunch of engine changes to the base game that will be released as 2.0 for free. Then the actual expansion is implemented as mods. So some of the FFF topics like this are about things that will come to the base game when the expansion launches.
deleted 3y ago
[deleted]
AndrewOMartin 3y ago
For anyone who hasn't played this before, you should know that the references to addiction are not jokes (for many people, at least). I played for a while, and had a blast doing so, but after I had achieved the simplest nominal 'victory' and noticed I was still compelled to disposed of a couple of good weekends towards meeting some self imposed challenged I felt it was time to stop. I didn't have the discipline to just play occasionally, but I did have the discipline to delete the game and all my saves and blueprints one day, and I'm very glad I did. I recommend any player at least consider whether they should do the same. Repeat, I had a great time, the community is great too, but the only way I could sustainably stop myself from turning that into a big waste of time was to delete the game.

141 points by ibobev 3y ago

When they say they removing their Allegro dependency "eliminated 123,024 lines of code from the game" I assume this is because they'd vendored in the code?

xboxnolifes 3y ago
Vendored or not, it's code that ends up as part of the game.
- db48x 3y ago
  But they probably wouldn’t have bothered to count the lines of code in the library if they hadn’t checked a copy of it into their source code.
Arnavion 3y ago
Yes. I believe they also vendor SDL, because they've mentioned in the past making local changes to SDL for their needs. There's also a comment in the Reddit discussion by the author of the Linux about deleting a large number of lines of code when he removed their vendored FreeType, so it's likely the case for Allegro too. https://old.reddit.com/r/factorio/comments/1cdifrh/friday_facts_408_statistics_improvements_linux/l1d48sa/?context=3 https://old.reddit.com/r/factorio/comments/1cdifrh/friday_fa...
raiguard 3y ago
Yes, we vendor and statically link as much as possible to avoid dependency hell across distros and platforms.

hoten 3y ago

I'm a heavy user of and minor contributor to the Allegro library. We thought Factorio had got rid of Allegro long ago, but apparently some vestigial component remained. So long!

philsnow 3y ago
Allegro has been around for decades. I still remember using it with DJGPP in the mid-90s, which is at least four forevers ago. One thing I remember making was a fire/heat visualization, not optimized at all but that didn’t matter much because I was targeting 320x240x8bpp.
raiguard 3y ago
Thanks for making such a great library! It served us well for many years. Edit: My brain skipped over the "user of and minor" part of the sentence. Still, thanks for contributing to Allegro!

lupusreal 3y ago

If they drop X11 support that will finally be the end of my cracktorio addiction.

db48x 3y ago
Same here.
jprete 3y ago
I don't see anything to indicate that they're dropping X11 support? It sounds like they're dropping dependencies that would require X11 to be installed.
- db48x 3y ago
  No, it doesn’t sound like they’ll be dropping X11 support any time soon. At least, not before the 2.0 release. It is nevertheless one of those worrisome things.
  - bombcar 3y ago
    They’re dropping a hard dependency on X11 and instead using a library that interfaces with X11 or Wayland for it. It still runs fine on X11.
  - Arnavion 3y ago
    There is a comment in the forum discussion by the author of the Linux section that X11 support is not going away, simply because SDL supports it. https://forums.factorio.com/viewtopic.php?p=609071#p609071 https://forums.factorio.com/viewtopic.php?p=609071#p609071
earthling8118 3y ago
It's time to make WaylandX, for running Wayland applications on X. Well, I happily use Wayland so I won't use it, but maybe you would.
- diath 3y ago

SCUSKU 3y ago

Will never cease to be amazed by the performance that Factorio gets.

nubinetwork 3y ago

> The game has decorations now, but the theme doesn't match. Thanks GNOME! Yeah good luck with that one :)

IAmLiterallyAB 3y ago

Mandatory CSD was always a bad idea imo and this is yet another reason

lpribis 3y ago
Sorry, what is CSD? I can't find any reference to this acronym.
- runiq 3y ago
  Client-side decorations: The title bar, the min-max-close buttons, and the window border.

replyifuagree 3y ago

I covet that guys job big time. However, I'm actually too busy playing factorio to do his job so there's that.

KeplerBoy 3y ago

Using fork to save the game state in the background is really clever. love it.

bno1 3y ago
I wonder how they deal with multi-threading. Forking a multi-threaded process is very error-prone (e.g. allocator locks can remain orphaned).
- xboxnolifes 3y ago
  Is that relevant if the forked process only has to save the game-state to disk and then die? I'd think reading the game-state wouldn't require some mutex lock.
- gizmo686 3y ago
  The good thing about this usecase is that they do not need to solve the general problem of forking a multithreades program. The child process is doing a very specific job that amounts to serializing the contents of memory and writing it to a file. Once that is done is can simply exit, and all the orphans are cleaned up. While this is admittadly more complicated, it is the same general idea behind fork+exec. Sure, your serialization logic cannot call malloc, but how often do you really need dynamic memory allocation. And if you really do need to, you can use a simple mmap backed bump allocator (or any other custom allocator you want) For datastructure locks protecting gamestate; just wait until the gamestate is in a consistent state before forking. Of course, if you try reusing existing code, or forget about the limitations while writing new code; it would be easy to introduce non-deterministic bugs.
  - nh2 3y ago
    It still seems super error prone: It is very difficult to write general purpose C++ code that does not call malloc somehere. Something fails and you want to display a message `"Error: " + reason`? Bam, malloc right here, serialisation process may hang forever in a malloc lock. You quit the game, the parent process exits, and the serialisation process gets reparented to init, invisibly using up your RAM until you reboot. fork()+exec() works in C because C has no invisible memory allocation, and even there you'd usually try to not call any function in between the two to be very sure. Using fork() without being 100% sure there can be no malloc usually means inviting years of rare, hard-to-reproduce random weird bug reports. Beyond that, as the post mentions, fork() needs "requires a significant amount of RAM to work" if many pages are touched due to copy-on-write, and copy-on-write also slows down the main game. It seems much safer to use a thread for saving the game state.

doesnt_know 3y ago

> Most desktop environments will allow windows to supply their own decorations if they wish but will provide a default implementation on the server side as an alternative. GNOME, in their infinite wisdom, have decided that all clients must provide their own decorations, and if a client does not, they will simply be missing I'm sure there was/is a good reason why this design decision was made, but it certainly seems a bit odd. Imagine if every game had to write decorations for every platform and desktop environment...

Arnavion 3y ago
Their reason was that applications want to control everything about how they look, so they don't want compositor-provided decorations that clash with the rest of the application window. Whether that's a *good* reason... Well lots of electrons have been used up in that discussion already. https://gitlab.gnome.org/GNOME/mutter/-/issues/217 https://gitlab.gnome.org/GNOME/mutter/-/issues/217 https://gitlab.gnome.org/GNOME/mutter/-/issues/1143 https://gitlab.gnome.org/GNOME/mutter/-/issues/1143 ... and probably many other places.
- naikrovek 3y ago
  Ok, so make it optional to use custom window chrome/decorations, rather than make it mandatory. GNOME devs are very arrogant. “Do it our way or go away.” They make it very hard to hold onto any hope that there will ever be a reasonable, cohesive, Linux desktop experience.
  - mksybr 3y ago
    KDE is reasonable and cohesive
    - failbuffer 3y ago
      I really like the hearty amount of configurability they make available thru the control panels. They're not afraid of giving you a lot of options, yet it's organized well, easy to use, and the defaults are sensible.

lucb1e 3y ago

Is this about the upcoming expansion or the base game? I am not reading FFF to avoid spoilers and instead get the full wow effect the first time I play the 2.0 Edit: expansion I guess, since the last 3 releases don't mention statistics (https://forums.factorio.com/viewforum.php?f=3 https://forums.factorio.com/viewforum.php?f=3)

reaperman 3y ago
Many of the changes affect the base game as well. Accumulator charge graphs were very overdue.
raiguard 3y ago
Everything I talked about here will apply to Factorio 2.0 as a whole, whether or not you have the expansion DLC.
positr0n 3y ago
Every FFF since they started doing them again has been about the expansion or base game updates that won't be released until the expansion, so I would continue to not read them :) The way they are doing it is a bunch of engine changes to the base game that will be released as 2.0 for free. Then the actual expansion is implemented as mods. So some of the FFF topics like this are about things that will come to the base game when the expansion launches.

deleted 3y ago

[deleted]

AndrewOMartin 3y ago

For anyone who hasn't played this before, you should know that the references to addiction are not jokes (for many people, at least). I played for a while, and had a blast doing so, but after I had achieved the simplest nominal 'victory' and noticed I was still compelled to disposed of a couple of good weekends towards meeting some self imposed challenged I felt it was time to stop. I didn't have the discipline to just play occasionally, but I did have the discipline to delete the game and all my saves and blueprints one day, and I'm very glad I did. I recommend any player at least consider whether they should do the same. Repeat, I had a great time, the community is great too, but the only way I could sustainably stop myself from turning that into a big waste of time was to delete the game.