6 ms·

All this time I had thought that Rocky Linux was winning the fork war over Alma (in the fight to be the successor to CentOS), but this post might change that wi

All this time I had thought that Rocky Linux was winning the fork war over Alma (in the fight to be the successor to CentOS), but this post might change that with a good chunk of the science community throwing their weight behind Alma. Do we have anyone else in the audience that has any insights over whether Alma is more prevalent over Rocky or is it the other way around? I know I can run Rocky Linux from DigitalOcean which I appreciate.

sliken 4y ago
We picked Rocky Linux because it was the first to get full support from Mellanox/Nvidia.
darthrupert 4y ago
> Alma Having just played through the F.E.A.R. games that name gives me weird feelings.
- themodelplumber 4y ago
  Having been assigned the name in a creepy religious temple ceremony we are in the same boat
- pelagicAustral 4y ago
  I love that game, but the first thing that came to mind was the satellite dish array somewhere in the desert.
- actualwitch 4y ago
  Having just watched through undone series that name also gives me weird feelings.
- spookie 4y ago
  Means soul in Spanish/Portuguese
friendzis 4y ago
> Do we have anyone else in the audience that has any insights over whether Alma is more prevalent over Rocky or is it the other way around? This is wrong reasoning. Distributions live on the shoulders of maintainers/developers. While popularity may be a factor attracting maintainers/developers it is not a deciding factor. If the less popular distribution attracts more maintenance/development effort (again, not necessarily head count) it will win back market share in medium term and once popular distribution will stagnate and fade.
rurban 4y ago
I have insights. Alma is done by professionals for professionals (the hosting industry). Those guys were behind CentOS, and are running 70% of the internet. Rocky is an unpaid community effort.
- dapozza 4y ago
  No. Rocky is done by the guy that started CentOS.
  - dotancohen 4y ago
    Is he paid to develop it? Serious question, my CentOS boxes have been running for years and I'm not looking to upgrade so I'm not up to date.
    - bclemens 4y ago
      Rocky Linux is developed by a large team, some who are paid too and some who are strictly volunteers, but yes, Gregory Kurtzer's own company (https://ciq.co https://ciq.co) has invested quite a bit into Rocky Linux. (Many developers, FIPS validation alone is upwards of a million USD, etc)
    - liamnal 4y ago
      He does not develop it whatsoever. Really all he does is talk, so basically zero development value. He certainly likes to take credit though for all the work all of rocky's volunteers do.
      - bclemens 4y ago
        Yes, Gregory Kurtzer personally helped in development (primarily the packaging / tools during 8.3 and then 8.4). I know, I was there lol. For example, see the commit log to the early set of Rocky Linux devtools: https://github.com/rocky-linux/devtools/commits/main?after=f1a8d8f2b845ccb19f1f44950ea892510fbb40fc+34&branch=main&qualified_name=refs%2Fheads%2Fmain https://github.com/rocky-linux/devtools/commits/main?after=f... "All he does is talk" is unfairly dismissive. We all have our roles, and Greg's is not release engineering. The "taking credit" bit is an unfortunate misconception, media likes to attribute the entire project to gmk since he's a notable personality, but he himself does not.
bclemens 4y ago
Rocky is "winning" if by "winning" you mean more widely used. Here are some graphs charting usage through EPEL statistics: https://rocky-stats.tiuxo.com https://rocky-stats.tiuxo.com. The source used to generate the graphs is available at https://github.com/brianclemens/rocky-stats https://github.com/brianclemens/rocky-stats, however please be kind to the Fedora servers and don't download the stats database too often if you use it. Also Rocky Linux is far larger in terms of community size, etc.
- comfypotato 4y ago
  Wow: those Alma Linux curves look nice. The Rocky Linux trends are definitely more aggressive, but they look a little bit tulip-fever-ish. Also noteworthy is the linear nature of the RH progress. I welcome any further interpretation in a similar (or contradictory) vein to my thoughts.
- KronisLV 4y ago
  > Here are some graphs charting usage through EPEL statistics: https://rocky-stats.tiuxo.com https://rocky-stats.tiuxo.com. As someone who has had to use Oracle Linux for quite a few projects due to requirements, those graphs are actually a sobering look on things. Of course, all of those distros are reasonably similar at the end of the day, but it's pretty clear that the popularity of Rocky Linux, Alma Linux, and even CentOS Stream are all formidable.
- fomine3 4y ago
  Also note that genuine RHEL users generally don't want to use EPEL that is out of support. > The EPEL repository is used by a significant portion of Enterprise Linux users. The data they share are unbiased, and it is reasonable to assume approximately equal proportions of users of each Enterprise Linux distribution use the EPEL repository. From the article but I disagree.
  - carlwgeorge 4y ago
    While it's true that a good number of RHEL users don't use EPEL due to the lack of vendor escalation, there are many that do use it. In fact, they tell Red Hat that specific EPEL packages being available in the next version of EPEL block them from migrating to the next major version of RHEL. The feedback finally got loud enough that Red Hat decided to provide headcount to the Community Platform Engineering (CPE) group to create an EPEL team focused on improving EPEL (this is my team at work). https://communityblog.fedoraproject.org/cpe-to-staff-epel-work/ https://communityblog.fedoraproject.org/cpe-to-staff-epel-wo... That said, the "vendor escalate-able only" group is large enough that RHEL is significantly underrepresented in EPEL stats. My guess is that somewhere around 40-60% of RHEL customers use EPEL. For RHEL rebuilds, I would guess that it's probably in the high 90% range, so their numbers in EPEL countme stats are probably fairly accurate. CentOS Stream on the other hand is also underrepresented, as we have countme data for both EPEL 9 and CentOS Stream 9, which shows there are over twice as many instances without EPEL as instances with EPEL. Keep in mind that the countme data only includes systems connecting directly to Fedora's MirrorManager. Sites that run their own local mirrors are not included. We will never have a complete picture of popularity between these distros. For example, Facebook runs "millions" (a direct quote from their engineers' conference talks, they don't publish exact numbers) of CentOS Stream instances, which is more than everything else in the EPEL countme metrics combined.
rwmj 4y ago
Wouldn't it make sense for Alma & Rocky to now combine their efforts?
axytol 4y ago
It depends. As a counter example think of Scientific Linux [0] which was also an RH clone supported by Fermilab and CERN. It eventually got replaced by this "obscure" distro called CentOS. [0] https://en.m.wikipedia.org/wiki/Scientific_Linux https://en.m.wikipedia.org/wiki/Scientific_Linux
infinet 4y ago
Alma is not exactly prevalent. It however releases faster than Rocky. Alma's mailing list is also more informative. Its Announce list post security advisory frequently. Rocky's Announce list in comparison is quiet.
mhitza 4y ago
While I didn't have many projects recently that focused on EL distros. Last year, I wanted to publish a public AMI built on top of Rocky. However, someone at the time decided to configure the published Rocky Linux AMIs to disallow republishing, so that's the only time I actually picked Alma over Rocky Linux. After the dust settled behind the CentOS model changes, I think there might be a reasonable amount of users that also shifted to CentOS Stream and Fedora linux (Amazon Linux 2022 is also Fedora based instead of RedHat). For the scientific community something that changes way less is obviously a different story.

sliken 4y ago

We picked Rocky Linux because it was the first to get full support from Mellanox/Nvidia.

darthrupert 4y ago

> Alma Having just played through the F.E.A.R. games that name gives me weird feelings.

themodelplumber 4y ago
Having been assigned the name in a creepy religious temple ceremony we are in the same boat
pelagicAustral 4y ago
I love that game, but the first thing that came to mind was the satellite dish array somewhere in the desert.
actualwitch 4y ago
Having just watched through undone series that name also gives me weird feelings.
spookie 4y ago
Means soul in Spanish/Portuguese

friendzis 4y ago

> Do we have anyone else in the audience that has any insights over whether Alma is more prevalent over Rocky or is it the other way around? This is wrong reasoning. Distributions live on the shoulders of maintainers/developers. While popularity may be a factor attracting maintainers/developers it is not a deciding factor. If the less popular distribution attracts more maintenance/development effort (again, not necessarily head count) it will win back market share in medium term and once popular distribution will stagnate and fade.

rurban 4y ago

I have insights. Alma is done by professionals for professionals (the hosting industry). Those guys were behind CentOS, and are running 70% of the internet. Rocky is an unpaid community effort.

dapozza 4y ago
No. Rocky is done by the guy that started CentOS.
- dotancohen 4y ago
  Is he paid to develop it? Serious question, my CentOS boxes have been running for years and I'm not looking to upgrade so I'm not up to date.
  - bclemens 4y ago
    Rocky Linux is developed by a large team, some who are paid too and some who are strictly volunteers, but yes, Gregory Kurtzer's own company (https://ciq.co https://ciq.co) has invested quite a bit into Rocky Linux. (Many developers, FIPS validation alone is upwards of a million USD, etc)
  - liamnal 4y ago
    He does not develop it whatsoever. Really all he does is talk, so basically zero development value. He certainly likes to take credit though for all the work all of rocky's volunteers do.
    - bclemens 4y ago
      Yes, Gregory Kurtzer personally helped in development (primarily the packaging / tools during 8.3 and then 8.4). I know, I was there lol. For example, see the commit log to the early set of Rocky Linux devtools: https://github.com/rocky-linux/devtools/commits/main?after=f1a8d8f2b845ccb19f1f44950ea892510fbb40fc+34&branch=main&qualified_name=refs%2Fheads%2Fmain https://github.com/rocky-linux/devtools/commits/main?after=f... "All he does is talk" is unfairly dismissive. We all have our roles, and Greg's is not release engineering. The "taking credit" bit is an unfortunate misconception, media likes to attribute the entire project to gmk since he's a notable personality, but he himself does not.

bclemens 4y ago

Rocky is "winning" if by "winning" you mean more widely used. Here are some graphs charting usage through EPEL statistics: https://rocky-stats.tiuxo.com https://rocky-stats.tiuxo.com. The source used to generate the graphs is available at https://github.com/brianclemens/rocky-stats https://github.com/brianclemens/rocky-stats, however please be kind to the Fedora servers and don't download the stats database too often if you use it. Also Rocky Linux is far larger in terms of community size, etc.

comfypotato 4y ago
Wow: those Alma Linux curves look nice. The Rocky Linux trends are definitely more aggressive, but they look a little bit tulip-fever-ish. Also noteworthy is the linear nature of the RH progress. I welcome any further interpretation in a similar (or contradictory) vein to my thoughts.
KronisLV 4y ago
> Here are some graphs charting usage through EPEL statistics: https://rocky-stats.tiuxo.com https://rocky-stats.tiuxo.com. As someone who has had to use Oracle Linux for quite a few projects due to requirements, those graphs are actually a sobering look on things. Of course, all of those distros are reasonably similar at the end of the day, but it's pretty clear that the popularity of Rocky Linux, Alma Linux, and even CentOS Stream are all formidable.
fomine3 4y ago
Also note that genuine RHEL users generally don't want to use EPEL that is out of support. > The EPEL repository is used by a significant portion of Enterprise Linux users. The data they share are unbiased, and it is reasonable to assume approximately equal proportions of users of each Enterprise Linux distribution use the EPEL repository. From the article but I disagree.
- carlwgeorge 4y ago
  While it's true that a good number of RHEL users don't use EPEL due to the lack of vendor escalation, there are many that do use it. In fact, they tell Red Hat that specific EPEL packages being available in the next version of EPEL block them from migrating to the next major version of RHEL. The feedback finally got loud enough that Red Hat decided to provide headcount to the Community Platform Engineering (CPE) group to create an EPEL team focused on improving EPEL (this is my team at work). https://communityblog.fedoraproject.org/cpe-to-staff-epel-work/ https://communityblog.fedoraproject.org/cpe-to-staff-epel-wo... That said, the "vendor escalate-able only" group is large enough that RHEL is significantly underrepresented in EPEL stats. My guess is that somewhere around 40-60% of RHEL customers use EPEL. For RHEL rebuilds, I would guess that it's probably in the high 90% range, so their numbers in EPEL countme stats are probably fairly accurate. CentOS Stream on the other hand is also underrepresented, as we have countme data for both EPEL 9 and CentOS Stream 9, which shows there are over twice as many instances without EPEL as instances with EPEL. Keep in mind that the countme data only includes systems connecting directly to Fedora's MirrorManager. Sites that run their own local mirrors are not included. We will never have a complete picture of popularity between these distros. For example, Facebook runs "millions" (a direct quote from their engineers' conference talks, they don't publish exact numbers) of CentOS Stream instances, which is more than everything else in the EPEL countme metrics combined.

rwmj 4y ago

Wouldn't it make sense for Alma & Rocky to now combine their efforts?

axytol 4y ago

It depends. As a counter example think of Scientific Linux [0] which was also an RH clone supported by Fermilab and CERN. It eventually got replaced by this "obscure" distro called CentOS. [0] https://en.m.wikipedia.org/wiki/Scientific_Linux https://en.m.wikipedia.org/wiki/Scientific_Linux

infinet 4y ago

Alma is not exactly prevalent. It however releases faster than Rocky. Alma's mailing list is also more informative. Its Announce list post security advisory frequently. Rocky's Announce list in comparison is quiet.

mhitza 4y ago

While I didn't have many projects recently that focused on EL distros. Last year, I wanted to publish a public AMI built on top of Rocky. However, someone at the time decided to configure the published Rocky Linux AMIs to disallow republishing, so that's the only time I actually picked Alma over Rocky Linux. After the dust settled behind the CentOS model changes, I think there might be a reasonable amount of users that also shifted to CentOS Stream and Fedora linux (Amazon Linux 2022 is also Fedora based instead of RedHat). For the scientific community something that changes way less is obviously a different story.