7 ms·

Zircon Fair Scheduler

117 points by return_0e 7y ago

dpflan 7y ago
This post has sky-rocketed to the top! I'm genuinely curious: can someone explain what's cool/interesting/important about this (maybe EL5)? Thanks!
- terryschiavo22 7y ago
  Well, Google intends for this new OS to replace Android. They'll need to convince the public that this new OS is somehow better than Android, which everyone has come to know and love. It seems that they believe the best way to do that is a grassroots approach beginning with tech discussion hubs like HN and Reddit. Of course, they could have just meme'd hard about the fact that they're moving away from evil Oracle technology and we'd have already all been on board.
  - ASinclair 7y ago
    > Well, Google intends for this new OS to replace Android. Can you point me to anything that demonstrates that intention?
    - davidb_ 7y ago
      Currently it's just rumors that it will replace Android based on Fushia and Flutter's target devices. You can see a summary on [the wikipedia page](https://en.wikipedia.org/wiki/Google_Fuchsia https://en.wikipedia.org/wiki/Google_Fuchsia).
    - terryschiavo22 7y ago
      https://www.tomsguide.com/us/google-fuchsia-os-replace-android,news-28966.html https://www.tomsguide.com/us/google-fuchsia-os-replace-andro...
  - pjmlp 7y ago
    It is very easy to convince the public, OEMs are the ones that need to be convinced. ART is already being ported into Fuchsia and Linuxisms are not part of NDK stable APIs. So managed Android apps will just work, and NDK libraries only need to be recompiled.
    - endorphone 7y ago
      In that case it wouldn't have replaced Android at all -- It would have simply replaced the Android kernel, which happens to be Linux. And at that point you have to ask what is gained, and at this point the answer is "nothing".
      - monocasa 7y ago
        Well a change of the kernel's license from GPLv2 to BSD is certainly a change. I'm sure the OEMs think that gains them something.
        endorphone 7y ago
        Android OEMs are not in the business of writing kernels, though, and the changes they do are minimal. And their HAL/chipset code -- the thing they might actually care about as IP -- is not governed by the GPL at all, nor is any of the enormous volume of system and userspace code they write. It's a neat initiative and might yield something interesting, but if the Linux kernel was replaced by Fuscia the ramifications are seemingly very minor. Android's many issues have never been at the kernel level.
        monocasa 7y ago
        The components of their HAL/chipset code in the kernel are most of the time governed by the GPL. We can take some examples apart of need be.
        pjmlp 7y ago
        After Project Treble that only applies to the legacy HAL code.
  - endorphone 7y ago
    Nothing, at all, has demonstrated that they plan it to replace Android -- that was a narrative various tech blogs invented. Nor would it have any benefit in moving away from the Java inspired/cloned underpinnings of the Android user layer. Google has a variety of initiatives, and they really like reinventing things (which can sometimes yield great outcomes). This is a kernel that is in contrast with Linux.
    - terryschiavo22 7y ago
      https://www.tomsguide.com/us/google-fuchsia-os-replace-android,news-28966.html https://www.tomsguide.com/us/google-fuchsia-os-replace-andro... From the article, the OS allows for full compatibility with all Android apps. Furthermore, it notes that Google is going out of its way to avoid mentioning Android anymore. Of course, if I were Google and trying to sell the public on my new OS, I'd want them all to think that I'm not scrapping the old OS so that they feel they have a choice.
      - endorphone 7y ago
        "the OS allows for full compatibility with all Android apps" The article does not say that. The article mentions an ART target for Fuscia -- you have to build from something. That is approximately 0.1% of the way towards full compatibility. And for that matter you can run Android apps on a load of targets (although far from full compatibility), but that doesn't mean that they're replacing Android. Google may absolutely replace Android -- they've made loads and loads of mistakes along the way -- but the way people keep arguing it doesn't make sense, using examples that jettison the parts that work well and somehow keep the parts that don't work well (which includes ART, as an aside). And indeed I'm falling into this same trap while talking about Fuscia like it's a kernel, when really the kernel is a small part and they're, at a very small scale, spit-boarding a new take on virtually every part of the system. Regarding Google distancing itself from the Android name, that's just branding. To quote one analysis -- "Android sounds technical, has baggage, and might be stale". They've had enough missteps that it's an anchor more than a lift, so it makes sense that they stop highlighting it.
  - monocasa 7y ago
    Well, they're not moving away from Android per se. ART is being ported to run on fuschia.
  - shereadsthenews 7y ago
    > Android, which everyone has come to know and love. Haha, honestly now. If my Android didn't cost $700 I would long since have smashed it to bits. It's scheduler is totally garbage, to the point where Google's own media apps like YouTube and Music drop samples while the screen is redrawing. Who "loves" Android? To me it is the Win98 of mobile operating systems.
    - ubercore 7y ago
      That's a turn of phrase to indicate that not everyone loves, but it _is_ well known and probably the biggest consumer OS on the planet in terms of volume. Also though, I do love it. Different strokes and all
    - linuxftw 7y ago
      Android has been pretty solid for me. I'm on my second Android device, this one cost me like $150 or so. I've never had a higher-end phone, so I'm not missing anything with the mid-range. Web works great, as do most apps.
    - stronglikedan 7y ago
      When you only have two real choices, each with their own significant set of distinct problems, I think the term "love" can be substituted for "hate the least". Android, which everyone has come to know and hate the least. Well, not everyone, but my point is the same regardless.
      - pjmlp 7y ago
        I left Symbian for Android 2.1, after several Android devices, became part of the 10% of WP share in Europe, now still using one of them as secondary device.
  - ansible 7y ago
    It is sort of replacing Android, but not really. But it sort of is. The kernel is designed to have a stable binary interface for drivers. This has been a problem with Linux-based Android devices (all of them so far), because the OEMs (or more properly the chipset vendors like Qualcomm) will only support a particular chipset for a short amount of time (maybe a couple years, it depends). After that, it becomes hard to bring new kernels to the platform, so we all end up with phones stuck at whatever major release was out at the time, with low prospects of upgrades. If you can make a stable binary API, and furthermore keep to the micro-kernel model, then most of the OS can be easily upgraded, because you don't need (and aren't going to get) new versions of the device drivers for the chipset. Also, there's an effort towards better low-latency real-time support. This is critical for AR/VR applications with tight rendering deadlines.
- pjmlp 7y ago
  Depending on how internal politics play out at Google, Android's kernel might eventually be replaced by Fuchsia (ART is already being ported, commits are visible on AOSP). And Fuchsia is mikrokernel based.
- jitl 7y ago
  I don’t care about the android aspects. I’m just fascinated by OS development. Fuschia is a new capabilities-secure microckernel OS backed by Big Google, so it’s got a lot of potential and engineering support behind it. I read and upvote pretty much anything about Fuschia.
- rhizome 7y ago
  tl;dr: Good question, it's not answered anywhere in the family of repositories that Zircon touches.
VanillaCafe 7y ago
> A NOTE ABOUT DEADLINES: While fair scheduling is appropriate for the vast majority of workloads, there are some tasks that require very specific timing and/or do not adapt well to overload conditions. For example, these workloads include low-latency audio / graphics, high-frequency sensors, and high-rate / low-latency networking. These specialized tasks are better served with a deadline scheduler, which is planned for later in the Zircon scheduler development cycle. Those seem like important workloads. Does this imply that the deadline scheduler runs concurrently with the fair scheduler? Otherwise, what's the point of developing an ideal scheduler for common workloads if it cannot be used for critical workloads. Is it common to run two different schedulers in the same system?
- colechristensen 7y ago
  Different scheduling algorithms and implementations have tradeoffs. Pick one based on your workload. No need to have only one for all applications. You can approximate the choice down to two dimensions, latency vs. throughput. Pick your poison.
  - VanillaCafe 7y ago
    For instance, I assume for instance a common workload that would otherwise benefit from the fair scheduler has a fair chance of wanting to do low latency audio. I believe Android has had this issue. As soon as the platform cares about low latency audio it would need to abandon the fair scheduler?
    - colechristensen 7y ago
      Low latency audio would fit for doing live or studio audio production, not necessarily watching videos, listening to music or doing video calls (where there is already network delay an order of magnitude larger than the scheduler would impart). "Low" will have context-dependent meaning. If you're doing low latency audio or flight controls on an aircraft for example, you would absolutely need to abandon the fair scheduler for one that could guarantee the deadlines you need are met. Sacrificing "performance" that is throughput/efficiency/etc of your processor for timing performance. Think of the difference between doing statistics on a billion lines of data and flight controls on a missile. VERY different needs for scheduling, why not have a selectable algorithm?
bepvte 7y ago
https://cchalpha.blogspot.com/2019/03/bmq-scheduler-call-out-for-testing.html?m=1 https://cchalpha.blogspot.com/2019/03/bmq-scheduler-call-out... Someone made a scheduler on linux based on some of the ideas here. Its included in the postfactum (linux-pf) patch set i believe, which might have packages for your distro.
- coreytabaka 7y ago
  No, that's based on the deprecated multi-level round-robin scheduler, which we find tremendously amusing. :)
  - bepvte 7y ago
    Oh gosh well thank you for pointing that out
  - euyyn 7y ago
    Who's "we"?
impostir 7y ago
There are many references to multiple cpu systems throughout the document. Maybe I missed something, but I didn't know Fuschia was aimed at systems like that. I am no expert, but aren't the vast majority of multiple cpu systems servers or high-rnd workstations? If google can supply their own server os, Linux could lose a lot of support and funding
- loudmax 7y ago
  I'm no expert, but when they talk about multiple CPUs I understood them to mean multiple cores, not necessarily multiple CPUs on different motherboard sockets. Even midrange phones today typically have CPUs with multiple cores. Google may have some intention of running Fuschia on servers, but even if you're developing a kernel for pocket mobile devices, you're still going to want to handle multiple cores. Whether this signals Google's intentions to stop supporting Linux, who knows, but there are still a lot of other organizations invested in supporting Linux on servers.
- linuxftw 7y ago
  Hopefully the industry learned it's lesson with Google and it's handling of the Android Open Source Project [1]. I don't think manufacturers of devices can bet their futures on Google. 1: https://arstechnica.com/gadgets/2018/07/googles-iron-grip-on-android-controlling-open-source-by-any-means-necessary/ https://arstechnica.com/gadgets/2018/07/googles-iron-grip-on...
  - geodel 7y ago
    Well Samsung wrote their wonderful OS named Tizen. Not sure why they still sell Android phones. And if company as big as Samsung couldn't do it then lesson for industry would be that writing OS for their hardware is pretty much guaranteed failure.
    - jasonvorhe 7y ago
      Tizen was a security laughing stock, iirc. This is from 2017: https://arstechnica.com/gadgets/2017/04/samsungs-tizen-is-riddled-with-security-flaws-amateurishly-written/ https://arstechnica.com/gadgets/2017/04/samsungs-tizen-is-ri...

117 points by return_0e 7y ago

This post has sky-rocketed to the top! I'm genuinely curious: can someone explain what's cool/interesting/important about this (maybe EL5)? Thanks!

terryschiavo22 7y ago
Well, Google intends for this new OS to replace Android. They'll need to convince the public that this new OS is somehow better than Android, which everyone has come to know and love. It seems that they believe the best way to do that is a grassroots approach beginning with tech discussion hubs like HN and Reddit. Of course, they could have just meme'd hard about the fact that they're moving away from evil Oracle technology and we'd have already all been on board.
- ASinclair 7y ago
  > Well, Google intends for this new OS to replace Android. Can you point me to anything that demonstrates that intention?
  - davidb_ 7y ago
    Currently it's just rumors that it will replace Android based on Fushia and Flutter's target devices. You can see a summary on [the wikipedia page](https://en.wikipedia.org/wiki/Google_Fuchsia https://en.wikipedia.org/wiki/Google_Fuchsia).
  - terryschiavo22 7y ago
    https://www.tomsguide.com/us/google-fuchsia-os-replace-android,news-28966.html https://www.tomsguide.com/us/google-fuchsia-os-replace-andro...
- pjmlp 7y ago
  It is very easy to convince the public, OEMs are the ones that need to be convinced. ART is already being ported into Fuchsia and Linuxisms are not part of NDK stable APIs. So managed Android apps will just work, and NDK libraries only need to be recompiled.
  - endorphone 7y ago
    In that case it wouldn't have replaced Android at all -- It would have simply replaced the Android kernel, which happens to be Linux. And at that point you have to ask what is gained, and at this point the answer is "nothing".
    - monocasa 7y ago
      Well a change of the kernel's license from GPLv2 to BSD is certainly a change. I'm sure the OEMs think that gains them something.
      - endorphone 7y ago
        Android OEMs are not in the business of writing kernels, though, and the changes they do are minimal. And their HAL/chipset code -- the thing they might actually care about as IP -- is not governed by the GPL at all, nor is any of the enormous volume of system and userspace code they write. It's a neat initiative and might yield something interesting, but if the Linux kernel was replaced by Fuscia the ramifications are seemingly very minor. Android's many issues have never been at the kernel level.
        monocasa 7y ago
        The components of their HAL/chipset code in the kernel are most of the time governed by the GPL. We can take some examples apart of need be.
        pjmlp 7y ago
        After Project Treble that only applies to the legacy HAL code.
- endorphone 7y ago
  Nothing, at all, has demonstrated that they plan it to replace Android -- that was a narrative various tech blogs invented. Nor would it have any benefit in moving away from the Java inspired/cloned underpinnings of the Android user layer. Google has a variety of initiatives, and they really like reinventing things (which can sometimes yield great outcomes). This is a kernel that is in contrast with Linux.
  - terryschiavo22 7y ago
    https://www.tomsguide.com/us/google-fuchsia-os-replace-android,news-28966.html https://www.tomsguide.com/us/google-fuchsia-os-replace-andro... From the article, the OS allows for full compatibility with all Android apps. Furthermore, it notes that Google is going out of its way to avoid mentioning Android anymore. Of course, if I were Google and trying to sell the public on my new OS, I'd want them all to think that I'm not scrapping the old OS so that they feel they have a choice.
    - endorphone 7y ago
      "the OS allows for full compatibility with all Android apps" The article does not say that. The article mentions an ART target for Fuscia -- you have to build from something. That is approximately 0.1% of the way towards full compatibility. And for that matter you can run Android apps on a load of targets (although far from full compatibility), but that doesn't mean that they're replacing Android. Google may absolutely replace Android -- they've made loads and loads of mistakes along the way -- but the way people keep arguing it doesn't make sense, using examples that jettison the parts that work well and somehow keep the parts that don't work well (which includes ART, as an aside). And indeed I'm falling into this same trap while talking about Fuscia like it's a kernel, when really the kernel is a small part and they're, at a very small scale, spit-boarding a new take on virtually every part of the system. Regarding Google distancing itself from the Android name, that's just branding. To quote one analysis -- "Android sounds technical, has baggage, and might be stale". They've had enough missteps that it's an anchor more than a lift, so it makes sense that they stop highlighting it.
- monocasa 7y ago
  Well, they're not moving away from Android per se. ART is being ported to run on fuschia.
- shereadsthenews 7y ago
  > Android, which everyone has come to know and love. Haha, honestly now. If my Android didn't cost $700 I would long since have smashed it to bits. It's scheduler is totally garbage, to the point where Google's own media apps like YouTube and Music drop samples while the screen is redrawing. Who "loves" Android? To me it is the Win98 of mobile operating systems.
  - ubercore 7y ago
    That's a turn of phrase to indicate that not everyone loves, but it _is_ well known and probably the biggest consumer OS on the planet in terms of volume. Also though, I do love it. Different strokes and all
  - linuxftw 7y ago
    Android has been pretty solid for me. I'm on my second Android device, this one cost me like $150 or so. I've never had a higher-end phone, so I'm not missing anything with the mid-range. Web works great, as do most apps.
  - stronglikedan 7y ago
    When you only have two real choices, each with their own significant set of distinct problems, I think the term "love" can be substituted for "hate the least". Android, which everyone has come to know and hate the least. Well, not everyone, but my point is the same regardless.
    - pjmlp 7y ago
      I left Symbian for Android 2.1, after several Android devices, became part of the 10% of WP share in Europe, now still using one of them as secondary device.
- ansible 7y ago
  It is sort of replacing Android, but not really. But it sort of is. The kernel is designed to have a stable binary interface for drivers. This has been a problem with Linux-based Android devices (all of them so far), because the OEMs (or more properly the chipset vendors like Qualcomm) will only support a particular chipset for a short amount of time (maybe a couple years, it depends). After that, it becomes hard to bring new kernels to the platform, so we all end up with phones stuck at whatever major release was out at the time, with low prospects of upgrades. If you can make a stable binary API, and furthermore keep to the micro-kernel model, then most of the OS can be easily upgraded, because you don't need (and aren't going to get) new versions of the device drivers for the chipset. Also, there's an effort towards better low-latency real-time support. This is critical for AR/VR applications with tight rendering deadlines.
pjmlp 7y ago
Depending on how internal politics play out at Google, Android's kernel might eventually be replaced by Fuchsia (ART is already being ported, commits are visible on AOSP). And Fuchsia is mikrokernel based.
jitl 7y ago
I don’t care about the android aspects. I’m just fascinated by OS development. Fuschia is a new capabilities-secure microckernel OS backed by Big Google, so it’s got a lot of potential and engineering support behind it. I read and upvote pretty much anything about Fuschia.
rhizome 7y ago
tl;dr: Good question, it's not answered anywhere in the family of repositories that Zircon touches.

VanillaCafe 7y ago

> A NOTE ABOUT DEADLINES: While fair scheduling is appropriate for the vast majority of workloads, there are some tasks that require very specific timing and/or do not adapt well to overload conditions. For example, these workloads include low-latency audio / graphics, high-frequency sensors, and high-rate / low-latency networking. These specialized tasks are better served with a deadline scheduler, which is planned for later in the Zircon scheduler development cycle. Those seem like important workloads. Does this imply that the deadline scheduler runs concurrently with the fair scheduler? Otherwise, what's the point of developing an ideal scheduler for common workloads if it cannot be used for critical workloads. Is it common to run two different schedulers in the same system?

colechristensen 7y ago
Different scheduling algorithms and implementations have tradeoffs. Pick one based on your workload. No need to have only one for all applications. You can approximate the choice down to two dimensions, latency vs. throughput. Pick your poison.
- VanillaCafe 7y ago
  For instance, I assume for instance a common workload that would otherwise benefit from the fair scheduler has a fair chance of wanting to do low latency audio. I believe Android has had this issue. As soon as the platform cares about low latency audio it would need to abandon the fair scheduler?
  - colechristensen 7y ago
    Low latency audio would fit for doing live or studio audio production, not necessarily watching videos, listening to music or doing video calls (where there is already network delay an order of magnitude larger than the scheduler would impart). "Low" will have context-dependent meaning. If you're doing low latency audio or flight controls on an aircraft for example, you would absolutely need to abandon the fair scheduler for one that could guarantee the deadlines you need are met. Sacrificing "performance" that is throughput/efficiency/etc of your processor for timing performance. Think of the difference between doing statistics on a billion lines of data and flight controls on a missile. VERY different needs for scheduling, why not have a selectable algorithm?

bepvte 7y ago

https://cchalpha.blogspot.com/2019/03/bmq-scheduler-call-out-for-testing.html?m=1 https://cchalpha.blogspot.com/2019/03/bmq-scheduler-call-out... Someone made a scheduler on linux based on some of the ideas here. Its included in the postfactum (linux-pf) patch set i believe, which might have packages for your distro.

coreytabaka 7y ago
No, that's based on the deprecated multi-level round-robin scheduler, which we find tremendously amusing. :)
- bepvte 7y ago
  Oh gosh well thank you for pointing that out
- euyyn 7y ago
  Who's "we"?

impostir 7y ago

There are many references to multiple cpu systems throughout the document. Maybe I missed something, but I didn't know Fuschia was aimed at systems like that. I am no expert, but aren't the vast majority of multiple cpu systems servers or high-rnd workstations? If google can supply their own server os, Linux could lose a lot of support and funding

loudmax 7y ago
I'm no expert, but when they talk about multiple CPUs I understood them to mean multiple cores, not necessarily multiple CPUs on different motherboard sockets. Even midrange phones today typically have CPUs with multiple cores. Google may have some intention of running Fuschia on servers, but even if you're developing a kernel for pocket mobile devices, you're still going to want to handle multiple cores. Whether this signals Google's intentions to stop supporting Linux, who knows, but there are still a lot of other organizations invested in supporting Linux on servers.
linuxftw 7y ago
Hopefully the industry learned it's lesson with Google and it's handling of the Android Open Source Project [1]. I don't think manufacturers of devices can bet their futures on Google. 1: https://arstechnica.com/gadgets/2018/07/googles-iron-grip-on-android-controlling-open-source-by-any-means-necessary/ https://arstechnica.com/gadgets/2018/07/googles-iron-grip-on...
- geodel 7y ago
  Well Samsung wrote their wonderful OS named Tizen. Not sure why they still sell Android phones. And if company as big as Samsung couldn't do it then lesson for industry would be that writing OS for their hardware is pretty much guaranteed failure.
  - jasonvorhe 7y ago
    Tizen was a security laughing stock, iirc. This is from 2017: https://arstechnica.com/gadgets/2017/04/samsungs-tizen-is-riddled-with-security-flaws-amateurishly-written/ https://arstechnica.com/gadgets/2017/04/samsungs-tizen-is-ri...