5 ms·

Something I don't like: > The argument that a cooperative scheduling is doomed to fail is overblown. Apps are already very much cooperative. For proof, run a v

by danhau 3y ago

Something I don't like: > The argument that a cooperative scheduling is doomed to fail is overblown. Apps are already very much cooperative. For proof, run a version of that on your nice preemptive system: [...] Might fill your swap and crash unsaved work on other apps. The difference is that in a preemptive system, a `while (true)` might slow down the system, but in a cooperative system, the machine effectively halts. Like for good. If you're caught in a loop and don't yield control back, you're done. In terms of security, this would make denial of service attacks against such systems trivial. You could exploit any bug in any application and it would bubble up to the entire system. Or maybe I'm all wrong. I'm not an OS dev, so please someone correct me.

amelius 3y ago
There's a reason that after the cooperative multitasking experiment in Windows 3.1, the world went with preemptive multitasking.
- jacquesm 3y ago
  And let's not forget the Mac, which was a dead-end until 1997.
- Symmetry 3y ago
  We've had an important change to computers since then in that essentially every processor you might be running an OS on will have multiple threads of execution available. As long as some thread returns to the OS and the OS there does a check for excess resource usage it looks like Fomos's model could be made to work. Except in the case where you have some task that uses more threads than the system has which then all try to grab the same lost mutex or something. But you could always just ensure that the OS has at least one thread at any given time, which could be pretty expensive depending on how many threads are available.
- cmrdporcupine 3y ago
  It is true however that we're in a different world now with multicore processors. Back then preemption was about slicing up the usage of a single sequentially executing core. For very specialized (probably embedded, but maybe other) use cases, I can see the value of specialized operating system which dispenses with time slice scheduling within a core and just assigns waiting tasks to a free core when/if it becomes available. On systems with very high core counts, and with lots of short lived tasks, there could be some value to this. Even back then, on many machines and operating systems it's not like the whole world stopped when a process would not give up time slice. Things like mouse cursors, keyboard buffering, and often even some I/O were able to proceed because the hardware assisted in making that possible. The thing is, I don't think going cooperative simplifies much.. you still have to handle concurrent access to shared resources, and re-entrancy etc. By the time you've made your operating system able to handle that (memory, VM subsystem, I/O, etc.) from multiple cores... you may as well just go implement timeslicing as well.
  - spookie 3y ago
    I'm not sure what the advantages of this would be for something like a renderer for instance. Would synchronization be easier this way? I'm sorry for the disclaimer, but I must admit I really don't know and if someone answers I would like to forward my thanks
    - cmrdporcupine 3y ago
      It's interesting because these days many language runtimes provide their own internal "green threads" / "coroutines" / "async", in-process timeslicing anyways.
Ruddle 3y ago
Author here. Scheduling is a spectrum. Current OS are preemptive, but also a bit cooperative. An app can make the decision of yielding control. The opposite could work: OS is cooperative, unless some threshold of resource usage is triggered (a timer interrupt of instance). It then context switch to enter a, hopefully rare, failure mode, thus turning preemptive. Kill the app, and get back into cooperative mode. Let's call it optimistically cooperative & pessimistically preemptive.
- 0xfedbee 3y ago
  > some threshold of resource usage is triggered (a timer interrupt of instance) That's literally the definition of preemptive scheduling.
  - otabdeveloper4 3y ago
    No idea why you are downvoted for quoting the literal textbook definition.
  - earthboundkid 3y ago
    Yes, but "preemptive" sounds so aggressive. We prefer the term "alternative cooperation".
  - makapuf 3y ago
    It could also be cooperative scheduling with a watchdog with task killing abilities (like we have for memory usage, it's effectively cooperative + OOM)
- jacquesm 3y ago
  If you're serious about this: make it preemptive. Handle interrupts in the kernel and find a nice way to pass them on to user processes. Any kind of prioritization mechanism will require preemption. I've written an experimental OS myself and I know this is hard stuff but if you can't crack that this is likely to be DOA. If you just want to play around then that's fine of course, there is nothing to stop you from doing that but if you want to see any kind of external adoption it is more or less a must. Also: this is more like a multi-threaded app itself than an actual OS, for it to be a proper operating system you'd expect at a minimum memory barriers between applications. In that sense the bar for what an OS is has been raised quite a bit since the times of CP/M where it was more of a 'handy library to do some I/O for me' (on mainframes and mini computers there were already proper operating systems back then but on micro computers we had to wait until OS/9 to have something similar).
aseipp 3y ago
You're completely right, the argument is a red herring. People did not move away from cooperative multitasking for operating systems because it solved every "run-away usage of resource" problem for all time. They did it specifically because, as you note, it makes incredibly simple application-level errors, whether they are design-level (UI thread blocked) or simple errors (accidental infinite loop) totally devastating to the entire state of the system. This is pretty bad whenever your system is designed to run arbitrary computer programs.
treis 3y ago
I'm not an OS dev either but I think core count makes a difference here. In that a while(true) loop brings down your single core system but doesn't for a multi-core system. Could see the tradeoffs being different today than back when the fundamentals of the OSes we use today were built.
- LeFantome 3y ago
  This is a reasonable point. With containerization and the cloud, this abstraction is taken even further. In the end though, all this really does is perhaps change how aggressive a pre-emptive system needs to be. It could wait longer to pre-empt. Fundamentally though, the design would still need to be pre-emptive.
  - treis 3y ago
    I don't think it needs to be system pre-emptable. If we reserve 1 core for the OS then the user can always interact with the system. The OS can tell them "App X has pegged the 15 userland cores. Do you wish to kill it?". In practice, I'm not sure how relevant pre-empting is any more. Software is much better behaved than it used to and I rarely see these runaway processes anymore. And when they do it's usually pegging a single core at 100% while other processes share the remaining core.
gnulinux 3y ago
It's because you treat it as if it's all or nothing. You can effectively allow apps to cooperatively schedule themselves, but also make sure `while(true){}` doesn't infinitely loop the system. You can e.g. timeout each app, or if you feel very experimental, avant-garde etc, you could try detecting loops, and put a very generous timeout (since you'll miss some loops like `i=0;while(true){i+=1}` depending on the algo you use to detect loops). There are tons of other schemes, algorithms. Most of these ideas are implemented somewhere, but 99.9% of computer users using linux/OSX/Windows don't experience them. For the programmer, it's most convenient when the isolation between unrelated programs is maximal but isolation between related programs is minimal; but for the user it's the most convenient when programs are maximally isolated in terms of computation, but not for other things such as storage/permissions etc. Practically, this requires a complex scheduling mechanism that is more preemptive than cooperative, but also cooperative a little. E.g. in linux, it's not particularly hard for a rogue program to bring the system to halt with something like a fork bomb (especially if swap is turned on). Even if you have a preemptive scheduler, if your program has 99% of the threads running in the system, it's effectively cooperative because chances are you're scheduled most of the time anyway.
- DSMan195276 3y ago
  The problem is that it really _is_ an all or nothing, there are assumptions that cooperative programs will make that break if they can be preempted at any point. Even having a "little" preemption means programs have to be written to assume it can happen at any time (unless preemption means killing the program entirely).
  - getpokedagain 3y ago
    I get what you are saying but does every operating system have the goal of allowing unpredictable multiple user applications running? A cooperative system can easily be correct for something well managed like an IOT device with only certain tasks or something somewhat general purpose but still more restricted like a video game console.
    - 3y ago
pitust2 3y ago
> Or maybe I'm all wrong. I'm not an OS dev, so please someone correct me. Nope, you are completl right! A while (true) might slow down the system, but even that is not necessarily the case: I wrote a program to test this (based on the pseudocode in the readme) and my system is totally usable, with basically zero lag! This is the power of a well-written operating system that includes a mysterious concept called priorities. There is actually no discernable different when running the program (other than battery use and temperature going up). In fact, I am running that program as I right this because the difference is so negligable. program: https://paste.sr.ht/~pitust/47dc80ff09243b4bf41a08ae9434a32cc5957a62 https://paste.sr.ht/~pitust/47dc80ff09243b4bf41a08ae9434a32c... > In terms of security, this would make denial of service attacks against such systems trivial Denial of service attacks where you have access to the target system are not super hard to mount AFAIK.
- JoeAltmaier 3y ago
  But understand that modern CPUs have hyperthreading, meaning more than one independent thread of execution. A while(true) on one doesn't affect the other(s). So you won't notice much - other apps keep running. Not because it isn't a resource disaster. But because unless you measure something, you might not notice. Oh! You did measure something. Battery use and temp. There you go. It's a disaster. Some kind of management of such irresponsible threads is definitely a good idea.
  - pitust2 3y ago
    > But understand that modern CPUs have hyperthreading, meaning more than one independent thread of execution. I know. But spinning up threads in an infinite loop is going to get code running on all threads (and indeed, that can be confirmed on `htop`). Also, not all CPUs have hyperthreading: the apple M1 chip (which is indeed my CPU) does not have support for that. > Oh! You did measure something. Battery use and temp. So I didn't do any scientific measurements, but the temperature difference doesn't seem to be very significant. > It's a disaster. But cooperative scheduling isn't any better. In fact, it's even worse! Since if an app doesn't yield (and let's face it, no app is perfect), you can easily get a deadlock. Developing for such a system without a VM sounds like a nightmare too to be honest. > Some kind of management of such irresponsible threads is definitely a good idea. What kind of managment do you propose? I mean, if there is nothing else running on the system, it's probably okay to just let them keep running (it's not like they are harming anything except battery life, and you might want something that can max all cpu cores, like if you are running a compilation or a video game).

by danhau 3y ago

amelius 3y ago

There's a reason that after the cooperative multitasking experiment in Windows 3.1, the world went with preemptive multitasking.

jacquesm 3y ago
And let's not forget the Mac, which was a dead-end until 1997.
Symmetry 3y ago
We've had an important change to computers since then in that essentially every processor you might be running an OS on will have multiple threads of execution available. As long as some thread returns to the OS and the OS there does a check for excess resource usage it looks like Fomos's model could be made to work. Except in the case where you have some task that uses more threads than the system has which then all try to grab the same lost mutex or something. But you could always just ensure that the OS has at least one thread at any given time, which could be pretty expensive depending on how many threads are available.
cmrdporcupine 3y ago
It is true however that we're in a different world now with multicore processors. Back then preemption was about slicing up the usage of a single sequentially executing core. For very specialized (probably embedded, but maybe other) use cases, I can see the value of specialized operating system which dispenses with time slice scheduling within a core and just assigns waiting tasks to a free core when/if it becomes available. On systems with very high core counts, and with lots of short lived tasks, there could be some value to this. Even back then, on many machines and operating systems it's not like the whole world stopped when a process would not give up time slice. Things like mouse cursors, keyboard buffering, and often even some I/O were able to proceed because the hardware assisted in making that possible. The thing is, I don't think going cooperative simplifies much.. you still have to handle concurrent access to shared resources, and re-entrancy etc. By the time you've made your operating system able to handle that (memory, VM subsystem, I/O, etc.) from multiple cores... you may as well just go implement timeslicing as well.
- spookie 3y ago
  I'm not sure what the advantages of this would be for something like a renderer for instance. Would synchronization be easier this way? I'm sorry for the disclaimer, but I must admit I really don't know and if someone answers I would like to forward my thanks
  - cmrdporcupine 3y ago
    It's interesting because these days many language runtimes provide their own internal "green threads" / "coroutines" / "async", in-process timeslicing anyways.

Ruddle 3y ago

Author here. Scheduling is a spectrum. Current OS are preemptive, but also a bit cooperative. An app can make the decision of yielding control. The opposite could work: OS is cooperative, unless some threshold of resource usage is triggered (a timer interrupt of instance). It then context switch to enter a, hopefully rare, failure mode, thus turning preemptive. Kill the app, and get back into cooperative mode. Let's call it optimistically cooperative & pessimistically preemptive.

0xfedbee 3y ago
> some threshold of resource usage is triggered (a timer interrupt of instance) That's literally the definition of preemptive scheduling.
- otabdeveloper4 3y ago
  No idea why you are downvoted for quoting the literal textbook definition.
- earthboundkid 3y ago
  Yes, but "preemptive" sounds so aggressive. We prefer the term "alternative cooperation".
- makapuf 3y ago
  It could also be cooperative scheduling with a watchdog with task killing abilities (like we have for memory usage, it's effectively cooperative + OOM)
jacquesm 3y ago
If you're serious about this: make it preemptive. Handle interrupts in the kernel and find a nice way to pass them on to user processes. Any kind of prioritization mechanism will require preemption. I've written an experimental OS myself and I know this is hard stuff but if you can't crack that this is likely to be DOA. If you just want to play around then that's fine of course, there is nothing to stop you from doing that but if you want to see any kind of external adoption it is more or less a must. Also: this is more like a multi-threaded app itself than an actual OS, for it to be a proper operating system you'd expect at a minimum memory barriers between applications. In that sense the bar for what an OS is has been raised quite a bit since the times of CP/M where it was more of a 'handy library to do some I/O for me' (on mainframes and mini computers there were already proper operating systems back then but on micro computers we had to wait until OS/9 to have something similar).

aseipp 3y ago

You're completely right, the argument is a red herring. People did not move away from cooperative multitasking for operating systems because it solved every "run-away usage of resource" problem for all time. They did it specifically because, as you note, it makes incredibly simple application-level errors, whether they are design-level (UI thread blocked) or simple errors (accidental infinite loop) totally devastating to the entire state of the system. This is pretty bad whenever your system is designed to run arbitrary computer programs.

treis 3y ago

I'm not an OS dev either but I think core count makes a difference here. In that a while(true) loop brings down your single core system but doesn't for a multi-core system. Could see the tradeoffs being different today than back when the fundamentals of the OSes we use today were built.

LeFantome 3y ago
This is a reasonable point. With containerization and the cloud, this abstraction is taken even further. In the end though, all this really does is perhaps change how aggressive a pre-emptive system needs to be. It could wait longer to pre-empt. Fundamentally though, the design would still need to be pre-emptive.
- treis 3y ago
  I don't think it needs to be system pre-emptable. If we reserve 1 core for the OS then the user can always interact with the system. The OS can tell them "App X has pegged the 15 userland cores. Do you wish to kill it?". In practice, I'm not sure how relevant pre-empting is any more. Software is much better behaved than it used to and I rarely see these runaway processes anymore. And when they do it's usually pegging a single core at 100% while other processes share the remaining core.

gnulinux 3y ago

It's because you treat it as if it's all or nothing. You can effectively allow apps to cooperatively schedule themselves, but also make sure `while(true){}` doesn't infinitely loop the system. You can e.g. timeout each app, or if you feel very experimental, avant-garde etc, you could try detecting loops, and put a very generous timeout (since you'll miss some loops like `i=0;while(true){i+=1}` depending on the algo you use to detect loops). There are tons of other schemes, algorithms. Most of these ideas are implemented somewhere, but 99.9% of computer users using linux/OSX/Windows don't experience them. For the programmer, it's most convenient when the isolation between unrelated programs is maximal but isolation between related programs is minimal; but for the user it's the most convenient when programs are maximally isolated in terms of computation, but not for other things such as storage/permissions etc. Practically, this requires a complex scheduling mechanism that is more preemptive than cooperative, but also cooperative a little. E.g. in linux, it's not particularly hard for a rogue program to bring the system to halt with something like a fork bomb (especially if swap is turned on). Even if you have a preemptive scheduler, if your program has 99% of the threads running in the system, it's effectively cooperative because chances are you're scheduled most of the time anyway.

DSMan195276 3y ago
The problem is that it really _is_ an all or nothing, there are assumptions that cooperative programs will make that break if they can be preempted at any point. Even having a "little" preemption means programs have to be written to assume it can happen at any time (unless preemption means killing the program entirely).
- getpokedagain 3y ago
  I get what you are saying but does every operating system have the goal of allowing unpredictable multiple user applications running? A cooperative system can easily be correct for something well managed like an IOT device with only certain tasks or something somewhat general purpose but still more restricted like a video game console.
  - 3y ago

pitust2 3y ago

> Or maybe I'm all wrong. I'm not an OS dev, so please someone correct me. Nope, you are completl right! A while (true) might slow down the system, but even that is not necessarily the case: I wrote a program to test this (based on the pseudocode in the readme) and my system is totally usable, with basically zero lag! This is the power of a well-written operating system that includes a mysterious concept called priorities. There is actually no discernable different when running the program (other than battery use and temperature going up). In fact, I am running that program as I right this because the difference is so negligable. program: https://paste.sr.ht/~pitust/47dc80ff09243b4bf41a08ae9434a32cc5957a62 https://paste.sr.ht/~pitust/47dc80ff09243b4bf41a08ae9434a32c... > In terms of security, this would make denial of service attacks against such systems trivial Denial of service attacks where you have access to the target system are not super hard to mount AFAIK.

JoeAltmaier 3y ago
But understand that modern CPUs have hyperthreading, meaning more than one independent thread of execution. A while(true) on one doesn't affect the other(s). So you won't notice much - other apps keep running. Not because it isn't a resource disaster. But because unless you measure something, you might not notice. Oh! You did measure something. Battery use and temp. There you go. It's a disaster. Some kind of management of such irresponsible threads is definitely a good idea.
- pitust2 3y ago
  > But understand that modern CPUs have hyperthreading, meaning more than one independent thread of execution. I know. But spinning up threads in an infinite loop is going to get code running on all threads (and indeed, that can be confirmed on `htop`). Also, not all CPUs have hyperthreading: the apple M1 chip (which is indeed my CPU) does not have support for that. > Oh! You did measure something. Battery use and temp. So I didn't do any scientific measurements, but the temperature difference doesn't seem to be very significant. > It's a disaster. But cooperative scheduling isn't any better. In fact, it's even worse! Since if an app doesn't yield (and let's face it, no app is perfect), you can easily get a deadlock. Developing for such a system without a VM sounds like a nightmare too to be honest. > Some kind of management of such irresponsible threads is definitely a good idea. What kind of managment do you propose? I mean, if there is nothing else running on the system, it's probably okay to just let them keep running (it's not like they are harming anything except battery life, and you might want something that can max all cpu cores, like if you are running a compilation or a video game).