2 ms·

I agree with everything you've written here (mostly) but I think you are dodging the argument Linus is making. He isn't (at least in this post) discussing the

I agree with everything you've written here (mostly) but I think you are dodging the argument Linus is making. He isn't (at least in this post) discussing the value of offloading certain processes (indeed, he highlights GPUs and Vision Processing as great places for offloading). Instead, he talks about general processing tasks as not being great targets for parallelism.

vidarh 12y ago
The point is "general processing tasks" tends to include lots of very generic parts that are great to offload. E.g. go back to the 80's and it was not uncommon for loading data to involve having your main CPU load data a bit at a time via a GPIO pin, during which time your application code was blocked until the transfer was complete. Then a byte at time. Then a block at a time. Then suddenly we got DMA. These days we expect other threads to go on executing, and application code that wants to, will expect async background transfer of data to not consume all that much CPU. Here's a contrived example of possible many-wimp-core offloading for you (that would be complicated, but possible): Consider an architecture where many wimpy cores can "hang" on reads from a memory location, and start executing as soon as there's an update. A "smart" version would use a hyper-threading type approach to get better utilization. Congratulations, you know have a "spreadsheet CPU" that automatically handles dependencies (would need some deadlock/loop resolution mechanism) and does recalculation as parallel as possible by the combination of data dependencies and number of cores/threads. It's also incidentally an architecture where caches would be a nightmare, and where the performance of individual cores would not be a big deal. Of course not many of us have spreadsheets where recalculation times is an issue, and it's easy to handle recalculation on a single big CPU too. Where the sweet spot is in terms of power usage, latency and throughput is hard to say, though. (not saying it'd necessarily be a good idea, but I now want to do a test-implementation for my 16-core Parallela's just because). But PC users dismissed the parallelism of the Amiga too. Until Windows 95 had proper multi-tasking and they all had graphics and sound cards. Suddenly they saw the value. I don't think we'll know how far we can push offloading without trying.
- ghshephard 12y ago
  I think the line between "offloading" as opposed to "Breaking up General Purpose task" is clearer than you are making it out to be. IO controllers, Graphics, Sound, are all obvious targets for offloading. Perhaps the grey area (which is a much more specific than things you are talking about) are things like TCP Offloading (TOE) - Linux, currently seems to be opposed to the concept. http://www.linuxfoundation.org/collaborate/workgroups/networking/toe http://www.linuxfoundation.org/collaborate/workgroups/networ...
  - vidarh 12y ago
    Anything that takes time and that can be farmed out without creating lots of contention over access to the same memory is an "obvious target for offloading". Consider that many people were arguing that IO, graphics and sound offloading was totally unnecessary even in the face of seeing what it did for architectures like the Amiga until costs came down for it and CPU speeds remained unable to do the stuff that the offloading made possible. IO in particular seemed pointless to many people: After all you're still going to wait for your file to load, aren't you? But loading data can often be made into a massively parallel task: For starters, you can widen the amount of data transferred with each unit of time. But secondly: you rarely just want to dump your data into memory; you usually wants to do some processing on it (e.g. build up data structures). AmigaOS went further, and demonstrated that there were architectural benefits from even increased OS-level parallelism via multitasking for even basic stuff like terminal handling: One of the reasons the Amigas felt so fast for its time was that the OS pretty consistently traded total throughput for reduced latency by removing blocking throughout. E.g. the terminal/shell windows on the Amiga typically involved half a dozen tasks (Amiga threads/processes - no MMU so not really a distinction) at a minimum: one to handle keyboard and mouse inputs, one "cooking" the raw events into higher level events and responding to low level requests for windowing updates, one handling higher level events and responding with things like cursor movements, the shell itself, one mediating cut and paste requests (which again would involved multiple other tasks to store the cut/copied data to the clipboard device (which would usually be mapped on top of a ram disk, but could be put on any filesystem - potentially involving even more separate tasks)) Many of the "primitives" of that kind of architecture can be offloaded: The process appears largely sequential, but it has numerous points where it's possible to do things in parallel, and more importantly: even sequential operations can be interleaved to a large extent, so you can start processing later events sooner. Many contemporary systems appeared laggy in comparison despite higher throughput because AmigaOS interleaved so many operations by processing smaller subsets of the total processing in small slices in many individual tasks. While you can do that by simply taking the CPU away from a bigger tasks doing everything in parallel, that takes control over the "chunking" away from the developer. I did some work on the AROS (AmigaOS reimplementation) console handling a few years back, and it was amazing how much difference tuning the interactions between those components affected responsiveness (running on the same single core of an x86 box). The limit is whether 1) you can do things faster (reduce latency) - if you can do things faster with offloading, it's a candidate, 2) if your main CPU has other stuff it can do while waiting, if so, you have a candidate. Consider the complex font engines we run these days, for example. Prime candidate for offloading, because it's largely a pipeline: "Put this text here rendered with this font", and we usually render a lot of text with a small set of fonts. We treat it as a sequential task when we usually we can interleave it with other work and just need to be able to have sequencing points where we say "don't return until all the rendering tasks are complete". We can do this with multi-core architectures, but it's hard to do it efficiently without extremely cheap context switches (which are hard to do if you do it as a user-level task running under a memory protected general purpose OS), and we rarely want to dedicate cores of our big expensive CPUs to tasks like that. Have an array of cheap, wimpy cores, and it becomes a different calculation.

vidarh 12y ago

The point is "general processing tasks" tends to include lots of very generic parts that are great to offload. E.g. go back to the 80's and it was not uncommon for loading data to involve having your main CPU load data a bit at a time via a GPIO pin, during which time your application code was blocked until the transfer was complete. Then a byte at time. Then a block at a time. Then suddenly we got DMA. These days we expect other threads to go on executing, and application code that wants to, will expect async background transfer of data to not consume all that much CPU. Here's a contrived example of possible many-wimp-core offloading for you (that would be complicated, but possible): Consider an architecture where many wimpy cores can "hang" on reads from a memory location, and start executing as soon as there's an update. A "smart" version would use a hyper-threading type approach to get better utilization. Congratulations, you know have a "spreadsheet CPU" that automatically handles dependencies (would need some deadlock/loop resolution mechanism) and does recalculation as parallel as possible by the combination of data dependencies and number of cores/threads. It's also incidentally an architecture where caches would be a nightmare, and where the performance of individual cores would not be a big deal. Of course not many of us have spreadsheets where recalculation times is an issue, and it's easy to handle recalculation on a single big CPU too. Where the sweet spot is in terms of power usage, latency and throughput is hard to say, though. (not saying it'd necessarily be a good idea, but I now want to do a test-implementation for my 16-core Parallela's just because). But PC users dismissed the parallelism of the Amiga too. Until Windows 95 had proper multi-tasking and they all had graphics and sound cards. Suddenly they saw the value. I don't think we'll know how far we can push offloading without trying.

ghshephard 12y ago
I think the line between "offloading" as opposed to "Breaking up General Purpose task" is clearer than you are making it out to be. IO controllers, Graphics, Sound, are all obvious targets for offloading. Perhaps the grey area (which is a much more specific than things you are talking about) are things like TCP Offloading (TOE) - Linux, currently seems to be opposed to the concept. http://www.linuxfoundation.org/collaborate/workgroups/networking/toe http://www.linuxfoundation.org/collaborate/workgroups/networ...
- vidarh 12y ago
  Anything that takes time and that can be farmed out without creating lots of contention over access to the same memory is an "obvious target for offloading". Consider that many people were arguing that IO, graphics and sound offloading was totally unnecessary even in the face of seeing what it did for architectures like the Amiga until costs came down for it and CPU speeds remained unable to do the stuff that the offloading made possible. IO in particular seemed pointless to many people: After all you're still going to wait for your file to load, aren't you? But loading data can often be made into a massively parallel task: For starters, you can widen the amount of data transferred with each unit of time. But secondly: you rarely just want to dump your data into memory; you usually wants to do some processing on it (e.g. build up data structures). AmigaOS went further, and demonstrated that there were architectural benefits from even increased OS-level parallelism via multitasking for even basic stuff like terminal handling: One of the reasons the Amigas felt so fast for its time was that the OS pretty consistently traded total throughput for reduced latency by removing blocking throughout. E.g. the terminal/shell windows on the Amiga typically involved half a dozen tasks (Amiga threads/processes - no MMU so not really a distinction) at a minimum: one to handle keyboard and mouse inputs, one "cooking" the raw events into higher level events and responding to low level requests for windowing updates, one handling higher level events and responding with things like cursor movements, the shell itself, one mediating cut and paste requests (which again would involved multiple other tasks to store the cut/copied data to the clipboard device (which would usually be mapped on top of a ram disk, but could be put on any filesystem - potentially involving even more separate tasks)) Many of the "primitives" of that kind of architecture can be offloaded: The process appears largely sequential, but it has numerous points where it's possible to do things in parallel, and more importantly: even sequential operations can be interleaved to a large extent, so you can start processing later events sooner. Many contemporary systems appeared laggy in comparison despite higher throughput because AmigaOS interleaved so many operations by processing smaller subsets of the total processing in small slices in many individual tasks. While you can do that by simply taking the CPU away from a bigger tasks doing everything in parallel, that takes control over the "chunking" away from the developer. I did some work on the AROS (AmigaOS reimplementation) console handling a few years back, and it was amazing how much difference tuning the interactions between those components affected responsiveness (running on the same single core of an x86 box). The limit is whether 1) you can do things faster (reduce latency) - if you can do things faster with offloading, it's a candidate, 2) if your main CPU has other stuff it can do while waiting, if so, you have a candidate. Consider the complex font engines we run these days, for example. Prime candidate for offloading, because it's largely a pipeline: "Put this text here rendered with this font", and we usually render a lot of text with a small set of fonts. We treat it as a sequential task when we usually we can interleave it with other work and just need to be able to have sequencing points where we say "don't return until all the rendering tasks are complete". We can do this with multi-core architectures, but it's hard to do it efficiently without extremely cheap context switches (which are hard to do if you do it as a user-level task running under a memory protected general purpose OS), and we rarely want to dedicate cores of our big expensive CPUs to tasks like that. Have an array of cheap, wimpy cores, and it becomes a different calculation.