3 ms·

I do not believe 'pipelining' and parallelism are interchangeable models and conflating them is a mistake. For example, consider a parallel processing system th

I do not believe 'pipelining' and parallelism are interchangeable models and conflating them is a mistake. For example, consider a parallel processing system that in fact works strictly using a 'pipeline' of length 0, that is there is a hand-of from input to processing stage and processing of that input. And you can have n such parallel processing stages and voila 'parallelism'. Pipelines are strictly processing stages where the 'production of the input' and processing on the inputs are not synchronized. For example, one sends n requests to via a pipeline protocol to a remote server without waiting for acks for each input from the server. There may only be one such processing pipeline (and thus no parallelism) while there is pipelining.

bee_rider 2y ago
I don’t 100% follow your comment, so sorry if this is not quite right. But, I would consider pipelining to be a form or parallelism. It breaks up a task so that parts of it can be run simultaneously (in different stages of the pipeline, simultaneously). There are other ways to do parallelism of course, but it is a way. In your example, if there are multiple pipeline stages in this server, then the tasks should be worked on simultaneously, and so parallelism is occurring. Multi-core, SIMD, and pipelining. Parallelism has multiple dimensions.
- spc476 2y ago
  Not quite. Going back to the bottling example, a bottle has to be filled, capped, then labeled. At time 1, 1 bottle is filling up, pipeline is advanced. At time 2, bottle 2 is filling up, bottle 1 is being capped, pipeline is advanced. At time 3, bottle 3 is filling up, bottle 2 is being capped, and bottle 1 is being labeled. At time 4, bottle 1 is done. Each bottle has to go through the pipeline in sequence, and it takes three units of time for a bottle to go through the pipeline. Yes, once the pipeline is filled up, you have the three operations going on at the same time, but it's still a sequence of steps that need to be performed in order for any given bottle. To make it faster, you either have to decrease the time for a step (but you will always be capped with the slowest step), or go for parallelism---a separate pipeline (or pipelines). For example, with one pipeline, each step taking 1 unit of time, once the pipeline is filled, will take six units of time to make a six-pack (the first will take longer due to the latency in filling the pipeline). You can make five other pipelines, and then get a six-pack per unit of time (again, after filling all the pipelines). A single pipeline just makes the output have a predictable latency and time; multiple pipelines give you parallelism.
  - Joker_vD 2y ago
    > For example, with one pipeline, each step taking 1 unit of time, once the pipeline is filled, will take six units of time to make a six-pack Compared to 18 units of time needed to make a six-pack without pipelining. Gee, what a wondrous invention this "pipeline" is: having three workers means the work is accomplished thrice as fast, yet there is (according to you) no parallelism at all! So naturally, if we could introduce parallelism inside this single pipeline, we would be able to make another triple reduction in time, and get a production of six-pack take only 2 units of time.
  - Jtsummers 2y ago
    > To make it faster, you either have to decrease the time for a step (but you will always be capped with the slowest step), or go for parallelism---a separate pipeline (or pipelines). > A single pipeline just makes the output have a predictable latency and time; multiple pipelines give you parallelism. No, the pipeline does give you parallelism because you're doing three (in the bottling example) pieces of work simultaneously, that is: in parallel. Filling, capping, labeling are each being done on different bottles at the same time. How is that not parallelism? Let's use some numbers: Filling, capping, labeling take 30s, 15s, 15s each (arbitrary, chosen for easy math). Without a pipeline you will process 60 bottles per hour (say one station that does all 3 tasks and then kicks out the bottle, let's ignore transition times). With a pipeline you can get 120 per hour. You've improved throughput. The latency per bottle is still 60s. BTW, you can double throughput again without even needing a full second pipeline just by adding a second filling station (with my example numbers) and feeding filled bottles to the same capping and labeling stages, getting you to 240 per hour.
  - bee_rider 2y ago
    I think you are using an unconventional definition of the word parallelism, if you want to exclude > Yes, once the pipeline is filled up, you have the three operations going on at the same time, but it's still a sequence of steps that need to be performed in order for any given bottle. as not parallel. It seems to me that what you are calling parallelism, most people would instead call homogeneous parallelism. Which is a subset of parallelism.
  - hinkley 2y ago
Joker_vD 2y ago
This is your own personal definition of "pipelining" which most of other people wouldn't subscribe to. And to pick a nit, there is always some synchronization between the submission of an input and the processing of that input: the submission must happen before the processing, unfortunately.

bee_rider 2y ago

I don’t 100% follow your comment, so sorry if this is not quite right. But, I would consider pipelining to be a form or parallelism. It breaks up a task so that parts of it can be run simultaneously (in different stages of the pipeline, simultaneously). There are other ways to do parallelism of course, but it is a way. In your example, if there are multiple pipeline stages in this server, then the tasks should be worked on simultaneously, and so parallelism is occurring. Multi-core, SIMD, and pipelining. Parallelism has multiple dimensions.

spc476 2y ago
Not quite. Going back to the bottling example, a bottle has to be filled, capped, then labeled. At time 1, 1 bottle is filling up, pipeline is advanced. At time 2, bottle 2 is filling up, bottle 1 is being capped, pipeline is advanced. At time 3, bottle 3 is filling up, bottle 2 is being capped, and bottle 1 is being labeled. At time 4, bottle 1 is done. Each bottle has to go through the pipeline in sequence, and it takes three units of time for a bottle to go through the pipeline. Yes, once the pipeline is filled up, you have the three operations going on at the same time, but it's still a sequence of steps that need to be performed in order for any given bottle. To make it faster, you either have to decrease the time for a step (but you will always be capped with the slowest step), or go for parallelism---a separate pipeline (or pipelines). For example, with one pipeline, each step taking 1 unit of time, once the pipeline is filled, will take six units of time to make a six-pack (the first will take longer due to the latency in filling the pipeline). You can make five other pipelines, and then get a six-pack per unit of time (again, after filling all the pipelines). A single pipeline just makes the output have a predictable latency and time; multiple pipelines give you parallelism.
- Joker_vD 2y ago
  > For example, with one pipeline, each step taking 1 unit of time, once the pipeline is filled, will take six units of time to make a six-pack Compared to 18 units of time needed to make a six-pack without pipelining. Gee, what a wondrous invention this "pipeline" is: having three workers means the work is accomplished thrice as fast, yet there is (according to you) no parallelism at all! So naturally, if we could introduce parallelism inside this single pipeline, we would be able to make another triple reduction in time, and get a production of six-pack take only 2 units of time.
- Jtsummers 2y ago
  > To make it faster, you either have to decrease the time for a step (but you will always be capped with the slowest step), or go for parallelism---a separate pipeline (or pipelines). > A single pipeline just makes the output have a predictable latency and time; multiple pipelines give you parallelism. No, the pipeline does give you parallelism because you're doing three (in the bottling example) pieces of work simultaneously, that is: in parallel. Filling, capping, labeling are each being done on different bottles at the same time. How is that not parallelism? Let's use some numbers: Filling, capping, labeling take 30s, 15s, 15s each (arbitrary, chosen for easy math). Without a pipeline you will process 60 bottles per hour (say one station that does all 3 tasks and then kicks out the bottle, let's ignore transition times). With a pipeline you can get 120 per hour. You've improved throughput. The latency per bottle is still 60s. BTW, you can double throughput again without even needing a full second pipeline just by adding a second filling station (with my example numbers) and feeding filled bottles to the same capping and labeling stages, getting you to 240 per hour.
- bee_rider 2y ago
  I think you are using an unconventional definition of the word parallelism, if you want to exclude > Yes, once the pipeline is filled up, you have the three operations going on at the same time, but it's still a sequence of steps that need to be performed in order for any given bottle. as not parallel. It seems to me that what you are calling parallelism, most people would instead call homogeneous parallelism. Which is a subset of parallelism.
- hinkley 2y ago

Joker_vD 2y ago

This is your own personal definition of "pipelining" which most of other people wouldn't subscribe to. And to pick a nit, there is always some synchronization between the submission of an input and the processing of that input: the submission must happen before the processing, unfortunately.