5 ms·

Everything gets synchronized at some point. There is no sense splitting hairs on labels just because you don't have to do the synchronization yourself. Those i

Everything gets synchronized at some point. There is no sense splitting hairs on labels just because you don't have to do the synchronization yourself. Those independent tasks are going to have to have their output go somewhere.

melvinroest 6y ago
In some cases, perhaps most cases, this gets hard. In other cases this is still easy. For example, when the ordering of the output doesn't matter. I'm dealing with such a case in a data conversion program. A super simple example for when ordering doesn't matter is when you do parallel addition.
- CyberDildonics 6y ago
  I'm not sure exactly what you are saying, but not all synchronization is about ordering. Addition is still synchronization since eventually multiple numbers will need to be merged into one output.
  - melvinroest 6y ago
    Sure, and that can cause issues potentially. But since addition is commutative [1], you don't have to wait on an exact ordering and synchronize for that. All you have to synchronize for is waiting until all the numbers are added. [1] for HN'ers that don't know the word, it means a + b is the same as b + a.
    - CyberDildonics 6y ago
      What I said originally was that everything is synchronized at some point. You started talking about ordering and addition unprompted. Are you sure you meant to reply to this thread?
phkahler 6y ago
There are many times when the same task is performed with a number of objects. This trivial form of parallelism is nicely handled with OpenMP.
- CyberDildonics 6y ago
  It is, but that is just synchronization being handled by openMP. The indices for each chunk are given to the body of the loop automatically and waiting for all forks to finish is done automatically.
bob1029 6y ago
Not everything shares the same global state. Two completely independent computer systems that are not in communication with each other will never become synchronized, nor would it be expected that their state is merged in any way. This line of thinking can extend into abstractions existing on the same physical host. You don't even have to share global state to work towards the successful outcome of some common objective. There are many problems that can be solved with actors operating in parallel with absolutely zero coordination between them. This might seem like its a place where only trivial cases live, but I have found some fairly complex concurrency scenarios that can be refactored into this trivial case by decoupling actors/state and reorganizing other parts of the system. For example - Assume you were to develop a database engine that uses sequential integers for the primary key. You would probably have to develop a complex synchronization mechanism so that sequences are dealt out uniquely to concurrent callers. Or, you could sidestep the problem altogether by switching to GUIDs (V4 - Random) and allowing each participant to key its own entities. I feel these sorts of decisions should be considered as a primary measure before reaching for any form of shared state.
- CyberDildonics 6y ago
  > Two completely independent computer systems that are not in communication with each other will never become synchronized, This is not what is being talked about when people discuss parallelism and concurrency since it is trivial. In that example all the tricky stuff has already happened. Lots of people looking at a painting is easy. Lots of people painting a single picture is hard. Not handling the synchronization explicitly in your own program doesn't mean it isn't happening. If you write to a disk, to a network or even just display pixels from multiple threads on a monitor, the synchronization is being handled for you. > sidestep the problem altogether by switching to GUIDs (V4 - Random) and allowing each participant to key its own entities That would be a concurrent hash map which doesn't side step synchronization, since you still have to deal with collisions, space allocation, ownership and lifetimes. It also isn't the same as integer keys since you lose the ordering and lose the ability to do ranged queries. Where synchronization is happening and how it is happening is important, but the idea that it is being avoided is nonsense. Letting something else do it can be sound advice. This however, is like saying that garbage collection means that allocating and freeing memory doesn't happen.

melvinroest 6y ago

In some cases, perhaps most cases, this gets hard. In other cases this is still easy. For example, when the ordering of the output doesn't matter. I'm dealing with such a case in a data conversion program. A super simple example for when ordering doesn't matter is when you do parallel addition.

CyberDildonics 6y ago
I'm not sure exactly what you are saying, but not all synchronization is about ordering. Addition is still synchronization since eventually multiple numbers will need to be merged into one output.
- melvinroest 6y ago
  Sure, and that can cause issues potentially. But since addition is commutative [1], you don't have to wait on an exact ordering and synchronize for that. All you have to synchronize for is waiting until all the numbers are added. [1] for HN'ers that don't know the word, it means a + b is the same as b + a.
  - CyberDildonics 6y ago
    What I said originally was that everything is synchronized at some point. You started talking about ordering and addition unprompted. Are you sure you meant to reply to this thread?

phkahler 6y ago

There are many times when the same task is performed with a number of objects. This trivial form of parallelism is nicely handled with OpenMP.

CyberDildonics 6y ago
It is, but that is just synchronization being handled by openMP. The indices for each chunk are given to the body of the loop automatically and waiting for all forks to finish is done automatically.

bob1029 6y ago

Not everything shares the same global state. Two completely independent computer systems that are not in communication with each other will never become synchronized, nor would it be expected that their state is merged in any way. This line of thinking can extend into abstractions existing on the same physical host. You don't even have to share global state to work towards the successful outcome of some common objective. There are many problems that can be solved with actors operating in parallel with absolutely zero coordination between them. This might seem like its a place where only trivial cases live, but I have found some fairly complex concurrency scenarios that can be refactored into this trivial case by decoupling actors/state and reorganizing other parts of the system. For example - Assume you were to develop a database engine that uses sequential integers for the primary key. You would probably have to develop a complex synchronization mechanism so that sequences are dealt out uniquely to concurrent callers. Or, you could sidestep the problem altogether by switching to GUIDs (V4 - Random) and allowing each participant to key its own entities. I feel these sorts of decisions should be considered as a primary measure before reaching for any form of shared state.

CyberDildonics 6y ago
> Two completely independent computer systems that are not in communication with each other will never become synchronized, This is not what is being talked about when people discuss parallelism and concurrency since it is trivial. In that example all the tricky stuff has already happened. Lots of people looking at a painting is easy. Lots of people painting a single picture is hard. Not handling the synchronization explicitly in your own program doesn't mean it isn't happening. If you write to a disk, to a network or even just display pixels from multiple threads on a monitor, the synchronization is being handled for you. > sidestep the problem altogether by switching to GUIDs (V4 - Random) and allowing each participant to key its own entities That would be a concurrent hash map which doesn't side step synchronization, since you still have to deal with collisions, space allocation, ownership and lifetimes. It also isn't the same as integer keys since you lose the ordering and lose the ability to do ranged queries. Where synchronization is happening and how it is happening is important, but the idea that it is being avoided is nonsense. Letting something else do it can be sound advice. This however, is like saying that garbage collection means that allocating and freeing memory doesn't happen.