4 ms·

I think Joint Parallel is hard, as in when two threads on two different cores are trying to write+read to+from the same memory. My solutions are: 1) Use Java

by bullen 6y ago

I think Joint Parallel is hard, as in when two threads on two different cores are trying to write+read to+from the same memory. My solutions are: 1) Use Java on the server; it's GC:ed VM has a complex memory model that can handle fast lock-free concurrency relatively well, even using OO though you'll hit cache-misses. 2) Use C on the client, primitive variable types are generally atomic so if you know what you are doing (I don't because I haven't gotten that far yet) you should be able to joint parallelize cores with char, int and float arrays as data, also avoiding cache-misses! There are many more arguments for this division, Java doesn't crash (usually) and C is truly portable (you cannot run Java on certain devices iOS, Switch, PS4).

setpatchaddress 6y ago
You cannot generally trust that C primitive types are atomic.
- eschneider 6y ago
  Can't. You can't trust that unless the types are specifically declared atomic.
CyberDildonics 6y ago
This seems more like someone taking wild guesses instead of information from experience and fundamentals. First, I don't think "Joint Parallel" is a common term and everything gets synced eventually at some level. Your solutions are languages, which are separate from understanding parallelism and concurrency. Using java doesn't solve anything directly. > it's GC:ed VM has a complex memory model that can handle fast lock-free concurrency relatively well, even using OO though you'll hit cache-misses. This is almost like talking about the sound of a color. Lock free programming is not a factor of a language. If there are good lock free data structures that work for your specific problems, then that can go a long way to solving synchronization. OO and cache misses have nothing to do with anything here, that comes down to how something is written, not the language. > Use C on the client, primitive variable types are generally atomic so if you know what you are doing This is not true. C has nothing to with this, it is a matter of getting access to atomics. Also primitive types are not "generally atomic", 32 bit stores can be written atomically, but that doesn't address ordering, compare and swap or atomic adds. > you should be able to joint parallelize cores with char, int and float arrays as data, also avoiding cache-misses! This seems like real information fed through google translate a few times. 'joint parallelize' doesn't mean anything. Avoiding cache misses is a matter of accessing memory sequentially. Arrays are orthogonal to atomics, each element needs to be dealt with individually.
- bullen 6y ago
  I minted Joint Parallel and gave the definition! This code runs: http://move.rupy.se/file/atomic.txt http://move.rupy.se/file/atomic.txt And that's all you need to know really. As for experience, I wrote my own app server with distributed database and multiplayer protocol from scratch by myself and proved it with a MMO that has 300.000 customers: https://store.steampowered.com/app/486310/Meadow/ https://store.steampowered.com/app/486310/Meadow/ https://github.com/tinspin https://github.com/tinspin
  - CyberDildonics 6y ago
    The things you said in your comment above are untrue for very fundamental reasons. This isn't a judgement call of the best techniques based github histories. > I minted Joint Parallel and gave the definition! That means it's nonsense. It is an oxymoron, it contradicts itself. Parallelism by definition is about separation of dependencies and synchronization. Join implies the exact opposite. That's like saying Bright Black while telling people you invented a new color space. It is a hiccup to being taken seriously (to put it mildly). You have said stuff like 'java doesn't crash (usually)' and that multi-threading rendering adds at least 10 frames of latency to a game. Statements like these are not just incorrect, they indicated a lack of understanding by thinking mostly orthogonal concepts are somehow at the mercy of each other. I looked at your link. The first file I opened says "non-blocking and asynchronous" in the first line of the comment. https://github.com/tinspin/rupy/blob/master/src/se/rupy/http/Reply.java https://github.com/tinspin/rupy/blob/master/src/se/rupy/http... Your methods that are supposedly doing this use the synchronize keyword whose purpose is to "block (suspend execution)" of other threads that call the same method until the first thread is done. https://docs.oracle.com/javase/tutorial/essential/concurrency/syncmeth.html https://docs.oracle.com/javase/tutorial/essential/concurrenc...
    - bullen 6y ago
      Joint because Parallel by itself means nothing, if I setup two separate computers with a load balancer they are parallel. Joint as I said; and it's not rocket science so I'm getting a little annoyed; is when two separate cores work on the same real-time task that shares memory, most software 1) does not do this yet because it's hard 2) most programmers don't learn this because 1). For example Erlang, Rust and Go are terrible at sharing memory because they have simple memory models, they solve parallelism by synchronizing hard and slow with copying and locks. And yes games today have 10 frames of motion-to-photon latency, I measured it myself. Non-blocking there is referring to the IO, synchronized only blocks if there is contention, you have to compare this to the synchronous HTTP APIs to understand why I guess? wakeup() (and other methods) are synchronized because you don't want two threads to contest the hand-over point (in this case wakeup a HTTP response); all shared memory is done over java.util.concurrent data, which in most cases is atomic. You have to look at the fuse project to see most joint parallel code, not saying it's pretty; but it works at, just as these quotes imply, around 10x perf: "While I'm on the topic of concurrency I should mention my far too brief chat with Doug Lea. He commented that multi-threaded Java these days far outperforms C, due to the memory management and a garbage collector. If I recall correctly he said "only 12 times faster than C means you haven't started optimizing"." - Martin Fowler https://martinfowler.com/bliki/OOPSLA2005.html https://martinfowler.com/bliki/OOPSLA2005.html "Many lock-free structures offer atomic-free read paths, notably concurrent containers in garbage collected languages, such as ConcurrentHashMap in Java. Languages without garbage collection have fewer straightforward options, mostly because safe memory reclamation is a hard problem..." - Travis Downs https://travisdowns.github.io/blog/2020/07/06/concurrency-costs.html https://travisdowns.github.io/blog/2020/07/06/concurrency-co... Until you show me something joint parallel you have done and preferably proven (with say 450 concurrent real-time action users on one machine); I'm calling this settled, your arguments are thin! (also why am I arguing with a user called dildo?!)
maxlybbert 6y ago
I put off reading the book for ages because I was convinced that knowing about locks and atomic data accesses, and intentionally keeping things simple would be enough to stay out of trouble. The book certainly does cover how to coordinate between multiple threads, but it goes further by pointing out that the real secret to multithreaded programming is designing things so that coordination is kept to a minimum. For one thing, the coordination often has the side effect of purging your CPU caches! You definitely get a lot of cache misses. Even if you use atomic operations on primitive types. I believe you can get a lot out of the book even if you already know Java’s or POSIX’s threading models.

by bullen 6y ago

setpatchaddress 6y ago

You cannot generally trust that C primitive types are atomic.

eschneider 6y ago
Can't. You can't trust that unless the types are specifically declared atomic.

CyberDildonics 6y ago

This seems more like someone taking wild guesses instead of information from experience and fundamentals. First, I don't think "Joint Parallel" is a common term and everything gets synced eventually at some level. Your solutions are languages, which are separate from understanding parallelism and concurrency. Using java doesn't solve anything directly. > it's GC:ed VM has a complex memory model that can handle fast lock-free concurrency relatively well, even using OO though you'll hit cache-misses. This is almost like talking about the sound of a color. Lock free programming is not a factor of a language. If there are good lock free data structures that work for your specific problems, then that can go a long way to solving synchronization. OO and cache misses have nothing to do with anything here, that comes down to how something is written, not the language. > Use C on the client, primitive variable types are generally atomic so if you know what you are doing This is not true. C has nothing to with this, it is a matter of getting access to atomics. Also primitive types are not "generally atomic", 32 bit stores can be written atomically, but that doesn't address ordering, compare and swap or atomic adds. > you should be able to joint parallelize cores with char, int and float arrays as data, also avoiding cache-misses! This seems like real information fed through google translate a few times. 'joint parallelize' doesn't mean anything. Avoiding cache misses is a matter of accessing memory sequentially. Arrays are orthogonal to atomics, each element needs to be dealt with individually.

bullen 6y ago
I minted Joint Parallel and gave the definition! This code runs: http://move.rupy.se/file/atomic.txt http://move.rupy.se/file/atomic.txt And that's all you need to know really. As for experience, I wrote my own app server with distributed database and multiplayer protocol from scratch by myself and proved it with a MMO that has 300.000 customers: https://store.steampowered.com/app/486310/Meadow/ https://store.steampowered.com/app/486310/Meadow/ https://github.com/tinspin https://github.com/tinspin
- CyberDildonics 6y ago
  The things you said in your comment above are untrue for very fundamental reasons. This isn't a judgement call of the best techniques based github histories. > I minted Joint Parallel and gave the definition! That means it's nonsense. It is an oxymoron, it contradicts itself. Parallelism by definition is about separation of dependencies and synchronization. Join implies the exact opposite. That's like saying Bright Black while telling people you invented a new color space. It is a hiccup to being taken seriously (to put it mildly). You have said stuff like 'java doesn't crash (usually)' and that multi-threading rendering adds at least 10 frames of latency to a game. Statements like these are not just incorrect, they indicated a lack of understanding by thinking mostly orthogonal concepts are somehow at the mercy of each other. I looked at your link. The first file I opened says "non-blocking and asynchronous" in the first line of the comment. https://github.com/tinspin/rupy/blob/master/src/se/rupy/http/Reply.java https://github.com/tinspin/rupy/blob/master/src/se/rupy/http... Your methods that are supposedly doing this use the synchronize keyword whose purpose is to "block (suspend execution)" of other threads that call the same method until the first thread is done. https://docs.oracle.com/javase/tutorial/essential/concurrency/syncmeth.html https://docs.oracle.com/javase/tutorial/essential/concurrenc...
  - bullen 6y ago
    Joint because Parallel by itself means nothing, if I setup two separate computers with a load balancer they are parallel. Joint as I said; and it's not rocket science so I'm getting a little annoyed; is when two separate cores work on the same real-time task that shares memory, most software 1) does not do this yet because it's hard 2) most programmers don't learn this because 1). For example Erlang, Rust and Go are terrible at sharing memory because they have simple memory models, they solve parallelism by synchronizing hard and slow with copying and locks. And yes games today have 10 frames of motion-to-photon latency, I measured it myself. Non-blocking there is referring to the IO, synchronized only blocks if there is contention, you have to compare this to the synchronous HTTP APIs to understand why I guess? wakeup() (and other methods) are synchronized because you don't want two threads to contest the hand-over point (in this case wakeup a HTTP response); all shared memory is done over java.util.concurrent data, which in most cases is atomic. You have to look at the fuse project to see most joint parallel code, not saying it's pretty; but it works at, just as these quotes imply, around 10x perf: "While I'm on the topic of concurrency I should mention my far too brief chat with Doug Lea. He commented that multi-threaded Java these days far outperforms C, due to the memory management and a garbage collector. If I recall correctly he said "only 12 times faster than C means you haven't started optimizing"." - Martin Fowler https://martinfowler.com/bliki/OOPSLA2005.html https://martinfowler.com/bliki/OOPSLA2005.html "Many lock-free structures offer atomic-free read paths, notably concurrent containers in garbage collected languages, such as ConcurrentHashMap in Java. Languages without garbage collection have fewer straightforward options, mostly because safe memory reclamation is a hard problem..." - Travis Downs https://travisdowns.github.io/blog/2020/07/06/concurrency-costs.html https://travisdowns.github.io/blog/2020/07/06/concurrency-co... Until you show me something joint parallel you have done and preferably proven (with say 450 concurrent real-time action users on one machine); I'm calling this settled, your arguments are thin! (also why am I arguing with a user called dildo?!)

maxlybbert 6y ago

I put off reading the book for ages because I was convinced that knowing about locks and atomic data accesses, and intentionally keeping things simple would be enough to stay out of trouble. The book certainly does cover how to coordinate between multiple threads, but it goes further by pointing out that the real secret to multithreaded programming is designing things so that coordination is kept to a minimum. For one thing, the coordination often has the side effect of purging your CPU caches! You definitely get a lot of cache misses. Even if you use atomic operations on primitive types. I believe you can get a lot out of the book even if you already know Java’s or POSIX’s threading models.