Oh parallelism, the one thing I used to worry about more, and probably should (as far as using GPUs goes)... but also one I don't really sweat so much. Most of

Oh parallelism, the one thing I used to worry about more, and probably should (as far as using GPUs goes)... but also one I don't really sweat so much. Most of the reason, for me, was finding Elixir/Erlang AND their virtual machine BEAM. Until Elixir and BEAM for me, parallel programming had been difficult because the competing mental models in other languages pretty much make for a textbook fantasy. I'm sure some folks just hit rage land, "Pardon me, you contemptuously ignorant and vile buffoon, what did you say?" but if you'd be so kind as to let me qualify it... Parallelism on computers has more or less evolved from things being asynchronous which gave us the effect of parallelism (yay, for concurrency) to real parallelism with the proliferation of multi-core computers. This jump from emulated parallelism to real parallelism hasn't really been too thoughtfully designed into most languages, and for those starting out, depending on your language and runtime/vm, you might be in for quite an odyssey. We live in a world where almost every language you use has NOT been designed with parallelism as a first class problem. I'm going to try to give an abridged version of my odyssey, Please note I love anything I seem hate on in this tirade. First was Ruby, pure and simple, it has (system) processes (forking), threads, fibers, and libraries implementing actors and event-loops :naive-heart-eyes: but _none_ of them offer real parallelism inside the runtime itself. When one writes code with one of those mental models it falls apart at the metal (hey, GIL) and eventually you're left scratching your head, disillusioned with the world, and realize "why am I even doing this if my work is CPU bound? It won't matter." Next up, I knew the JVM had "REAL THREADS!" I needed to get experience with another language and thought "oh hell, yeah, baby, I'm gonna rock the shit outta these 1.. 2... 4 CORES" watch out CPU bound tasks. Then came reality creeping in like Santa Claus in July... "Creating threads is expensive? Oh. No, shit?" and "oh wow, so like, I now have between 1 and N threads, that I must to schedule, and now I lock my system somehow." I was told apply locks until the burning goes away, but it never does.. And in that a drunken haze of threading hell one night I saw what I thought was a beacon of hope, a JVM based Actor pattern named Akka. "I get the actor model, how could this go wrong?" So. Many. Ways. First and foremost is choking this new fangled "thread pool," Akka had given me with my CPU bound tasks. How does one then gloriously obliterate CPU bound tasks? Oh shit... One goes right back to where they started... in a separate thread. Or one can do as I did, and GALAXY BRAIN the solution: create a separate pool for my CPU bound tasks. This also introduces a new gang of friends "back pressure," "priority," and my favorite "contention." Eventually, I kind of accepted fate, knowing that some mental model even if totally contrived was better than no mental model-- right? Still I believed there had to be something better, and I kept reading about how awesome Erlang was for this sort of thing [1] at the time and Elixir had just come out. Peanut butter meet pickles. This shit was good. Erlang the language, and it's runtime/vm BEAM, were designed to work together to solve a problem space: asynchronous, fault tolerant, and distributed systems. At that point of realization, I blacked out from astonishment that people could write asynchronous code in a language and run-time designed to do so. Turns out its mental model was for me, the end all be all, because they (lang/vm) were, and this my big point, DESIGNED TOGETHER TO SOLVE THE PROBLEM. On BEAM a "process" can be pre-empted (queue choir of angels) and thus _everything_ is non-blocking. The actor pattern was finally able to hit the metal in my head and my beard gained substantial majesty. I write Elixir nearly all day long right now, and while I love it, I wish there were other options (I should probably check out Pony again) for me to use, but honestly, I don't know of any. I'm beyond thankful for Erlang and BEAM since it really helped out my software development game and understanding how parallelism works. P.S. Anyone know if is there a language designed for running on GPUs designed to make it easier to facilitate programming even if they're slightly slower to execute, e.g. the Erlang/OTP/Beam of GPUs? [1] BEAM is actually not great a CPU bound tasks, but that's a different story.