15 ms·

I took a class that went over this in depth like 3-4 years ago. Basically the message was that serial performance is saturating, and the only way to get speed i

by mjfl 9y ago

I took a class that went over this in depth like 3-4 years ago. Basically the message was that serial performance is saturating, and the only way to get speed improvements in the future is going to be by exploiting parallelism. However, most programmers, and programming languages, remain stuck in a serial-by-default paradigm. I'm surprised that there hasn't emerged a "parallel-by-default C++" kind of language + hardware system to exploit it to keep things going forward. I find the apparent stagnation extremely depressing.

quadcore 9y ago
https://golang.org/ https://golang.org/ In case you dont know, Golang goroutines are a marvel of parallelism. They are coroutines which are dispatched on a few OS threads. So you can use 100% of a multi-cores CPU and yet, spawn, say, 10K of those light threads without worrying about context switches PLUS have them all run concurrently. I've found that golang is one of those rare language, like Lisp, that actually change the way you think about programming. Makes you feel really more powerful. If you dont know that language, I suggest to run the following and watch your cpu activity and memory (or any metric): import "time" func main() { for i := 0; i < 10000; i++ { go func() { for { time.Sleep(time.Second) } }() } }
- dx034 9y ago
  And yet, Go's standard map doesn't allow concurrent access. They recently added a concurrent map feature but it's probably easier to add a lock to your code than refactoring it with the new map type. I would've preferred if they had introduced a map type that can be used exactly as the standard one (i.e. without function calls). Calling functions via go func() is really easy but handling data between goroutines can still cause headaches and is something that could be improved.
  - ta2384428 9y ago
    If you squint a little, you'll notice that go is pretty big on not hiding complexity. There are obvious exceptions to this such as the garbage collector, and heap/stack control. I'm unsure if the intent of not including a map function was due to this, however with a for loop such as for x := range ch { slice = append(slice, x) } , it is immediately obvious there are allocations happening in the background. The fact that map access, and slice access is not thread safe means there is no trickery going on in the background. The fact that it is not threadsafe means I don't need to worry about a lock if I only write to a map when it's created. Sure the compiler could take care of this - they have the race detector after all, but the compile speed is one of the design goals of go. I really like being able to compile in less than 1 second. `sync.Map` however calls a function which implies there is more going on in the background. If you follow the go mantra of share memory by communicate, don't communicate by sharing memory - handling data becomes a whole lot easier. It does allow you to share memory in case you do need the extra speed. Like most things, all designs are a matter of trade offs. Sacrifice one thing for another. There are other languages that provide the functionality you desire - but I understand the frustration when one thing is 90% what you want. Go obviously has room for improvement, and perhaps a native threadsafe map is one of those areas.
  - quadcore 9y ago
    Well, go isn't really a high-level language which maps can be seen as a feature of. The idea behind go, I believe, is to focus and provide innovative system-level features.
    - eropple 9y ago
      Go is not a language for "system-level features" in the first place. Its garbage collector largely precludes it from that in any modern context. That it can't do something effectively does not mean that it shouldn't or didn't mean to. (And users can't really fix it, because hey, no generics. Sigh.)
      - quadcore 9y ago
        Go is not a language for "system-level features" in the first place. Its garbage collector largely precludes it from that in any modern context. I'm pretty sure I disagree with that largely because Go is low-level enough so that the GC is easily handled. And users can't really fix it, because hey, no generics. Sigh. Users can fix it by writing languages on top of Go (particularly dynamic ones).
- tzahola 9y ago
  Your example will not run in parallel. The go runtime will schedule your goroutines concurrently, but they will be run by a single OS thread, and consequently on a single CPU core. Once you execute truly on multiple CPU cores (by increasing GOMAXPROCS), you'll be having the same kind of race conditions in Go as in any other imperative language (inb4 Rust Evangelism Strike Force saying "except Rust").
  - quadcore 9y ago
    Wrong. Goroutines are not simply coroutines. GOMAXPROCS defaults to number of cores.
    - tzahola 9y ago
      Prove it. Prove it by replacing Sleep in your example with some number crunching, and show how it scales with the number of cores in your CPU.
      - deleted 9y ago
        [deleted]
      - gall0ws 9y ago
        https://play.golang.org/p/o8odLCRqUMA https://play.golang.org/p/o8odLCRqUMA
      - ta2384428 9y ago
        https://imgur.com/a/DNpw3 https://imgur.com/a/DNpw3 Running the following code. https://play.golang.org/p/k_rRxNAyb0i https://play.golang.org/p/k_rRxNAyb0i I can assure you that go runs across all processors by default. https://docs.google.com/document/d/1At2Ls5_fhJQ59kDK2DFVhFu3g5mATSXqqV5QrxinasI/edit https://docs.google.com/document/d/1At2Ls5_fhJQ59kDK2DFVhFu3...
        grkvlt 9y ago
        That's not what he's saying. He knows go can use all processors if GOMAXPROCS is set correctly, the argument seems to be that there will be race conditions just like any other threading, which seems pretty self evident to me: yes, multi-threaded code can have concurrency issues, film at 11...
- VHRanger 9y ago
  How is that an example an improvement over c++? C++ has had pragmatic omp parallel for for two decades
  - ta2384428 9y ago
    I'm not familiar with OMP, so please forgive me if I'm incorrect - but OMP from my quick search appears to be thread based, while go routines are much lighter weight than threads. They have their own scheduler - which like most things is both good and bad depending on what you want them for. If you use them as intended, the internal scheduler is a great design decision. This means I can happily spawn 10,000 without concern. Additionally, combining them with the power of channels makes quick work of many tasks. Channels of course can be implemented in C++ too, but having the compiler take care of it for you with additional tools such as the race detector is very handy. For a large set of problems, they are very nice to work with.
    - pjmlp 9y ago
      You can do the same in C++ on Windows with PPL, UNIX/Windows with Intel TBB, or any of the fiber/co-routine libraries. Then there is the ongoing work to add async/await patterns into C++20.
      - quadcore 9y ago
        As a matter of fact, you can do the same in assembly if you want to. Notice how you can always have that answer when it comes to programming languages: you can do the same in. The point is that, the way it's made in go, is awfully handy.
        pjmlp 9y ago
        I hardly see a difference from go func () { } and Task.Run(() => { }) With the benefit that on the latter example, the runtime allows me to customise how scheduling is done.
- magnetic 9y ago
  In my experience the facilities to throw tasks into a scheduler that will run them in parallel was never the hard thing to accomplish (regardless of the language: some may have built-in capabilities, others may have syntactic sugar provided by a lib, but at the end of the day, most systems provide some kind of runTask(f) method). What's really hard is to break down a problem into parallelizable chunks, figure out as much independent work as possible to reduce the touchpoints, and coordinate all those tasks such that they keep the CPU as busy as possible and as a whole finish as early as possible. Beyond this "parallel breakdown design", it's the little touchpoints with shared data structures and synchronization that creates the difficulty of implementation, and I haven't seen any language or system that does magic there.
  - quadcore 9y ago
    throw tasks into a scheduler that will run them in parallel was never the hard thing to accomplish Common mistake number 1. Goroutines are running in parallel AND concurrently - they are coroutines (common mistake number 2 is to think they are only coroutines). I suggest not to underestimate that, it's the big deal. Additionnaly, it's important to note that goroutines yields on sleeps (any kind of sleeps / waits, like disk reads, network requests, channel writes/reads, etc), and while it may sound like a detail, it's a wonder of cpu control. Due to that, there is also a rare elegance to the way Go solve sharing data via channels. What's really hard is to break down a problem into parallelizable chunks, figure out as much independent work as possible to reduce the touchpoints, and coordinate all those tasks such that they keep the CPU as busy as possible and as a whole finish as early as possible. That's exactly what Go is a wonder for due to the combination of parallelism, concurrency, yield-on-sleep and channels.
- brobinson 9y ago
  >marvel of parallelism If Go's m:n is blowing your mind, you should check out Erlang. Spawning a million "processes" is not a big deal.
  - quadcore 9y ago
    Cool, thanks for the suggestion.
- Const-me 9y ago
  > golang is one of those rare language Not sure how rare, .NET does that as well. Below is your sample translated to C#. It requires C# 7.1 because async main, but the rest of the stuff is available for many years, since 2010. static async Task routine() { await Task.Delay(TimeSpan.FromSeconds(1)); } static Task Main() { return Task.WhenAll(Enumerable.Range(0, 1000).Select(i => routine())); }
  - quadcore 9y ago
    It's unclear to me if these parallel tasks are also coroutines (there seems to be a call to yield but its unclear what it really does), and if they yield on any sleeps which is a key features of goroutines.
    - Const-me 9y ago
      > if these parallel tasks are also coroutines I don’t have hands-on experience with golang but based on what I know about it yes, they are. > there seems to be a call to yield but its unclear what it really does You mean “await”? It waits for the completion of whatever is on the right side of “await”. If the result is already available, it just continues. If the result is not yet available (e.g. Task.Delay creates a task that will complete in some moment in the future), the control goes away from the async method to the scheduler. The scheduler can then run some other task on the same OS thread. When the result of that operation becomes available (in the sample code, when 1 second delay passes), the scheduler resumes execution of that async method, on the statement after the await. > if they yield on any sleeps No, not any sleep. You can call Thread.Sleep() which will put the whole OS thread to sleep. It’s up to programmers to avoid calling blocking APIs from their async methods, i.e. use await Task.Delay() instead of Thread.Sleep(), await stream.ReadAsync() instead of stream.Read(), and so on.
wiz21c 9y ago
Is it depressing or is it a kind of signal we should listen to and try to interpret ?
pjmlp 9y ago
A few attempts have already been done, the issue is with developer adoption, not lack of trying. One of the best examples was StarLisp for the Connection Machine.
- convolvatron 9y ago
  starlisp required that you rephrased your problem to be large vector, with support for turning off logical cpus, very much like programming a modern gpu. if your problem could be recast that way it was pretty nice. however the generalized scatter/gather was so expensive, it had to be used very sparingly. you had to use the grey coded nearest neighbor hypercube network as much as possible. the really* cool language from Hillis and Steele was cmlisp, but I don't know how far they got, they never released anything.
madez 9y ago
While Rust doesn't promote a specific model for parallelism, it's stronger compiler helps a lot with that. https://doc.rust-lang.org/beta/nomicon/concurrency.html https://doc.rust-lang.org/beta/nomicon/concurrency.html https://doc.rust-lang.org/book/second-edition/ch16-01-threads.html https://doc.rust-lang.org/book/second-edition/ch16-01-thread... I don't think fixing C(++) can give what Rust can give, because Rust has a clean start with these strong guarantees built-in while for C(++) it would always be an addon. Defaults are powerful.
- beojan 9y ago
  C++ has parallel algorithms built in (http://en.cppreference.com/w/cpp/algorithm http://en.cppreference.com/w/cpp/algorithm). Parallelism is complicated though, and easy parallelism pretty much requires a functional style. Things like Haskell's Accelerate library (https://www.stackage.org/package/accelerate https://www.stackage.org/package/accelerate) seem the ideal way forward to me.
  - smitherfield 9y ago
    Should be noted that so far only Visual Studio (partially) implements C++17 parallel algorithms, although there are several third-party implementations.[1] Apples to oranges comparison of course, since Rayon isn't [planned to become?] part of the Rust standard library. [1] http://www.bfilipek.com/2017/08/cpp17-details-parallel.html#current-implementation http://www.bfilipek.com/2017/08/cpp17-details-parallel.html#...
    - beojan 9y ago
      If you're including third-party libraries, you have things like ArrayFire and Intel TBB.
- chrismorgan 9y ago
  A really powerful outworking of it is seen in the Rayon crate, where you can change a sequential iterator into a parallel iterator just by adding the crate, importing the trait and changing `iter` to `par_iter`. If it’s not thread-safe to do, then it won’t compile. (That’s the big difference from C++.) If it is, it will, and it’ll be smart about how it runs, spreading the load across all available cores pretty much optimally, or not bothering with multiple threads if it’s not going to be worth it (e.g. single-threaded, or only one item in the iterator). And all that with close enough to no overhead. Basically, Rayon makes data parallelism really easy in a way that few if any other languages do. I’d love to have an equivalent in Python or Node, but it’s just not possible to achieve such a thing in most languages—even if you ignore the thread safety aspect. Parallelism hasn’t seen a great deal of use until it’s urgently needed, because it’s hard to get right in most environments, and you normally need to substantially refactor code to make it happen. My hope is that with the likes of Rayon, parallelism can be a much more natural thing that people that care even a little about performance will just do, because it’s so easy to do. https://crates.io/crates/rayon https://crates.io/crates/rayon
elorant 9y ago
Microsoft has developed Parallel Linq which is a subset of .net framework and works like a charm.
ddorian43 9y ago
Seastar framework
- krylon 9y ago
  Honest question: Is that name a pun on C* (https://en.wikipedia.org/wiki/C* https://en.wikipedia.org/wiki/C*)?
jokoon 9y ago
I wonder if the future will be massively parallel, when CUDA and opencl came out I thought that future processors would have more and more core, so if you follow Moore's law, CPUs would see their core count double each 6 months. The problem is that GPU don't have error correcting codes, so you cannot really run application code on a GPU. The problem with parallelism is that C-like language don't fit well, only functional languages do. If you want to use multi-threading you have to forget about state and only work with input/output paradigms. For OSes it might mean a deep re-design, but I don't really know. A possible design would be a small but very fast CPU that only takes care or scheduling and task control, and another chip with many cores that deal with payloads and user software. AMD had some kind of hybrid chip that planned to do both graphics and task, but it was thrown away. Going parallel would require to change both hardware and software, and by software I mean stateless.
- tzahola 9y ago
  Intel Larrabee also got cancelled.
  - pjmlp 9y ago
    Kind of, it became Xeon Phi.
- zmonx 9y ago
  These are very good points! I would generalize "functional" to declarative though: Logic programming languages like Prolog and Mercury are also much more amenable to parallelization than C-like languages. In fact, different Prolog clauses could in principle be executed in parallel without changing the declarative meaning of the program, at least as long as you stay in the so-called pure subset of the language which imposes certain restrictions on the code.
- pjmlp 9y ago
  I always like to think of FP dataflow pipelines as doing digital circuits modeling.
- pjc50 9y ago
blauditore 9y ago
I think part of the reason why many programmers haven't wanted to deal with parallel execution is because concurrency is not easy to handle. It has several pitfalls and can be painful to debug. Also, it needs proactive efforts implementing it, so as long as not required, devs just stick to serial execution. Now, with helpful systems like no-side-effect functional languages and reactive stream frameworks, a lot of gory detail can be abstracted away. I think this has recently lead to more parallel-by-default software development.
kbart 9y ago
While modern languages support for parallelism is adequate, tools are still lacking imho. I avoid parallelism when it's not necessarily, because debugging all these race conditions, deadlocks, synchronization etc. is a nightmare.
- pjmlp 9y ago
  That is mostly an issue with the "avoid IDE" crowd. While IDE tooling can still be improved, the parallel debugging tools in .NET and Java eco-systems are already quite good. On VS, I can have at any given moment a graphical snapshot on how all threads and tasks are interacting with each other, or just execute some of the threads. It doesn't solve everything, but it makes it easier than a typical gdb session.
  - kbart 9y ago
    .NET has one of the best ecosystem overall, so it's more an exception than a rule (can't comment on Java as I don't work with it). As I'm mostly in low level, embedded system programming, you are still stuck with gdb, Valgrind and other primitive tools there, because if there's some legacy "IDE" at all, it's most often just some half-assed Eclipse plugin.
    - pjmlp 9y ago
      At least looking to their product sites, Microchip and Green Hills seem to have quite good tooling, then again I don't have embedded experience on modern systems, beyond mobile devices.
      - kbart 9y ago
        Do Microchip even produces multi-core MCU's? Haven't seen one, though I worked mostly with Cortex-A series so might miss something.
        pjmlp 9y ago
        As I said, I don't have much experience in real embedded domain outside mobile devices (iOS, Android, UWP), but aren't Cortex-A5 supposed to handle up to 4 cores?
Sean1708 9y ago
Chapel[0] and Fortress[1] come to mind. Wikipedia also has a list of parallel programming languages[2] (although it seems to play somewhat fast and loose with the definition of "parallel programming language"). [0]: https://en.wikipedia.org/wiki/Chapel_(programming_language) https://en.wikipedia.org/wiki/Chapel_(programming_language) [1]: https://en.wikipedia.org/wiki/Fortress_(programming_language) https://en.wikipedia.org/wiki/Fortress_(programming_language... [2]: https://en.wikipedia.org/wiki/List_of_concurrent_and_parallel_programming_languages https://en.wikipedia.org/wiki/List_of_concurrent_and_paralle...
adrianN 9y ago
Most software is fast enough when written in a naive sequential style. For the parts that parallelize well and matter, there are already decently mature ways of using all cores. Languages like Go, Rust, and Erlang make it fairly easy to write concurrent programs.
tyingq 9y ago
"there hasn't emerged a "parallel-by-default C++" kind of language + hardware system to exploit it" That's roughly what Intel tried with Itanium. I don't know if whatever barriers they hit are still barriers today.
- pjc50 9y ago
  Not really - Itanium tried to move the instruction-level parallelism and the burden of re-ordering execution to the compiler. You could only execute the maximum six instructions per cycle if there were no data dependencies. So it only really works for certain kinds of algorithmic data-heavy execution; that's why VLIW is only found in DSP architectures today. Good SE answer here: https://softwareengineering.stackexchange.com/questions/279334/why-was-the-itanium-processor-difficult-to-write-a-compiler-for https://softwareengineering.stackexchange.com/questions/2793... especially the ones focusing on cache misses.
  - tyingq 9y ago
    "Itanium tried to move the instruction-level parallelism and the burden of re-ordering execution to the compiler" That sounds a lot like "parallel-by-default C++" kind of language + hardware system to exploit it" Just swapping "compiler" for language. They didn't succeed, but they did try Edit: helping me understand where I'm off might be more helpful than a downvote. Does swapping "compiler" for "language" not respresent what Intel was trying to do?
Pica_soO 9y ago
The sequential programming by default is because most people think sequential by default. Even though claims of multi-tasking etc persist, the truth is good parallel programmers are a rare thing. Many ordinary programmers already get into Hot-Water when they use two threads and access data where a Semaphore might be needed. In addition many algorithms are sequential, so parallelizing them is tricky or gives you no true reward due to cross-thread communication. Add to that the OO-Software Structures that subtle encourage using sequential programming. I think the future in parallel-programming is actually hiding the parallel programming completely - accept the fact that most humans are not made for it, allow for experts to unlock the ability to override that behavior- let compilers go as far as they can and live with the results. It will suffer the same fate as functional programming. Really useful, but never dominant, due to limitations in the applying humans.
pjc50 9y ago
Unfortunately it requires more than just a change of language, it requires a change in mode of thinking by developers. People are very used to reasoning in terms of "do X, then Y, then Z" or "compute a value X then do A or B on the basis of that". In order to achieve automatic parallelism you need e.g. a type+proof system that can determine that X/Y/Z are independent, or a system that can partially execute both A and B then retire the branch not taken - without invoking security bugs!
- pjmlp 9y ago
  I think it is easier to understand by those of us that also had electronic design as part of the engineering degree.
  - deleted 9y ago
    [deleted]
MichaelMoser123 9y ago
> However, most programmers, and programming languages, remain stuck in a serial-by-default paradigm You still have to decide upon the unit of work that is going to be sent to a different thread/core/processor/NUMA node/whatever. The different units of work that are distributed should not share state; one really doesn't want to be sharing a lot of state between different processors, because synchronizing the processors memory caches in NUMA is a extremely slow. I guess it is really hard to break up both the program and data and decide upon the optimal granularity of the work units, it is not something that can be easily done behind the scenes - human intervention is still required.
z3t4 9y ago
I think JavaScript is nice because it's async in nature. Concurrency is hard so it's nice to deal with it using a simple language, so that everything besides the business logic is abstracted. Yes you do not get the same performance, but CPU cores are relatively cheap compared to engineer salaries.
- pdpi 9y ago
  The notion that cpu cores are cheap compared to engineer salaries only scales so far. Scaling upwards, your opinion on that changes when a single engineer’s service is running on 10k machines. At the other end of the spectrum, if you’re developing high-performance applications for small systems (desktop, laptops, mobile), Your workload isn’t going to look like tens of thousands of concurrent independent requests, So the approach of getting parallelism by deploying multiple copies of the application no longer works
- dx034 9y ago
  > but CPU cores are relatively cheap compared to engineer salaries I often experienced that this backfired. Single machines are still constrained in their power and while it's easy to spin up additional VMs in the cloud, scaling a program properly to run on dozens of machines takes a lot of work. It can be faster to develop a program that is really efficient and can solve the problem on one machine than to develop faster only to then spend the time scaling it to a large fleet of servers.
  - z3t4 9y ago
    I kinda regret adding the note about performance, because switching to a lower level language used to yield orders of magnitude more performance, but optimizers have evolved and today there's not much difference. Sometimes the higher level language will be even faster because of optimizations. And bad performance is often not to blame on the language, instead blame the programmer or more likely the business people as they think it's great to waste resources as it gives them an excuse to charge more.
    - ska 9y ago
swiley 9y ago
There are limits to how much parallelism will improve things, see amdahl's law.
- kamaal 9y ago
  Sure, but let's start moving towards parallelism first. May be a few decades from now we would be close to breaking those limits.
  - Something1234 9y ago
    That law is a fact of life. Its a form of diminishing returns, there's only so much you can do.
Santosh83 9y ago
One reason is that our thinking process is inherently serial and we do really badly at multitasking naturally. At least those involving deliberate thought. And our programs are almost an extension of our way of thinking, so we will have to push the boundaries of how we think and model systems in our mind before we can build excellent parallel programming languages. Not that it isn't being done, but the weight of the "serial" legacy is long...
- monocasa 9y ago
  It's not just our thought process; there are upper limits to the gains from increased parallelism. https://en.wikipedia.org/wiki/Amdahl%27s_law https://en.wikipedia.org/wiki/Amdahl%27s_law
UncleSlacky 9y ago
This issue was discussed in my Occam class ~28 years ago. Occam itself is an example of a concurrent/"parallel-by-default" programming language intended for a Transputer hardware environment (now retargetable to x86 etc.), but it's not the easiest of languages to learn: https://en.wikipedia.org/wiki/Occam_(programming_language) https://en.wikipedia.org/wiki/Occam_(programming_language) https://en.wikipedia.org/wiki/Transputer https://en.wikipedia.org/wiki/Transputer https://en.wikipedia.org/wiki/KRoC https://en.wikipedia.org/wiki/KRoC
digi_owl 9y ago
I can't shake the feel that this parallelism thing will be nothing but a wild goose chase, because nature seems to be highly serial in all but the most macroscopic of senses.
- pjc50 9y ago
  All of your cells and neurons operate in parallel.
  - adwn 9y ago
    And yet you only have a single train of thought [1] – or multiple interleaved trains, but that's concurrency, not parallelism. [1] Plus a couple of background processes, like breathing, that execute in parallel.
    - boomlinde 9y ago
      That's a huge simplification not pertaining to the fundamental nature of the brain. Maybe it is in the nature of our consciousness (which in itself is probably only an abstract concept) to perceive the processes it emerges from as a sequence of single trains of thought rather than a chaotic, continuous consolidation of processes both internal to the brain/body and outside of it. You could look at society as a whole and see the zeitgeist as a singular "train of thought", but you'd probably still recognize humans as individual agents. I think we have a bias towards thinking of ourselves as the ultimate individuals, neither recognizing the processes within us (like those of our cells) or the processes beyond us (like those of a group of people, animals, plants etc.) as having a similar nature. This is probably a genetically advantageous trait.
      - adwn 9y ago
        > That's a huge simplification not pertaining to the fundamental nature of the brain. I disagree. Conscious thinking is a crucial process that's inherently single-threaded, even though it runs on highly parallel hardware (the billions of neurons). However, I must admit that my point doesn't necessarily contradict pjc's point, and I don't really agree with digi_owl's claim that parallelism is a "wild goose chase".
skrebbel 9y ago
> Basically the message was that serial performance is saturating, and the only way to get speed improvements in the future is going to be by exploiting parallelism. People have been warning us about this for about 10 years now, but I still don't see those 64-core CPUs I was promised anywhere. If we had the amount of parallelism we were told we were going to get, we could give every app its own core. OSes could even consider disabling context switches altogether for the majority of apps. Instead, we're left complaining about Electron apps like it matters. That said, I'm not sure what stagnation you refer to. There's a reason languages like Rust, Elixir/Erlang and Go are getting popular. My PHP app could handle hundreds of concurrent connections on a single machine, my Elixir app handles hundreds of thousands. Yet, processors didn't get 1000x faster (and Elixir isn't even a particularly fast language). This is the opposite of stagnation, it's progress.
- skummetmaelk 9y ago
  They have been here for a while, just for a very specific market. Threadripper and EPYC exist now though. With 32 and 64 logical cores respectively.
  - skrebbel 9y ago
    Sure, there's been high end niche products for anything forever. You could buy a 64 core computer in 1990 (they'd call it a supercomputer, but same thing). The people telling us we had to go change our code to use parallel processing fast predicted a significantly faster increase in amount-of-cores on commodity hardware. Instead, CPUs stopped getting faster and hardly gained more parallellism.
    - pdpi 9y ago
      You could buy a 64 processor machine in 1990, but it wouldn’t have been a 64 core machine in the sense we’re talking about — a single socket system. This isn’t some trivial distinction either, as the whole memory architecture is very different indeed for the two scenarios.
otabdeveloper1 9y ago
I'm surprised that there hasn't emerged a "parallel-by-default C++" kind of language + hardware system to exploit it to keep things going forward. There has. GPU programming is exactly that. CPU-heavy tasks (games, bitcoin mining, machine learning) have already migrated.
jonathanstrange 9y ago
I agree, programming languages have not caught up yet. The right kind of language looks at serial program formulations and based on flow-analysis automatically identifies parallelizable fragments that are large enough to benefit from multicore, then schedules these fragments e.g. by using work-stealing in a system of green threads, i.e., mapping green threads to OS cores as efficiently as possible. Something like that. In a good parallel language there need to be many immutable constructs by default, exception handling is tricky, and ordinary flow control needs to be compatible by default with parallel evaluation. The languages I've seen such as Parasail are not yet production ready. Making the programmer control parallelism can be okay, like in Go and Ada, but in the end it should be automatic. Edit: The problem is also that finding a neat way of solving the problem academically does not readily translate into an efficient implementation, so much that I wonder whether green threads are actually worth it over OS threads. In most languages/VMs they aren't but Go seems to be an exception.
Entalpi 9y ago
X10 is kind what you are describing. https://en.wikipedia.org/wiki/X10_(programming_language) https://en.wikipedia.org/wiki/X10_(programming_language)
BenoitP 9y ago
I'd say exploiting parallelism is not the only way at all. Parallelism is only one way to compute differently. Specialization of hardware to specific workloads will explode in the next years as we can't rely anymore on Moore's law. This will happen on RISC-V, IMHO. We already have these: * Rendering, medium precision mathematics: GPU * Low precision mathematics: TPU * Software Defined Networking: Microsoft is deploying FPGAs, AWS has its own hardware We could have: * Databases: Projections, hashing, sorting in hardware. * Dynamic runtimes: Hardware implemented memory models, HW assisted GC, code caches and user-level interrupts for the JITs. Here is the J extension RISC-V working group: [1] etc. Also, why not have the usual hot paths in Node.js|Spring Framework|Django directly etched into hardware? HW http header parsing surely could bring benefit to them all. ---- Of course language and programmers will have to adapt, but in a lot of cases the runtimes will take care of it automatically. [1] https://groups.google.com/a/groups.riscv.org/forum/#!msg/hw-dev/itbdrL_zdrc/t4B5fdbgAAAJ https://groups.google.com/a/groups.riscv.org/forum/#!msg/hw-...
- microcolonel 9y ago
  Currently working on software for RISC-V and I can definitely see some opportunities here. One thing that's becoming apparent though, is that some level of abstraction needs to be available in order for many of these things to become useful. RISC-V has a very exciting Vector processing extension, which is intended to replace packed SIMD in most cases, but some software systems assume packed SIMD is the only way to get more FP performance. For example, WebAssembly specifically exposes SIMD primitives, which means that it may be necessary to work backwards from those SIMD primitives to make use of a true vector machine. I think many people simply underestimate the cost of adopting a new programming model. > Also, why not have the usual hot paths in Node.js|Spring Framework|Django directly etched into hardware? HW http header parsing surely could bring benefit to them all. Well, in all the listed cases here, the CPU is not the bottleneck on throughput. As far as I can tell problem with HTTP is not that headers take too long to parse, it's that memory is still too slow, and context switches cost us precious time. The problem with Node is not that the hardware doesn't adequately model the semantics, it's that dynamic, weak typing makes it hard for any system (software or hardware) to understand what type things are. Update: The J extension seems interesting, and I've read some research (not thoroughly) recently showing considerable power and time savings from hardware GC primitives. I'm excited to see what goes on in that committee.
Androider 9y ago
With just Chrome and IntelliJ running on Ubuntu, I have 281 processes and kernel workers running (as reported by ps). Just stitching the apps together for display on your screen requires 3-4 different processes (and a GPU). That's why 4 logical cores is a bare minimum these days for a desktop, even if no individual application takes advantage of more than a single core. On the server side, when we deploy a Node.js web service on AWS we start one instance per logical core, for 4-64 processes all independently serving connections. It seems the process has become the new thread, the smallest unit you should design for. So today's workloads actually make pretty good use of all those cores. Unless you're doing high performance computing and need to squeeze every last drop of performance, processes are a straightforward way to parallelize.
- rbanffy 9y ago
  Having 281 processes is not a biggie. Having 281 processes that can actually use a piece of the CPUs is quite a different thing. I think we should, really, start thinking about such things. Maybe prefixing instructions with the execution unit that should handle them (and overflow back to the first one in a circle if we have more EU's in software than the actual hardware provides), separating dependencies within code flow in a more explicit way and, at the same time, not bothering with creating threads.
quietbritishjim 9y ago
One option is that you write in a high-level language where the top-level control code is single threaded, but you call APIs that perform multi-threaded operations seamlessly. The prototypical example of this is Python with numpy/blas (or with deep learning libraries like TensorFlow).
PaulHoule 9y ago
Materials other than silicon can support higher clock rates. From reading the open literature and advertisements by foundry companies, I think you could make a 6502 equivalent processor with Indium Phosphide with 64kb of static RAM that clocks at 30 GHz. With a more refined process you might push 90 GHz and a much more complex processor. Yes, InP is more expensive than Silicon but part of that is the low volume that InP parts are made in. Advances in Silicon are getting much more expensive, and one InP microprocessor could do the work of ten Silicon-based cores so you can save on die area without the "race to the bottom" in size. The main issue with high clocks is fast access to memory, probably you would need an optical interface to off-chip RAM, also I don't know what the InP equivalent of DRAM is. (Something like Optane?)
- deepnotderp 9y ago
  1. Compound semiconductors often suffer from p-type and n-type conduction problems. Plus they're ridiculously expensive and power hungry. They also often lack a native oxide. 2. The problem today in CPUs is not really clock speed but much more the memory access latency, optane is much slower* than DRAM and has much lower endurance. *Even though silicon HKMG transistors use high k gate dielectrics now, they still use a silicon dioxide interfacial layer.
contravariant 9y ago
Haskell kind of qualifies. You can't have race conditions when everything is immutable. Of course this makes some other things a bit more difficult.
kamaal 9y ago
New languages are coming up, it will just take long adoption cycles, given the library and surrounding tooling ecosystem has to be mature enough to go with it. Prominent examples being Go and Perl 6. Perl 6 especially, given how audacious the project is. There are performance issues with it currently though, from what I hear they are working to fix it soon.
- b2gills 9y ago
  I remember someone posting some Perl 6 code and C code that did the same thing. The Perl 6 code was shorter, easier to understand, more correct (Unicode), and was reported to be faster for what they were doing. There are things which are slower, but since it is a higher level language it may be easier to try multiple algorithms one of which may be significantly faster. It also has many useful features included, which can be optimized in ways that aren't recommended for user code. (writing the algorithm in NQP) There is also a code specilizer (spesh) and a JIT. Basically for many things it can be fast enough. Also if you profile your code and find something that is egregiously slow you should report it. Many times such things get optimized quickly.
Bitcoin_McPonzi 9y ago
In fact, there is! Erlang is extremely parallel. Processes are lightweight, share nothing, and run on as many CPUs you have.
- djsumdog 9y ago
  And also its derivatives, like Elixir.
  - losteric 9y ago
    Elixir is amazing! The original author, José Valim, took a brilliant approach: Take a 30 year-old battle-tested highly-parallel VM, and build a modern language on top of it. Syntax is inspired by the good parts of Ruby (very clean), but nothing comes close in terms of the ease of parallelism... it's so natural, and insanely fast. The unit tests are what convinced me this will be the next big thing. Beautifully clear syntax, succinct tests, and most importantly: Parallel out of the box. Hundreds of unit tests run instantaneously. Ruby TDD setups run tests that changed with maybe 1-2s lag... Elixir runs all the tests so fast that, at the beginning, I wasn't sure the tests were running.
gameswithgo 9y ago
shaders?
nikofeyn 9y ago
> I'm surprised that there hasn't emerged a "parallel-by-default C++" kind of language + hardware system to exploit it to keep things going forward. there has and it's called labview, although by hardware you may have meant processors. labview has many quirks, but it surprisingly gets many things right, even in futuristic ways. when i move back to text-based languages it's always a jolt primarily due to the serial nature of them, even those that support asynchronous computation. it's really hard to recalibrate to having to assign things to temporary variables and the like. and the lower dimensions of a text file compared to a higher dimensional canvas is something that sticks out as a limiting factor in supporting parallel by default. > I find the apparent stagnation extremely depressing. agreed.
TheLostOne 9y ago
With the web and requests the default would look to be parallel for most developers. The container runtime (Ruby, Python, Java) just abstracts away the parallelism on different cores.
eloisant 9y ago
There are many languages that handle well parallelism, but not all problems really need parallelism in the program itself. For example, for web programming, you can throw several machines (or processes) of your serial program to have it run in parallel for all practical purpose.
dragontamer 9y ago
> I'm surprised that there hasn't emerged a "parallel-by-default C++" kind of language + hardware system to exploit it to keep things going forward. CUDA. OpenCL. Vulkan Compute Shaders. DirectCompute. C++ AMP. AMD's ROCm. Intel's SPMD. Khronos SYCL.
0xcde4c3db 9y ago
Beyond the issues of languages and mindset, many problem domains parallelize poorly. Too many important phenomena fundamentally involve feedback loops evolving over time, and when that happens you can't just compute f(t) and f(t + 1) on different cores. At that point, throwing more cores at the problem might let you make the model bigger, but will very quickly hit a wall in terms of making it faster.
tbrock 9y ago
Golangs’s web server, by default serves every request in a new goroutine (thread of execution) making it parallel by default with no effort on the user’s behalf. Obviously this problem domain is easily parallelized but it’s nice to see parallelism be the refs to standard when possible and reasonable to do so.
- ninkendo 9y ago
  Just to be clear, goroutines aren't threads (although they are multiplexed onto threads), and they're only pre-empted at certain points in the go runtime (function calls are the big ones.) If your request spins in a for-loop doing lots of work without function calls, other goroutines on the same thread won't get a chance to run, and you'll be limited to GOMAXPROCS simultaneous requests. In practice this never really happens though.
ajross 9y ago
> I'm surprised that there hasn't emerged a "parallel-by-default C++" kind of language + hardware system There has. It's called a GPU. Things like OpenCL and CUDA are the new languages.
- vvanders 9y ago
  Yeah, the tricky bit is it's not just a new language but also laying out your data in a new way that compatible with vectorization/etc. Modern OO/etc techniques love to litter pointers to random places in memory at nearly every step. It's partly what made the PS3 so hard to write for, the SPUs only have 256kb of directly addressable memory, everything else is DMA'd. That said when you had your code+data fitting in 256kb it screamed everywhere else as well since you fit in L1+L2 cache neatly.
- fulafel 9y ago
  But it would be much better if a single language + hardware system emerged, rather than a multitude of mutually incompatible hardware systems and languages. With the current fragmented and sometimes proprietary forest of programming platforms, an application needs to be quite specialized to warrant investment in GPU compute outside the original niche of graphics acceleration. There are other giant problems too, after you get over rewriting your application for numerous different platforms - atrocious quality of GPU drivers causing OS crashes for users, lack of any common way to debug GPU code, the colourful quality of compilers, the wildly different performance characteristics of different platforms necessitating per-platform algorithm changes, etc... Consider what a minority of applications bother to even put in the work to exploit large amounts of CPU parallelism, which is vastly easier. There is after all >10x parallelism available on a typical PC CPU, after you count cores, threads and SIMD lanes.
  - ajross 9y ago
    > But it would be much better if a single language + hardware system emerged There will, but it takes time. The world of scalar hardware in the 1970's was no less fragmented. Honestly most of the incompatibilities in the SIMD world at this point are bugs and not fundamental problems. The vector world has settled on a broad architecture at this point for most things.
TickleSteve 9y ago
look at VHDL or Verilog programming for FPGAs. Its effectively exactly what you're talking about.
tomc1985 9y ago
This argument made sense 10 years ago. We've had dual cores for more than a decade, and CPU speeds stopped growing years ago. If you still can't think in threads and their primitives, or require crutches to handle multithreaded situations, then the problem is with you.
jbergens 9y ago
I guess it's not going to help a lot except in special cases. See for example what Linus had to say about many cores and the long discussion after. https://www.realworldtech.com/forum/?threadid=146066&curpostid=146227 https://www.realworldtech.com/forum/?threadid=146066&curpost...

by mjfl 9y ago

quadcore 9y ago

https://golang.org/ https://golang.org/ In case you dont know, Golang goroutines are a marvel of parallelism. They are coroutines which are dispatched on a few OS threads. So you can use 100% of a multi-cores CPU and yet, spawn, say, 10K of those light threads without worrying about context switches PLUS have them all run concurrently. I've found that golang is one of those rare language, like Lisp, that actually change the way you think about programming. Makes you feel really more powerful. If you dont know that language, I suggest to run the following and watch your cpu activity and memory (or any metric): import "time" func main() { for i := 0; i < 10000; i++ { go func() { for { time.Sleep(time.Second) } }() } }

dx034 9y ago
And yet, Go's standard map doesn't allow concurrent access. They recently added a concurrent map feature but it's probably easier to add a lock to your code than refactoring it with the new map type. I would've preferred if they had introduced a map type that can be used exactly as the standard one (i.e. without function calls). Calling functions via go func() is really easy but handling data between goroutines can still cause headaches and is something that could be improved.
- ta2384428 9y ago
  If you squint a little, you'll notice that go is pretty big on not hiding complexity. There are obvious exceptions to this such as the garbage collector, and heap/stack control. I'm unsure if the intent of not including a map function was due to this, however with a for loop such as for x := range ch { slice = append(slice, x) } , it is immediately obvious there are allocations happening in the background. The fact that map access, and slice access is not thread safe means there is no trickery going on in the background. The fact that it is not threadsafe means I don't need to worry about a lock if I only write to a map when it's created. Sure the compiler could take care of this - they have the race detector after all, but the compile speed is one of the design goals of go. I really like being able to compile in less than 1 second. `sync.Map` however calls a function which implies there is more going on in the background. If you follow the go mantra of share memory by communicate, don't communicate by sharing memory - handling data becomes a whole lot easier. It does allow you to share memory in case you do need the extra speed. Like most things, all designs are a matter of trade offs. Sacrifice one thing for another. There are other languages that provide the functionality you desire - but I understand the frustration when one thing is 90% what you want. Go obviously has room for improvement, and perhaps a native threadsafe map is one of those areas.
- quadcore 9y ago
  Well, go isn't really a high-level language which maps can be seen as a feature of. The idea behind go, I believe, is to focus and provide innovative system-level features.
  - eropple 9y ago
    Go is not a language for "system-level features" in the first place. Its garbage collector largely precludes it from that in any modern context. That it can't do something effectively does not mean that it shouldn't or didn't mean to. (And users can't really fix it, because hey, no generics. Sigh.)
    - quadcore 9y ago
      Go is not a language for "system-level features" in the first place. Its garbage collector largely precludes it from that in any modern context. I'm pretty sure I disagree with that largely because Go is low-level enough so that the GC is easily handled. And users can't really fix it, because hey, no generics. Sigh. Users can fix it by writing languages on top of Go (particularly dynamic ones).
tzahola 9y ago
Your example will not run in parallel. The go runtime will schedule your goroutines concurrently, but they will be run by a single OS thread, and consequently on a single CPU core. Once you execute truly on multiple CPU cores (by increasing GOMAXPROCS), you'll be having the same kind of race conditions in Go as in any other imperative language (inb4 Rust Evangelism Strike Force saying "except Rust").
- quadcore 9y ago
  Wrong. Goroutines are not simply coroutines. GOMAXPROCS defaults to number of cores.
  - tzahola 9y ago
    Prove it. Prove it by replacing Sleep in your example with some number crunching, and show how it scales with the number of cores in your CPU.
    - deleted 9y ago
      [deleted]
    - gall0ws 9y ago
      https://play.golang.org/p/o8odLCRqUMA https://play.golang.org/p/o8odLCRqUMA
    - ta2384428 9y ago
      https://imgur.com/a/DNpw3 https://imgur.com/a/DNpw3 Running the following code. https://play.golang.org/p/k_rRxNAyb0i https://play.golang.org/p/k_rRxNAyb0i I can assure you that go runs across all processors by default. https://docs.google.com/document/d/1At2Ls5_fhJQ59kDK2DFVhFu3g5mATSXqqV5QrxinasI/edit https://docs.google.com/document/d/1At2Ls5_fhJQ59kDK2DFVhFu3...
      - grkvlt 9y ago
        That's not what he's saying. He knows go can use all processors if GOMAXPROCS is set correctly, the argument seems to be that there will be race conditions just like any other threading, which seems pretty self evident to me: yes, multi-threaded code can have concurrency issues, film at 11...
VHRanger 9y ago
How is that an example an improvement over c++? C++ has had pragmatic omp parallel for for two decades
- ta2384428 9y ago
  I'm not familiar with OMP, so please forgive me if I'm incorrect - but OMP from my quick search appears to be thread based, while go routines are much lighter weight than threads. They have their own scheduler - which like most things is both good and bad depending on what you want them for. If you use them as intended, the internal scheduler is a great design decision. This means I can happily spawn 10,000 without concern. Additionally, combining them with the power of channels makes quick work of many tasks. Channels of course can be implemented in C++ too, but having the compiler take care of it for you with additional tools such as the race detector is very handy. For a large set of problems, they are very nice to work with.
  - pjmlp 9y ago
    You can do the same in C++ on Windows with PPL, UNIX/Windows with Intel TBB, or any of the fiber/co-routine libraries. Then there is the ongoing work to add async/await patterns into C++20.
    - quadcore 9y ago
      As a matter of fact, you can do the same in assembly if you want to. Notice how you can always have that answer when it comes to programming languages: you can do the same in. The point is that, the way it's made in go, is awfully handy.
      - pjmlp 9y ago
        I hardly see a difference from go func () { } and Task.Run(() => { }) With the benefit that on the latter example, the runtime allows me to customise how scheduling is done.
magnetic 9y ago
In my experience the facilities to throw tasks into a scheduler that will run them in parallel was never the hard thing to accomplish (regardless of the language: some may have built-in capabilities, others may have syntactic sugar provided by a lib, but at the end of the day, most systems provide some kind of runTask(f) method). What's really hard is to break down a problem into parallelizable chunks, figure out as much independent work as possible to reduce the touchpoints, and coordinate all those tasks such that they keep the CPU as busy as possible and as a whole finish as early as possible. Beyond this "parallel breakdown design", it's the little touchpoints with shared data structures and synchronization that creates the difficulty of implementation, and I haven't seen any language or system that does magic there.
- quadcore 9y ago
  throw tasks into a scheduler that will run them in parallel was never the hard thing to accomplish Common mistake number 1. Goroutines are running in parallel AND concurrently - they are coroutines (common mistake number 2 is to think they are only coroutines). I suggest not to underestimate that, it's the big deal. Additionnaly, it's important to note that goroutines yields on sleeps (any kind of sleeps / waits, like disk reads, network requests, channel writes/reads, etc), and while it may sound like a detail, it's a wonder of cpu control. Due to that, there is also a rare elegance to the way Go solve sharing data via channels. What's really hard is to break down a problem into parallelizable chunks, figure out as much independent work as possible to reduce the touchpoints, and coordinate all those tasks such that they keep the CPU as busy as possible and as a whole finish as early as possible. That's exactly what Go is a wonder for due to the combination of parallelism, concurrency, yield-on-sleep and channels.
brobinson 9y ago
>marvel of parallelism If Go's m:n is blowing your mind, you should check out Erlang. Spawning a million "processes" is not a big deal.
- quadcore 9y ago
  Cool, thanks for the suggestion.
Const-me 9y ago
> golang is one of those rare language Not sure how rare, .NET does that as well. Below is your sample translated to C#. It requires C# 7.1 because async main, but the rest of the stuff is available for many years, since 2010. static async Task routine() { await Task.Delay(TimeSpan.FromSeconds(1)); } static Task Main() { return Task.WhenAll(Enumerable.Range(0, 1000).Select(i => routine())); }
- quadcore 9y ago
  It's unclear to me if these parallel tasks are also coroutines (there seems to be a call to yield but its unclear what it really does), and if they yield on any sleeps which is a key features of goroutines.
  - Const-me 9y ago
    > if these parallel tasks are also coroutines I don’t have hands-on experience with golang but based on what I know about it yes, they are. > there seems to be a call to yield but its unclear what it really does You mean “await”? It waits for the completion of whatever is on the right side of “await”. If the result is already available, it just continues. If the result is not yet available (e.g. Task.Delay creates a task that will complete in some moment in the future), the control goes away from the async method to the scheduler. The scheduler can then run some other task on the same OS thread. When the result of that operation becomes available (in the sample code, when 1 second delay passes), the scheduler resumes execution of that async method, on the statement after the await. > if they yield on any sleeps No, not any sleep. You can call Thread.Sleep() which will put the whole OS thread to sleep. It’s up to programmers to avoid calling blocking APIs from their async methods, i.e. use await Task.Delay() instead of Thread.Sleep(), await stream.ReadAsync() instead of stream.Read(), and so on.

wiz21c 9y ago

Is it depressing or is it a kind of signal we should listen to and try to interpret ?

pjmlp 9y ago

A few attempts have already been done, the issue is with developer adoption, not lack of trying. One of the best examples was StarLisp for the Connection Machine.

convolvatron 9y ago
starlisp required that you rephrased your problem to be large vector, with support for turning off logical cpus, very much like programming a modern gpu. if your problem could be recast that way it was pretty nice. however the generalized scatter/gather was so expensive, it had to be used very sparingly. you had to use the grey coded nearest neighbor hypercube network as much as possible. the really* cool language from Hillis and Steele was cmlisp, but I don't know how far they got, they never released anything.

madez 9y ago

While Rust doesn't promote a specific model for parallelism, it's stronger compiler helps a lot with that. https://doc.rust-lang.org/beta/nomicon/concurrency.html https://doc.rust-lang.org/beta/nomicon/concurrency.html https://doc.rust-lang.org/book/second-edition/ch16-01-threads.html https://doc.rust-lang.org/book/second-edition/ch16-01-thread... I don't think fixing C(++) can give what Rust can give, because Rust has a clean start with these strong guarantees built-in while for C(++) it would always be an addon. Defaults are powerful.

beojan 9y ago
C++ has parallel algorithms built in (http://en.cppreference.com/w/cpp/algorithm http://en.cppreference.com/w/cpp/algorithm). Parallelism is complicated though, and easy parallelism pretty much requires a functional style. Things like Haskell's Accelerate library (https://www.stackage.org/package/accelerate https://www.stackage.org/package/accelerate) seem the ideal way forward to me.
- smitherfield 9y ago
  Should be noted that so far only Visual Studio (partially) implements C++17 parallel algorithms, although there are several third-party implementations.[1] Apples to oranges comparison of course, since Rayon isn't [planned to become?] part of the Rust standard library. [1] http://www.bfilipek.com/2017/08/cpp17-details-parallel.html#current-implementation http://www.bfilipek.com/2017/08/cpp17-details-parallel.html#...
  - beojan 9y ago
    If you're including third-party libraries, you have things like ArrayFire and Intel TBB.
chrismorgan 9y ago
A really powerful outworking of it is seen in the Rayon crate, where you can change a sequential iterator into a parallel iterator just by adding the crate, importing the trait and changing `iter` to `par_iter`. If it’s not thread-safe to do, then it won’t compile. (That’s the big difference from C++.) If it is, it will, and it’ll be smart about how it runs, spreading the load across all available cores pretty much optimally, or not bothering with multiple threads if it’s not going to be worth it (e.g. single-threaded, or only one item in the iterator). And all that with close enough to no overhead. Basically, Rayon makes data parallelism really easy in a way that few if any other languages do. I’d love to have an equivalent in Python or Node, but it’s just not possible to achieve such a thing in most languages—even if you ignore the thread safety aspect. Parallelism hasn’t seen a great deal of use until it’s urgently needed, because it’s hard to get right in most environments, and you normally need to substantially refactor code to make it happen. My hope is that with the likes of Rayon, parallelism can be a much more natural thing that people that care even a little about performance will just do, because it’s so easy to do. https://crates.io/crates/rayon https://crates.io/crates/rayon

elorant 9y ago

Microsoft has developed Parallel Linq which is a subset of .net framework and works like a charm.

ddorian43 9y ago

Seastar framework

krylon 9y ago
Honest question: Is that name a pun on C* (https://en.wikipedia.org/wiki/C* https://en.wikipedia.org/wiki/C*)?

jokoon 9y ago

I wonder if the future will be massively parallel, when CUDA and opencl came out I thought that future processors would have more and more core, so if you follow Moore's law, CPUs would see their core count double each 6 months. The problem is that GPU don't have error correcting codes, so you cannot really run application code on a GPU. The problem with parallelism is that C-like language don't fit well, only functional languages do. If you want to use multi-threading you have to forget about state and only work with input/output paradigms. For OSes it might mean a deep re-design, but I don't really know. A possible design would be a small but very fast CPU that only takes care or scheduling and task control, and another chip with many cores that deal with payloads and user software. AMD had some kind of hybrid chip that planned to do both graphics and task, but it was thrown away. Going parallel would require to change both hardware and software, and by software I mean stateless.

tzahola 9y ago
Intel Larrabee also got cancelled.
- pjmlp 9y ago
  Kind of, it became Xeon Phi.
zmonx 9y ago
These are very good points! I would generalize "functional" to declarative though: Logic programming languages like Prolog and Mercury are also much more amenable to parallelization than C-like languages. In fact, different Prolog clauses could in principle be executed in parallel without changing the declarative meaning of the program, at least as long as you stay in the so-called pure subset of the language which imposes certain restrictions on the code.
pjmlp 9y ago
I always like to think of FP dataflow pipelines as doing digital circuits modeling.
pjc50 9y ago

blauditore 9y ago

I think part of the reason why many programmers haven't wanted to deal with parallel execution is because concurrency is not easy to handle. It has several pitfalls and can be painful to debug. Also, it needs proactive efforts implementing it, so as long as not required, devs just stick to serial execution. Now, with helpful systems like no-side-effect functional languages and reactive stream frameworks, a lot of gory detail can be abstracted away. I think this has recently lead to more parallel-by-default software development.

kbart 9y ago

While modern languages support for parallelism is adequate, tools are still lacking imho. I avoid parallelism when it's not necessarily, because debugging all these race conditions, deadlocks, synchronization etc. is a nightmare.

pjmlp 9y ago
That is mostly an issue with the "avoid IDE" crowd. While IDE tooling can still be improved, the parallel debugging tools in .NET and Java eco-systems are already quite good. On VS, I can have at any given moment a graphical snapshot on how all threads and tasks are interacting with each other, or just execute some of the threads. It doesn't solve everything, but it makes it easier than a typical gdb session.
- kbart 9y ago
  .NET has one of the best ecosystem overall, so it's more an exception than a rule (can't comment on Java as I don't work with it). As I'm mostly in low level, embedded system programming, you are still stuck with gdb, Valgrind and other primitive tools there, because if there's some legacy "IDE" at all, it's most often just some half-assed Eclipse plugin.
  - pjmlp 9y ago
    At least looking to their product sites, Microchip and Green Hills seem to have quite good tooling, then again I don't have embedded experience on modern systems, beyond mobile devices.
    - kbart 9y ago
      Do Microchip even produces multi-core MCU's? Haven't seen one, though I worked mostly with Cortex-A series so might miss something.
      - pjmlp 9y ago
        As I said, I don't have much experience in real embedded domain outside mobile devices (iOS, Android, UWP), but aren't Cortex-A5 supposed to handle up to 4 cores?

Sean1708 9y ago

Chapel[0] and Fortress[1] come to mind. Wikipedia also has a list of parallel programming languages[2] (although it seems to play somewhat fast and loose with the definition of "parallel programming language"). [0]: https://en.wikipedia.org/wiki/Chapel_(programming_language) https://en.wikipedia.org/wiki/Chapel_(programming_language) [1]: https://en.wikipedia.org/wiki/Fortress_(programming_language) https://en.wikipedia.org/wiki/Fortress_(programming_language... [2]: https://en.wikipedia.org/wiki/List_of_concurrent_and_parallel_programming_languages https://en.wikipedia.org/wiki/List_of_concurrent_and_paralle...

adrianN 9y ago

Most software is fast enough when written in a naive sequential style. For the parts that parallelize well and matter, there are already decently mature ways of using all cores. Languages like Go, Rust, and Erlang make it fairly easy to write concurrent programs.

tyingq 9y ago

"there hasn't emerged a "parallel-by-default C++" kind of language + hardware system to exploit it" That's roughly what Intel tried with Itanium. I don't know if whatever barriers they hit are still barriers today.

pjc50 9y ago
Not really - Itanium tried to move the instruction-level parallelism and the burden of re-ordering execution to the compiler. You could only execute the maximum six instructions per cycle if there were no data dependencies. So it only really works for certain kinds of algorithmic data-heavy execution; that's why VLIW is only found in DSP architectures today. Good SE answer here: https://softwareengineering.stackexchange.com/questions/279334/why-was-the-itanium-processor-difficult-to-write-a-compiler-for https://softwareengineering.stackexchange.com/questions/2793... especially the ones focusing on cache misses.
- tyingq 9y ago
  "Itanium tried to move the instruction-level parallelism and the burden of re-ordering execution to the compiler" That sounds a lot like "parallel-by-default C++" kind of language + hardware system to exploit it" Just swapping "compiler" for language. They didn't succeed, but they did try Edit: helping me understand where I'm off might be more helpful than a downvote. Does swapping "compiler" for "language" not respresent what Intel was trying to do?

Pica_soO 9y ago

The sequential programming by default is because most people think sequential by default. Even though claims of multi-tasking etc persist, the truth is good parallel programmers are a rare thing. Many ordinary programmers already get into Hot-Water when they use two threads and access data where a Semaphore might be needed. In addition many algorithms are sequential, so parallelizing them is tricky or gives you no true reward due to cross-thread communication. Add to that the OO-Software Structures that subtle encourage using sequential programming. I think the future in parallel-programming is actually hiding the parallel programming completely - accept the fact that most humans are not made for it, allow for experts to unlock the ability to override that behavior- let compilers go as far as they can and live with the results. It will suffer the same fate as functional programming. Really useful, but never dominant, due to limitations in the applying humans.

pjc50 9y ago

Unfortunately it requires more than just a change of language, it requires a change in mode of thinking by developers. People are very used to reasoning in terms of "do X, then Y, then Z" or "compute a value X then do A or B on the basis of that". In order to achieve automatic parallelism you need e.g. a type+proof system that can determine that X/Y/Z are independent, or a system that can partially execute both A and B then retire the branch not taken - without invoking security bugs!

pjmlp 9y ago
I think it is easier to understand by those of us that also had electronic design as part of the engineering degree.
- deleted 9y ago
  [deleted]

MichaelMoser123 9y ago

> However, most programmers, and programming languages, remain stuck in a serial-by-default paradigm You still have to decide upon the unit of work that is going to be sent to a different thread/core/processor/NUMA node/whatever. The different units of work that are distributed should not share state; one really doesn't want to be sharing a lot of state between different processors, because synchronizing the processors memory caches in NUMA is a extremely slow. I guess it is really hard to break up both the program and data and decide upon the optimal granularity of the work units, it is not something that can be easily done behind the scenes - human intervention is still required.

z3t4 9y ago

I think JavaScript is nice because it's async in nature. Concurrency is hard so it's nice to deal with it using a simple language, so that everything besides the business logic is abstracted. Yes you do not get the same performance, but CPU cores are relatively cheap compared to engineer salaries.

pdpi 9y ago
The notion that cpu cores are cheap compared to engineer salaries only scales so far. Scaling upwards, your opinion on that changes when a single engineer’s service is running on 10k machines. At the other end of the spectrum, if you’re developing high-performance applications for small systems (desktop, laptops, mobile), Your workload isn’t going to look like tens of thousands of concurrent independent requests, So the approach of getting parallelism by deploying multiple copies of the application no longer works
dx034 9y ago
> but CPU cores are relatively cheap compared to engineer salaries I often experienced that this backfired. Single machines are still constrained in their power and while it's easy to spin up additional VMs in the cloud, scaling a program properly to run on dozens of machines takes a lot of work. It can be faster to develop a program that is really efficient and can solve the problem on one machine than to develop faster only to then spend the time scaling it to a large fleet of servers.
- z3t4 9y ago
  I kinda regret adding the note about performance, because switching to a lower level language used to yield orders of magnitude more performance, but optimizers have evolved and today there's not much difference. Sometimes the higher level language will be even faster because of optimizations. And bad performance is often not to blame on the language, instead blame the programmer or more likely the business people as they think it's great to waste resources as it gives them an excuse to charge more.
  - ska 9y ago

swiley 9y ago

There are limits to how much parallelism will improve things, see amdahl's law.

kamaal 9y ago
Sure, but let's start moving towards parallelism first. May be a few decades from now we would be close to breaking those limits.
- Something1234 9y ago
  That law is a fact of life. Its a form of diminishing returns, there's only so much you can do.

Santosh83 9y ago

One reason is that our thinking process is inherently serial and we do really badly at multitasking naturally. At least those involving deliberate thought. And our programs are almost an extension of our way of thinking, so we will have to push the boundaries of how we think and model systems in our mind before we can build excellent parallel programming languages. Not that it isn't being done, but the weight of the "serial" legacy is long...

monocasa 9y ago
It's not just our thought process; there are upper limits to the gains from increased parallelism. https://en.wikipedia.org/wiki/Amdahl%27s_law https://en.wikipedia.org/wiki/Amdahl%27s_law

UncleSlacky 9y ago

This issue was discussed in my Occam class ~28 years ago. Occam itself is an example of a concurrent/"parallel-by-default" programming language intended for a Transputer hardware environment (now retargetable to x86 etc.), but it's not the easiest of languages to learn: https://en.wikipedia.org/wiki/Occam_(programming_language) https://en.wikipedia.org/wiki/Occam_(programming_language) https://en.wikipedia.org/wiki/Transputer https://en.wikipedia.org/wiki/Transputer https://en.wikipedia.org/wiki/KRoC https://en.wikipedia.org/wiki/KRoC

digi_owl 9y ago

I can't shake the feel that this parallelism thing will be nothing but a wild goose chase, because nature seems to be highly serial in all but the most macroscopic of senses.

pjc50 9y ago
All of your cells and neurons operate in parallel.
- adwn 9y ago
  And yet you only have a single train of thought [1] – or multiple interleaved trains, but that's concurrency, not parallelism. [1] Plus a couple of background processes, like breathing, that execute in parallel.
  - boomlinde 9y ago
    That's a huge simplification not pertaining to the fundamental nature of the brain. Maybe it is in the nature of our consciousness (which in itself is probably only an abstract concept) to perceive the processes it emerges from as a sequence of single trains of thought rather than a chaotic, continuous consolidation of processes both internal to the brain/body and outside of it. You could look at society as a whole and see the zeitgeist as a singular "train of thought", but you'd probably still recognize humans as individual agents. I think we have a bias towards thinking of ourselves as the ultimate individuals, neither recognizing the processes within us (like those of our cells) or the processes beyond us (like those of a group of people, animals, plants etc.) as having a similar nature. This is probably a genetically advantageous trait.
    - adwn 9y ago
      > That's a huge simplification not pertaining to the fundamental nature of the brain. I disagree. Conscious thinking is a crucial process that's inherently single-threaded, even though it runs on highly parallel hardware (the billions of neurons). However, I must admit that my point doesn't necessarily contradict pjc's point, and I don't really agree with digi_owl's claim that parallelism is a "wild goose chase".

skrebbel 9y ago

> Basically the message was that serial performance is saturating, and the only way to get speed improvements in the future is going to be by exploiting parallelism. People have been warning us about this for about 10 years now, but I still don't see those 64-core CPUs I was promised anywhere. If we had the amount of parallelism we were told we were going to get, we could give every app its own core. OSes could even consider disabling context switches altogether for the majority of apps. Instead, we're left complaining about Electron apps like it matters. That said, I'm not sure what stagnation you refer to. There's a reason languages like Rust, Elixir/Erlang and Go are getting popular. My PHP app could handle hundreds of concurrent connections on a single machine, my Elixir app handles hundreds of thousands. Yet, processors didn't get 1000x faster (and Elixir isn't even a particularly fast language). This is the opposite of stagnation, it's progress.

skummetmaelk 9y ago
They have been here for a while, just for a very specific market. Threadripper and EPYC exist now though. With 32 and 64 logical cores respectively.
- skrebbel 9y ago
  Sure, there's been high end niche products for anything forever. You could buy a 64 core computer in 1990 (they'd call it a supercomputer, but same thing). The people telling us we had to go change our code to use parallel processing fast predicted a significantly faster increase in amount-of-cores on commodity hardware. Instead, CPUs stopped getting faster and hardly gained more parallellism.
  - pdpi 9y ago
    You could buy a 64 processor machine in 1990, but it wouldn’t have been a 64 core machine in the sense we’re talking about — a single socket system. This isn’t some trivial distinction either, as the whole memory architecture is very different indeed for the two scenarios.

otabdeveloper1 9y ago

I'm surprised that there hasn't emerged a "parallel-by-default C++" kind of language + hardware system to exploit it to keep things going forward. There has. GPU programming is exactly that. CPU-heavy tasks (games, bitcoin mining, machine learning) have already migrated.

jonathanstrange 9y ago

I agree, programming languages have not caught up yet. The right kind of language looks at serial program formulations and based on flow-analysis automatically identifies parallelizable fragments that are large enough to benefit from multicore, then schedules these fragments e.g. by using work-stealing in a system of green threads, i.e., mapping green threads to OS cores as efficiently as possible. Something like that. In a good parallel language there need to be many immutable constructs by default, exception handling is tricky, and ordinary flow control needs to be compatible by default with parallel evaluation. The languages I've seen such as Parasail are not yet production ready. Making the programmer control parallelism can be okay, like in Go and Ada, but in the end it should be automatic. Edit: The problem is also that finding a neat way of solving the problem academically does not readily translate into an efficient implementation, so much that I wonder whether green threads are actually worth it over OS threads. In most languages/VMs they aren't but Go seems to be an exception.

Entalpi 9y ago

X10 is kind what you are describing. https://en.wikipedia.org/wiki/X10_(programming_language) https://en.wikipedia.org/wiki/X10_(programming_language)

BenoitP 9y ago

I'd say exploiting parallelism is not the only way at all. Parallelism is only one way to compute differently. Specialization of hardware to specific workloads will explode in the next years as we can't rely anymore on Moore's law. This will happen on RISC-V, IMHO. We already have these: * Rendering, medium precision mathematics: GPU * Low precision mathematics: TPU * Software Defined Networking: Microsoft is deploying FPGAs, AWS has its own hardware We could have: * Databases: Projections, hashing, sorting in hardware. * Dynamic runtimes: Hardware implemented memory models, HW assisted GC, code caches and user-level interrupts for the JITs. Here is the J extension RISC-V working group: [1] etc. Also, why not have the usual hot paths in Node.js|Spring Framework|Django directly etched into hardware? HW http header parsing surely could bring benefit to them all. ---- Of course language and programmers will have to adapt, but in a lot of cases the runtimes will take care of it automatically. [1] https://groups.google.com/a/groups.riscv.org/forum/#!msg/hw-dev/itbdrL_zdrc/t4B5fdbgAAAJ https://groups.google.com/a/groups.riscv.org/forum/#!msg/hw-...

microcolonel 9y ago
Currently working on software for RISC-V and I can definitely see some opportunities here. One thing that's becoming apparent though, is that some level of abstraction needs to be available in order for many of these things to become useful. RISC-V has a very exciting Vector processing extension, which is intended to replace packed SIMD in most cases, but some software systems assume packed SIMD is the only way to get more FP performance. For example, WebAssembly specifically exposes SIMD primitives, which means that it may be necessary to work backwards from those SIMD primitives to make use of a true vector machine. I think many people simply underestimate the cost of adopting a new programming model. > Also, why not have the usual hot paths in Node.js|Spring Framework|Django directly etched into hardware? HW http header parsing surely could bring benefit to them all. Well, in all the listed cases here, the CPU is not the bottleneck on throughput. As far as I can tell problem with HTTP is not that headers take too long to parse, it's that memory is still too slow, and context switches cost us precious time. The problem with Node is not that the hardware doesn't adequately model the semantics, it's that dynamic, weak typing makes it hard for any system (software or hardware) to understand what type things are. Update: The J extension seems interesting, and I've read some research (not thoroughly) recently showing considerable power and time savings from hardware GC primitives. I'm excited to see what goes on in that committee.

Androider 9y ago

With just Chrome and IntelliJ running on Ubuntu, I have 281 processes and kernel workers running (as reported by ps). Just stitching the apps together for display on your screen requires 3-4 different processes (and a GPU). That's why 4 logical cores is a bare minimum these days for a desktop, even if no individual application takes advantage of more than a single core. On the server side, when we deploy a Node.js web service on AWS we start one instance per logical core, for 4-64 processes all independently serving connections. It seems the process has become the new thread, the smallest unit you should design for. So today's workloads actually make pretty good use of all those cores. Unless you're doing high performance computing and need to squeeze every last drop of performance, processes are a straightforward way to parallelize.

rbanffy 9y ago
Having 281 processes is not a biggie. Having 281 processes that can actually use a piece of the CPUs is quite a different thing. I think we should, really, start thinking about such things. Maybe prefixing instructions with the execution unit that should handle them (and overflow back to the first one in a circle if we have more EU's in software than the actual hardware provides), separating dependencies within code flow in a more explicit way and, at the same time, not bothering with creating threads.

quietbritishjim 9y ago

One option is that you write in a high-level language where the top-level control code is single threaded, but you call APIs that perform multi-threaded operations seamlessly. The prototypical example of this is Python with numpy/blas (or with deep learning libraries like TensorFlow).

PaulHoule 9y ago

Materials other than silicon can support higher clock rates. From reading the open literature and advertisements by foundry companies, I think you could make a 6502 equivalent processor with Indium Phosphide with 64kb of static RAM that clocks at 30 GHz. With a more refined process you might push 90 GHz and a much more complex processor. Yes, InP is more expensive than Silicon but part of that is the low volume that InP parts are made in. Advances in Silicon are getting much more expensive, and one InP microprocessor could do the work of ten Silicon-based cores so you can save on die area without the "race to the bottom" in size. The main issue with high clocks is fast access to memory, probably you would need an optical interface to off-chip RAM, also I don't know what the InP equivalent of DRAM is. (Something like Optane?)

deepnotderp 9y ago
1. Compound semiconductors often suffer from p-type and n-type conduction problems. Plus they're ridiculously expensive and power hungry. They also often lack a native oxide. 2. The problem today in CPUs is not really clock speed but much more the memory access latency, optane is much slower* than DRAM and has much lower endurance. *Even though silicon HKMG transistors use high k gate dielectrics now, they still use a silicon dioxide interfacial layer.

contravariant 9y ago

Haskell kind of qualifies. You can't have race conditions when everything is immutable. Of course this makes some other things a bit more difficult.

kamaal 9y ago

New languages are coming up, it will just take long adoption cycles, given the library and surrounding tooling ecosystem has to be mature enough to go with it. Prominent examples being Go and Perl 6. Perl 6 especially, given how audacious the project is. There are performance issues with it currently though, from what I hear they are working to fix it soon.

b2gills 9y ago
I remember someone posting some Perl 6 code and C code that did the same thing. The Perl 6 code was shorter, easier to understand, more correct (Unicode), and was reported to be faster for what they were doing. There are things which are slower, but since it is a higher level language it may be easier to try multiple algorithms one of which may be significantly faster. It also has many useful features included, which can be optimized in ways that aren't recommended for user code. (writing the algorithm in NQP) There is also a code specilizer (spesh) and a JIT. Basically for many things it can be fast enough. Also if you profile your code and find something that is egregiously slow you should report it. Many times such things get optimized quickly.

Bitcoin_McPonzi 9y ago

In fact, there is! Erlang is extremely parallel. Processes are lightweight, share nothing, and run on as many CPUs you have.

djsumdog 9y ago
And also its derivatives, like Elixir.
- losteric 9y ago
  Elixir is amazing! The original author, José Valim, took a brilliant approach: Take a 30 year-old battle-tested highly-parallel VM, and build a modern language on top of it. Syntax is inspired by the good parts of Ruby (very clean), but nothing comes close in terms of the ease of parallelism... it's so natural, and insanely fast. The unit tests are what convinced me this will be the next big thing. Beautifully clear syntax, succinct tests, and most importantly: Parallel out of the box. Hundreds of unit tests run instantaneously. Ruby TDD setups run tests that changed with maybe 1-2s lag... Elixir runs all the tests so fast that, at the beginning, I wasn't sure the tests were running.

gameswithgo 9y ago

shaders?

nikofeyn 9y ago

> I'm surprised that there hasn't emerged a "parallel-by-default C++" kind of language + hardware system to exploit it to keep things going forward. there has and it's called labview, although by hardware you may have meant processors. labview has many quirks, but it surprisingly gets many things right, even in futuristic ways. when i move back to text-based languages it's always a jolt primarily due to the serial nature of them, even those that support asynchronous computation. it's really hard to recalibrate to having to assign things to temporary variables and the like. and the lower dimensions of a text file compared to a higher dimensional canvas is something that sticks out as a limiting factor in supporting parallel by default. > I find the apparent stagnation extremely depressing. agreed.

TheLostOne 9y ago

With the web and requests the default would look to be parallel for most developers. The container runtime (Ruby, Python, Java) just abstracts away the parallelism on different cores.

eloisant 9y ago

There are many languages that handle well parallelism, but not all problems really need parallelism in the program itself. For example, for web programming, you can throw several machines (or processes) of your serial program to have it run in parallel for all practical purpose.

dragontamer 9y ago

> I'm surprised that there hasn't emerged a "parallel-by-default C++" kind of language + hardware system to exploit it to keep things going forward. CUDA. OpenCL. Vulkan Compute Shaders. DirectCompute. C++ AMP. AMD's ROCm. Intel's SPMD. Khronos SYCL.

0xcde4c3db 9y ago

Beyond the issues of languages and mindset, many problem domains parallelize poorly. Too many important phenomena fundamentally involve feedback loops evolving over time, and when that happens you can't just compute f(t) and f(t + 1) on different cores. At that point, throwing more cores at the problem might let you make the model bigger, but will very quickly hit a wall in terms of making it faster.

tbrock 9y ago

Golangs’s web server, by default serves every request in a new goroutine (thread of execution) making it parallel by default with no effort on the user’s behalf. Obviously this problem domain is easily parallelized but it’s nice to see parallelism be the refs to standard when possible and reasonable to do so.

ninkendo 9y ago
Just to be clear, goroutines aren't threads (although they are multiplexed onto threads), and they're only pre-empted at certain points in the go runtime (function calls are the big ones.) If your request spins in a for-loop doing lots of work without function calls, other goroutines on the same thread won't get a chance to run, and you'll be limited to GOMAXPROCS simultaneous requests. In practice this never really happens though.

ajross 9y ago

> I'm surprised that there hasn't emerged a "parallel-by-default C++" kind of language + hardware system There has. It's called a GPU. Things like OpenCL and CUDA are the new languages.

vvanders 9y ago
Yeah, the tricky bit is it's not just a new language but also laying out your data in a new way that compatible with vectorization/etc. Modern OO/etc techniques love to litter pointers to random places in memory at nearly every step. It's partly what made the PS3 so hard to write for, the SPUs only have 256kb of directly addressable memory, everything else is DMA'd. That said when you had your code+data fitting in 256kb it screamed everywhere else as well since you fit in L1+L2 cache neatly.
fulafel 9y ago
But it would be much better if a single language + hardware system emerged, rather than a multitude of mutually incompatible hardware systems and languages. With the current fragmented and sometimes proprietary forest of programming platforms, an application needs to be quite specialized to warrant investment in GPU compute outside the original niche of graphics acceleration. There are other giant problems too, after you get over rewriting your application for numerous different platforms - atrocious quality of GPU drivers causing OS crashes for users, lack of any common way to debug GPU code, the colourful quality of compilers, the wildly different performance characteristics of different platforms necessitating per-platform algorithm changes, etc... Consider what a minority of applications bother to even put in the work to exploit large amounts of CPU parallelism, which is vastly easier. There is after all >10x parallelism available on a typical PC CPU, after you count cores, threads and SIMD lanes.
- ajross 9y ago
  > But it would be much better if a single language + hardware system emerged There will, but it takes time. The world of scalar hardware in the 1970's was no less fragmented. Honestly most of the incompatibilities in the SIMD world at this point are bugs and not fundamental problems. The vector world has settled on a broad architecture at this point for most things.

TickleSteve 9y ago

look at VHDL or Verilog programming for FPGAs. Its effectively exactly what you're talking about.

tomc1985 9y ago

This argument made sense 10 years ago. We've had dual cores for more than a decade, and CPU speeds stopped growing years ago. If you still can't think in threads and their primitives, or require crutches to handle multithreaded situations, then the problem is with you.

jbergens 9y ago

I guess it's not going to help a lot except in special cases. See for example what Linus had to say about many cores and the long discussion after. https://www.realworldtech.com/forum/?threadid=146066&curpostid=146227 https://www.realworldtech.com/forum/?threadid=146066&curpost...