3 ms·
"The Japanese thought they could parallelize Prolog programs for their Fifth Generation Computing Project in the 1980s but found out quickly that you couldn't."
by jerf 3y ago
"The Japanese thought they could parallelize Prolog programs for their Fifth Generation Computing Project in the 1980s but found out quickly that you couldn't."
One of the lessons I'm still trying to absorb is how convinced we all were (and many still are) that there must be a ton of implicit parallelism in the world, but once we went looking for it, it turned out there was hardly any.
I've been chewing on this for years, trying to figure out whether this is something true about the world, true about the problems we try to solve, or because of the pervasive use of the imperative paradigm which is highly ordered and makes it very easy to impose ordering constraints. Now, clearly, the latter is a non-trivial component... but I still struggle with whether it is a full answer, because when people sat down with clean sheets of paper and tried to solve problems with extensive parallelism, they've largely failed to get more than low single-digit speedup factors. Non-imperative paradigms like logic programming have been tried, especially in these projects.
It isn't exactly news that our intuitive beliefs about our code and the real characteristics it has is quite out of whack. This underlies the pervasive advice to just go grab a profiler whenever you're trying to accelerate some code because even experts in the field with decades of experience are routinely completely wrong about what is slow in some bit of code. This is one particular aspect I'm still struggling with. It still feels like there should be so much more parallelism available....
- PaulHoule 3y agoPart of the "low code" puzzle is that part of the job of the professional programmer is to determine what sequence events need to happen in time. There are some cases (make and the Spring Framework) where the user specifies the dependencies between things and the framework does a topological sort to figure out what order to do them in. Even though this could be a basis for a whole paradigm of programming, professional programmers usually don't find it hard to figure out how to order things so there is little motivation to address this problem systematically. However, I think it is on the list of problems that non-professional programmers (say the subject matter expert who wants to learn Python to put their skills on wheels) get hung up. Instead it's seen as a special case that makes high-complexity systems scalable (... Spring lets you describe the parts of a system and how they are related without writing structurally unstable initialization code.)
- nerdponx 3y agoI think part of the problem is that there is plenty parallelism available, but it's not "high impact" parallelism, in that it's not worth whatever overhead you might incur (whether in language complexity or runtime overhead). I think most "business logic" is fairly linear most of the time, and that even if it's highly branched, usually you don't have a large number of concurrently executing units of logic. You do have things like iterating/mapping over arrays, but that's precisely what I think is generally not "high impact" to parallelize. And when it is high impact, we do in fact have a rich space of tools available in the programming and software world, depending on the nature of the task at hand. I know nothing why this particular Prolog parallelization project failed, but the above is generally my intuition about parallelism in computer programming, so I imagine it's at least somewhat relevant in this case.
- PaulHoule 3y agoOddly superscalar CPUs find quite a bit of parallelism at the micro level in ordinary scalar code although that is tied up very much with managing the comparatively slow motion of data from RAM to CPU and it is something the processor has to discover on the fly as the compiler can’t really know what will be in the cache every time.