12 ms·

Disagree on that last part. If more programmers understood cache coherency, maybe their programs would not run like a giant turd.

• codetrotter 7y ago
  I agree with you Jonathan but am wondering, will Jai help programmers write programs with better cache coherency even if said programmers don’t understand cache coherency well? Or is that orthogonal to the goals of Jai?
  • cma 7y ago
    If it still plans on wrapping SOA in something that looks in use like AOS, it could make people less aware of how their code is impacting cache (would now have to look at the definitions instead of seeing the array form at point of usage). But if it is enough more ergonomic to write AOS code it might still be worth it and increase uptake.
    • Sean1708 7y ago
      The developer would still need to understand enough to be able to choose between them though, even if using them is identical.
  • naikrovek 7y ago
    Like any reasonable language, I imagine JAI will allow a developer to write software with the cache in mind. It will not force them to, nor will it prevent them from doing otherwise.
  • BubRoss 7y ago
    Only if he releases it
• strstr 7y ago
  Most engineers don't write code with hard performance constraints. Game devs probably need to be fighting to get every frame. For the bulk of the eng I work with the concept of StoreLoad reordering on x86 would be an academic distraction.
  • kasey_junk 7y ago
    The problem is you never know when you’ll go from “dev who doesn’t care” to “dev who does”. While I agree that the details of StoreLoad are likely a distraction the big picture concepts of cache coherence presented in this article are table stakes for performant systems.
    • strstr 7y ago
      While that might be true in practice, I do think not knowing when you transition between those two indicates a failing by your coworkers/mentors. I encourage people I work with to read many of the books in this book series. I particularly encourage them to read “Hardware and Software Support for Virtualization”, since it’s basically a book on their job.
      • steev 7y ago
        Just to be clear, the book series you are referring to is "Synthesis Lectures on Computer Architecture" published by Morgan-Claypool, right? If you had to rank them in importance for the average engineer, how would you rank them?
    • onion2k 7y ago
      The problem is you never know when you’ll go from “dev who doesn’t care” to “dev who does”. This is true for literally every hard problem in dev though, and the implication that you need to grasp everything just in case you need it is silly. The problem space in compsci is too big to know everything. We have to choose.
      • afiori 7y ago
        Part of the reason this kind of blogs exist is to give you more information to choose properly.
  • AnIdiotOnTheNet 7y ago
    > Most engineers don't write code with hard performance constraints. Only because most software "engineers" don't give two shits about the actual user experience of their glacially slow over-engineered garbage.
• fulafel 7y ago
  Are the details really helpful for performance work? MOSI, MOESI, MERSI, MESIF etc are 99% irrelevant to having the right metal model. "Dirtying cache lines across different cores/threads is slow" is most of what you need to know. Within the same core the coherency protocols between levels of cache is not really visible at all to software even as varying performance artifacts. Of course you might end up analyzing assembly level perf traces in a hot path for some game console with a less known CPU architecture and making a cross cpu cache miss slightly less slow could just maybe be helped by the detailed understanding of the machine model, but by that time you're already far in the not-giant-turd territory (at least if you're optimizing the right thing). Of course computer architecture is fascinating and fun to learn about.
  • vardump 7y ago
    > Of course you might end up analyzing assembly level perf traces in a hot path for some game console with a less known CPU architecture Or if you work in finance, mining, oil industry, medical, biosciences and countless other fields where you need to get good performance out of your hardware. Yes, even if you use GPUs, they're no magic bullet, they also have their architectural bottlenecks, strengths and weaknesses. Or if you care about power consumption. There are a lot of reasons to optimize hot paths and inner loops. CPU single core performance isn't improving much anymore, and we need to make better use of what we have.
    • fulafel 7y ago
      I was trying to say that even in those cases you almost never benefit from knowing the details of cache coherency.
      • vardump 7y ago
        That's a very bold statement. You could just as well for example say that front end Javascript developer almost never needs to understand event callbacks or how DOM works. If you write multithreaded high performance code, yeah, you do need to know about cache coherency at varying levels of detail. Sometimes rough rules of thumb work, not too often you need to understand all those annoying performance destroying details that leak through cache abstraction.
• jacobush 7y ago
  Maybe but it feels like most are stuck in environments which will do bad things to their cache coherence. It's fine if you are doing some data processing in C. If you are using .NET with a bunch of magic libraries or Javascript or whatever, sure it will help, but to actually make an impact you have to be very careful.
• devnonymous 7y ago
  I disagree on this. More programmers should understand the performance characteristics of the abstraction layers that they rely on. Else we have the case of jerk programmers who insist on redesigning / rewriting / refactoring to optimise for CPU caches while still running the apps via a bunch of docker containers each based of the centos image to run one simple binary that probably needs only glibc.
• DrScientist 7y ago
  Surely the whole point of good system design is a set of logical abstractions, where you need to understand the logical model and not the internal details - as these are free to be evolved. Of course performance matters, but surely having performance tests, rather than trying to second guess what the whole stack below you might be doing, is 1. more efficient 2. more accurate 3. more likely to detect changes in a timely way. That's not to say, you shouldn't be curious and deep understanding isn't a good thing. Just saying understanding inside-out the abstraction you are working with ( eg Java Memory Model ) it's performance characteristics ( from real world testing ) - is more important than some passing knowledge of real world CPU design. This app I am using right now is in a webbrowser - not sure how understanding cache coherency helps in a single threaded javascript.
  • skohan 7y ago
    > Surely the whole point of good system design is a set of logical abstractions, where you need to understand the logical model and not the internal details In my opinion an important part of being a good programmer is understanding - at least at a broad level - how the set of abstractions you're working on top of work. At the end of the day, our job is making computer hardware operate on data in memory. The more that we forget that, and think about computing as some abstract endeavor performed in Plato's heaven, the more tendency we have for bloat and inefficiency to creep into our various abstractions. In other words, I think it's better to think about abstractions as a tool for interacting with hardware, not as something to save us from dealing with hardware.
• pingyong 7y ago
  Eh, idk. Most programs that run like a giant turd do it because they load 15 megabytes of Javascript libraries to call two functions, or something to that effect. Computers are so fast now that you really need to be doing something unbelievably stupid for things in consumer programs to not be instantaneous.
  • claudius 7y ago
    Like start Firefox? I have no idea why it takes multiple seconds to bring up a blank window when starting a comparably useful program like Claws Mail is absolutely instantaneous.
    • pingyong 7y ago
      To be honest, yes, generally programs that don't start instantaneously do something very unnecessary and stupid that has nothing to do with actual CPU performance. As in they read thousands of tiny files or they decompress files or they wait for some sort of answer from the network, or they make 10k+ expensive API calls or all of the above. Firefox is so big it might fall into the "all of the above" category, but to answer that question definitively you'd have to analyze what Firefox actually does. And of course, making the decompression 15-20% faster by optimizing the decompression code (which is usually not even written by the developers of said software but just some external library) won't even make a difference because 20% less than 5 seconds is still 4 seconds which is way too long for a program to start. Instead using a different compression algorithm that increases file size by 25% but decompression speed by 10x would actually start solving the problem, with the next step being to ask why the program needs to read so much damn data at the start in the first place. But since NVMe SSDs and Intel CPUs with very high boost clocks are quickly becoming the norm now even for laptops I don't see much of that happening, because Firefox starts pretty quickly (~1 second) on those machines.
      • bzbarsky 7y ago
        Fwiw, Firefox stores almost everything it will need on startup in one large file (omni.ja) precisely to avoid the "thousands of tiny files" problem. That data is uncompressed, precisely to avoid the decompression problem. The low-hanging fruit has largely been picked. As for why so much data needs to be read... I just checked, and on Mac the main Firefox library (the executable itself is mostly a stub) is 120MB. So that's going to take a second or three just to read in at typical HDD speeds (faster on a good SSD), and then the dynamic linker has to do its thing on that big library, which is not instantaneous either.