7 ms·

Kind of tangential, but who are these people who are so comfortable with disassembling a high level language binary, reading assembly, and then making statement

by vsnf 3y ago

Kind of tangential, but who are these people who are so comfortable with disassembling a high level language binary, reading assembly, and then making statements about branch prediction and other such low level esoterica? I've only ever meet people like that maybe two or thee times in my career, and yet it seems like every other blog post I read in certain language circles everyone is some kind of ASM and Reverse Engineering expert.

Smaug123 3y ago
There are many ways you can get to this point (e.g. I just sort of picked this kind of thing up), but an example of a course which is designed precisely to give you these skills is Casey Muratori's "Performance-Aware Programming".
distcs 3y ago
I don't think I'd call "branch prediction" as "low level esoterica". It is a basic fact about how CPUs are implemented since many decades now. I learnt these things in my university coursework. Any module on CPU or computer system architecture is going to teach you all this stuff. But I'm sure you could learn these things from books on this topic too.
- mcv 3y ago
  I didn't, and frankly, half of the articles I read about it make me think branch prediction is a bug. I mean, I know it's meant to improve performance, which is great, but it has to make assumptions about what's going to happen before it knows it, and those assumptions are going to be wrong. How wrong? How can we con it into making better assumptions? Suddenly programming becomes about second guessing the compiler. And remember Spectre and Meltdown? Security vulnerabilities caused by branch prediction. If I recall correctly, the pipeline was executing code it wasn't meant to execute because it's executing it before it knows the result of the check that decides if it has to execute it. Programming is a lot easier if the actual control flow is as linear as I'm writing it. My broad takeaway of the whole ordeal is that I'm basically avoiding if-statements these days. I feel like I can't trust them anymore.
  - superb_dev 3y ago
    You shouldn’t be thinking about the branch predictor this much. The predictor is pretty good at guessing, unless you need to eek every last shred of performance you can ignore it
  - Galanwe 3y ago
    > it has to make assumptions about what's going to happen before it knows it, and those assumptions are going to be wrong. How wrong Speculative/optimistic techniques are not limited to branch prediction, you encounter various forms of it pretty much everywhere, without knowing it. * Your hard drives have a read ahead buffer, fetching in advance future data, just because most reads are immediately followed by an other one for the data just after. * Your CPU instructions are pre fetched, because most instructions are _not_ jumps, so you will 99% of the time just execute the instruction right after it. * Instruction reordering where your compiler will decide that future instructions not affected by previous instructions could be run in advance. * Overall any kind of caching is some form of optimistic technique. You are preparing future results. If you think about it, optimistic/speculative techniques are ubiquitous, and used even at _high_ abstraction levels. The famous python mantra of "better ask for forgiveness than permission" embodies that spirit. It encourages a coding style of "try: do() except: nope()" rather than "if check: do()". Standing "against optimistic/speculative techniques" is standing against transactions, rollbacks, caches. It's just not a viable line of thinking IMHO.
  - layer8 3y ago
    Without branch prediction and speculative execution, performance would always be as bad as with a misprediction. So every correct prediction is a win, and that’s why it’s being done. > My broad takeaway of the whole ordeal is that I'm basically avoiding if-statements these days. That seems like an overreaction. For most applications, cache locality and using the right algorithms is more important, if performance is an issue at all.
  - flohofwoe 3y ago
    > My broad takeaway of the whole ordeal is that I'm basically avoiding if-statements these days. I feel like I can't trust them anymore. Such broad "optimization rules" usually don't make much sense on any compiler created in the last 25 years. An optimizing compiler will analyse and optimize your program's control flow at a very high level, not just naively insert a conditional branch for each if (and that's also true in "low level" languages like C) If you actually care about such things, you really need to look at the compiler output and incrementally tweak your code (but then other compilers or even compile options might destroy such tweaks again), there hardly is a single optimal source code solution across CPU architectures or even just different CPU models of the same architecture (just don't do any obviously stupid things which make the life of the compiler harder than need be)
buro9 3y ago
This feels semi-normal to me... just have the curiosity to ask "why?" and the bias-to-action to move to "I'm going to find out". You encounter far far more dead-ends than anyone ever says, and every unsolved mystery is a mild nerd snipe, an open case, that years from now you'll see someone else explain something you realise it answers that question from years prior. For me, the hard bit is not over-indexing on this... you learn things, but biasing too much for them is a sure fire way to over-engineer or increase complexity to the point where something is now worse for you knowing something. But once in a while that tiny thing you learned years before is a 20% savings across the board with associated performance increase and everyone wondering how on Earth you could possibly have made those jumps. Also related... incidents. "Why" and "I'm going to find out" is the best way these things don't recur in future. A high degree of observation and understanding is a happy engineer life as it can improve what can often be the most stressful parts of the work (on-call, etc). That XKCD comic about everyone learning something for the first time factors too... there is stuff you know that others do not, share it.
- totetsu 3y ago
  https://imgs.xkcd.com/comics/ten_thousand.png https://imgs.xkcd.com/comics/ten_thousand.png for those who haven't heard of it.
  - teddyh 3y ago
    Canonical link: <https://xkcd.com/1053/ https://xkcd.com/1053/>
  - Chris2048 3y ago
    I always felt this one was a bit smug when applied to occupations (which it often is). People paid to do a job should know the basics of that occupation.
- djtango 3y ago
  I remember someone saying the difference between Physics and Computer Science is that in CS we are the masters of the universe - there are no laws of Physics that bind us. For me that means that in our world of computers there is infinite curiosities to discover. (Not that the same isn't true for the natural world too)
deaddodo 3y ago
I think you'll find that the deeper you go into "traditional" computer scientists, the more you'll find the problem-solvers, hackers and tinkerers that post these types of blogs. Especially in odd cases where a random print statement doubles your profiled performance. That being said, of all the people at all of the tech companies I've worked at, maybe ~5% of them had this sort of mentality and drive to execute on it.
keyle 3y ago
They're a dying breed. We're forgetting how to look under the hood and understand "why something works". Case in point, I'm slowly being replaced by Salesforce muppets for all my projects at work. They're little code monkeys with amazon ebook type knowledge, projects cost 20x more and I look like the mad scientist for speaking the truth. The products are worse in every possible metrics, I'm not crazy. The politics at play is the reason why I'm losing ground, not logic. Cabinet designers are being replaced by Ikea flat pack artists in the software world. All we can do is stand by and watch. And in regards to this blog, when Medium eventually go, that knowledge will go too. Blogs have died, personal websites as well, and their ability to be found in Google is almost non-existent. Sorry I don't have anything more positive to add, except maybe that they're still there, slowly being alienated by the modern tech world!
- mcv 3y ago
  > We're forgetting how to look under the hood and understand "why something works". Partly because that's often not what we're supposed to do; the stuff under the hood "just works" and we're meant to use it to write features, not worry about optimising the stuff that happens under the hood. And partly it's because the stuff under the hood is increasingly weird and bizarre. Branch prediction is weird, and I still don't understand why that extra print statement changes the branch prediction. Why does it predict `v > maxV` is true when the alternative is to print something, but it doesn't predict that when the alternative is to do nothing? Is it because printing is expensive, and therefore the branch predictor is going to strongly prefer avoiding that? It's weird that we'd basically have to deceive our code into compiling into a more performant form. I don't want to have to second guess the compiler.
  - chii 3y ago
    > not worry about optimising the stuff that happens under the hood. no one is asking you to do this optimization all the time. But you need to know what and how, if you were to be tasked. Then as you get more senior, you will have to also be able to judge whether it is possible, or worth the effort to devote to the optimization, should a scenario arises. > second guess the compiler. you aint second guessing it, you're examining the output, and understanding it. Then potentially recognizing when or if the compiler is outputting bad stuff. That's not second guessing.
kuroguro 3y ago
There's some bias as topics like these are often on top of HN. Extrapolate the 2 or 3 people you've met to all the programmers in the world - that's how you get some amazing in depth blog posts every other week.
BiteCode_dev 3y ago
The HN bubble doesn't have only disadvantages.
secondcoming 3y ago
It’s not all that uncommon outside of web stuff
sailorganymede 3y ago
These people genuinely care about their craft to the point they ask: what happened. They then look into it. People who tend to care tend to talk about what they care about. So, you wind up getting a lot of blogs who sort of self select themselves to do this kind of stuff. And everyone feeds off that energy and gets better :)
crvdgc 3y ago
This topic among others was covered by a Computer Architecture class back in my university. That class is mandatory for all CS students. Though I have to add that it's never been brought up after I started to work.
lordlimecat 3y ago
....And then feel OK resorting to ChatGPT for the explanation. Seriously that threw me, and maybe it makes sense in this context but it seems strange for someone with such an apparent depth of technical knowledge leaning on an LLM for anything.
- layer8 3y ago
  It seems they didn’t want to bother coming up with their own explanation, but then why not just link to https://en.wikipedia.org/wiki/Branch_predictor https://en.wikipedia.org/wiki/Branch_predictor?
  - xdavidliu 3y ago
    pasting LLM answers is sort of a tongue-in-cheek rhetorical flourish these days
ColinWright 3y ago
My first machine was a TRS-80. My first large program was a compiler from TRS-80 BASIC to Z80. I subsequently disassembled the ROM in the machine to figure out how things worked. These skills stay with you, and if you read articles like this then you can keep broadly up-to-date with the insanity that is current CPUs. Things like pipe-lines, branch prediction, and different levels of cacheing are optimisations that you can acquire as you go. If you're an auto-didact web developer then you never have the opportunity to learn these skills, or the need to do so. I know a lot of people who are comfortable with doing this, but in my case it's a generational thing. If you want to do it then you can. It's not hard, it's just a different skill from those you already have, though sufficiently related that you wouldn't be starting from scratch. But starting with modern CPUs can be hard. Learning the basics from older, simpler CPUs can help. Doing some kind of embedded programming might be the way to get started, or working on an emulator. As always, YMMV.
- pohl 3y ago
  A compiler from what to Z80? (I think there’s a word missing in what you wrote.) I started on a Model I and I’m curious about your compiler. I miss some aspects of those old machines. It was great, as a learner, that the ROM and a DOS like L-DOS was small enough to fit in your head. It can be intimidating how large everything is today.
  - ColinWright 3y ago
    Now edited ... thank you. The compiler was from a limited subset of TRS-80 BASIC to Z80. It was written in its own subset, so it could compile itself. I had 16K of RAM, and that had to fit the BASIC source, the compiled version, and then that compiled version compiled the BASIC source to another compiled version in a different location, which could then be saved on tape. Interesting times, and so foreign to today's experiences that it's hard for people to follow the details. I could write it up, but I'm pretty sure no one would derive any value from reading it. I sometimes wonder if the hand-written syntax flow charts and first draft are still somewhere in my piles of undiscarded papers. I doubt it ... it as 1979 when I was doing it, and I've moved continents since then.
rofo1 3y ago
There are many engineers that can do this, it's just that writing javascript pays 3 times or more so the focus goes there :) If you visit #c / #asm on any popular IRC network, you'll find a lot of skilled people that can do this routinely.
2-718-281-828 3y ago
> but who are these people who are so comfortable with disassembling a high level language binary, reading assembly, and then making statements about branch prediction and other such low level esoterica? i don't see any assembly here. the analysis is done by using a profiler. a very common tool available for most programming languages. https://timotijhof.net/wp-content/uploads/2020_profiling_figure1_flamegraph_intro.png https://timotijhof.net/wp-content/uploads/2020_profiling_fig...
layer8 3y ago
People who maintain a blog and whose blog posts are widely circulated are already highly selected. Conversely, people not comfortable with that stuff will also tend to be less comfortable writing a technical blog. Also, if you read the other subthreads, there are a number of points to be criticized about this writeup.
fit2rule 3y ago
[dead]
deleted 3y ago
[deleted]
amirxyz 3y ago
I think it depends on what niche you're in. I'm in the storage business and while not everyone, but quite a few of the people in my group would not be afraid of running a disaseembler, or talk about branch prediction, or unaligned integer access. `likely` and `unlikely` are very common in the code, and so on. AFAIK linux kernel people are very similar, and I assume a few other niches of system programming as well.

by vsnf 3y ago

Smaug123 3y ago

There are many ways you can get to this point (e.g. I just sort of picked this kind of thing up), but an example of a course which is designed precisely to give you these skills is Casey Muratori's "Performance-Aware Programming".

distcs 3y ago

I don't think I'd call "branch prediction" as "low level esoterica". It is a basic fact about how CPUs are implemented since many decades now. I learnt these things in my university coursework. Any module on CPU or computer system architecture is going to teach you all this stuff. But I'm sure you could learn these things from books on this topic too.

mcv 3y ago
I didn't, and frankly, half of the articles I read about it make me think branch prediction is a bug. I mean, I know it's meant to improve performance, which is great, but it has to make assumptions about what's going to happen before it knows it, and those assumptions are going to be wrong. How wrong? How can we con it into making better assumptions? Suddenly programming becomes about second guessing the compiler. And remember Spectre and Meltdown? Security vulnerabilities caused by branch prediction. If I recall correctly, the pipeline was executing code it wasn't meant to execute because it's executing it before it knows the result of the check that decides if it has to execute it. Programming is a lot easier if the actual control flow is as linear as I'm writing it. My broad takeaway of the whole ordeal is that I'm basically avoiding if-statements these days. I feel like I can't trust them anymore.
- superb_dev 3y ago
  You shouldn’t be thinking about the branch predictor this much. The predictor is pretty good at guessing, unless you need to eek every last shred of performance you can ignore it
- Galanwe 3y ago
  > it has to make assumptions about what's going to happen before it knows it, and those assumptions are going to be wrong. How wrong Speculative/optimistic techniques are not limited to branch prediction, you encounter various forms of it pretty much everywhere, without knowing it. * Your hard drives have a read ahead buffer, fetching in advance future data, just because most reads are immediately followed by an other one for the data just after. * Your CPU instructions are pre fetched, because most instructions are _not_ jumps, so you will 99% of the time just execute the instruction right after it. * Instruction reordering where your compiler will decide that future instructions not affected by previous instructions could be run in advance. * Overall any kind of caching is some form of optimistic technique. You are preparing future results. If you think about it, optimistic/speculative techniques are ubiquitous, and used even at _high_ abstraction levels. The famous python mantra of "better ask for forgiveness than permission" embodies that spirit. It encourages a coding style of "try: do() except: nope()" rather than "if check: do()". Standing "against optimistic/speculative techniques" is standing against transactions, rollbacks, caches. It's just not a viable line of thinking IMHO.
- layer8 3y ago
  Without branch prediction and speculative execution, performance would always be as bad as with a misprediction. So every correct prediction is a win, and that’s why it’s being done. > My broad takeaway of the whole ordeal is that I'm basically avoiding if-statements these days. That seems like an overreaction. For most applications, cache locality and using the right algorithms is more important, if performance is an issue at all.
- flohofwoe 3y ago
  > My broad takeaway of the whole ordeal is that I'm basically avoiding if-statements these days. I feel like I can't trust them anymore. Such broad "optimization rules" usually don't make much sense on any compiler created in the last 25 years. An optimizing compiler will analyse and optimize your program's control flow at a very high level, not just naively insert a conditional branch for each if (and that's also true in "low level" languages like C) If you actually care about such things, you really need to look at the compiler output and incrementally tweak your code (but then other compilers or even compile options might destroy such tweaks again), there hardly is a single optimal source code solution across CPU architectures or even just different CPU models of the same architecture (just don't do any obviously stupid things which make the life of the compiler harder than need be)

buro9 3y ago

This feels semi-normal to me... just have the curiosity to ask "why?" and the bias-to-action to move to "I'm going to find out". You encounter far far more dead-ends than anyone ever says, and every unsolved mystery is a mild nerd snipe, an open case, that years from now you'll see someone else explain something you realise it answers that question from years prior. For me, the hard bit is not over-indexing on this... you learn things, but biasing too much for them is a sure fire way to over-engineer or increase complexity to the point where something is now worse for you knowing something. But once in a while that tiny thing you learned years before is a 20% savings across the board with associated performance increase and everyone wondering how on Earth you could possibly have made those jumps. Also related... incidents. "Why" and "I'm going to find out" is the best way these things don't recur in future. A high degree of observation and understanding is a happy engineer life as it can improve what can often be the most stressful parts of the work (on-call, etc). That XKCD comic about everyone learning something for the first time factors too... there is stuff you know that others do not, share it.

totetsu 3y ago
https://imgs.xkcd.com/comics/ten_thousand.png https://imgs.xkcd.com/comics/ten_thousand.png for those who haven't heard of it.
- teddyh 3y ago
  Canonical link: <https://xkcd.com/1053/ https://xkcd.com/1053/>
- Chris2048 3y ago
  I always felt this one was a bit smug when applied to occupations (which it often is). People paid to do a job should know the basics of that occupation.
djtango 3y ago
I remember someone saying the difference between Physics and Computer Science is that in CS we are the masters of the universe - there are no laws of Physics that bind us. For me that means that in our world of computers there is infinite curiosities to discover. (Not that the same isn't true for the natural world too)

deaddodo 3y ago

I think you'll find that the deeper you go into "traditional" computer scientists, the more you'll find the problem-solvers, hackers and tinkerers that post these types of blogs. Especially in odd cases where a random print statement doubles your profiled performance. That being said, of all the people at all of the tech companies I've worked at, maybe ~5% of them had this sort of mentality and drive to execute on it.

keyle 3y ago

They're a dying breed. We're forgetting how to look under the hood and understand "why something works". Case in point, I'm slowly being replaced by Salesforce muppets for all my projects at work. They're little code monkeys with amazon ebook type knowledge, projects cost 20x more and I look like the mad scientist for speaking the truth. The products are worse in every possible metrics, I'm not crazy. The politics at play is the reason why I'm losing ground, not logic. Cabinet designers are being replaced by Ikea flat pack artists in the software world. All we can do is stand by and watch. And in regards to this blog, when Medium eventually go, that knowledge will go too. Blogs have died, personal websites as well, and their ability to be found in Google is almost non-existent. Sorry I don't have anything more positive to add, except maybe that they're still there, slowly being alienated by the modern tech world!

mcv 3y ago
> We're forgetting how to look under the hood and understand "why something works". Partly because that's often not what we're supposed to do; the stuff under the hood "just works" and we're meant to use it to write features, not worry about optimising the stuff that happens under the hood. And partly it's because the stuff under the hood is increasingly weird and bizarre. Branch prediction is weird, and I still don't understand why that extra print statement changes the branch prediction. Why does it predict `v > maxV` is true when the alternative is to print something, but it doesn't predict that when the alternative is to do nothing? Is it because printing is expensive, and therefore the branch predictor is going to strongly prefer avoiding that? It's weird that we'd basically have to deceive our code into compiling into a more performant form. I don't want to have to second guess the compiler.
- chii 3y ago
  > not worry about optimising the stuff that happens under the hood. no one is asking you to do this optimization all the time. But you need to know what and how, if you were to be tasked. Then as you get more senior, you will have to also be able to judge whether it is possible, or worth the effort to devote to the optimization, should a scenario arises. > second guess the compiler. you aint second guessing it, you're examining the output, and understanding it. Then potentially recognizing when or if the compiler is outputting bad stuff. That's not second guessing.

kuroguro 3y ago

There's some bias as topics like these are often on top of HN. Extrapolate the 2 or 3 people you've met to all the programmers in the world - that's how you get some amazing in depth blog posts every other week.

BiteCode_dev 3y ago

The HN bubble doesn't have only disadvantages.

secondcoming 3y ago

It’s not all that uncommon outside of web stuff

sailorganymede 3y ago

These people genuinely care about their craft to the point they ask: what happened. They then look into it. People who tend to care tend to talk about what they care about. So, you wind up getting a lot of blogs who sort of self select themselves to do this kind of stuff. And everyone feeds off that energy and gets better :)

crvdgc 3y ago

This topic among others was covered by a Computer Architecture class back in my university. That class is mandatory for all CS students. Though I have to add that it's never been brought up after I started to work.

lordlimecat 3y ago

....And then feel OK resorting to ChatGPT for the explanation. Seriously that threw me, and maybe it makes sense in this context but it seems strange for someone with such an apparent depth of technical knowledge leaning on an LLM for anything.

layer8 3y ago
It seems they didn’t want to bother coming up with their own explanation, but then why not just link to https://en.wikipedia.org/wiki/Branch_predictor https://en.wikipedia.org/wiki/Branch_predictor?
- xdavidliu 3y ago
  pasting LLM answers is sort of a tongue-in-cheek rhetorical flourish these days

ColinWright 3y ago

My first machine was a TRS-80. My first large program was a compiler from TRS-80 BASIC to Z80. I subsequently disassembled the ROM in the machine to figure out how things worked. These skills stay with you, and if you read articles like this then you can keep broadly up-to-date with the insanity that is current CPUs. Things like pipe-lines, branch prediction, and different levels of cacheing are optimisations that you can acquire as you go. If you're an auto-didact web developer then you never have the opportunity to learn these skills, or the need to do so. I know a lot of people who are comfortable with doing this, but in my case it's a generational thing. If you want to do it then you can. It's not hard, it's just a different skill from those you already have, though sufficiently related that you wouldn't be starting from scratch. But starting with modern CPUs can be hard. Learning the basics from older, simpler CPUs can help. Doing some kind of embedded programming might be the way to get started, or working on an emulator. As always, YMMV.

pohl 3y ago
A compiler from what to Z80? (I think there’s a word missing in what you wrote.) I started on a Model I and I’m curious about your compiler. I miss some aspects of those old machines. It was great, as a learner, that the ROM and a DOS like L-DOS was small enough to fit in your head. It can be intimidating how large everything is today.
- ColinWright 3y ago
  Now edited ... thank you. The compiler was from a limited subset of TRS-80 BASIC to Z80. It was written in its own subset, so it could compile itself. I had 16K of RAM, and that had to fit the BASIC source, the compiled version, and then that compiled version compiled the BASIC source to another compiled version in a different location, which could then be saved on tape. Interesting times, and so foreign to today's experiences that it's hard for people to follow the details. I could write it up, but I'm pretty sure no one would derive any value from reading it. I sometimes wonder if the hand-written syntax flow charts and first draft are still somewhere in my piles of undiscarded papers. I doubt it ... it as 1979 when I was doing it, and I've moved continents since then.

rofo1 3y ago

There are many engineers that can do this, it's just that writing javascript pays 3 times or more so the focus goes there :) If you visit #c / #asm on any popular IRC network, you'll find a lot of skilled people that can do this routinely.

2-718-281-828 3y ago

> but who are these people who are so comfortable with disassembling a high level language binary, reading assembly, and then making statements about branch prediction and other such low level esoterica? i don't see any assembly here. the analysis is done by using a profiler. a very common tool available for most programming languages. https://timotijhof.net/wp-content/uploads/2020_profiling_figure1_flamegraph_intro.png https://timotijhof.net/wp-content/uploads/2020_profiling_fig...

layer8 3y ago

People who maintain a blog and whose blog posts are widely circulated are already highly selected. Conversely, people not comfortable with that stuff will also tend to be less comfortable writing a technical blog. Also, if you read the other subthreads, there are a number of points to be criticized about this writeup.

fit2rule 3y ago

[dead]

deleted 3y ago

[deleted]

amirxyz 3y ago

I think it depends on what niche you're in. I'm in the storage business and while not everyone, but quite a few of the people in my group would not be afraid of running a disaseembler, or talk about branch prediction, or unaligned integer access. `likely` and `unlikely` are very common in the code, and so on. AFAIK linux kernel people are very similar, and I assume a few other niches of system programming as well.