6 ms·

Before writing compilers, I think it's best to understand computer architectures, and what needs to be generated by the compiler to yield the most efficient mac

Before writing compilers, I think it's best to understand computer architectures, and what needs to be generated by the compiler to yield the most efficient machine code. Unfortunately, in my experience, computer architecture and even systems programming are domains that schools/universities appear to systematically increasingly de-prioritize, presumably because it is seen as too technical. That knowledge however is instrumental in landing some of the best jobs in the industry.

zarathustreal 3y ago
In my experience, no one tends to see it as “too technical.” The typical (and imo obvious, logical) reason not to emphasize low-level details is because they’re implementation-specific, immediately outdated, and not particularly generalizable. As an aside, I also tend not to get along with the “systems programmer” types because they tend to make knowing these kinds of extremely specific factoids their entire professional personality. You end up with people that can write assembly but have no idea what a functor is.
- stevesimmons 3y ago
  Excellent final sentence!
- IshKebab 3y ago
  There are things that are generally true across almost all architectures and have been for a long time. Like cache lines, cache hierarchies, false sharing, alignment, branch prediction, pipelining, etc. But I agree it isn't really necessary to write a compiler. Compilers tend to be pretty much the worst case from a microarchitectural point of view anyway - full of trees of small objects etc.
- mgaunard 3y ago
  I don't think they are. So long as you were not trained before the 70s, any knowledge there is still very relevant today. ISAs are still very similar, and so is the relative performance of the various operations. Registers, caches, pipelines, superscalar execution... it's all still mostly the same, and still drives what the codegen should look like. And yet you have so many people that don't know the considerations of what makes a simple loop fast or not.
  - zarathustreal 3y ago
    Knowing "the considerations of what makes a simple loop fast or not" is irrelevant to solving a computational problem. We as programmers should not need to worry about details of the hardware or toolchain (e.g. compiler) we're using. Granted, the reality is that most people are doing the equivalent of digital plumbing, not necessarily solving computational problems.
    - mgaunard 3y ago
      The role of the compiler is to automatically perform the task of translating a program in a given programming language to a program that runs on a computer, making the best possible use of that computer's resources. Writing a compiler with shallow understanding of the computer you're targeting is definitely possible, but ultimately wasteful.
    - sakjur 3y ago
      Plumbers have done more to save lives and enable the luxuries we take for granted than most. I would be lucky if my work could compare to that. I think we need software engineers coming from electrical engineering, mathematics, humanities, and all walks of life. If you’re not interested in compilers, maybe the comments for a link about compilers isn’t your place to be.
      - aswanson 3y ago
        I come from an EE background. I didn't understand the importance of computation in general until entering industry, and I want to understand compilers before I leave this earthly plane, just for the sake of understanding them.
- andai 3y ago
  What's the purpose of functors? It appears to have a different meaning in different programming languages.
  - mgaunard 3y ago
    Functors come from category theory. They're relevant in programming languages that are based on that, e.g. Ocaml, Haskell, and other sorts of overly academic and impractical esoteric stuff. Like most ideas in programming based on theory, it's ultimately a very trivial thing, whose theoretical foundation is quite unimportant outside of helping design a minimal set of operations for your programming language. It's not particularly important in the grand scheme of things. People even build and use monads with no understanding of the background behind it.
    - chongli 3y ago
      People even build and use monads with no understanding of the background behind it. And then they miss the common pattern between them, so they miss the opportunity for abstraction and a common interface over the pattern. That’s the whole point of category theory: recognizing large classes (in the mathematical sense) of objects by their universal properties. It’s useful in programming language design because it gives you the ability to build useful, generic, water-tight abstractions. A lot of abstractions built without any regard to the mathematics behind them wind up being leaky and difficult to apply appropriately because they rely on vague intuitions rather than simple mathematical properties.
      - quickthrower2 3y ago
        I feel there is impedance mismatch between the mathematical category theory side and what is needed for programs. A case in point is the infamous complex diagram for the Lens library in Haskell. This is abstracting the notion of getters and setters to some extreme and is several hours of work to really fathom. Compare to Go where the tutorial does balanced tree comparisons as an intro example to coroutines, when the language gets out of your way I feel it is much nicer. Haskell is more of a playpen for PL ideas. Some of those ideas get promoted into the mainstream when shown to be very useful! So Haskell is very valuable in that sense but can be difficult to write code in. Especially if you want to use libraries that may use complex category theoretic libraries.
- sspiff 3y ago
  I have been programming since I was 11 years old and I've been professionally programming in various languages and domains for roughly 15 years, yet I had to Google what a functor was. Your attack on systems programmers holding specific knowledge as holy while pointing to their ignorance about another bit of knowledge you consider more relevant seems kind of ironic to me.
- hnthrowaway0328 3y ago
  I guess it's poison and meat. Seems to be my idea job. I have always wanted to write C and assembly language programs in my career but don't care much about functional programming.
- deleted 3y ago
  [deleted]
- anthomtb 3y ago
  > You end up with people that can write assembly but have no idea what a functor is New challenge to self: write a functor in pure assembly.
  - galdosdi 3y ago
    What, such as a linked list? This is a first week assignment in a systems course.
  - anthomtb 3y ago
    After some reading I have come to two conclusions: 1) I have written many functors in my life without describing them as such 2) "Pure assembly" functor doesn't make much sense. Functors, both in the CS and mathematics sense, require the concept of type. Which is absent once you are dealing with stacks and registers. My best idea is something when you have mixed-width registers - eg, say you've got some 128 bit and 256 bit registers. Your functor could apply a mapping from one to the other (and would probably be nothing more than a move instruction with the appropriate sign extension).
- hardware2win 3y ago
  Haha, I agree The kind of ppl who always when arguing about programming langs or compilers starts referring to C or CPP standard or its ecosystem as a holy bible
- never_inline 3y ago
  > You end up with people that can write assembly but have no idea what a functor is. Thank gods that's good thing.
sabas123 3y ago
> Before writing compilers, I think it's best to understand computer architectures, and what needs to be generated by the compiler to yield the most efficient machine code. In my opinion one of the biggest industry advancements we have had is that compilers are now available without such level of low-level detail. There is still so much work to be done on a compiler level that really shouldn't concern themselves with the (micro)-architectural level of computers.
- mgaunard 3y ago
  It seems the trend is in increased diversity of hardware, with the dominance of x86 being reconsidered with smartphones, energy-optimal cloud, GPU computing, and ML accelerators. LLVM allows hardware manufacturers to more easily provide mainstream language support for their platform than before, but the problem here was mostly that GCC was hostile to modular design, not really theoretical advances. In terms of frontends, I guess we're seeing more languages reach C-level performance thanks to LLVM again. But in terms of optimizations driven by theory? There were some significant advancements in generic auto-parallelization for imperative languages, about a decade ago I think. And it it doesn't magically solve the codegen problem, and remains hampered by language semantics that are not always parallelization-friendly. There were a bunch of improvements which were driven by making languages more hardware-aware, e.g. the concurrent C++ model in 2011 which was widely copied to other low-level programming languages. We're also seeing more and more libraries that are specifically designed to target hardware features better. So ultimately it looks like most of the advances are driven by better integration of how the hardware works throughout the compiler, language and community.
  - shrimp_emoji 3y ago
    > I guess we're seeing more languages reach C-level performance thanks to LLVM again. If they made JavaScript as fast as C, the software industry will become a cheap perversion of what it once was, and I'm picking up my toys and going home.
dmichulke 3y ago
> That knowledge however is instrumental in landing some of the best jobs in the industry. Could you elaborate what you think are the best jobs in the industry and why?
- JonChesterfield 3y ago
  Compilers are amplifiers. Compiler optimisations give you faster/cheaper execution of every program. Type systems and linters detect errors without writing or running tests. Programming languages are tools for thought. Matching one to the domain makes the domain easier to reason about, lets people solve bigger problems. Whether that correlates with "best jobs" is subjective. The value proposition of making other programmers more productive is really good. If you think that's an exaggeration, consider writing a web browser in machine code.
  - bee_rider 3y ago
    I agree that “best jobs” is a bit ambiguous, but I think “landing the best jobs” is unambiguously getting a job that benefits the job-getter to an unusual extent (very well paying, or at least well paying and very stable). It is an expression. The surprising thing about this, I think, is that hardware is generally expected to be quite underpaid around here, I think, compared to programming.
  - dmichulke 3y ago
    Fair enough, the points you mention definitely help. But to get "the best jobs in the industry" you also need a demand/supply imbalance and I wonder if that exists and where, or whether something different was meant by OP (i.e. not just plain $$)
xw390111 3y ago
> schools/universities appear to systematically increasingly de-prioritize, presumably because it is seen as too technical. The statement is correct: universities are de-prioritizing systems programming, but not for the reason you state. Since I've been working with Universities, I've come to appreciate their fundamental problem. There's a lot of potential material to cover in a finite number of hours, which is horrifying short if you're the one to allocate them. The amount of information one could know in our field grows exponentially with time, and we're well past the point of overflow. So yes, systems programming is getting less class time in the general track, but that's only because it's less relevant to more and more students every year, so it makes sense.
seanmcdirmid 3y ago
GPU related tasks are still fairly in need of lots of computer architecture and compiler skills. Yes, you don’t need to study x86 or MIPs, but CUDA presents an even weirder architecture than those.

zarathustreal 3y ago

In my experience, no one tends to see it as “too technical.” The typical (and imo obvious, logical) reason not to emphasize low-level details is because they’re implementation-specific, immediately outdated, and not particularly generalizable. As an aside, I also tend not to get along with the “systems programmer” types because they tend to make knowing these kinds of extremely specific factoids their entire professional personality. You end up with people that can write assembly but have no idea what a functor is.

stevesimmons 3y ago
Excellent final sentence!
IshKebab 3y ago
There are things that are generally true across almost all architectures and have been for a long time. Like cache lines, cache hierarchies, false sharing, alignment, branch prediction, pipelining, etc. But I agree it isn't really necessary to write a compiler. Compilers tend to be pretty much the worst case from a microarchitectural point of view anyway - full of trees of small objects etc.
mgaunard 3y ago
I don't think they are. So long as you were not trained before the 70s, any knowledge there is still very relevant today. ISAs are still very similar, and so is the relative performance of the various operations. Registers, caches, pipelines, superscalar execution... it's all still mostly the same, and still drives what the codegen should look like. And yet you have so many people that don't know the considerations of what makes a simple loop fast or not.
- zarathustreal 3y ago
  Knowing "the considerations of what makes a simple loop fast or not" is irrelevant to solving a computational problem. We as programmers should not need to worry about details of the hardware or toolchain (e.g. compiler) we're using. Granted, the reality is that most people are doing the equivalent of digital plumbing, not necessarily solving computational problems.
  - mgaunard 3y ago
    The role of the compiler is to automatically perform the task of translating a program in a given programming language to a program that runs on a computer, making the best possible use of that computer's resources. Writing a compiler with shallow understanding of the computer you're targeting is definitely possible, but ultimately wasteful.
  - sakjur 3y ago
    Plumbers have done more to save lives and enable the luxuries we take for granted than most. I would be lucky if my work could compare to that. I think we need software engineers coming from electrical engineering, mathematics, humanities, and all walks of life. If you’re not interested in compilers, maybe the comments for a link about compilers isn’t your place to be.
    - aswanson 3y ago
      I come from an EE background. I didn't understand the importance of computation in general until entering industry, and I want to understand compilers before I leave this earthly plane, just for the sake of understanding them.
andai 3y ago
What's the purpose of functors? It appears to have a different meaning in different programming languages.
- mgaunard 3y ago
  Functors come from category theory. They're relevant in programming languages that are based on that, e.g. Ocaml, Haskell, and other sorts of overly academic and impractical esoteric stuff. Like most ideas in programming based on theory, it's ultimately a very trivial thing, whose theoretical foundation is quite unimportant outside of helping design a minimal set of operations for your programming language. It's not particularly important in the grand scheme of things. People even build and use monads with no understanding of the background behind it.
  - chongli 3y ago
    People even build and use monads with no understanding of the background behind it. And then they miss the common pattern between them, so they miss the opportunity for abstraction and a common interface over the pattern. That’s the whole point of category theory: recognizing large classes (in the mathematical sense) of objects by their universal properties. It’s useful in programming language design because it gives you the ability to build useful, generic, water-tight abstractions. A lot of abstractions built without any regard to the mathematics behind them wind up being leaky and difficult to apply appropriately because they rely on vague intuitions rather than simple mathematical properties.
    - quickthrower2 3y ago
      I feel there is impedance mismatch between the mathematical category theory side and what is needed for programs. A case in point is the infamous complex diagram for the Lens library in Haskell. This is abstracting the notion of getters and setters to some extreme and is several hours of work to really fathom. Compare to Go where the tutorial does balanced tree comparisons as an intro example to coroutines, when the language gets out of your way I feel it is much nicer. Haskell is more of a playpen for PL ideas. Some of those ideas get promoted into the mainstream when shown to be very useful! So Haskell is very valuable in that sense but can be difficult to write code in. Especially if you want to use libraries that may use complex category theoretic libraries.
sspiff 3y ago
I have been programming since I was 11 years old and I've been professionally programming in various languages and domains for roughly 15 years, yet I had to Google what a functor was. Your attack on systems programmers holding specific knowledge as holy while pointing to their ignorance about another bit of knowledge you consider more relevant seems kind of ironic to me.
hnthrowaway0328 3y ago
I guess it's poison and meat. Seems to be my idea job. I have always wanted to write C and assembly language programs in my career but don't care much about functional programming.
deleted 3y ago
[deleted]
anthomtb 3y ago
> You end up with people that can write assembly but have no idea what a functor is New challenge to self: write a functor in pure assembly.
- galdosdi 3y ago
  What, such as a linked list? This is a first week assignment in a systems course.
- anthomtb 3y ago
  After some reading I have come to two conclusions: 1) I have written many functors in my life without describing them as such 2) "Pure assembly" functor doesn't make much sense. Functors, both in the CS and mathematics sense, require the concept of type. Which is absent once you are dealing with stacks and registers. My best idea is something when you have mixed-width registers - eg, say you've got some 128 bit and 256 bit registers. Your functor could apply a mapping from one to the other (and would probably be nothing more than a move instruction with the appropriate sign extension).
hardware2win 3y ago
Haha, I agree The kind of ppl who always when arguing about programming langs or compilers starts referring to C or CPP standard or its ecosystem as a holy bible
never_inline 3y ago
> You end up with people that can write assembly but have no idea what a functor is. Thank gods that's good thing.

sabas123 3y ago

> Before writing compilers, I think it's best to understand computer architectures, and what needs to be generated by the compiler to yield the most efficient machine code. In my opinion one of the biggest industry advancements we have had is that compilers are now available without such level of low-level detail. There is still so much work to be done on a compiler level that really shouldn't concern themselves with the (micro)-architectural level of computers.

mgaunard 3y ago
It seems the trend is in increased diversity of hardware, with the dominance of x86 being reconsidered with smartphones, energy-optimal cloud, GPU computing, and ML accelerators. LLVM allows hardware manufacturers to more easily provide mainstream language support for their platform than before, but the problem here was mostly that GCC was hostile to modular design, not really theoretical advances. In terms of frontends, I guess we're seeing more languages reach C-level performance thanks to LLVM again. But in terms of optimizations driven by theory? There were some significant advancements in generic auto-parallelization for imperative languages, about a decade ago I think. And it it doesn't magically solve the codegen problem, and remains hampered by language semantics that are not always parallelization-friendly. There were a bunch of improvements which were driven by making languages more hardware-aware, e.g. the concurrent C++ model in 2011 which was widely copied to other low-level programming languages. We're also seeing more and more libraries that are specifically designed to target hardware features better. So ultimately it looks like most of the advances are driven by better integration of how the hardware works throughout the compiler, language and community.
- shrimp_emoji 3y ago
  > I guess we're seeing more languages reach C-level performance thanks to LLVM again. If they made JavaScript as fast as C, the software industry will become a cheap perversion of what it once was, and I'm picking up my toys and going home.

dmichulke 3y ago

> That knowledge however is instrumental in landing some of the best jobs in the industry. Could you elaborate what you think are the best jobs in the industry and why?

JonChesterfield 3y ago
Compilers are amplifiers. Compiler optimisations give you faster/cheaper execution of every program. Type systems and linters detect errors without writing or running tests. Programming languages are tools for thought. Matching one to the domain makes the domain easier to reason about, lets people solve bigger problems. Whether that correlates with "best jobs" is subjective. The value proposition of making other programmers more productive is really good. If you think that's an exaggeration, consider writing a web browser in machine code.
- bee_rider 3y ago
  I agree that “best jobs” is a bit ambiguous, but I think “landing the best jobs” is unambiguously getting a job that benefits the job-getter to an unusual extent (very well paying, or at least well paying and very stable). It is an expression. The surprising thing about this, I think, is that hardware is generally expected to be quite underpaid around here, I think, compared to programming.
- dmichulke 3y ago
  Fair enough, the points you mention definitely help. But to get "the best jobs in the industry" you also need a demand/supply imbalance and I wonder if that exists and where, or whether something different was meant by OP (i.e. not just plain $$)

xw390111 3y ago

> schools/universities appear to systematically increasingly de-prioritize, presumably because it is seen as too technical. The statement is correct: universities are de-prioritizing systems programming, but not for the reason you state. Since I've been working with Universities, I've come to appreciate their fundamental problem. There's a lot of potential material to cover in a finite number of hours, which is horrifying short if you're the one to allocate them. The amount of information one could know in our field grows exponentially with time, and we're well past the point of overflow. So yes, systems programming is getting less class time in the general track, but that's only because it's less relevant to more and more students every year, so it makes sense.

seanmcdirmid 3y ago

GPU related tasks are still fairly in need of lots of computer architecture and compiler skills. Yes, you don’t need to study x86 or MIPs, but CUDA presents an even weirder architecture than those.