7 ms·

What Is Systems Programming, Really? (2018)

130 points by generichuman 7y ago

dang 7y ago
Discussed at the time: https://news.ycombinator.com/item?id=17948265 https://news.ycombinator.com/item?id=17948265
streetcat1 7y ago
If your domain of discourse is the computer itself or the functionality of the computer in the world (store/compute/ communicate) than it is system programming. Else it is application programming. It has nothing to do with the actual programming language.
- tyingq 7y ago
  I think anyone's opinion on what makes something "systems programming" has a lot to do with what it meant when they learned the term. The quotes from Stroustrup and Alexandrescu, for example, very much call out language features.
  - streetcat1 7y ago
    Right. They confuse "system programming languages" with "system programming"
    - icedchai 7y ago
      There are languages that are clearly better suited for systems programming. You might call these "systems programming languages." Would you write a file system driver in PHP?
      - giancarlostoro 7y ago
        > There are languages that are clearly better suited for systems programming. You might call these "systems programming languages." Would you write a file system driver in PHP? I wouldnt but that sounds like something we would all read about on HN.
        tyingq 7y ago
        The latest version of PHP comes with FFI, so I suspect that's coming :)
      - streetcat1 7y ago
        Openstack was written in python... Personally I use go for everything.
        mekster 7y ago
        Isn't garbage collection the problem for being in the system programming?
        streetcat1 7y ago
        No. It might be a problem for real-time systems, which are a subset. For example, storage systems would do fine.
        gmfawcett 7y ago
        Not necessarily. There have been real-world operating systems that employed garbage collectors (e.g., Oberon, Genera).
        pjmlp 7y ago
        No, that is a common myth from the GC-hate crowd. Here is some stuff to get you started, "The Cedar Programming Environment: A Midterm Report and Examination " https://pdfs.semanticscholar.org/33ff/212245ff8af0359e9118832c9f29ee1fd291.pdf https://pdfs.semanticscholar.org/33ff/212245ff8af0359e911883... http://www.projectoberon.com/ http://www.projectoberon.com/ http://www.ocp.inf.ethz.ch/wiki/Documentation/Front http://www.ocp.inf.ethz.ch/wiki/Documentation/Front https://www.whoishostingthis.com/resources/modula-3/ https://www.whoishostingthis.com/resources/modula-3/ http://joeduffyblog.com/2015/11/03/blogging-about-midori/ http://joeduffyblog.com/2015/11/03/blogging-about-midori/ https://www.infoq.com/presentations/csharp-systems-programming/ https://www.infoq.com/presentations/csharp-systems-programmi...
      - saagarjha 7y ago
        I'm sure you could probably get that to work with FUSE.
      - pjc50 7y ago
        I actually wrote a filesystem driver in Perl using FUSE for my final year of university. It seemed like it would be much less painful when it crashed, given that the objective was to prototype an idea rather than ship a product.
      - bakul 7y ago
        You can write a user level filesystem in any programming language if it can talk the 9p protocol!
- burntoutfire 7y ago
  So, if a programmer at say Amazon writes a small python utility for deployment on AWS, is that systems programming?
  - tom_mellior 7y ago
    Your phrasing suggests (I might be wrong) that (a) the expected answer is no, in your optinion, and that (b) Python is a relevant factor. If that is so, I wonder if you would consider an equivalent program written in C to be systems programming. (My personal opinion is that the whole discussion is moot, a "systems programming" category is ill-defined and not needed.)
    - burntoutfire 7y ago
      My personal definition of systems programming a programming where one calls a lot of OS APIs directly. Which would make the AWS deployment Python script (or even C program) example not s.p..
Taniwha 7y ago
It's really (or was back in the day) simply short for "operating systems programming"
- haecceity 7y ago
  That makes sense. But everything we program are systems so shortening it like that isn't really helping anyone.
aazaa 7y ago
> To summarize, what we call “systems programming” I think should be called “low-level programming.” Computer systems design as a field is too important not to have its own name. Clearly separating these two ideas provides a greater conceptual clarity on the space of programming language design, and it also opens the door to sharing insights across the two spaces: how can we design the system around the machine, and vice versa? Alternate definition whose source I can't recall: Systems programming is everything that isn't application programming. Maybe it's time to update that with: Systems programming is everything that isn't application programming or low-level programming.
- Izkata 7y ago
  So, HTML? ;) Slightly more seriously, systems programming seems to me to be approaching the lowest levels, so including "not low-level programming" would give the wrong impression that it includes the really high-levels ones.
whateveracct 7y ago
Maybe not the thread to post this, but I've seen too often "systems programming" be intertwined with arrogance and gatekeeping. As if these skills are special and inherently more difficult to pick up by developers without them. But tbh all software engineers are programming abstract machines at various levels. Systems work is just a different abstract machine to grok. My education is in relevant skills (CompE), but the attitude I keep running into in this domain has really started to disgust me, even if it isn't everyone. When it's people I've seen in leadership, that's enough :/
- gdy 7y ago
  "As if these skills are special and inherently more difficult to pick up by developers without them" It's because they are.
  - saagarjha 7y ago
    Source?
    - whateveracct 7y ago
      There isn't one. The only source is the hivemind of systems programming elitism. I've seen it permeate every systems engineering shop I've encountered.
      - gdy 7y ago
        If you are saying it is happening everywhere you come, maybe it's you who is doing something wrong socially?
        whateveracct 7y ago
        I get where you're coming from, but I don't see how me joining a company could cause its leaders and employees to exhibit the sort of arrogance and gatekeeping I'm describing.
lesserknowndan 7y ago
I think it would be a unfortunate for people to start redefining what "Systems Programming" means because it is vague enough now as it is - and we don't have the luxury of being able to go back and update the term in the historic literature. My understanding is that, in effect, "Systems Programming" is analogous to "Operating Systems Programming", or rather you might say (given the increasing size of operating systems these days) that "Systems Programming" is a subset of "Operating Systems Programming" that deals with the direct manipulation of memory in order to provide a useful abstraction for "user" software - remembering that most hardware interaction is via memory. Obviously, in order to do "Systems Programming" you need a "Systems Programming Language" that allows such direct access to memory. These days, "Systems Programmers" would be those working on the Linux kernel, or microkernels, or graphics drivers (basically those who have complete access to memory) though those developing services one layer up might also consider themselves to be such (especially micro-kernal-based operating systems). It is probably worth emphasising that "System" is a generic term that is used in a lot of different contexts. I think "Systems Programming" is used in the context of a "Computer System", while modern day cloud programming would be better described as "Distributed Systems Programming" as it is in the context of a "Distributed System" (though I've probably just annoyed a whole bunch of distributed systems researchers). I think that what the author of the article is talking about is "System Architecture" (or perhaps "Distributed Systems Architecture"). Should we talk about this as programming at all?
- astrobe_ 7y ago
  > I think it would be a unfortunate for people to start redefining what "Systems Programming" means because it is vague enough now as it is Indeed. When people call something "systems programming" when it is just e.g. server programming, it feels like when you are being told that "hacker" means the same thing as "cracker" because that's what most people think it means. In my slightly angry opinion, "Systems programming" is one of those terms that are convenient because they are vague and look cool. The plural on "systems" is suspicious in itself. It's not just one system they are working on, but many. Impressive, isn't it? The funny thing is, if you look up the definition of "system": a regularly interacting or interdependent group of items forming a unified whole ... not so many people actually do that. I think programmers usually work on one thing, one component of a whole system (guess why you are part of a team?). I'm not sure about how uncommon the opposite is because I actually do that - I have worked at the driver level, OS configuration level and at the application level in order to make a product work - and I am not that much of an exceptional programmer.
forrestthewoods 7y ago
I've primarily writing C++ for over 15 years. I don't think I'd ever heard the term "systems programming" until the last few years when Rust talked about it. I don't have a super formal CS education and most of my career has been in gamedev. What is sytems programming? I don't know. I don't care either. It's an arbitrary label that is rarely used and doesn't affect anyone in any way.
s1k3b8 7y ago
Systems programming concerns the code between hardware and the application programmer. So pretty much kernel programming. Application programming concerns the code between kernel and the ordinary user. Essentially, systems programmers write software that application programmers can interface with. Application programmers write software that ordinary users interface with. It gets fuzzy at the edges like most definitions, but that's how I think about it generally.
- vnorilo 7y ago
  I've always thought of compiler programming as systems programming. I guess it fits your definition of being something application folks interface with in addition to kernel.
kartayyar 7y ago
One litmus test that a former coworker applied was "how resource constrained is the application"? I think conceptually this is a better test, more so than say if you have access to a raw pointer - because more important parts of how to handle core issues like scheduling, storage management, concurrency, parallelism and distributed computation manifest themselves in many forms. I started my career doing C programming in a OS / kernel environment. In retrospect a lot of pieces of applications I've worked on are what I think of as systems, and many parts of kernels are more applications than systems.
- tom_mellior 7y ago
  How resource constrained is the Linux kernel when running on a modern x86_64 machine with 16 GB of RAM? Not very. So is Linux kernel development for this platform not "systems programming"? If you're going to say something like "but the kernel should be fast when servicing syscalls", you are right, but that is not a resource constaint, it's a user requirement. LibreOffice should also be fast when reacting to my keypresses, but I hope we agree that LibreOffice development would fall into the "applications development" camp, if we have to put things into bins. The truth is, we don't have to put things into bins. There is no need for a muddy, fuzzy term like "systems programming". If you're doing "kernel programming" or "embedded systems programming on resource constrained microcontrollers", you can just say that directly.
luord 7y ago
I was really liking this article, until the "dynamic typing sucks" and "functional programming is the path to enlightenment" stuff came up at the end. Specially jarring because it's out of nowhere. That conclusion does not follow from everything that comes before.

130 points by generichuman 7y ago

dang 7y ago

Discussed at the time: https://news.ycombinator.com/item?id=17948265 https://news.ycombinator.com/item?id=17948265

If your domain of discourse is the computer itself or the functionality of the computer in the world (store/compute/ communicate) than it is system programming. Else it is application programming. It has nothing to do with the actual programming language.

tyingq 7y ago
I think anyone's opinion on what makes something "systems programming" has a lot to do with what it meant when they learned the term. The quotes from Stroustrup and Alexandrescu, for example, very much call out language features.
- streetcat1 7y ago
  Right. They confuse "system programming languages" with "system programming"
  - icedchai 7y ago
    There are languages that are clearly better suited for systems programming. You might call these "systems programming languages." Would you write a file system driver in PHP?
    - giancarlostoro 7y ago
      > There are languages that are clearly better suited for systems programming. You might call these "systems programming languages." Would you write a file system driver in PHP? I wouldnt but that sounds like something we would all read about on HN.
      - tyingq 7y ago
        The latest version of PHP comes with FFI, so I suspect that's coming :)
    - streetcat1 7y ago
      Openstack was written in python... Personally I use go for everything.
      - mekster 7y ago
        Isn't garbage collection the problem for being in the system programming?
        streetcat1 7y ago
        No. It might be a problem for real-time systems, which are a subset. For example, storage systems would do fine.
        gmfawcett 7y ago
        Not necessarily. There have been real-world operating systems that employed garbage collectors (e.g., Oberon, Genera).
        pjmlp 7y ago
        No, that is a common myth from the GC-hate crowd. Here is some stuff to get you started, "The Cedar Programming Environment: A Midterm Report and Examination " https://pdfs.semanticscholar.org/33ff/212245ff8af0359e9118832c9f29ee1fd291.pdf https://pdfs.semanticscholar.org/33ff/212245ff8af0359e911883... http://www.projectoberon.com/ http://www.projectoberon.com/ http://www.ocp.inf.ethz.ch/wiki/Documentation/Front http://www.ocp.inf.ethz.ch/wiki/Documentation/Front https://www.whoishostingthis.com/resources/modula-3/ https://www.whoishostingthis.com/resources/modula-3/ http://joeduffyblog.com/2015/11/03/blogging-about-midori/ http://joeduffyblog.com/2015/11/03/blogging-about-midori/ https://www.infoq.com/presentations/csharp-systems-programming/ https://www.infoq.com/presentations/csharp-systems-programmi...
    - saagarjha 7y ago
      I'm sure you could probably get that to work with FUSE.
    - pjc50 7y ago
      I actually wrote a filesystem driver in Perl using FUSE for my final year of university. It seemed like it would be much less painful when it crashed, given that the objective was to prototype an idea rather than ship a product.
    - bakul 7y ago
      You can write a user level filesystem in any programming language if it can talk the 9p protocol!
burntoutfire 7y ago
So, if a programmer at say Amazon writes a small python utility for deployment on AWS, is that systems programming?
- tom_mellior 7y ago
  Your phrasing suggests (I might be wrong) that (a) the expected answer is no, in your optinion, and that (b) Python is a relevant factor. If that is so, I wonder if you would consider an equivalent program written in C to be systems programming. (My personal opinion is that the whole discussion is moot, a "systems programming" category is ill-defined and not needed.)
  - burntoutfire 7y ago
    My personal definition of systems programming a programming where one calls a lot of OS APIs directly. Which would make the AWS deployment Python script (or even C program) example not s.p..

Taniwha 7y ago

It's really (or was back in the day) simply short for "operating systems programming"

haecceity 7y ago
That makes sense. But everything we program are systems so shortening it like that isn't really helping anyone.

aazaa 7y ago

> To summarize, what we call “systems programming” I think should be called “low-level programming.” Computer systems design as a field is too important not to have its own name. Clearly separating these two ideas provides a greater conceptual clarity on the space of programming language design, and it also opens the door to sharing insights across the two spaces: how can we design the system around the machine, and vice versa? Alternate definition whose source I can't recall: Systems programming is everything that isn't application programming. Maybe it's time to update that with: Systems programming is everything that isn't application programming or low-level programming.

Izkata 7y ago
So, HTML? ;) Slightly more seriously, systems programming seems to me to be approaching the lowest levels, so including "not low-level programming" would give the wrong impression that it includes the really high-levels ones.

whateveracct 7y ago

Maybe not the thread to post this, but I've seen too often "systems programming" be intertwined with arrogance and gatekeeping. As if these skills are special and inherently more difficult to pick up by developers without them. But tbh all software engineers are programming abstract machines at various levels. Systems work is just a different abstract machine to grok. My education is in relevant skills (CompE), but the attitude I keep running into in this domain has really started to disgust me, even if it isn't everyone. When it's people I've seen in leadership, that's enough :/

gdy 7y ago
"As if these skills are special and inherently more difficult to pick up by developers without them" It's because they are.
- saagarjha 7y ago
  Source?
  - whateveracct 7y ago
    There isn't one. The only source is the hivemind of systems programming elitism. I've seen it permeate every systems engineering shop I've encountered.
    - gdy 7y ago
      If you are saying it is happening everywhere you come, maybe it's you who is doing something wrong socially?
      - whateveracct 7y ago
        I get where you're coming from, but I don't see how me joining a company could cause its leaders and employees to exhibit the sort of arrogance and gatekeeping I'm describing.

lesserknowndan 7y ago

I think it would be a unfortunate for people to start redefining what "Systems Programming" means because it is vague enough now as it is - and we don't have the luxury of being able to go back and update the term in the historic literature. My understanding is that, in effect, "Systems Programming" is analogous to "Operating Systems Programming", or rather you might say (given the increasing size of operating systems these days) that "Systems Programming" is a subset of "Operating Systems Programming" that deals with the direct manipulation of memory in order to provide a useful abstraction for "user" software - remembering that most hardware interaction is via memory. Obviously, in order to do "Systems Programming" you need a "Systems Programming Language" that allows such direct access to memory. These days, "Systems Programmers" would be those working on the Linux kernel, or microkernels, or graphics drivers (basically those who have complete access to memory) though those developing services one layer up might also consider themselves to be such (especially micro-kernal-based operating systems). It is probably worth emphasising that "System" is a generic term that is used in a lot of different contexts. I think "Systems Programming" is used in the context of a "Computer System", while modern day cloud programming would be better described as "Distributed Systems Programming" as it is in the context of a "Distributed System" (though I've probably just annoyed a whole bunch of distributed systems researchers). I think that what the author of the article is talking about is "System Architecture" (or perhaps "Distributed Systems Architecture"). Should we talk about this as programming at all?

astrobe_ 7y ago
> I think it would be a unfortunate for people to start redefining what "Systems Programming" means because it is vague enough now as it is Indeed. When people call something "systems programming" when it is just e.g. server programming, it feels like when you are being told that "hacker" means the same thing as "cracker" because that's what most people think it means. In my slightly angry opinion, "Systems programming" is one of those terms that are convenient because they are vague and look cool. The plural on "systems" is suspicious in itself. It's not just one system they are working on, but many. Impressive, isn't it? The funny thing is, if you look up the definition of "system": a regularly interacting or interdependent group of items forming a unified whole ... not so many people actually do that. I think programmers usually work on one thing, one component of a whole system (guess why you are part of a team?). I'm not sure about how uncommon the opposite is because I actually do that - I have worked at the driver level, OS configuration level and at the application level in order to make a product work - and I am not that much of an exceptional programmer.

forrestthewoods 7y ago

I've primarily writing C++ for over 15 years. I don't think I'd ever heard the term "systems programming" until the last few years when Rust talked about it. I don't have a super formal CS education and most of my career has been in gamedev. What is sytems programming? I don't know. I don't care either. It's an arbitrary label that is rarely used and doesn't affect anyone in any way.

s1k3b8 7y ago

Systems programming concerns the code between hardware and the application programmer. So pretty much kernel programming. Application programming concerns the code between kernel and the ordinary user. Essentially, systems programmers write software that application programmers can interface with. Application programmers write software that ordinary users interface with. It gets fuzzy at the edges like most definitions, but that's how I think about it generally.

vnorilo 7y ago
I've always thought of compiler programming as systems programming. I guess it fits your definition of being something application folks interface with in addition to kernel.

kartayyar 7y ago

One litmus test that a former coworker applied was "how resource constrained is the application"? I think conceptually this is a better test, more so than say if you have access to a raw pointer - because more important parts of how to handle core issues like scheduling, storage management, concurrency, parallelism and distributed computation manifest themselves in many forms. I started my career doing C programming in a OS / kernel environment. In retrospect a lot of pieces of applications I've worked on are what I think of as systems, and many parts of kernels are more applications than systems.

tom_mellior 7y ago
How resource constrained is the Linux kernel when running on a modern x86_64 machine with 16 GB of RAM? Not very. So is Linux kernel development for this platform not "systems programming"? If you're going to say something like "but the kernel should be fast when servicing syscalls", you are right, but that is not a resource constaint, it's a user requirement. LibreOffice should also be fast when reacting to my keypresses, but I hope we agree that LibreOffice development would fall into the "applications development" camp, if we have to put things into bins. The truth is, we don't have to put things into bins. There is no need for a muddy, fuzzy term like "systems programming". If you're doing "kernel programming" or "embedded systems programming on resource constrained microcontrollers", you can just say that directly.

luord 7y ago

I was really liking this article, until the "dynamic typing sucks" and "functional programming is the path to enlightenment" stuff came up at the end. Specially jarring because it's out of nowhere. That conclusion does not follow from everything that comes before.