3 ms·

Most people don't get into systems programming by writing a database completely on their own, and it's fair to say that the line between application and systems

by wheels 5y ago

Most people don't get into systems programming by writing a database completely on their own, and it's fair to say that the line between application and systems programming is fuzzy. Here are a bunch of things I would broadly consider systems programming: - Kernel and driver development - Low level library development (pretty much anything involving bit wrangling) - Platform abstraction libraries - Database development (not usage) - Message queuing systems - Daemon / server development Perhaps the recurring pattern there, and differentiated from application programming is that most of those are tools for other applications, rather than applications themselves. I don't think that all library / daemon development is systems programming, but a whole lot of it is. I'm coming from a background of having done 5 of the 6 of those groups (though there could obviously be more listed there). For most significant projects, as noted, there will be internal APIs that wrap system APIs (or in the case of the kernel, where the system APIs are totally irrelevant, except for the parts that implement the system calls). Generally someone first jumping into those projects isn't going to immediately start hacking on the internal libraries. As it were, I have actually written a database from scratch [1]. That database uses Qt and Qt-like APIs internally where possible. There are places in there where you need to know the intricacies of mmap, fsync and similar, but they're compartmentalized to a couple of classes. I'd still call code that isn't in those couple of classes "systems programming". [1] https://blog.directededge.com/2009/02/27/on-building-a-stupidly-fast-graph-database/ https://blog.directededge.com/2009/02/27/on-building-a-stupi...

wheels 5y ago
I realize it's unlikely anyone will see this, but to amplify this point a little, I thought I'd poke through the Posgres source (which I'm not otherwise familiar with). By my count, mmap is only called in 4 places, in three files. The implementation behind this file is one of them: https://doxygen.postgresql.org/fd_8h.html https://doxygen.postgresql.org/fd_8h.html That is the actual file API that you'd use inside of Postgres. While the API there isn't going to win any beauty pageants, it's significantly more sane than raw POSIX. For comparison, this is the interface to mmaped files within my company's internal database: https://gist.github.com/scotchi/83609f5eb9b98ac3b4b4476ed621a033 https://gist.github.com/scotchi/83609f5eb9b98ac3b4b4476ed621... There's a grand total of 1 call to mmap in all of our source. Most of the time for writing code using mapped files here, all you'd see is the interface above.

by wheels 5y ago

wheels 5y ago

I realize it's unlikely anyone will see this, but to amplify this point a little, I thought I'd poke through the Posgres source (which I'm not otherwise familiar with). By my count, mmap is only called in 4 places, in three files. The implementation behind this file is one of them: https://doxygen.postgresql.org/fd_8h.html https://doxygen.postgresql.org/fd_8h.html That is the actual file API that you'd use inside of Postgres. While the API there isn't going to win any beauty pageants, it's significantly more sane than raw POSIX. For comparison, this is the interface to mmaped files within my company's internal database: https://gist.github.com/scotchi/83609f5eb9b98ac3b4b4476ed621a033 https://gist.github.com/scotchi/83609f5eb9b98ac3b4b4476ed621... There's a grand total of 1 call to mmap in all of our source. Most of the time for writing code using mapped files here, all you'd see is the interface above.