4 ms·
I disagree. These are complex to make robust, but they are not hard concepts. I think everyone should write their own GC, write a filesystem, and handle their
by pwang 13y ago
I disagree.
These are complex to make robust, but they are not hard concepts. I think everyone should write their own GC, write a filesystem, and handle their own memory layer, at least once.
Treating lower levels of the stack as "too complex, there be dragons, just write some Javascript that hits RESTful API for JSON" is a great way to ensure you never progress as a software dev or an engineer.
I agree that reinventing the wheel poorly - without understanding geometry or physics - is a terrible idea. But I also think that we don't have enough people nowadays making toy wheels to learn about geometry and physics.
For each of the things you mentioned, there are concrete and good reasons why they grow complexity: filesystems need to be robust to media corruption and power faults; memory allocators need to be highly tuned to OS and architecture memory layouts; GCs need to be optimized for the code patterns of the particular language they serve, as well as understand the same OS and hardware memory constraints as allocators.
But much of the complexity of these layers is not intrinsic to the problem, and not related to the above. In fact, most of the complexity and bugs of production code at these layers have to do with legacy compatibility, or code inheritance. (If you look at what Theo says about this specific OpenSSL bug, it arises because the OpenSSL team wrote their own allocator that doesn't use malloc() protection bits because those are not compatible across some platforms.)
As the old saying goes: in science, we advance by standing on each others' shoulders; in software, we stand on each others' toes.
- arethuza 13y agoI think the important part of the previous comment is "At a first approximation" - of course there is nothing wrong in writing code with the primary goal of personal understanding, but re-inventing the wheel when there are potentially very serious consequences for others is hubris (unless, of course, you have a really good reason why, NIH not being a good reason in my book). [NB Ken Shirriff's blog articles on Bitcoin are a superb example of someone coding in an effort to understand a system - but I don't think he is attempting to write a real Bitcoin library]
- LaGrange 13y ago> I think everyone should write their own GC, write a filesystem, and handle their own memory layer, at least once. Of course. But not in production. Which is a detail missed by the snarky one-liner, maybe because one-liners suck. Still, the point isn't that people shouldn't ever write them, just that they shouldn't actually use the ones they developed while not being a part of a team of experts specializing in the issue at hand.
- jwr 13y ago> These are complex to make robust, but they are not hard concepts. Yes, yes, I didn't quantify my statement with "for production use". Your sentence I quoted above was exactly my point: they aren't necessarily hard (or at least they don't seem hard), but they are very, very difficult to get right (robust). Getting systems like these to work well in practical production use is more than 80% of the effort. I wasn't saying that no one should learn how a garbage collector works by implementing one. My point was that no one should implement their own garbage collector for a production system unless they already became an expert in the field of garbage collectors by implementing them for the last 10 years or so. Same goes for memory allocators. If someone thinks these are simple systems, it means they don't know what they don't know.