3 ms·

If you don't learn about grammars, lexers, parsers, ASTs,... you're at a huge disadvantage in many situations. 'Compilers' was my favorite course in university,

If you don't learn about grammars, lexers, parsers, ASTs,... you're at a huge disadvantage in many situations. 'Compilers' was my favorite course in university, and I've used most of that knowledge many times. I was also often the only person on the team who had enough experience to immediately start doing it (say, writing a parser for something, or writing a clang-tidy plugin to detect and fix some issue in the codebase). You're equally likely to need this sort of knowledge when writing software for non-technical users (say in finance), and when you write dev tools, linters, checkers, fixers, etc. If you're doing anything web-related or even vaguely server-client, you're at a huge disadvantage if you didn't have a good networks course. You'll likely be stuck in a loop of half-knowledge and cargo culting. I had a terrible network course, skipped on deep diving into networking for years (I worked in embedded; SPI, I2C, RS-232/422 and CAN were all I needed), and only relatively recently figured out how much I had missed. I see the author mentioned operating systems as a second-tier course. I find it to be the differentiator (that, and C++ knowledge) when it comes to any serious project, especially understanding of processes, threads, IPC, and such. Databases are very easy to learn on your own, and are generally easy to learn, even the theoretical aspect (relational algebra, b-trees,...). Most of the time, and in most shitty universities (the majority of universities everywhere if we're honest with ourselves), for most courses, you're better off picking a couple of good books on whatever subject (classics are classics for a reason) and just learning. That + OCW from MIT, Stanford, Harvard for assignments for practice. The best courses in my university blatantly stole materials from MIT and Stanford.

ryanlpeterman 4y ago
Networking is worth an honorable mention I agree. > If you don't learn about grammars, lexers, parsers, ASTs,... you're at a huge disadvantage in many situations. Can you get an example? I don't recall a time where I used this knowledge in the last 5 years as a systems generalist in big tech.
- sn9 4y ago
  https://steve-yegge.blogspot.com/2007/06/rich-programmer-food.html https://steve-yegge.blogspot.com/2007/06/rich-programmer-foo...
- cue_the_strings 4y ago
  Your (non-technical) clients want to query the company's data, which is stored in who-knows-what way (Redis + Mongo + Postgres + Kafka + hdf5 + network filesystems with Apache Parquet files, whatever wakes you up at night). They just want to write filters in a trivial `some_field > 3 and otherField * 5 < thirdField` way. You implement this, but they keep asking for more functionality and syntax to this little DSL. The obvious answer is writing a parser, what the parser generates is dependent on the backend. Maybe you generate a lambda filter for the collection, maybe you generate an intermediary AST which you then traverse to generate a query for some DB or multiple of them. I've had similar requests in both medical and finance fields. Your predecessors had zero regard for C programming best practices, leaving you with a pile of supremely brittle code that just barely works, but you have to tiptoe around it in silk gloves whenever you want to modify it in any way, because of unreasonable standards on ownership, calling conventions, concurrency, whatever. They might have had a reason in the beginning, but now it took over creating a monster that no regular programmer can work on. They didn't think that the code will ever need to be multithreaded. This is no environment for a sane person. A young, inexperienced fool would suggest to rewrite it from scratch. You know better. The whole business logic of a working, profitable business is imprisoned in this pile of shit, and yet it's too far gone to continue forward with it. Nobody understands all the silly rules of engagement with the codebase. You decide to: 1) break everything into chunks, 2) write extensive unit tests, 3) use automated tools to fix the whole codebase yourself to some sane standard. Some of the automation can be just grep + xargs + sed, some is much more complex. Regex replacements are unreasonably efficient, but not omnipotent. The only sane way to fix 40k LOC at a time is to write tools that operate on the AST, find toxic patterns and smartly replace them. This takes dozens of iterations, but you can always roll back with git. You're not transforming the code incrementally, you're writing a transformer. Eventually, your fixers are good enough that they fix 95% of the issues, you fix the rest manually. Your company can now hire any old mediocre C dev off the street, and this dev can productively contribute to the rejuvinated codebase. The company no longer needs psychics, regular C devs will do. If they stray from the path, the newly set up CI with dozens of custom clang-tidy plugins swiftly puts them back in their place. Your company is in a highly regulated space that has insane coding standards. There are (shitty) compliance tools that cost >$100k a year for your team. You're able to replicate 80% of the functionality of the tools, fix some issues and generate compliance reports from the code on this in ~8 months. Slap on some CI, and the bean counters don't even have to do anything but download the latest PDF report.
veqq 4y ago
> C++ Do you literally mean 1) C++, 2) just a lower level language with control over those or 3) reading the relevant libraries or system code implementing IPC etc. (isn't it in C not C++?)?
- cue_the_strings 4y ago
  Well, I (and also my current employer and the whole team) think that C++ is the best proxy for general programming ability and understanding of CS. A person that understands the intricacies of C++ and all of its aspects, including the STL, template metaprogramming, design patterns, policy based design, concepts, concurrency, etc. is a person demonstrably capable of reasoning about complex systems, rulesets, different intersecting approaches and such, not to mention an intuitive understanding of the meat and potatoes of resource management and code generation. Even if you're hiring for something else and adjacent, like systems C or C# for finance, you'll be hard pressed to find a better measure of general programming ability and understanding of CS concepts. If you're really familiar with C++, you're at peace with best practice deprecation, handling legacy code, compromises of language design, etc. You're a grownup. Another great measure is knowledge of (Linux) systems programming, this also tends to weed out people with inferior understanding of, you know, programming and deploying software on actual computers. The interfaces here are obviously in C. There are many things that aren't congruent with C++ ability, such as general functional programming ability, or hiring for JS frontend or Erlang roles and such, but for most industry jobs, it's a useful determinator. I don't know any good C++ programmers that aren't great and all-rounded overall developers. I can't say the same for any other language or technology.

ryanlpeterman 4y ago

Networking is worth an honorable mention I agree. > If you don't learn about grammars, lexers, parsers, ASTs,... you're at a huge disadvantage in many situations. Can you get an example? I don't recall a time where I used this knowledge in the last 5 years as a systems generalist in big tech.

sn9 4y ago
https://steve-yegge.blogspot.com/2007/06/rich-programmer-food.html https://steve-yegge.blogspot.com/2007/06/rich-programmer-foo...
cue_the_strings 4y ago
Your (non-technical) clients want to query the company's data, which is stored in who-knows-what way (Redis + Mongo + Postgres + Kafka + hdf5 + network filesystems with Apache Parquet files, whatever wakes you up at night). They just want to write filters in a trivial `some_field > 3 and otherField * 5 < thirdField` way. You implement this, but they keep asking for more functionality and syntax to this little DSL. The obvious answer is writing a parser, what the parser generates is dependent on the backend. Maybe you generate a lambda filter for the collection, maybe you generate an intermediary AST which you then traverse to generate a query for some DB or multiple of them. I've had similar requests in both medical and finance fields. Your predecessors had zero regard for C programming best practices, leaving you with a pile of supremely brittle code that just barely works, but you have to tiptoe around it in silk gloves whenever you want to modify it in any way, because of unreasonable standards on ownership, calling conventions, concurrency, whatever. They might have had a reason in the beginning, but now it took over creating a monster that no regular programmer can work on. They didn't think that the code will ever need to be multithreaded. This is no environment for a sane person. A young, inexperienced fool would suggest to rewrite it from scratch. You know better. The whole business logic of a working, profitable business is imprisoned in this pile of shit, and yet it's too far gone to continue forward with it. Nobody understands all the silly rules of engagement with the codebase. You decide to: 1) break everything into chunks, 2) write extensive unit tests, 3) use automated tools to fix the whole codebase yourself to some sane standard. Some of the automation can be just grep + xargs + sed, some is much more complex. Regex replacements are unreasonably efficient, but not omnipotent. The only sane way to fix 40k LOC at a time is to write tools that operate on the AST, find toxic patterns and smartly replace them. This takes dozens of iterations, but you can always roll back with git. You're not transforming the code incrementally, you're writing a transformer. Eventually, your fixers are good enough that they fix 95% of the issues, you fix the rest manually. Your company can now hire any old mediocre C dev off the street, and this dev can productively contribute to the rejuvinated codebase. The company no longer needs psychics, regular C devs will do. If they stray from the path, the newly set up CI with dozens of custom clang-tidy plugins swiftly puts them back in their place. Your company is in a highly regulated space that has insane coding standards. There are (shitty) compliance tools that cost >$100k a year for your team. You're able to replicate 80% of the functionality of the tools, fix some issues and generate compliance reports from the code on this in ~8 months. Slap on some CI, and the bean counters don't even have to do anything but download the latest PDF report.

veqq 4y ago

> C++ Do you literally mean 1) C++, 2) just a lower level language with control over those or 3) reading the relevant libraries or system code implementing IPC etc. (isn't it in C not C++?)?

cue_the_strings 4y ago
Well, I (and also my current employer and the whole team) think that C++ is the best proxy for general programming ability and understanding of CS. A person that understands the intricacies of C++ and all of its aspects, including the STL, template metaprogramming, design patterns, policy based design, concepts, concurrency, etc. is a person demonstrably capable of reasoning about complex systems, rulesets, different intersecting approaches and such, not to mention an intuitive understanding of the meat and potatoes of resource management and code generation. Even if you're hiring for something else and adjacent, like systems C or C# for finance, you'll be hard pressed to find a better measure of general programming ability and understanding of CS concepts. If you're really familiar with C++, you're at peace with best practice deprecation, handling legacy code, compromises of language design, etc. You're a grownup. Another great measure is knowledge of (Linux) systems programming, this also tends to weed out people with inferior understanding of, you know, programming and deploying software on actual computers. The interfaces here are obviously in C. There are many things that aren't congruent with C++ ability, such as general functional programming ability, or hiring for JS frontend or Erlang roles and such, but for most industry jobs, it's a useful determinator. I don't know any good C++ programmers that aren't great and all-rounded overall developers. I can't say the same for any other language or technology.