5 ms·

I took a compiler construction course this year, and successfully passed it. We implemented an imperative/functional hybrid, with dozens of the implied (the Pro

I took a compiler construction course this year, and successfully passed it. We implemented an imperative/functional hybrid, with dozens of the implied (the Prof had warned us) swearing/cursing/regretting/all-nightering along the way. The employed toolchain comprised flex, bison, and LLVM, all wrapped with C++ and running on x86_64. I still don't feel any confidence around user-space code. Lots of stuff was abstracted away--especially during the code generation part. Before taking this course, I had imagined that every component of my compiler would be hand-crafted by yours truly. Note that I do not expect an undergrad course to demand such levels of skill. The thing I want to say, I guess, is this gut feeling that there are projects more appropriate than a compiler to draw your first userLand blood. Like a memory allocator, for instance; in my opinion, systems programming carries far all-arounder knowledge on its shoulders, with knowledge meaning a reliable representation of what takes place under the hood of a modern computer, in a comfy level of abstraction (program, threads, data, memory hierarchy, user, kernel, etc). I am afraid, however, that I'm just too amateur to actually judge, and that I'm projecting a considerable amount of CS-revolving insecurities on the above statements. Any help with clarifying that is deeply appreciated.

Banana699 5y ago
>flex, bison, and LLVM Don't feel bad for wiring up ready components, the common wisdom (which I imbibed by reading lots of tutorials and half-reading random interesting sections of compiler books and telling myself this is learning) in compiler land is that the things that happen at the very beginning and the very last are not the core of the problem and abstracting them out is safe and even preferable for a language designer to do. The "Parsers are uninteresting" meme is off course very widespread (often unfairly, parsers and parsing algorithms can be a gorgeous rabbit hole), and I feel that a corresponding "Native code generation is uninteresting" meme is beginning to take hold. Despite much rote repetition, they are mostly right. Parsers and code generation have two things in common : 1- They are very well formalized, with sophisticated algorithms that could take a declarative notation and generate what it describes. (grammars for parsers, 3-address codes and other virtual assembly languages for code generators). 2- They are relatively stable, there are only a few major families of syntax styles, and only a few major mainstream architectures. So you have a complex task, but once done it's relatively stable and static. It's algorithms and architectures are very complex and error-prone, but they could be painted over with well defined specifications that other tools can consume and do the heavy lifting for you. Sounds like the perfect storm for automation. In contrast, lots of problems in language design are those quick shifting sands of ergonomics and other problems where the tradeoff landscape is so large and varied there is no obvious "one way to do it well". Packaging and importing libraries, cheap editor and IDE integration, how to make human-friendly also machine-friendly, etc.... This is not to say that parsers or code generators are "Solved", only that they are just slightly above the nastiest chess or alphaGo game, sometimes manageable, sometimes ferociously difficult, at other times downright impossible. Always interesting and worth exploring, but never ambiguous or murky. Tooling, semantics and other design/social problems are more like how to best raise your kids or how to make a just and lawful society, sometimes people can't even agree on what seems to be the problem. So, naturally, the constant formal parts of the compiler got formalized, was hidden behind declarative specifications, and further exploration and implementation was relegated to academia and R&D departments. The social aspects and murky design problems are what drives people designing new languages. (for purposes other than learning)
cloogshicer 5y ago
I feel your pain. I recommend reading the absolutely wonderful book "Crafting Interpreters" by Bob Nystrom. For me, it demystified a lot of what you mentioned.

Banana699 5y ago

>flex, bison, and LLVM Don't feel bad for wiring up ready components, the common wisdom (which I imbibed by reading lots of tutorials and half-reading random interesting sections of compiler books and telling myself this is learning) in compiler land is that the things that happen at the very beginning and the very last are not the core of the problem and abstracting them out is safe and even preferable for a language designer to do. The "Parsers are uninteresting" meme is off course very widespread (often unfairly, parsers and parsing algorithms can be a gorgeous rabbit hole), and I feel that a corresponding "Native code generation is uninteresting" meme is beginning to take hold. Despite much rote repetition, they are mostly right. Parsers and code generation have two things in common : 1- They are very well formalized, with sophisticated algorithms that could take a declarative notation and generate what it describes. (grammars for parsers, 3-address codes and other virtual assembly languages for code generators). 2- They are relatively stable, there are only a few major families of syntax styles, and only a few major mainstream architectures. So you have a complex task, but once done it's relatively stable and static. It's algorithms and architectures are very complex and error-prone, but they could be painted over with well defined specifications that other tools can consume and do the heavy lifting for you. Sounds like the perfect storm for automation. In contrast, lots of problems in language design are those quick shifting sands of ergonomics and other problems where the tradeoff landscape is so large and varied there is no obvious "one way to do it well". Packaging and importing libraries, cheap editor and IDE integration, how to make human-friendly also machine-friendly, etc.... This is not to say that parsers or code generators are "Solved", only that they are just slightly above the nastiest chess or alphaGo game, sometimes manageable, sometimes ferociously difficult, at other times downright impossible. Always interesting and worth exploring, but never ambiguous or murky. Tooling, semantics and other design/social problems are more like how to best raise your kids or how to make a just and lawful society, sometimes people can't even agree on what seems to be the problem. So, naturally, the constant formal parts of the compiler got formalized, was hidden behind declarative specifications, and further exploration and implementation was relegated to academia and R&D departments. The social aspects and murky design problems are what drives people designing new languages. (for purposes other than learning)

cloogshicer 5y ago

I feel your pain. I recommend reading the absolutely wonderful book "Crafting Interpreters" by Bob Nystrom. For me, it demystified a lot of what you mentioned.