4 ms·

A college student in CS here. I'm always impressed by people with deep understanding of programming language internals and try to pick up as much about programm

A college student in CS here. I'm always impressed by people with deep understanding of programming language internals and try to pick up as much about programming language internal workings and compilers as I can. How does one get really good at this? Is it by spending a lot of time programming and building stuff? Is it by reading books/blogs/articles about programming languages? Any recommendations for such resources?

kamaal 13y ago
One way is as you said is by spending a lot of time programming and building stuff. The slide deck says, the only way is the experience. You need to read and learn endlessly over the years. Like you I hope there was one book, that does this. Makes a nice idea for a community project. Compiling such massive wisdom into a book can't be done by a person alone. Currently its a bit like alchemy, looks very little chemistry and much more magic. And to learn there are hardly any resources beyond your regular C books, which more or less keep talking of the same things.
- RogerL 13y ago
  I always found it extremely useful to 1) learn to program in assembly 2) inspect the assembly generated from your compiler 3) trace a call all the way from user land down into the kernel, and back up 4) don't forget networks. Whether that means a single wire instrumented with a 'scope, or a network switch instrumented with wireshark, knowing how data gets around is important. Until you can do that on your platform, you don't really understand it (which is fine for many jobs out there, I'm just addressing the question of how to learn it). I suspect the shortest and best path is to write your own (toy/emulator) assembler and compiler, and to do some embedded work with no OS. That'll at least expose you to all the various issues; you might not implement register allocation well (or at all), but you'll have had to think about it and the implications (example: passing function parameters on the stack vs in registers). That's one of the advantages of a University education to my way of thinking. You will be put through all these paces if it isn't just a Java accreditation program. There are good books documenting, say, the Windows or Linux internals, so you can gain a good understanding of things like virtual memory, device drivers and so on. It is not as hard as it might seem. This is all basic information, one fact built on top of another. I have no idea how the ARM pipeline works, but that doesn't worry me. I know what pipelines are, some of the tradeoffs they have, and if I need to get down and dirty with a cute little ARM processor I'll know what I need to learn. You can feel sort of overwhelmed if you try to think how pressing on these keys get turned into truetype fonts on a graphical screen, and then sent across the internet to anyone bored enough to read my ramblings, while at the same time my computer updates the clock, checks my mail, and does a dozen other things, but piece by piece it is all discrete, well contained, and completely understandable. Reading Stroustrup is pretty much mandatory to understand C++. I personally wouldn't bother with the standards (other than a brief go-over); so much of the code in those slides are just things you should never, ever program. Which is to say I disagree with the slides in large measure; it's almost suspicious to have some of that knowledge! Either your coworkers are writing truly atrocious code, or you are spending time learning corners of the language which is time that almost certainly could be better spent elsewhere, such as learning about memory mapped files or something. As others have said, knowing where the boundaries of where undefined behavior lies is important, but knowing how to avoid straying even close to that territory is far more important (always initialize variables. Even static ones! You are going to cost some sorry bastard half a day of debugging when they erroneously think the problem in the code is an uninitiated variable and keep chasing that until they finally figure it out)
banachtarski 13y ago
The points made in this (excellent I thought) presentation are very subtle and I'm not sure I would have come across them before organically. Because the behavior outlined in the presentation tends toward the exception, it's more efficient to learn it by reading. Effective C++ is a great place to start if you are trying to hone your C++ skills (practically every C++ programmer in the industry considers it mandatory). Keep in mind though, that knowing about evaluation order, and stack frames, and sequencing, and linker optimizations is all great and all, but I definitely consider it icing on the cake for a working software engineer for most positions. If you're a senior guy, sure, you should know this stuff. But the first thing to do is learn to actually write programs well. Authoring your own projects and contributing to open source is a great way to do it.
chipsy 13y ago
The top option for learning is to work off a spec to implement your own compiler. A good second place would be exhaustively studying and maintaining an existing compiler. Regular use of a language can build certain kinds of knowledge about the internals, but it won't be as well-rounded a study as actually working with them directly.
drblast 13y ago
You'll learn a lot of this via debugging or reverse engineering. Do you REALLY know how your compiler, linker, and loader work? If you do, you'll understand everything they wrote about static variables, stack frames, etc. The best exercise to start with is this one on creating the smallest ELF executable possible: http://www.muppetlabs.com/~breadbox/software/tiny/teensy.html http://www.muppetlabs.com/~breadbox/software/tiny/teensy.htm... And after that I'd do the same thing on Windows with a PE file. Then try to break things. Try to write a C program that has a buffer overflow bug in it and exploit that to change the flow of program execution. Then exploit the same bug to open an xterm or calc.exe. Getting really familiar with a good debugger will help you a lot with these things. After you have a decent understanding of program loading and execution, you can dive into the more language lawyery things they're talking about so you can answer the questions about the nineteen different meanings of "static" like the hacker girl. With C, that's a worthwhile goal. With C++, good fucking luck.
laichzeit0 13y ago
Get hold of an embedded microprocessor and write some silly app for it. E.g. ZigBee or Digi's rabbit microprocessors. I learned all my "deep C" on the rabbit. The architecture is based on Z80 and the compiler is a really shitty non-standard C compiler with some custom syntax. You will very quickly learn to throw away many assumptions you might have about how things work. Learn how to write mutithreaded code without an operating system or a thread library (hint: co-routines). Learn how memory layout, very quickly get a feel for the time/space tradeoffs of different algorithms, etc.
pjungwir 13y ago
Everyone else's advice to learn by doing is great, but there is also _Expert C Programming_ by Peter van der Linden: http://www.amazon.com/Expert-Programming-Peter-van-Linden/dp/0131774298 http://www.amazon.com/Expert-Programming-Peter-van-Linden/dp... The presentation's "Deep C" pun is a reference to this book, and if you make it all the way to slide 444 you'll see it mentioned as further reading. It's a wonderful book for understanding C (not C++) and what's really going on.
- bloodorange 13y ago
  It'll take a team of ten authors and a couple of decades to write something of that kind for C++. Though the language is full of horrors, I still quite enjoy C++ (esp. with C++11 and am looking forward to some C++14 features...)
  - qznc 13y ago
    I know at least one guy, who is on it. http://herbsutter.com/gotw/ http://herbsutter.com/gotw/
    - bloodorange 13y ago
      Nice! Thanks for the link.

kamaal 13y ago

One way is as you said is by spending a lot of time programming and building stuff. The slide deck says, the only way is the experience. You need to read and learn endlessly over the years. Like you I hope there was one book, that does this. Makes a nice idea for a community project. Compiling such massive wisdom into a book can't be done by a person alone. Currently its a bit like alchemy, looks very little chemistry and much more magic. And to learn there are hardly any resources beyond your regular C books, which more or less keep talking of the same things.

RogerL 13y ago
I always found it extremely useful to 1) learn to program in assembly 2) inspect the assembly generated from your compiler 3) trace a call all the way from user land down into the kernel, and back up 4) don't forget networks. Whether that means a single wire instrumented with a 'scope, or a network switch instrumented with wireshark, knowing how data gets around is important. Until you can do that on your platform, you don't really understand it (which is fine for many jobs out there, I'm just addressing the question of how to learn it). I suspect the shortest and best path is to write your own (toy/emulator) assembler and compiler, and to do some embedded work with no OS. That'll at least expose you to all the various issues; you might not implement register allocation well (or at all), but you'll have had to think about it and the implications (example: passing function parameters on the stack vs in registers). That's one of the advantages of a University education to my way of thinking. You will be put through all these paces if it isn't just a Java accreditation program. There are good books documenting, say, the Windows or Linux internals, so you can gain a good understanding of things like virtual memory, device drivers and so on. It is not as hard as it might seem. This is all basic information, one fact built on top of another. I have no idea how the ARM pipeline works, but that doesn't worry me. I know what pipelines are, some of the tradeoffs they have, and if I need to get down and dirty with a cute little ARM processor I'll know what I need to learn. You can feel sort of overwhelmed if you try to think how pressing on these keys get turned into truetype fonts on a graphical screen, and then sent across the internet to anyone bored enough to read my ramblings, while at the same time my computer updates the clock, checks my mail, and does a dozen other things, but piece by piece it is all discrete, well contained, and completely understandable. Reading Stroustrup is pretty much mandatory to understand C++. I personally wouldn't bother with the standards (other than a brief go-over); so much of the code in those slides are just things you should never, ever program. Which is to say I disagree with the slides in large measure; it's almost suspicious to have some of that knowledge! Either your coworkers are writing truly atrocious code, or you are spending time learning corners of the language which is time that almost certainly could be better spent elsewhere, such as learning about memory mapped files or something. As others have said, knowing where the boundaries of where undefined behavior lies is important, but knowing how to avoid straying even close to that territory is far more important (always initialize variables. Even static ones! You are going to cost some sorry bastard half a day of debugging when they erroneously think the problem in the code is an uninitiated variable and keep chasing that until they finally figure it out)

banachtarski 13y ago

The points made in this (excellent I thought) presentation are very subtle and I'm not sure I would have come across them before organically. Because the behavior outlined in the presentation tends toward the exception, it's more efficient to learn it by reading. Effective C++ is a great place to start if you are trying to hone your C++ skills (practically every C++ programmer in the industry considers it mandatory). Keep in mind though, that knowing about evaluation order, and stack frames, and sequencing, and linker optimizations is all great and all, but I definitely consider it icing on the cake for a working software engineer for most positions. If you're a senior guy, sure, you should know this stuff. But the first thing to do is learn to actually write programs well. Authoring your own projects and contributing to open source is a great way to do it.

chipsy 13y ago

The top option for learning is to work off a spec to implement your own compiler. A good second place would be exhaustively studying and maintaining an existing compiler. Regular use of a language can build certain kinds of knowledge about the internals, but it won't be as well-rounded a study as actually working with them directly.

drblast 13y ago

You'll learn a lot of this via debugging or reverse engineering. Do you REALLY know how your compiler, linker, and loader work? If you do, you'll understand everything they wrote about static variables, stack frames, etc. The best exercise to start with is this one on creating the smallest ELF executable possible: http://www.muppetlabs.com/~breadbox/software/tiny/teensy.html http://www.muppetlabs.com/~breadbox/software/tiny/teensy.htm... And after that I'd do the same thing on Windows with a PE file. Then try to break things. Try to write a C program that has a buffer overflow bug in it and exploit that to change the flow of program execution. Then exploit the same bug to open an xterm or calc.exe. Getting really familiar with a good debugger will help you a lot with these things. After you have a decent understanding of program loading and execution, you can dive into the more language lawyery things they're talking about so you can answer the questions about the nineteen different meanings of "static" like the hacker girl. With C, that's a worthwhile goal. With C++, good fucking luck.

laichzeit0 13y ago

Get hold of an embedded microprocessor and write some silly app for it. E.g. ZigBee or Digi's rabbit microprocessors. I learned all my "deep C" on the rabbit. The architecture is based on Z80 and the compiler is a really shitty non-standard C compiler with some custom syntax. You will very quickly learn to throw away many assumptions you might have about how things work. Learn how to write mutithreaded code without an operating system or a thread library (hint: co-routines). Learn how memory layout, very quickly get a feel for the time/space tradeoffs of different algorithms, etc.

pjungwir 13y ago

Everyone else's advice to learn by doing is great, but there is also _Expert C Programming_ by Peter van der Linden: http://www.amazon.com/Expert-Programming-Peter-van-Linden/dp/0131774298 http://www.amazon.com/Expert-Programming-Peter-van-Linden/dp... The presentation's "Deep C" pun is a reference to this book, and if you make it all the way to slide 444 you'll see it mentioned as further reading. It's a wonderful book for understanding C (not C++) and what's really going on.

bloodorange 13y ago
It'll take a team of ten authors and a couple of decades to write something of that kind for C++. Though the language is full of horrors, I still quite enjoy C++ (esp. with C++11 and am looking forward to some C++14 features...)
- qznc 13y ago
  I know at least one guy, who is on it. http://herbsutter.com/gotw/ http://herbsutter.com/gotw/
  - bloodorange 13y ago
    Nice! Thanks for the link.