6 ms·

As a front-end web developer with no formal computer science background or traditional programming experience I find these kinds of articles extremely valuable.

As a front-end web developer with no formal computer science background or traditional programming experience I find these kinds of articles extremely valuable. I like to understand as much as possible, at least conceptually, what happens throughout the stack even if I don't touch it. Does anyone have any links to anything similar? perhaps for the Linux kernel or other lower level systems but with a top down overview like this? Especially anything that would build on this article. Effects and unexpected phenomena that manifest in networks like this also would be interesting.

foota 9y ago
You might find this interesting in the vein of weird networking things: https://www.ibiblio.org/harris/500milemail.html https://www.ibiblio.org/harris/500milemail.html
Quequau 9y ago
As an old dude who went through engineering before networks were a major thing and worked some years on embedded devices, I also find these sorts of articles extremely interesting.
zxcmx 9y ago
IMHO the classic is still TCP/IP illustrated by Richard W. Stevens.
Radle 9y ago
I agree, I had a course about this topic in university and I was like "This is why I am here". There are things you don't know that you don't know. And those always get you.
_pmf_ 9y ago
Pick a simple driver (for example, for a serial device / UART; [0] should be the Raspberry Pi's UART); prefer ARM, since this is more straightforward. From there, you can work your way up from how data arrives in controller registers, to how the driver handles the interrupt (you'll learn about ISRs and DSRs and how to read a microcontroller datasheet), to how the driver notifies the application program thread waiting on the data (you'll learn about scheduling), to how the read syscall transfers memory from kernel space to userspace. I'd not recommend diving directly into the network driver stack, since this has a lot or intermediary layers that do not further basic understanding[1]. Perhaps you should first try to get familiar with the Linux kernel's way of doing OO in C (if you are unfamiliar with this style; if you have done any modern C programming, it should not be a problem). (Do not start with the platform startup code; this would probably be too much for the start.) [0] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/tty/serial/amba-pl011.c?h=v4.12-rc3 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin... (maybe a bit too complex, since it uses DMA) [1] To be fair, the serial I/O layer has the tty layer as intermediary, so it's admittedly only slightly better
readittwice 9y ago
Same for me, although I studied computer science. I enjoy to read such nice written articles about technology. Even though most of the text was about stuff I already knew, there still was something new (maybe I just forgot?) for me: retransmission via ACK messages. But even if not, I still appreciate reading about this stuff again to freshen my knowledge. If I don't work on/with these topics regularly, I tend to forget about some of the details.
dominotw 9y ago
Linux Inside gitbook https://www.gitbook.com/book/0xax/linux-insides/details https://www.gitbook.com/book/0xax/linux-insides/details
avichalp 9y ago
Check these comics they are mostly about linux and networking: https://jvns.ca/zines/ https://jvns.ca/zines/
- sakawa 9y ago
  I really hope that's not my bias; I really like the format, but I find them lacking of some background information, or sometimes going.. stupid. ._.
  - always_good 9y ago
    What bias could you be referring to? And what does "sometimes going stupid" mean?
indigochill 9y ago
Depending on how deep you want to go: https://www.coursera.org/learn/build-a-computer https://www.coursera.org/learn/build-a-computer - This is a fantastic no-prerequisites course that has you actually implementing a functional CPU in 6 weeks starting from building basic logic gates out of NAND gates. There's a hands-on project every week to help you solidify your understanding of the lectures. The resources are also all available at their site: http://www.nand2tetris.org/ http://www.nand2tetris.org/. In case that sounds intimidating, I have no formal CS education either and couldn't implement any of the basic logic gates before I took the course, and I built a CPU by the end so I'm confident others can too. They have a "Part 2" to this course which has you build from the CPU to a functional operating system which would get you an introduction to how kernels work in general, although they recommend some programming experience for that part. Hacking: The Art of Exploitation - Despite the name, this book is actually an excellent introduction to low-level behavior both on the CPU and on networks (it's split into distinct sections to cover each topic). I would recommend picking this up after the course linked above, because it's somewhat brief with its explanation of CPU architecture. A beginner's course in C programming might also be advisable since this does use some basic C code which might be challenging if you've never been exposed to it before.
falcolas 9y ago
Even as a relatively network-savy developer, the 8 to 10 bit translation done over the wire was news to me. I love these kinds of implementation details.
- packetized 9y ago
  If you really want a fun time read about bipolar encoding & bipolar violations in T1s/E1s, which were once a source of much consternation to me - in a former life. https://en.m.wikipedia.org/wiki/Bipolar_violation https://en.m.wikipedia.org/wiki/Bipolar_violation
rectang 9y ago
For Linux/Unix, I'm going to recommend a book, which for now is probably too ambitious: "Computer Systems: A Programmer's Perspective" by Bryant and O'Hallaron, a.k.a. "CSAPP". http://csapp.cs.cmu.edu/3e/perspective.html http://csapp.cs.cmu.edu/3e/perspective.html This book is targeted at working programmers: Most books on systems—computer architecture, compilers, operating systems, and networking—are written as if the reader were going to design and implement such a system. We call this the “builder's persepective.” We believe that students should first learn about systems in terms of how they affect the behavior and performance of their programs—a “programmer's perspective.” I'm recommending CSAPP to give you an idea of where the end lies. If you grok its material, you will know more about OS kernels than would be required for any typical front-end or back-end web development job.
- eriknstr 9y ago
  I have a copy of an old edition of that book on my bookshelf, published 2003. I haven't gotten around to read it yet. I bought it because the cover stood out to me at a flea-market, and also because it was one of only a few books there which were about computers and I wanted to pick up some computer books for cheap. Here is a photo of my copy of the book: http://i.imgur.com/2sd5ivf.jpg http://i.imgur.com/2sd5ivf.jpg. Don't the front look nice? :)
- arawde 9y ago
  This was the textbook for my intermediate operating systems class, and I'd second this recommendation.
  - closeparen 9y ago
    This was the textbook for my first-year intro to systems and I'd recommend it too!
- mafuyu 9y ago
  I took this course with O'Hallaron and it's fantastic. I highly recommend doing the labs that come with it- they're fun and give you lots of working experience with the material.
dcosson 9y ago
After you read a bit on it, you can really learn a lot by opening wireshark and watching what's going on, googling for tutorials and documentation to understand the settings. As a front end developer there is actually a lot of networking involved making requests to the servers. It's really helpful to have a high level picture of how http works, and what common networking failure modes are, so you can use better abstractions in your code, expect common failures and shown nice error messages or retry requests, test your app's connections using wirshark to make sure there aren't extra round trips slowing it down, etc.
jsingleton 9y ago
Shameless plug but I cover a lot of lower level network and hardware effects for web developers in this book: https://unop.uk/book/ https://unop.uk/book/ It's mainly aimed at people interested in ASP.NET Core but there are a lot of general performance tips in there. I have a EEE background and I think understanding things down to the transistor level can really help you write better software. BTW Gary's WAT video is great fun if you haven't seen it.
teh_klev 9y ago
I can thoroughly recommend "Internet Routing Architectures 2nd Edition" by Sam Halabi. It's an older text, but much of it is still relevant: http://amzn.eu/5tCOQfW http://amzn.eu/5tCOQfW It was my goto book when I worked on BGP-4 odds and sods.
ajdecon 9y ago
For networking in particular, Andrew Tanenbaum's book "Computer Networking" [0] is quite good, and covers everything from the physical layer on up. The most recent edition is from 2010, but much of this doesn't change quickly so it's still quite relevant. I've found it extremely helpful at times. [0] https://www.goodreads.com/book/show/8515228-computer-networks https://www.goodreads.com/book/show/8515228-computer-network...
jmagoon 9y ago
I'm pretty surprised that no one has recommended the other articles on the same site. They are pretty excellent: https://www.destroyallsoftware.com/compendium https://www.destroyallsoftware.com/compendium The screencasts are fantastic as well.
- Sundiata 9y ago
  At $29 a month it's pretty steep though. I'm unable to afford it right now, however after reading the linked article I'm tempted to get a subscription.

foota 9y ago

You might find this interesting in the vein of weird networking things: https://www.ibiblio.org/harris/500milemail.html https://www.ibiblio.org/harris/500milemail.html

Quequau 9y ago

As an old dude who went through engineering before networks were a major thing and worked some years on embedded devices, I also find these sorts of articles extremely interesting.

zxcmx 9y ago

IMHO the classic is still TCP/IP illustrated by Richard W. Stevens.

Radle 9y ago

I agree, I had a course about this topic in university and I was like "This is why I am here". There are things you don't know that you don't know. And those always get you.

_pmf_ 9y ago

Pick a simple driver (for example, for a serial device / UART; [0] should be the Raspberry Pi's UART); prefer ARM, since this is more straightforward. From there, you can work your way up from how data arrives in controller registers, to how the driver handles the interrupt (you'll learn about ISRs and DSRs and how to read a microcontroller datasheet), to how the driver notifies the application program thread waiting on the data (you'll learn about scheduling), to how the read syscall transfers memory from kernel space to userspace. I'd not recommend diving directly into the network driver stack, since this has a lot or intermediary layers that do not further basic understanding[1]. Perhaps you should first try to get familiar with the Linux kernel's way of doing OO in C (if you are unfamiliar with this style; if you have done any modern C programming, it should not be a problem). (Do not start with the platform startup code; this would probably be too much for the start.) [0] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/tty/serial/amba-pl011.c?h=v4.12-rc3 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin... (maybe a bit too complex, since it uses DMA) [1] To be fair, the serial I/O layer has the tty layer as intermediary, so it's admittedly only slightly better

readittwice 9y ago

Same for me, although I studied computer science. I enjoy to read such nice written articles about technology. Even though most of the text was about stuff I already knew, there still was something new (maybe I just forgot?) for me: retransmission via ACK messages. But even if not, I still appreciate reading about this stuff again to freshen my knowledge. If I don't work on/with these topics regularly, I tend to forget about some of the details.

dominotw 9y ago

Linux Inside gitbook https://www.gitbook.com/book/0xax/linux-insides/details https://www.gitbook.com/book/0xax/linux-insides/details

avichalp 9y ago

Check these comics they are mostly about linux and networking: https://jvns.ca/zines/ https://jvns.ca/zines/

sakawa 9y ago
I really hope that's not my bias; I really like the format, but I find them lacking of some background information, or sometimes going.. stupid. ._.
- always_good 9y ago
  What bias could you be referring to? And what does "sometimes going stupid" mean?

indigochill 9y ago

Depending on how deep you want to go: https://www.coursera.org/learn/build-a-computer https://www.coursera.org/learn/build-a-computer - This is a fantastic no-prerequisites course that has you actually implementing a functional CPU in 6 weeks starting from building basic logic gates out of NAND gates. There's a hands-on project every week to help you solidify your understanding of the lectures. The resources are also all available at their site: http://www.nand2tetris.org/ http://www.nand2tetris.org/. In case that sounds intimidating, I have no formal CS education either and couldn't implement any of the basic logic gates before I took the course, and I built a CPU by the end so I'm confident others can too. They have a "Part 2" to this course which has you build from the CPU to a functional operating system which would get you an introduction to how kernels work in general, although they recommend some programming experience for that part. Hacking: The Art of Exploitation - Despite the name, this book is actually an excellent introduction to low-level behavior both on the CPU and on networks (it's split into distinct sections to cover each topic). I would recommend picking this up after the course linked above, because it's somewhat brief with its explanation of CPU architecture. A beginner's course in C programming might also be advisable since this does use some basic C code which might be challenging if you've never been exposed to it before.

falcolas 9y ago

Even as a relatively network-savy developer, the 8 to 10 bit translation done over the wire was news to me. I love these kinds of implementation details.

packetized 9y ago
If you really want a fun time read about bipolar encoding & bipolar violations in T1s/E1s, which were once a source of much consternation to me - in a former life. https://en.m.wikipedia.org/wiki/Bipolar_violation https://en.m.wikipedia.org/wiki/Bipolar_violation

rectang 9y ago

For Linux/Unix, I'm going to recommend a book, which for now is probably too ambitious: "Computer Systems: A Programmer's Perspective" by Bryant and O'Hallaron, a.k.a. "CSAPP". http://csapp.cs.cmu.edu/3e/perspective.html http://csapp.cs.cmu.edu/3e/perspective.html This book is targeted at working programmers: Most books on systems—computer architecture, compilers, operating systems, and networking—are written as if the reader were going to design and implement such a system. We call this the “builder's persepective.” We believe that students should first learn about systems in terms of how they affect the behavior and performance of their programs—a “programmer's perspective.” I'm recommending CSAPP to give you an idea of where the end lies. If you grok its material, you will know more about OS kernels than would be required for any typical front-end or back-end web development job.

eriknstr 9y ago
I have a copy of an old edition of that book on my bookshelf, published 2003. I haven't gotten around to read it yet. I bought it because the cover stood out to me at a flea-market, and also because it was one of only a few books there which were about computers and I wanted to pick up some computer books for cheap. Here is a photo of my copy of the book: http://i.imgur.com/2sd5ivf.jpg http://i.imgur.com/2sd5ivf.jpg. Don't the front look nice? :)
arawde 9y ago
This was the textbook for my intermediate operating systems class, and I'd second this recommendation.
- closeparen 9y ago
  This was the textbook for my first-year intro to systems and I'd recommend it too!
mafuyu 9y ago
I took this course with O'Hallaron and it's fantastic. I highly recommend doing the labs that come with it- they're fun and give you lots of working experience with the material.

dcosson 9y ago

After you read a bit on it, you can really learn a lot by opening wireshark and watching what's going on, googling for tutorials and documentation to understand the settings. As a front end developer there is actually a lot of networking involved making requests to the servers. It's really helpful to have a high level picture of how http works, and what common networking failure modes are, so you can use better abstractions in your code, expect common failures and shown nice error messages or retry requests, test your app's connections using wirshark to make sure there aren't extra round trips slowing it down, etc.

jsingleton 9y ago

Shameless plug but I cover a lot of lower level network and hardware effects for web developers in this book: https://unop.uk/book/ https://unop.uk/book/ It's mainly aimed at people interested in ASP.NET Core but there are a lot of general performance tips in there. I have a EEE background and I think understanding things down to the transistor level can really help you write better software. BTW Gary's WAT video is great fun if you haven't seen it.

teh_klev 9y ago

I can thoroughly recommend "Internet Routing Architectures 2nd Edition" by Sam Halabi. It's an older text, but much of it is still relevant: http://amzn.eu/5tCOQfW http://amzn.eu/5tCOQfW It was my goto book when I worked on BGP-4 odds and sods.

ajdecon 9y ago

For networking in particular, Andrew Tanenbaum's book "Computer Networking" [0] is quite good, and covers everything from the physical layer on up. The most recent edition is from 2010, but much of this doesn't change quickly so it's still quite relevant. I've found it extremely helpful at times. [0] https://www.goodreads.com/book/show/8515228-computer-networks https://www.goodreads.com/book/show/8515228-computer-network...

jmagoon 9y ago

I'm pretty surprised that no one has recommended the other articles on the same site. They are pretty excellent: https://www.destroyallsoftware.com/compendium https://www.destroyallsoftware.com/compendium The screencasts are fantastic as well.

Sundiata 9y ago
At $29 a month it's pretty steep though. I'm unable to afford it right now, however after reading the linked article I'm tempted to get a subscription.