6 ms·

How the Internet works

45 points by Jonhoo 11y ago

vezzy-fnord 11y ago
I'd honestly recommend that people just pick up a copy of and devote some time getting through Andrew S. Tanenbaum and David J. Wetherall's Computer Networks.
- dayjah 11y ago
  I'd also recommend "How the world was one" ( http://www.amazon.com/How-World-Was-Arthur-Clarke/dp/0553074407 http://www.amazon.com/How-World-Was-Arthur-Clarke/dp/0553074... ) It does an astonishingly good job of describing how networks evolved and why the function as they do today (as of mid-90's - not much has changed AFAICT).
  - danso 11y ago
    Just bought a used paperback version from Amazon, thanks!
- m-app 11y ago
  Also, Tubes by Andrew Blum to get a taste of the physical part of the Internet. (http://www.amazon.com/Tubes-A-Journey-Center-Internet/dp/0061994952 http://www.amazon.com/Tubes-A-Journey-Center-Internet/dp/006...)
- draker 11y ago
  Any recommendations that aren't textbooks? Even the Kindle version of that book is $100.
  - irl_zebra 11y ago
    So I just copy/pasted the book recommendation into search, and the first result was a pdf of the entire book hosted on an edu server. Forgive my ignorance, but would this be a pirated copy or something? This often happens when I'm searching for book recommendations like this and I just assumed older versions were pushed out into the public domain or something.
jasode 11y ago
>In the interest of making the topic easier to comprehend, we will start with a fairly high-level overview, and then dive into each individual component as we go along. Our journey starts with a user called Alice. Alice owns a laptop, and wants to send e-mail to a person called Bob. ... [wall of text...] I understand the enthusiasm to spread knowledge but a wall of text like that isn't going to be helpful for novices. When technical people try to impart knowledge, the text they write usually suffers from the Curse of Knowledge[1]. For example, "DHCP" and "UDP" is mentioned several times which is a level of detail that is not necessary for an introductory overview. Instead, to explain how the internet works would require lots of diagrams, pictures, videos, etc. illustrating different scenarios. E.g., sending SMTP is different from HTTP and both use DNS. All these "mental buckets" (and many others) are difficult to separate for novices. Instead of drilling into a timeline of bytes sent from Alice to Bob, new concepts can be taught as "progressive layers" of detail (with the timeline rewound each time.) Answering the question of "how the internet works" is so open-ended that apparently, some tech interviewers like to use it for evaluating job candidates. [1]https://en.wikipedia.org/wiki/Curse_of_knowledge https://en.wikipedia.org/wiki/Curse_of_knowledge
- Jonhoo 11y ago
  I see your point, but I think it's still important to give an end-to-end description, allowing people to tie it all together. The wall of text, if you'd read it, uses progressive layers in several places (like core Internet routing), specifically to try and avoid information overload for the reader. If a reader were to actually use networks for something (like try to write a program that uses TCP), they would obviously need much more detailed knowledge. At that point, diagrams become necessary, as do much deeper explanations, but the goal of this post was to give an overview of what things are needed to provide end-to-end communication, and why.
- fumar 11y ago
  Agree on all your points. I also think the general structure can be improved by adding a table of contents. This will better guide the user through the wall of text.
- rahimnathwani 11y ago
pyvpx 11y ago
The "Tier" system is out-dated at best and shouldn't be regurgitated in this day and age.
- Jonhoo 11y ago
  I'd be happy to get rid of it if it is indeed out-dated as you say, though that was not my impression. Could you give me some references giving information about how core Internet is now being done?
  - dvanduzer 11y ago
    Plenty of people will still talk about providers that way, but it has never been about topology the way you describe. BGP is a mesh, not a tree. Packets that do have to leave one network are routed to the closest BGP peer. Plenty of small providers peer directly with other small providers (and always have). It's also quite misleading to talk about the hostname once you've mentioned BGP. At that layer, the routers have completely "forgotten" the hostname, and are only looking at IP address/prefix to move that packet. I understand why one might want to illustrate the fact that mail delivered via the SMTP protocol might be retrieved from the same server via the HTTP protocol running in different ports. But this is not a feature of TCP. The term "TCP multiplexing" commonly refers to reusing the same end-to-end TCP session for multiple application requests (e.g. more than one HTTP GET to the same server/port). Very different than IP multiplexing. But email is also a very bad example to use to illustrate IP multiplexing, because email uses a store-and-forward mechanism very unlike HTTP lookups. A sender's email client just doesn't do lookups for a remote email provider. A mail client has a provider-specified SMTP server for all outbound mail traffic, regardless of destination. The client does use DNS to find that server's IP address. But the sender's provider's outbound email server performs an entirely different category of DNS lookup to figure out the destination mail server for the domain. This difference makes it a particularly bad idea to mention SMTP at all, only to say that it's unimportant. I'm sorry this is such harsh criticism. It is extremely difficult to learn this material, let alone teach it to someone else in a digestible fashion. So please don't let this discourage you from trying to write a document like this. But it would be a lot more effective if you just didn't try at all to explain quite so much material in one post. Hope that helps a little?
heydonovan 11y ago
Have you seen Code.org's video on routing? Might be worth linking to, as it is very simple to understand: https://www.youtube.com/watch?v=AYdF7b3nMto https://www.youtube.com/watch?v=AYdF7b3nMto
aristus 11y ago
http://carlos.bueno.org/2013/03/internet-shape.html http://carlos.bueno.org/2013/03/internet-shape.html
markc 11y ago
Apologies in advance for a near useless comment, but this typo? made me smile: "When Bob "sumbits" this form, another HTTP request will be sent" It took a moment to decide that it probably wasn't intentional :)
- Jonhoo 11y ago
  Hehe, thanks, fixed.

45 points by Jonhoo 11y ago

I'd honestly recommend that people just pick up a copy of and devote some time getting through Andrew S. Tanenbaum and David J. Wetherall's Computer Networks.

dayjah 11y ago
I'd also recommend "How the world was one" ( http://www.amazon.com/How-World-Was-Arthur-Clarke/dp/0553074407 http://www.amazon.com/How-World-Was-Arthur-Clarke/dp/0553074... ) It does an astonishingly good job of describing how networks evolved and why the function as they do today (as of mid-90's - not much has changed AFAICT).
- danso 11y ago
  Just bought a used paperback version from Amazon, thanks!
m-app 11y ago
Also, Tubes by Andrew Blum to get a taste of the physical part of the Internet. (http://www.amazon.com/Tubes-A-Journey-Center-Internet/dp/0061994952 http://www.amazon.com/Tubes-A-Journey-Center-Internet/dp/006...)
draker 11y ago
Any recommendations that aren't textbooks? Even the Kindle version of that book is $100.
- irl_zebra 11y ago
  So I just copy/pasted the book recommendation into search, and the first result was a pdf of the entire book hosted on an edu server. Forgive my ignorance, but would this be a pirated copy or something? This often happens when I'm searching for book recommendations like this and I just assumed older versions were pushed out into the public domain or something.

jasode 11y ago

>In the interest of making the topic easier to comprehend, we will start with a fairly high-level overview, and then dive into each individual component as we go along. Our journey starts with a user called Alice. Alice owns a laptop, and wants to send e-mail to a person called Bob. ... [wall of text...] I understand the enthusiasm to spread knowledge but a wall of text like that isn't going to be helpful for novices. When technical people try to impart knowledge, the text they write usually suffers from the Curse of Knowledge[1]. For example, "DHCP" and "UDP" is mentioned several times which is a level of detail that is not necessary for an introductory overview. Instead, to explain how the internet works would require lots of diagrams, pictures, videos, etc. illustrating different scenarios. E.g., sending SMTP is different from HTTP and both use DNS. All these "mental buckets" (and many others) are difficult to separate for novices. Instead of drilling into a timeline of bytes sent from Alice to Bob, new concepts can be taught as "progressive layers" of detail (with the timeline rewound each time.) Answering the question of "how the internet works" is so open-ended that apparently, some tech interviewers like to use it for evaluating job candidates. [1]https://en.wikipedia.org/wiki/Curse_of_knowledge https://en.wikipedia.org/wiki/Curse_of_knowledge

Jonhoo 11y ago
I see your point, but I think it's still important to give an end-to-end description, allowing people to tie it all together. The wall of text, if you'd read it, uses progressive layers in several places (like core Internet routing), specifically to try and avoid information overload for the reader. If a reader were to actually use networks for something (like try to write a program that uses TCP), they would obviously need much more detailed knowledge. At that point, diagrams become necessary, as do much deeper explanations, but the goal of this post was to give an overview of what things are needed to provide end-to-end communication, and why.
fumar 11y ago
Agree on all your points. I also think the general structure can be improved by adding a table of contents. This will better guide the user through the wall of text.
rahimnathwani 11y ago

pyvpx 11y ago

The "Tier" system is out-dated at best and shouldn't be regurgitated in this day and age.

Jonhoo 11y ago
I'd be happy to get rid of it if it is indeed out-dated as you say, though that was not my impression. Could you give me some references giving information about how core Internet is now being done?
- dvanduzer 11y ago
  Plenty of people will still talk about providers that way, but it has never been about topology the way you describe. BGP is a mesh, not a tree. Packets that do have to leave one network are routed to the closest BGP peer. Plenty of small providers peer directly with other small providers (and always have). It's also quite misleading to talk about the hostname once you've mentioned BGP. At that layer, the routers have completely "forgotten" the hostname, and are only looking at IP address/prefix to move that packet. I understand why one might want to illustrate the fact that mail delivered via the SMTP protocol might be retrieved from the same server via the HTTP protocol running in different ports. But this is not a feature of TCP. The term "TCP multiplexing" commonly refers to reusing the same end-to-end TCP session for multiple application requests (e.g. more than one HTTP GET to the same server/port). Very different than IP multiplexing. But email is also a very bad example to use to illustrate IP multiplexing, because email uses a store-and-forward mechanism very unlike HTTP lookups. A sender's email client just doesn't do lookups for a remote email provider. A mail client has a provider-specified SMTP server for all outbound mail traffic, regardless of destination. The client does use DNS to find that server's IP address. But the sender's provider's outbound email server performs an entirely different category of DNS lookup to figure out the destination mail server for the domain. This difference makes it a particularly bad idea to mention SMTP at all, only to say that it's unimportant. I'm sorry this is such harsh criticism. It is extremely difficult to learn this material, let alone teach it to someone else in a digestible fashion. So please don't let this discourage you from trying to write a document like this. But it would be a lot more effective if you just didn't try at all to explain quite so much material in one post. Hope that helps a little?

heydonovan 11y ago

Have you seen Code.org's video on routing? Might be worth linking to, as it is very simple to understand: https://www.youtube.com/watch?v=AYdF7b3nMto https://www.youtube.com/watch?v=AYdF7b3nMto

aristus 11y ago

http://carlos.bueno.org/2013/03/internet-shape.html http://carlos.bueno.org/2013/03/internet-shape.html

markc 11y ago

Apologies in advance for a near useless comment, but this typo? made me smile: "When Bob "sumbits" this form, another HTTP request will be sent" It took a moment to decide that it probably wasn't intentional :)

Jonhoo 11y ago
Hehe, thanks, fixed.