4 ms·

I watched simple text-based internet protocols eat the efficient, ASN.1 OSI protocols for breakfast. From the OSI side of the fence. You can talk about efficie

by jbert 13y ago

I watched simple text-based internet protocols eat the efficient, ASN.1 OSI protocols for breakfast. From the OSI side of the fence. You can talk about efficiency until you're blue in the face, but human-readability is very useful. I used to be able to pretty much parse a hexdump of an X.400 P1 PDU by eye (certainly if I could reformat in a text editor), but even today I found it useful to eyeball a recalcitrant programs's HTTP request/response cycle by simply catching it under strace and grepping for "HTTP". It's a massive reduction of friction to have human-readable protocols. I used to make the argument that it didn't matter and I was wrong.

zxcdw 13y ago
> * but human-readability is very useful.* For who, how often and when exactly? As far as I have understood, we are doing something wrong if we have to deal with debugging protocols which implement an abstraction, rather than just let the abstraction do it's job. This is like saying that "Well for programmers x86 assembly instruction mnemonics are useful compared to machine code bytes!", to which one could say that for an average programmer that makes no sense. I didn't quite care about stuff like this until I got introduced to information theory and really started thinking about what it means to send and receive information. The amount of totally unnecessary waste is astounding.
- dsr_ 13y ago
  I'm starting to feel like I start all my HN posts with "Hi, I'm a sysadmin." Hi, I'm a sysadmin. Sometimes we have to debug things that are on the other side of the world. When we do that, we need to have a mental model of what we're doing. You're familiar with that as a software developer: you have a mental model of the capabilities of the language that you are using, and you are fitting that in with the mental model of the problem you are solving and the context of existing program code. To sysadmins, protocols are like programming languages. That's why we like text-based protocols: we can fit them in our heads and type them out, slowly, like ancient creaky teletypes that make lots of mistakes and pause in between commands. Meanwhile, we're running down checklists: can we connect? OK, there's no IP-based packet filtering. What does the banner say? Can it support STARTTLS? OK, disconnect and try again with telnet-tls. Hey, it doesn't negotiate... Totally unnecessary waste? Sure, assuming someone has already written a tool which does all the testing for you, and you know that tool exists, and you have access to it right now over your tiny smartphone's 3G connection at 0400 while you thought you were on vacation.
  - zxcdw 13y ago
    Hi, I'm a programmer. As a sysadmin, do you really have to deal with HTTP protocol by hand on such basis that a tool could not do the task for you if the protocol wasn't human readable?
    - dsr_ 13y ago
      No. I need to deal with a dozen protocols in an emergency situation where I can't guarantee the appropriate tool is going to be available. (When it's not an emergency, yay tools!) For the same reason, I can use vi with no vim extensions. For the same reason, I like configuration files written in text formats, not binary blobs. Binary would be faster, sure. When it breaks, you need the precise tools to know what you're doing. I commend to you RFC 3117 - http://www.rfc-editor.org/in-notes/rfc3117.txt http://www.rfc-editor.org/in-notes/rfc3117.txt - as a discussion on how to figure out what a network protocol has to do, and how.
      - weavejester 13y ago
        Under what circumstances are you logging into machines with only telnet available? Presumably you also lack SSH, otherwise you could use a tunnel, so I assume you're physically logging into a bare-bones OS that lacks even basic functionality. Is that really something you do often? And if so, why?
  - lstamour 13y ago
    Don't want to debug it? Don't turn it on. The only reason you would is if it had more benefits than drawbacks. Standards bodies make standards, they don't force people to pick one.
  - snuxoll 13y ago
    If you are using telnet on your smartphone at 0400 on vacation you should probably have access to an SSH client that you can use to get on a real system to run curl on.
- laumars 13y ago
  Well actually, your assembly example is a good one because C#, Go and a bunch of other similar languages all have refractor information compiled into the output binary because debugging machine code is a pig (even with assembly mnemonics). So even the average developer enjoys the luxury of wasted information in modern specifications.
  - zxcdw 13y ago
    My point was that a average programmer does not understand a thing about assembly languages, because they are abstracted away by well working implementations of compilers and debuggers. I live under an assumption that such is also the case with HTTP, or at least should be, because for me it seems that a protocol as simple as HTTP would be abstracted away ages ago by libraries and implementations.
    - laumars 13y ago
      Plain text HTTP (and all the other ASCII-based networking protocols) is already a layer of abstraction on top of TCP/IP data packets. So a better comparison would be HTTP as the higher level programming language and the raw TCP/IP packets as the assembly. In which case we're back to the point that developers would care if they had to write websites in binary. The problem with many web developers these days is that they don't understand or don't care about the networking side of things. Web development is such a high level view of programming that many developers who've only grown up with targeting the web, those kinds of developers don't also appreciate just how many layers of abstraction there are between them and the users navigating their site. As far as they're concerned, they just bang out some PHP, copy the files onto some shared hosting provider and let the sys admins worry about the rest. Which is fine if that's all they want to do, but there's a whole plethora of technology at work - even beneath the HTTP protocol. As for tools to query HTTP, I swear by curl: curl -i --silent example.com | head # http headers (written by web app) curl -I example.com # http headers (written by web daemon) curl -v --silent example.com | more # verbose output; great for tracking down faults curl -H "host:example.com" ip.address # set the host header; useful when using named based virtual hosts curl -A "opera mobile" example.com # sets the user agent; useful for working around mobile / desktop redirects ...etc. Rarely does a day go by and I'm not using curl.
- jbert 13y ago
  > For who, how often and when exactly? As far as I have understood, we are doing something wrong if we have to deal with debugging protocols which implement an abstraction, rather than just let the abstraction do it's job. Any time you're debugging "one layer above" and you're in Sherlock Holmes territory (i.e. the problem seems impossible, so one of your basic assumptions must be incorrect) you have to check stuff. And if you check your protocol by using the protocol-parsing tool which comes with your protocol implementation, you're not doing an independent test that it's actually working OK. As a more concrete example, I'm using a library doing AWS request/responses over HTTP. There are various places I could add instrumentation to dump information but: - it takes work to add or enable. I can grab the on-the-wire protocol with strace or tcpdump (this a reason why it's always good to provide a non-SSL option for your protocol) - whatever is causing the problem could be below the layer I'm logging at - they are all error prone. Maybe I miss a part of what goes on the wire. The data-on-the-wire is the only thing which matters for the protocol. The other end has no additional state. When you're debugging, you have to validate stuff. And look for patterns. Human-readable protocols facilitate both of those difficult activities, reducing friction.
  - zxcdw 13y ago
    Human readability too could mean base64 encoding. Instead of ACKNOWLEDGE you say ACK or mere A. Instead of REQUEST PAGE FROM <path>, you say RPF <path> and so on. This is my main point of hatred towards "human readable formats", because they waste bytes for no reason. I can run the water for as long as I want without absolutely no consequences for me, but why would I do it if I can avoid it? Why would I not save resources whenever I can, even though I don't need to do it? It's more a philosophical question, to which I would answer with "save anything you can, whenever you can and make no waste.". Very simple.
    - jbert 13y ago
      I don't disagree that human-readable doesn't mean "verbose", but in answer to: > Why would I not save resources whenever I can, even though I don't need to do it? It's a cost-benefit. You don't spend your evenings clipping coupons all the coupons you can (which would save you some pennies). Or if you do, you don't stay up late to do it. The benefit to you of some free time is greater than the benefit of saving the pennies. Similarly, the cost of mild verbosity is balanced against the benefit. Basically, I don't buy an absolutist position of "save wherever you can". There are costs to saving, make a judgement whether it is worth it.
pbsdp 13y ago
The problem with ASN.1 was ASN.1, BER, and DER themselves not the fact that a binary encoding was used.

by jbert 13y ago

zxcdw 13y ago

> * but human-readability is very useful.* For who, how often and when exactly? As far as I have understood, we are doing something wrong if we have to deal with debugging protocols which implement an abstraction, rather than just let the abstraction do it's job. This is like saying that "Well for programmers x86 assembly instruction mnemonics are useful compared to machine code bytes!", to which one could say that for an average programmer that makes no sense. I didn't quite care about stuff like this until I got introduced to information theory and really started thinking about what it means to send and receive information. The amount of totally unnecessary waste is astounding.

dsr_ 13y ago
I'm starting to feel like I start all my HN posts with "Hi, I'm a sysadmin." Hi, I'm a sysadmin. Sometimes we have to debug things that are on the other side of the world. When we do that, we need to have a mental model of what we're doing. You're familiar with that as a software developer: you have a mental model of the capabilities of the language that you are using, and you are fitting that in with the mental model of the problem you are solving and the context of existing program code. To sysadmins, protocols are like programming languages. That's why we like text-based protocols: we can fit them in our heads and type them out, slowly, like ancient creaky teletypes that make lots of mistakes and pause in between commands. Meanwhile, we're running down checklists: can we connect? OK, there's no IP-based packet filtering. What does the banner say? Can it support STARTTLS? OK, disconnect and try again with telnet-tls. Hey, it doesn't negotiate... Totally unnecessary waste? Sure, assuming someone has already written a tool which does all the testing for you, and you know that tool exists, and you have access to it right now over your tiny smartphone's 3G connection at 0400 while you thought you were on vacation.
- zxcdw 13y ago
  Hi, I'm a programmer. As a sysadmin, do you really have to deal with HTTP protocol by hand on such basis that a tool could not do the task for you if the protocol wasn't human readable?
  - dsr_ 13y ago
    No. I need to deal with a dozen protocols in an emergency situation where I can't guarantee the appropriate tool is going to be available. (When it's not an emergency, yay tools!) For the same reason, I can use vi with no vim extensions. For the same reason, I like configuration files written in text formats, not binary blobs. Binary would be faster, sure. When it breaks, you need the precise tools to know what you're doing. I commend to you RFC 3117 - http://www.rfc-editor.org/in-notes/rfc3117.txt http://www.rfc-editor.org/in-notes/rfc3117.txt - as a discussion on how to figure out what a network protocol has to do, and how.
    - weavejester 13y ago
      Under what circumstances are you logging into machines with only telnet available? Presumably you also lack SSH, otherwise you could use a tunnel, so I assume you're physically logging into a bare-bones OS that lacks even basic functionality. Is that really something you do often? And if so, why?
- lstamour 13y ago
  Don't want to debug it? Don't turn it on. The only reason you would is if it had more benefits than drawbacks. Standards bodies make standards, they don't force people to pick one.
- snuxoll 13y ago
  If you are using telnet on your smartphone at 0400 on vacation you should probably have access to an SSH client that you can use to get on a real system to run curl on.
laumars 13y ago
Well actually, your assembly example is a good one because C#, Go and a bunch of other similar languages all have refractor information compiled into the output binary because debugging machine code is a pig (even with assembly mnemonics). So even the average developer enjoys the luxury of wasted information in modern specifications.
- zxcdw 13y ago
  My point was that a average programmer does not understand a thing about assembly languages, because they are abstracted away by well working implementations of compilers and debuggers. I live under an assumption that such is also the case with HTTP, or at least should be, because for me it seems that a protocol as simple as HTTP would be abstracted away ages ago by libraries and implementations.
  - laumars 13y ago
    Plain text HTTP (and all the other ASCII-based networking protocols) is already a layer of abstraction on top of TCP/IP data packets. So a better comparison would be HTTP as the higher level programming language and the raw TCP/IP packets as the assembly. In which case we're back to the point that developers would care if they had to write websites in binary. The problem with many web developers these days is that they don't understand or don't care about the networking side of things. Web development is such a high level view of programming that many developers who've only grown up with targeting the web, those kinds of developers don't also appreciate just how many layers of abstraction there are between them and the users navigating their site. As far as they're concerned, they just bang out some PHP, copy the files onto some shared hosting provider and let the sys admins worry about the rest. Which is fine if that's all they want to do, but there's a whole plethora of technology at work - even beneath the HTTP protocol. As for tools to query HTTP, I swear by curl: curl -i --silent example.com | head # http headers (written by web app) curl -I example.com # http headers (written by web daemon) curl -v --silent example.com | more # verbose output; great for tracking down faults curl -H "host:example.com" ip.address # set the host header; useful when using named based virtual hosts curl -A "opera mobile" example.com # sets the user agent; useful for working around mobile / desktop redirects ...etc. Rarely does a day go by and I'm not using curl.
jbert 13y ago
> For who, how often and when exactly? As far as I have understood, we are doing something wrong if we have to deal with debugging protocols which implement an abstraction, rather than just let the abstraction do it's job. Any time you're debugging "one layer above" and you're in Sherlock Holmes territory (i.e. the problem seems impossible, so one of your basic assumptions must be incorrect) you have to check stuff. And if you check your protocol by using the protocol-parsing tool which comes with your protocol implementation, you're not doing an independent test that it's actually working OK. As a more concrete example, I'm using a library doing AWS request/responses over HTTP. There are various places I could add instrumentation to dump information but: - it takes work to add or enable. I can grab the on-the-wire protocol with strace or tcpdump (this a reason why it's always good to provide a non-SSL option for your protocol) - whatever is causing the problem could be below the layer I'm logging at - they are all error prone. Maybe I miss a part of what goes on the wire. The data-on-the-wire is the only thing which matters for the protocol. The other end has no additional state. When you're debugging, you have to validate stuff. And look for patterns. Human-readable protocols facilitate both of those difficult activities, reducing friction.
- zxcdw 13y ago
  Human readability too could mean base64 encoding. Instead of ACKNOWLEDGE you say ACK or mere A. Instead of REQUEST PAGE FROM <path>, you say RPF <path> and so on. This is my main point of hatred towards "human readable formats", because they waste bytes for no reason. I can run the water for as long as I want without absolutely no consequences for me, but why would I do it if I can avoid it? Why would I not save resources whenever I can, even though I don't need to do it? It's more a philosophical question, to which I would answer with "save anything you can, whenever you can and make no waste.". Very simple.
  - jbert 13y ago
    I don't disagree that human-readable doesn't mean "verbose", but in answer to: > Why would I not save resources whenever I can, even though I don't need to do it? It's a cost-benefit. You don't spend your evenings clipping coupons all the coupons you can (which would save you some pennies). Or if you do, you don't stay up late to do it. The benefit to you of some free time is greater than the benefit of saving the pennies. Similarly, the cost of mild verbosity is balanced against the benefit. Basically, I don't buy an absolutist position of "save wherever you can". There are costs to saving, make a judgement whether it is worth it.

pbsdp 13y ago

The problem with ASN.1 was ASN.1, BER, and DER themselves not the fact that a binary encoding was used.