9 ms·
This idea is bad for a couple reasons. First, ASCII is inefficient. People don't interpret HTTP, computers do. Web servers and browsers. People only look at HT
by peterwwillis 13y ago
This idea is bad for a couple reasons.
First, ASCII is inefficient. People don't interpret HTTP, computers do. Web servers and browsers. People only look at HTTP when they want to troubleshoot without any tools. With real tools, you can find out what's broken much quicker. And there's plenty of things you can miss without a real HTTP interpreter. Most hackers prefer to think of themselves as wizards that can spy 0's and 1's and tell you what the weather is. It doesn't make for a better protocol, though.
Second, we can already break HTTP responses up in multiple parts, using a novel idea called "multipart". It sucks and nobody has used it since HTML/JS found new ways of providing content. http://www.w3.org/Protocols/rfc1341/7_2_Multipart.html http://www.w3.org/Protocols/rfc1341/7_2_Multipart.html
Third, it's a hack. If you want to improve the protocol, improve the protocol, don't just hack onto it to make it do what you want. I could make a horse and buggy go 60mph, but would it be a good idea? How about just designing a better buggy that is intended to go 60mph?
Fourth, fixed-length records are the wave of the future! It solves crazy problems like header injection and request integrity checking. Moreover, it makes for simpler, more efficient parsing of requests.
Fifth, redundancies introduced from the beginning of time need to go away, like terminating every record with "\r\n", or passing the same headers on every single damn request when once should be just fine for a stream of requests. Little inefficiencies like this don't go away if you just hack onto the same old protocol forever.
Sixth, the flow control improvements can make different applications more efficient by both not having to hold state of where and when traffic is coming and improving flow across disparate network hops.
Seventh, as to a general question "why use binary when we can compress stuff", what is smaller: a string of bits, or a compressed string of digits? Add to this that every header could have a 32-bit identifier (4 bytes) and you've got more efficient compression than gzip. Of course, most people would argue that you don't save enough time or bandwidth or cpu cycles to make a difference, but it would make for easier-to-write parsers, which would make working with the protocol in general more attractive. But then you have your binary-detractor-wizard-hackers and the whole conversation becomes an infinite loop.
- deleted 13y ago[deleted]
- romaniv 13y agoThe same arguments can be applied to HTML, CSS, JSON, RSS and so on. I fail to see the crucial difference between those and HTTP. Or would you say the web as a whole should be binary? Fifth, redundancies introduced from the beginning of time need to go away I wholeheartedly agree with this, but it doesn't automatically warrant binary encoding. Human readability is a huge bonus in any protocol or format. Not because normally people read those protocols, but because people read ASCII and therefore they have good tools to work with ASCII.
- avolcano 13y agoMost people aren't writing HTTP requests by hand in a text editor.
- skriticos2 13y agoPeople are actually writing HTML/CSS/JS, so no. They have a whole other set of issues like being XML based (for HTML), but they do the job and are not likely to experience fundamental change in the next 5 years in broad adoption.
- bajsejohannes 13y ago> First, ASCII is inefficient. People don't interpret HTTP, computers do. I can't tell you how many times I've manually read HTTP. To be sure, it's insignificant compared to how many HTTP headers have passed through my computer unseen by me. ASCII may be inefficient, but computers are really fast, people are not. I don't have any measurements, but in making a browser, HTTP header parsing/writing was never near a performance issue. Bandwidth wise it also tiny compared to that image file you'd inevitably download for every page visit. And sometimes you don't have tools. Sometimes you don't want tools. Sometimes you want to use tools that work with text to analyse your problem. (You other arguments might still stand, though :) )
- FlyingAvatar 13y ago> HTTP header parsing/writing was never near a performance issue. Since header lengths are not limited, and a single TCP packet's payload is quite limited, long headers can cause very measurable latency difference. Additionally, while I agree the generation / parsing overhead is probably quite small, saving it for every HTTP request is still a boon. I'm also curious where you are reading raw HTTP from? For me it's primarily in two situations, reading a packet capture from WireShark, or in the browser's debugger. In both cases, the tool will end up translating the request for me.
- alayne 13y agoI guess a lot of people here never had to work with the X protocols like X.400 and X.500. When you need specialized tools for every protocol and encoding format, development is a real drag. Just because I use Charles or Wireshark doesn't mean that I only want to use those specialized tools. I have definitely been in situations where I'm doing something like running nc as a proxy and looking at raw HTTP. I wouldn't choose to throw that away and revert to the bad old days without a big win.
- BHSPitMonkey 13y agoShould your willingness or unwillingness to use a tool for these (rare) scenarios influence the design of the protocol in any substantial way?
- antitrust 13y ago> First, ASCII is inefficient. Why aren't we using binary file formats anymore? XML is more easily debugged when the machine tools don't work. It's also more accessible, more easily created and modified, and thus, more available to a wider range of people than just web design professionals.
- smsm42 13y agoASCII is inefficient, but nobody cares. For vast majority of Web users, which aren't working on scales of Google and Amazon, difference in performance couldn't even be measured. And for them easiness of use - low barrier of entry, basically all you need is a basic ASCII tool and you're ready to go - is vastly more important than completely immeasurable performance gains from using opaque protocols. Of course, you can say TCP/IP is still binary, and it is true. But TCP/IP tools are built in every OS in existence now, so they do not form a real entrance barrier. Would HTTP tools be in the same position? I'm not sure - most HTTP tools right now are not standard and do not cover even HTTP/1.1 completely, what reason is there to expect they'd cover the whole 2.0 protocol properly and be widely standard and available on the level tcp/ip tools are? Which means much higher barrier of entry.
- zanny 13y agoI'd figure the inefficiency cost of ascii vs binary http headers over yottabytes of packets every year would add up. It hurts your bandwidth, it wastes electricity on the wire, and it wastes processing power. An insignificant smidgen on average, but add it up and it would probably be substantial. That is always my stance on things - if one computer is going to run something, write it in python, make it bleed memory, just make it work. If it is going to run on a million, you have to consider the raw power waste of inefficient programming. If it is going to run on trillions of devices for decades, your choices are few in my mind.
- smsm42 13y agoYottabytes only come into play if everybody switches. But complexity of binary protocols would work against that. So probably only very large sites would implement it - and even for them, is parsing HTTP costs that much? >>> If it is going to run on trillions of devices for decades, your choices are few in my mind. The history suggests otherwise - majority of mass-produced software is not written with performance as an ultimate concern. You would find a lot of software written in languages like Python or Java, even though using C or assembly would probably produce better performance. But using C or assembly that software would probably never be produced because its complexity would be harder to manage. Of course, performance does matter - even writing in Python, you have to worry about performance. But here we effectively see an argument saying "since we have a lot of software in Python, if we switch it to C we'll have massive performance gains". I think it is a wrong line of argument, if we switch to C a lot of this software wouldn't be written. (Note it's not against Python or C - I use both and they both are great in their areas :) So I guess optimized protocol does have its uses for high-volume websites - but I am concerned its advantages would be offset by its complexity. The designation of it as HTTP/2.0 implies it is the next version of HTTP - but it's rather a rather different thing with different use case. I'd rather have it as a separate protocol for high-traffic websites.
- brokenparser 13y agoA binary protocol is bad for a couple of reasons, too. First, OSI layer 6 called and it wants its old job back. It sat around connecting layers 5 and 7 peacefully since the dawn of the ARPANET, all while the HTTPbis guys were passing around messages back and forth trying to obsolete it. Second, you can't type a binary protocol. Yet, you have to somehow make the server work without a client or vice versa (for the initial implementation). That's going to be a lot more difficult. With HTTP, you could literally hook up a teletype to the Internet, let it print incoming requests and type the response back to the user agent. I've done this occasionally on a terminal emulator for debugging purposes. Third, the Internet is big-endian while most common processors in use today are little-endian. This is going to haunt peoples lives forever because you have to continuously convert between the two and although the conversions are orthogonal, the methods aren't idempotent (as opposed to converting a string to ASCII, or a text buffer to DOS style line endings). You mention 32-bit identifiers as opposed to a string of digits. This is more error-prone than you think, two's complement isn't the only integer representation out there. Implementations written in C would have to deal with their underlying architecture as the standard allows for 3 different representations (so the compiler wouldn't help you out). Then there's signed and unsigned, either of which might not be available in the implementers programming language. You end up unpacking the identifier by hand, which may end up being slower than just looping through a string. ASCII is hardly an inefficient serialisation format. Fourth, any fixed-length records are going to be useless at some point in the future. Several versions later the fixed length records are going to either point to an extra set of records tailing the 2.0 records or will simply have a (designated) backwards compatible value for consumption by older peers. With HTTP, we can add a header anywhere in the request or response except for the very top. We can even shuffle them at will without adverse effects on peers. Fifth, it doesn't make sense to optimise a tiny fraction of the entire HTTP session. Any benefits are too small to be worthwhile and would therefore result in a net-negative to most implementers. Sixth, you can still make improvements to HTTP without moving to a binary protocol. Not sending the same headers on every single request isn't one of them. HTTP is essentially a stateless protocol and every request could be handled by a different server. You can architect clusters of servers routing incoming requests however you please and satisfy every one of them correctly and efficiently. For starters, you can replace any underlying protocol in the stack with a more cluster-friendly protocol in transit. Seventh, just because no one is using a given content type in HTTP (I think you were referring to multipart/related) doesn't mean the protocol used to transfer that content is bad. Heck, it's not even part of any HTTP standard.
- guelo 13y agoPlain-text formats have always been slower for things that are not plain text. But even 30 years ago, when computers were even slower, Unix designers decided plain text was still the way to go, because it was easier to debug and easier for humans to work with. No specialized tools required, no poring over hex dumps. HTML won over other document formats. JSON and XML won over other binary formats. Any coder can look at JSON and see what is being transferred, without the aid of anything but a text editor. Plain-text marshalling formats for binary data (e.g., base64) are still useful for pasting data into an email or adding ssh keys to authorized_keys with "cat >>". Tool support is not going to make SPDY any nicer.
- peterwwillis 13y agoThings have changed in 30 years. Unix designers didn't have the time or resources to write elaborate tools, nor the need for complicated software. Back then you would use telnet to browse Gopher or send your mail. Things are different now. I dare you to read a 3KB JSON file without a parser. Base64 was a hack for text-based protocols. Tool support will make it a lot nicer than no support.
- nickik 13y agoSimple protocol, makes it simpler to write tools, simpler tools are easier to change, upgread and its simpler to add features, faster development, faster improvment. Better live for programmer. Also sometimes you need something spcific and then you have to option to code it up yourself, or change a open source library quickly and efficently.
- zobzu 13y agoAnd reading such comments all I think is: "The history is doomed to repeat itself".
- sergiosgc 13y agoThe main "thing" that changed in 30 years is computational power, which is now several orders of magnitude greater. If 30 years ago computers sporting the power of timex watches spared the cycles for text protocol overhead, I fail to see the need to squeeze, in today's hardware, that last drop of performance. The advantages of text based protocol remain the same. The disadvantage is lessened by faster CPUs.
- kragen 13y ago> Fourth, fixed-length records are the wave of the future! 1960s-style arbitrary field size limitations: the wave of the future! No doubt any day now we'll reorganize the internet around shipping punched-card images around, too. We could call the project the "Because It's Time Network", or BITNET. > Of course, most people would argue that you don't save enough time or bandwidth or cpu cycles to make a difference, but it would make for easier-to-write parsers. Most people would argue that, yes; famously, Kernighan and Plauger did argue that in The Elements of Programming Style in 1974. By "easier-to-write parsers", you mean easier than dict(((key.lower(), value.lstrip()) for key, value in (line.split(':', 1) for line in header.split('\r\n'))))? Because I think that's going to be a pretty tough bar to fit under. (Yeah, I know you need another couple of lines of code if you're going to handle indented continuation lines, but you can get rid of those without returning your protocol design to the Summer of Love. You could also get rid of the .lower() and the .lstrip() while you're at it.) > Add to this that every header could have a 32-bit identifier (4 bytes) Padding out sub-byte-sized values to fill out fixed-width fields: the Intelligent Man's Approach to Saving Bandwidth! Or you could just use one-letter names in an ASCII protocol. > [Fixed-length fields] solves crazy problems like header injection and request integrity checking. Clearly we've never had parsing bugs in binary protocols full of fixed-width fields, now have we? Surely not bugs that produced security holes? Except maybe TCP, IP, and DNS. And X.400, and X.500, and X.509, and some of those were the fault of ASN.1 BER and DER, which are hardly fixed-width formats. And surely silently truncating a value to put it into a fixed-width field would never change its semantics, right? > as to a general question "why use binary when we can compress stuff", what is smaller: a string of bits, or a compressed string of digits? Well, let's see. How many files do I have here? $ find | wc 5946 17074 330147 Each one has a ctime, an mtime, an atime, and an inode number. The ctime is generally going to be the same as the mtime in this case, so we'll leave it out. I think they're technically 64-bit values in the current inode structure, but let's count them as 32 bits instead, since none of my files are from after 2038. The inode number is also 32 bits. So if we take these three 32-bit values per file, we have 5946×12 bytes, or 71352 bytes. And if we print them out as digits and compress them? $ find -printf %.10A@%.10T@%i | gzip -9c | wc -c 46278 But wait, that string of digits isn't parsable; it's just digits. So let's add delimiters. $ find -printf %.10A@\ %.10T@\ %i\\n | head -3 1373396230 1373338685 2359306 1372967497 1372967489 5246218 1365458166 1365458157 5248264 $ find -printf %.10A@\ %.10T@\ %i\\n | gzip -9c | wc -c 47506 So a compressed string of digits is a lot smaller in this case. But you could argue that that's just because my data is highly redundant, since most of the timestamps are going to be within the current few years, which is true. But then, most data is highly redundant. How bad can it get, in the worst case of representing uniformly distributed random 32-bit values as compressed strings of digits, with spaces between? It adds about 44% overhead: $ dd </dev/urandom bs=1024 count=1 | od -w1024 -l | tr -s ' ' ' ' | gzip -9c | wc -c 1+0 records in 1+0 records out 1024 bytes (1.0 kB) copied, 0.000622635 s, 1.6 MB/s 1460 > Little inefficiencies like this don't go away if you just hack onto the same old protocol forever. But you know what? They do go away if you tunnel over a deflate-compressed VPN, or if you increase the available bandwidth by a few percent. Maybe instead of trying to take us back to bug-prone 1960s designs you should be working on that.
- stiff 13y agowhat is smaller: a string of bits, or a compressed string of digits? What are you talking about? You could have had something on your mind, but this is a terrible terrible juxtaposition of words...
- rfugger 13y agoThe Art of Unix Programming by Eric Raymond makes an excellent argument for textual protocols in Chapter 5: http://www.catb.org/~esr/writings/taoup/html/textualitychapter.html http://www.catb.org/~esr/writings/taoup/html/textualitychapt...
- neuroscr 13y agoReal tools? I would argue that there are way more real tools for debugging ASCII-based protocols than there will ever be for binary-based one. ASCII-based protocol is highly compose-able by allows us to separate intent (contents of the message) from ever-evolving encodings (rebasing, encryption, compression, etc). In this world of fighting complexity, why are we not favoring the simple?
- zobzu 13y agoI strongly disagree with "protocols are observed with tools we don't need ASCII". That's pretty annoying to see this kind of thinking. The reason why everyone codes in JS and uses HTML, CSS is because its ASCII. Its easy to understand, hack, etc. Same reason python is so popular. Even Go, is pretty simple like that. Sure its languages vs protocols, but the reasoning is exactly the same. And in fact, the comparison works with protocols as well: SMTP, IMAP, HTTP, IRC are EXTREMELY easy to understand and code for. Binary protocols are a huge PITA to code for. The argument that you're going to use a lib or whatever tool just doesn't hold any water. You want to understand what exactly happens. Thats how everyone learns, etc. I could write my own SMTP, IRC clients when i was 10. I could understand it. It works. No way I could understand fully the documented binary protocols. I tried, and it was just too painful and not fun at all (hey, I was 10). I'm not certain the added performance of using a binary format and some of the other advantages are really good enough to make the world unable to understand whats going on anymore by just looking at it. Sure purely technically speaking, it sounds like "binary is the way to go" for pure performance. But if you think about it, making hacking around that stuff a niche thing is perhaps a much greater loss. Even the reliability of a binary protocol is VERY arguable. In fact I'll put a last comparison. Shell pipes and ASCII. Many tried to replace them with smart binary protocols, objects, etc. Its cool. Its more powerful. More efficient. At the end of the day tho, a quick hack with regular pipes transferring ascii is just easier to understand and we all use those - not the fancy binary objects.