17 ms·

Plain Text Protocols

194 points by tate 6y ago

rasengan 6y ago
IRC is also a simple plaintext protocol. It’s simplicity helped an entire generation of programmers to appear.
- stagas 6y ago
  IRC was the first protocol I implemented back in the days. It's so simple you don't even need a client, telnet is enough to get you chatting. I miss this directness.
icy 6y ago
While HTTP is plaintext alright, it's neither simple nor easy to parse. Probably wouldn't put it in the same group as statsd/InfluxDB line protocols which can be parsed in a few lines of code.
- je_bailey 6y ago
  I really don’t understand what you mean by this. I’ve never thought http was difficult. What is it that you find problematic?
  - nly 6y ago
    Legacy HTTP/1.1 suffers a few issues, see the current RFC errata: https://www.rfc-editor.org/errata_search.php?rfc=7230&rec_status=15&presentation=records https://www.rfc-editor.org/errata_search.php?rfc=7230&rec_st... There are issues particularly around how whitespace and obsolete line folding should be handled Various whitespace issues in node.js: https://github.com/nodejs/http-parser/issues?q=is%3Aissue+whitespace https://github.com/nodejs/http-parser/issues?q=is%3Aissue+wh... Spec clarification: https://github.com/httpwg/http-core/issues/53 https://github.com/httpwg/http-core/issues/53 Node.js's parser was at one point replacing all white space in headers with a single space character, even though until recently this was non-conformant (you were only supposed to do so with obs-fold). It did this so it didn't have to buffer characters (since http-parser is a streaming parser). It's not as trivial as a few string splits. Node.js's parser is ~2,500 lines of C code.
blainsmith 6y ago
Thanks for sharing this post.
hermanradtke 6y ago
"Plaintext" is ASCII binary that is overwhelmingly English. The reason people like plaintext is that we have the tooling to see bits of the protocol as it comes over the wire. If we had good tooling for other protocols, then the barrier to entry would be lower as well.
- gambler 6y ago
  ASCII has a built-in markup language and a processing control protocol that most people aren't even aware of and most tools out there don't support. This is significant. Look at the parts that are used and parts that aren't. What is the difference between them?
  - Robotbeat 6y ago
    I don’t know. Can you tell me? ;)
    - TeMPOraL 6y ago
      The bits that aren't used don't correspond to printable characters :).
    - tdeck 6y ago
      I assume the parent is referring to the various control characters like "START OF HEADING", "START OF TEXT", "RECORD SEPARATOR", etc... I haven't seen most of these used for their original control purpose but they date back a long way: https://ascii.cl/control-characters.htm https://ascii.cl/control-characters.htm
      - hakfoo 6y ago
        I've seen them in some vendor-specific data formats in the financial space. They seem to be from an era when the formatting models were either fixed-width fields, or a serial set of variable width fields delineated by FIELD SEPERATOR, GROUP SEPERATOR, etc. What both models lacked was a good way to handle optional/sparse fields. If you have a data structure with 40 sub-fields, a JSON, XML or YAML notation can encode "Just subfield 26" pretty efficiently, but the FIELD SEPERATOR model usually involves having dozens of empty fields to space the one you want, and a lot of delicacy if the field layout changes.
dexen 6y ago
HTTP/2 is a good example of how to handle textual-to-binary transition. The original HTTP/1 was textual (if a bit convoluted), and that helped it to become a lingua franca; helped cooperation and data interchange between applications proliferate; everybody was able to proverbially "scratch his itch". Gradually tooling grew up around it too. The HTTP/2 is binary, and also quite complex - however most of that is hidden behind the already established tooling, and from developer's perspective, the protocol is seen as "improved HTTP/1 with optional extras". The protocol appears textual for all developmental intents and purposes - because the transition was handled (mostly) smoothly in the tooling. The key APIs remained the same for quick, mundane use cases - and got extended for advanced use cases. There's a lesson in that having been a success. Unpopular opinion warning: contrast the success of HTTP/2 with the failure of IPv6 to maintain backward compatibility at the API level - which hampered its ability to be seamlessly employed in applications.
- gambler 6y ago
  >however most of that is hidden behind the already established tooling ...and so everyone without Google-level funding is stuck with this tooling. Thus, control over the protocol that millions of people could use to communicate with one another directly (by making websites and possibly servers) is ceded to a handful of centralized authorities that can handle the complexity and also happen to benefit from the new features. I remember how Node when it was just rising in popularity was usually demonstrated by writing a primitive HTTP server that served a "hello world" HTTP page. There were no special libraries involved, so it was super-easy to understand what's going on. We're moving away from able to do things of this sort without special tooling and almost no one seems to notice or care.
  - espadrine 6y ago
    > I remember how Node when it was just rising in popularity was usually demonstrated by writing a primitive HTTP server that served a "hello world" HTTP page. That is still possible in the exact same way. But a toy is just a toy. All websites should encrypt their content with TLS. In fact, all protocols should encrypt their communications. The result? Sure, it is a binary stream of random-looking bits. Yet to me, what matters about text protocols is not the ASCII encoding. It is the ability to read and edit the raw representation. As long as your protocol has an unambiguous one-to-one textual representation with two-way conversion, I can inspect it and modify it with no headache. An outstanding example of that is WASM, which converts to and from WAT: https://en.wikipedia.org/wiki/WebAssembly#Code_representation https://en.wikipedia.org/wiki/WebAssembly#Code_representatio...
jacques_chester 6y ago
Plaintext protocols are textbook case of a negative externality. Programmers who work on them capture value, but impose higher costs (vs a well-suited binary protocol) for parsing, serialising, transmission, storage, computation etc onto everyone else.
- andrewflnr 6y ago
  Interesting perspective, but considering how much everyone seems to want software, I think we have to say that the commons also captures a lot of the same value from plain text that the programmers do. That might work out to less of a negative externality than just a trade-off for everyone, especially when you consider the positive effects on the commons of the network effects that text makes easier.
  - jacques_chester 6y ago
    Interesting counterpoint. It would come down to how things net out. And it won't be stable over time. I'm still of the view that programmers systematically overvalue their convenience, because it's a value/cost tradeoff that they directly experience.
    - andrewflnr 6y ago
      Fair. I do think the long term goal should be a compact binary format with equal or better tooling as plain text. Goodness knows there are enough formats and structural editors out there, so in principle we only need to standardize on one, but it seems none of them are actually quite good enough yet.
      - jacques_chester 6y ago
        I am generally of the view that Avro is Good Enough for most things that plaintext is used for and is pretty well-supported. Arrow looks very promising for cases where fast raw data shipment is the goal.
jpalomaki 6y ago
These look easy, but I don’t think they always are. Think about CSV for example. Looks simple to create and parse. In reality these simple implementations will give you lot of headache when they don’t handle edge cases (escaping, linefeeds etc).
- u801e 6y ago
  If they only used the ASCII record separator instead of a comma.
  - edflsafoiewq 6y ago
    Then you couldn't type it in a text editor.
    - u801e 6y ago
      I guess it depends on a text editor. Mapping a key to insert that character is one possible solution.
    - jeffbee 6y ago
      Standard unix-style input accepts ctrl-^ as ASCII RS and ctrl-_ as ASCII US. If you want your terminal to accept an ASCII US literally — so that you can use it as the -t argument to sort, for example — you would use ctrl-v ctrl-_ to give it the literal character. $ hd ^^^_ 00000000 1e 1f 0a 00000003 $ sort -t "^_" -k 2,2n a^_42 z^_5 n^_7 z5 n7 a42
  - leejoramo 6y ago
    I always preferred tabs to CVS, but how I wish our industry had made use of the ASCII record separator character. How many hours would I and my teams have saved in the last 20 years?
  - SahAssar 6y ago
    You would still need to be able to handle escaping, right? Otherwise you couldn't have a string with a record separator within a CSV column.
kstenerud 6y ago
Plain text protocols are: - human readable - good for quick protyping - good for inspection while debugging They are also: - complicated and slow to parse - more bloated than binary We benefited from text protocols because we had plenty of headroom: memory was cheap enough, storage was cheap enough, network was cheap enough, power was cheap enough for our use cases. But that's not quite so true anymore when you have to scale to support many connected systems and handle lots of data. The honeymoon's almost over. These are some of the reasons why I'm building https://concise-encoding.org https://concise-encoding.org
- slezyr 6y ago
  > They are also: - Get funny when you need to pass binary data.
  - bombcar 6y ago
    They also all eventually encode control data as text, which then causes errors with parsing some data that coincidentally has those same control characters in it. Just look at the garble URLs you see sometimes, more percent signs than a Macy's sale.
- bullen 6y ago
  - complicated and slow to parse - more bloated than binary Not neccessarily, you can write fast compact protocols with text... sending integer and float data as text is not the bottleneck in any system: See my root comment!
  - kstenerud 6y ago
    Floating point numbers are INCREDIBLY complicated to scan (and print). https://code.woboq.org/userspace/glibc/stdio-common/vfscanf-internal.c.html#1903 https://code.woboq.org/userspace/glibc/stdio-common/vfscanf-... Compare that with reading 4 bytes directly into an ieee754 float32. If your messages are short, the benefits of fast codecs are outweighed by the inefficiencies in the communication system (most of your time, processing power and bandwidth are taken up by setting up and tearing down the communication medium). If it takes 7 "administration" packets to send 1 data packet, your codec won't be your bottleneck (in which case you probably don't care about efficiency anyway, and this discussion is not for you).
u801e 6y ago
POP, IMAP and NNTP are also plain text protocols. What's interesting about SMTP as well as NNTP is that the data phase in the former as well as the post phase in the latter allow for all ascii characters to be transmitted as is without any issue other than NUL. The period needs to be escaped in certain cases such that a CRLF.CRLF doesn't prematurely end the data phase or article post. Clients actually emply "dot-stuffing" to address that case, meaning that any line that starts with a period is modified such that it starts with 2 periods before being transmitted to the server. When a client receives the email or article, it will remove the extra period so that the lines start with a single period.
Pinus 6y ago
The obvious risk with plain-text protocols is that you don't write a rigorous spec, and don't write a strict parser, but some hack up least-effort thing with a few string.split() and whatever. This means there is a lot of slack in what is actually accepted, and unless you are in full control of both ends of the protocol, that slack will be taken advantage of, and unless you are more powerful than whoever is at the other end (which you aren't, if they're your clients and you are not Google or Facebook), you have to support it forever. So write plain-text protocols if you like, but make sure to have a rigorous spec, and a parser with the persicketyness of... I don't know.
- rini17 6y ago
  That works until next time someone considers the strictness of the existing protocol too overwhelming or complicated and invents a new "simpler" one.
  - GoblinSlayer 6y ago
    TCP is strict. If you want something rich, you just run another protocol on top of the strict one. There can be many layers: TCP -> TLS -> HTTP -> JSON -> Base64 -> AES -> PNG -> Deflate -> Picture.
- mumblemumble 6y ago
  I'm thinking here of "standards" like csv and mbox that are almost impossible to handle with 100% reliability if you don't control all the programs that are producing them. It can get even worse with some niche products. I used to work with a piece of legal software that defined its own text format, and had a nasty habit of exporting files that it couldn't import. There was a defined spec, but it was riddled with ambiguities. I'm coming to think that, when it comes to text formats, it's LL(k) | GTFO.
  - deleted 6y ago
    [deleted]
- wtetzner 6y ago
superkuh 6y ago
Plain text protocols are in serious danger as the (confused) desire for TLS-only everywhere spreads with the best intentions. The problem is that the security TLS-only brings to protocols like HTTP(s) also brings with it massive centralization in cert authorities which provide single points of technical, social, and political failure. If the TLS-everywhere people succeed in their misguided cargo-cult efforts to kill of HTTP and other plain text connections everywhere, if browsers make HTTPS only default, then the web will lose even more of it's distributed nature. But it's not only the web (HTTP) that is under attack from the centralization of TLS, even SMTP with SMTPS might fall to it eventually. Right now you can self sign on your mailserver and it works just fine. But I imagine "privacy" advocates will want to exchange distributed security of centralized privacy there soon too. TLS is great. I love it and it has real, important uses. But TLS only is terrible for the internet. We need plain text protocols too. HTTP+HTTPS for life.
- minitoar 6y ago
  I sort of thought TLS everywhere was more about encryption than authentication.
  - bombcar 6y ago
    If it was only authentication then they'd be perfectly fine with unsigned certs. But they're not.
- jl6 6y ago
  Gemini uses TLS and it is common practice for Gemini clients to use self-signed certificates and TOFU. No dependency on centralized CAs.
  - elric 6y ago
    TOFU seems to work pretty well for SSH. AFAIK not many people actively verify host fingerprints on first use. It doesn't protect against MITM attacks on the first connection, but I wonder if that's not a case of better being the enemy of good to some extent?
bullen 6y ago
HTTP is unbeatable when you remove optional headers, not because of bandwidth; but because there are robust servers that can multi-thread joint-memory access with non-blocking IO and atomic concurrency. I use comet-stream for real-time 3D Action MMO data, so I have my own text based protocol wrapped in 2x sockets HTTP: F.ex. message = "move|<session>|<x>,<y>,<z>|<x>,<y>,<z>,<w>|walk": ### push (client -> server): "GET /push?data=" + message + " HTTP/1.1\r\nHost: my.host.name", then you get back "200 OK\r\nContent-Length: <length>\r\n\r\n<content>". In this case "0\r\n\r\n". ### pull (server -> client): Just one request pulls infinite response chunks: "GET /pull HTTP/1.1\r\nHost: my.host.name\r\nAccept: text/event-stream" then you get back "200 OK\r\nTransfer-Encoding: chunked\r\n\r\n". while(true) { <hex_length> + "\r\n" "data:" + message + "\n\n\r\n\r\n" } Simple and efficient! Text- + Event- based protocols over HTTP way outscale Binary- + Tick- based ones for compressed (as in averaged, not zipped) real-time data.
mattlondon 6y ago
Don't forget Gopher! Orignal: https://tools.ietf.org/html/rfc1436 https://tools.ietf.org/html/rfc1436 Gopher+: https://github.com/gopher-protocol/gopher-plus https://github.com/gopher-protocol/gopher-plus I feel like there is a lot of potential to "rejuvenate" Gopher somewhat in today's internet. No javascript, no cookies, no ads, no auto-play videos etc. There are some nice new modern GUIs like https://gophie.org https://gophie.org that are cross platform and modern. Fun fact: Redis (yes - that Redis) just added Gopher protocol support (https://redis.io/topics/gopher https://redis.io/topics/gopher)
- jl6 6y ago
  I hope you are aware of Gemini which aims to do exactly that?
- hliyan 6y ago
  I sometimes wonder if the web would have been a better place had CSS and everything but the most basic HTML tags didn't exist.
  - berniemadoff69 6y ago
    yes
- tdeck 6y ago
  But also no hyperlinks, no mixing text with images, and no unicode support in the menus.
  - mattlondon 6y ago
    I think there is probably a lot of mileage that can be gained serving markdown over gopher with embedded gopher:// hyperlinks and images and utf8 and everything else markdown supports via gopher itself. Gopher0 already has sort-of support for HTML file types so this would not be such a wild divergence from the original design. Not serving HTML provides some basic guarantees (no JavaScript, no tracking pixelsnetc) Gopher+ allows for quite flexible (albeit clunky) attributes for menu items so I can imagine an attribute on a menu directing compatible browsers to the markdown version of the menu, but allowing old clients to just view the traditional menu. This kinda relegates the gopher menu to a sort of old school directory listing type things we used to see in HTTP, but there is room for some fanciness via gopher+ to style menus themselves if browsers support that too! All of this is possible in Gopher+ if clients support it (...and there is an appetite for it). Perhaps we need some sort of "agreement"/Python-PIP-style thing to define sets of common Gopher+ attributes for all of this sort of thing.
guerrilla 6y ago
I think the author would like TreeNotation [1] https://faq.treenotation.org/ https://faq.treenotation.org/
andrewmcwatters 6y ago
What's sort of interesting is that there aren't too many overwhelming reasons why someone couldn't come up with a piece of software that autodetected a binary format and translated it to something readable in a GUI. I mean we know what the binary layout is of things, so I never understood (outside of the time that it would take to build such a utility) why I've never been able to find something that says, "Oh yeah, that binary string contains three doubles, separated by a NULL character, with an int, followed by a UTF-8 compatible string." Such a tool would be incredibly useful for reverse engineering proprietary formats, and yet I don't know of a good one, so if it exists it's at least obscure enough for it to have escaped my knowledge for well over a decade.
- nograpes 6y ago
  There is a command-line program called "file" that attempts to determine the file type (format). It uses a series of known formats and returns the first matching one. I have found it useful to reverse engineer proprietary formats.
  - andrewmcwatters 6y ago
    Yeah, but that's for known formats. If I said I have a buffer of 512 bytes and piped it through to some cli, that would be fine if it could tell me how many ints, chars, floats, doubles, compressed bits of data, CRC32s, UTF-8 strings, etc. it contained, but there's few utilities out there that will do that.
    - magmastonealex 6y ago
      I'm curious how you'd propose doing that. If I give you a buffer of 5 bytes: [0x68 0x65 0x6c 0x6c 0x6f] there are a ton of ways to interpret that. - The ascii string "hello" - 5 single-byte integers - 2 two-byte integers and 0x6c as a delimiter - 1 four-byte integer and ending in the char "o" - 1 32-bit float, and one single-byte integer etc. Or are you hoping for something that will provide you with all the possible combinations? That would produce pages of output for any decently-sized binary blob.
nly 6y ago
Plain text protocols, unless they can be expressed in, say, 3 grammar rules are almost always more pain than they're worth. These days just go and use a flexible binary option like protobufs, flatbuffers, avro, etc.
evmar 6y ago
My favorite plain text protocol is HTML server sent events, within HTTP. It's really trivial to make a server produce these -- it's just some simple newline-delimited printfs() to the socket -- and they manifest client-side as simple event objects. https://html.spec.whatwg.org/multipage/server-sent-events.html#parsing-an-event-stream https://html.spec.whatwg.org/multipage/server-sent-events.ht...
- jeffbee 6y ago
  Don't you have to deal with the problem of having a double newline in the field? Any time the value has a newline you have to restart with the field name, colon, and space. So it's not quite trivial to produce.
- jacques_chester 6y ago
  They are simple, but it's very slow, in my experience.
  - remram 6y ago
    Slower than websockets or WebRTC?
    - jacques_chester 6y ago
      The usage I've seen is to send log lines. In that case it's a lot slower than bundling stuff up into single requests/responses over HTTP.
      - remram 6y ago
        You mean, using server-sent events for things that are not events but data, one row at a time? Of course that'd be slow, but that's equivalent to sending one websocket frame for each row, or one file per row over HTTP multipart. Hardly talks to the speed of either protocol.
        6y ago
unnouinceput 6y ago
None mention JSON? Alright, then I'll do it.
zby 6y ago
OK - but what is Plain Text? Is it ASCII or maybe should it also include UTF8 or other Unicode encodings? What is the difference between the bits that form HTTP/1 and the bits that form a binary protocol like HTTP/2?
InfiniteRand 6y ago
It would be good to have a standard checklist of edge cases to handle with plaintext protocol design. Anyone know of one? I'm thinking along the lines of: 1. Control characters 2. Whitespace normalization 3. Newline normalization 4. Options for compression 5. Escaping characters significant to the protocol 6. Encoding characters outside of the normal character range 7. Dealing with ambiguous characters (not really an issue with strict ASCII) 8. Internationalization (which is intertwined with the previous items) 9. Dealing with invalid characters I'm not saying plain text doesn't have its advantages, I'm just saying there are issues you need to consider.
- chubot 6y ago
  Some comments on those issues here: https://www.arp242.net/the-art-of-unix-programming/#id2907428 https://www.arp242.net/the-art-of-unix-programming/#id290742... https://lobste.rs/s/rzhxyk/plain_text_protocols#c_5vx4ez https://lobste.rs/s/rzhxyk/plain_text_protocols#c_5vx4ez
- deleted 6y ago
  [deleted]
psim1 6y ago
SIP and its cousin SDP are wonderfully readable plaintext protocols used for VoIP. If you don’t think so, have a look at SIP’s predecessor H.323.
leastsquare 6y ago
I also wonder if letsencrypt will be dropped at some point so the SSL mafia can start squeezing everyone who has been shamed into using SSL.
IshKebab 6y ago
Please avoid plain text protocols! > * Simple to implement. > * Fast to parse. > * Human readable. I'll give him one out of three. They are human readable. The are clearly not fast to parse - binary protocols are obviously faster. The biggest problem is "simple to implement". That's just nonsense. Using text introduces so many ambiguities and tricky edge cases around quoting, whitespace, case, line endings, character encodings, escaping, etc. etc. etc. Soooo much more complicated than a simple binary system (e.g. Protobuf). There was a link here only recently about how Go might introduce security issues by lowercasing HTTP headers. Would a binary protocol have had that issue? I seriously doubt it. Don't use plain text protocols!!
- friend-monoid 6y ago
  I’ve written hundreds of parsers at this point, for many kinds of protocols - http, mqtt, canbus, midi, various strange binary stuff, cli’s, just lots and lots of parsing. I’ve found that text protocols are usually easier to parse, depending on the situation. Some binary protocols are easy because you can just cast the read buffer to a struct, but even that can fail when doing a partial read (which is the norm more then the exception in embedded spaces). And even in the best of cases you usually need to do a lot of post-processing to make sure that the data you read is correct. In the worst of cases you’re bitshifting and you have conditionals everywhere and you’re just not going to have a fun time. Canbus and midi are a lot easier to write parsers for then e.g. protobuf and mqtt. For text, stuff like http, redis, and those types of protocols are easy, but the text parsing world has json, which is the absolute worst of pains.
  - IshKebab 6y ago
    Canbus is binary isn't it? And Protobuf is very simple. In any case the problem is that it's easy to write a parser for text data that works for the data you have. It's extremely difficult to write a parser that exactly matches the spec and gets all the quoting/charset/etc. stuff right.
    - friend-monoid 6y ago
AceJohnny2 6y ago
Tangentially, I'm a bit surprised that we've completely dropped the ASCII separator control characters: 28/0x1C FS (File Separator), 29/0x1D GS (Group Separator), 30/0x1E RS (Record Separator), 21/0x1F US (Unit Separator). It's a pity, because I usually see Space (32/0x20) as the record separator, which I suppose is convenient because it works well in a standard text editor, but it does mean we've built up decades of habit/trauma-avoidance about avoiding spaces in names, replacing them with underscores (_) or dashes (-)...
- AceJohnny2 6y ago
  BTW, at least in a Unix terminal you can input the separator characters using Ctl with another char (because terminals inherit from the time when modifier keys just set/unset bits in the input), so: - 28/0x1C FS = Ctl-\ - 29/0x1D GS = Ctl-] - 30/0x1E RS = Ctl-^ - 31/0x1F US = Ctrl-_
jchw 6y ago
Honestly, I dislike plaintext formats a lot. It is more accessible because it’s human readable. But, this only extends to humans who happen to speak the language the protocol uses for keywords. While it’s not a huge ask, I still suggest this is mostly not that interesting of a benefit. Parsing and emitting plaintext formats, meanwhile, is a rabbit hole. It’s human readable which makes you tempted to make it human writable. Should you accept extraneous whitespace? Tabs vs spaces? Terminating new line? Unix or DOS line endings? Case sensitivity? Unicode normalized forms? Etc. Binary data may seem less accessible, but I blame the libraries. There’s tons of easy (if inefficient) ways to parse text. You can use string.split, atoi and scanf in your language of choice. What is there for binary? In Go, the encoding/binary package actually implements something really cool. A simple reflection-based mechanism that can read and write binary data into a structure in a defined and simple way. lunixbochs extended this to struc[1], which adds additional tags for advanced reading and writing of binary structures, including variable length structures. I went further and maybe a bit off into the deep end with Restruct[2], a similar concept but with a lot more features, designed specifically so I could handle advanced structures quickly. The end result is that I can define some Go structs with integers, strings, byte arrays and corresponding tags, and be able to serialize and deserialize from those structures to their corresponding binary representation. For an overdone demo of what you could do with Restruct for example, see this (incomplete) PNG demo: https://github.com/go-restruct/restruct/blob/master/formats/png/png.go https://github.com/go-restruct/restruct/blob/master/formats/... (It is mainly incomplete because I had moved focus to develop a codegen for restruct, to improve runtime performance, although such work has since stalled.) Most of my inspiration for newer advanced features like expressions comes from Kaitai Struct[3], an excellent project I have also contributed to a bit. I learned about Kaitai after having written Restruct thanks to a friend, though its lack of write capabilities has lead me to continue using Restruct for many hacks, such as some tools I wrote that modified FL Studio Project files. [1]: https://pkg.go.dev/github.com/lunixbochs/struc https://pkg.go.dev/github.com/lunixbochs/struc [2]: https://pkg.go.dev/github.com/go-restruct/restruct https://pkg.go.dev/github.com/go-restruct/restruct [3]: https://kaitai.io https://kaitai.io
LinuxBender 6y ago
[Edit] Ignore me, I thought we were talking plain text vs encrypted, not binary. :-)
- remram 6y ago
  I'm not sure how the protocol being text rather than binary helps the MITM? Are you misunderstanding "plain text" to mean "unencrypted" rather than "not binary"?
  - LinuxBender 6y ago
    Oh yes, I guess I missed the point they were making. In that case, I don't have an opinion. Binary in most protocols has been a solved problem for a very long time.
deleted 6y ago
[deleted]
doggodaddo78 6y ago
STOMP POP3 SMTP NNTP
Animats 6y ago
This holds up until you need to escape something.
DorothyOns80 6y ago
Local Moms Need Аnonуmous SЕХ! Just mеssаge аnd аsk to fuск! - https://xlink.cc/sXE1m https://xlink.cc/sXE1m

194 points by tate 6y ago

rasengan 6y ago

IRC is also a simple plaintext protocol. It’s simplicity helped an entire generation of programmers to appear.

stagas 6y ago
IRC was the first protocol I implemented back in the days. It's so simple you don't even need a client, telnet is enough to get you chatting. I miss this directness.

icy 6y ago

While HTTP is plaintext alright, it's neither simple nor easy to parse. Probably wouldn't put it in the same group as statsd/InfluxDB line protocols which can be parsed in a few lines of code.

je_bailey 6y ago
I really don’t understand what you mean by this. I’ve never thought http was difficult. What is it that you find problematic?
- nly 6y ago
  Legacy HTTP/1.1 suffers a few issues, see the current RFC errata: https://www.rfc-editor.org/errata_search.php?rfc=7230&rec_status=15&presentation=records https://www.rfc-editor.org/errata_search.php?rfc=7230&rec_st... There are issues particularly around how whitespace and obsolete line folding should be handled Various whitespace issues in node.js: https://github.com/nodejs/http-parser/issues?q=is%3Aissue+whitespace https://github.com/nodejs/http-parser/issues?q=is%3Aissue+wh... Spec clarification: https://github.com/httpwg/http-core/issues/53 https://github.com/httpwg/http-core/issues/53 Node.js's parser was at one point replacing all white space in headers with a single space character, even though until recently this was non-conformant (you were only supposed to do so with obs-fold). It did this so it didn't have to buffer characters (since http-parser is a streaming parser). It's not as trivial as a few string splits. Node.js's parser is ~2,500 lines of C code.

blainsmith 6y ago

Thanks for sharing this post.

hermanradtke 6y ago

"Plaintext" is ASCII binary that is overwhelmingly English. The reason people like plaintext is that we have the tooling to see bits of the protocol as it comes over the wire. If we had good tooling for other protocols, then the barrier to entry would be lower as well.

gambler 6y ago
ASCII has a built-in markup language and a processing control protocol that most people aren't even aware of and most tools out there don't support. This is significant. Look at the parts that are used and parts that aren't. What is the difference between them?
- Robotbeat 6y ago
  I don’t know. Can you tell me? ;)
  - TeMPOraL 6y ago
    The bits that aren't used don't correspond to printable characters :).
  - tdeck 6y ago
    I assume the parent is referring to the various control characters like "START OF HEADING", "START OF TEXT", "RECORD SEPARATOR", etc... I haven't seen most of these used for their original control purpose but they date back a long way: https://ascii.cl/control-characters.htm https://ascii.cl/control-characters.htm
    - hakfoo 6y ago
      I've seen them in some vendor-specific data formats in the financial space. They seem to be from an era when the formatting models were either fixed-width fields, or a serial set of variable width fields delineated by FIELD SEPERATOR, GROUP SEPERATOR, etc. What both models lacked was a good way to handle optional/sparse fields. If you have a data structure with 40 sub-fields, a JSON, XML or YAML notation can encode "Just subfield 26" pretty efficiently, but the FIELD SEPERATOR model usually involves having dozens of empty fields to space the one you want, and a lot of delicacy if the field layout changes.

dexen 6y ago

HTTP/2 is a good example of how to handle textual-to-binary transition. The original HTTP/1 was textual (if a bit convoluted), and that helped it to become a lingua franca; helped cooperation and data interchange between applications proliferate; everybody was able to proverbially "scratch his itch". Gradually tooling grew up around it too. The HTTP/2 is binary, and also quite complex - however most of that is hidden behind the already established tooling, and from developer's perspective, the protocol is seen as "improved HTTP/1 with optional extras". The protocol appears textual for all developmental intents and purposes - because the transition was handled (mostly) smoothly in the tooling. The key APIs remained the same for quick, mundane use cases - and got extended for advanced use cases. There's a lesson in that having been a success. Unpopular opinion warning: contrast the success of HTTP/2 with the failure of IPv6 to maintain backward compatibility at the API level - which hampered its ability to be seamlessly employed in applications.

gambler 6y ago
>however most of that is hidden behind the already established tooling ...and so everyone without Google-level funding is stuck with this tooling. Thus, control over the protocol that millions of people could use to communicate with one another directly (by making websites and possibly servers) is ceded to a handful of centralized authorities that can handle the complexity and also happen to benefit from the new features. I remember how Node when it was just rising in popularity was usually demonstrated by writing a primitive HTTP server that served a "hello world" HTTP page. There were no special libraries involved, so it was super-easy to understand what's going on. We're moving away from able to do things of this sort without special tooling and almost no one seems to notice or care.
- espadrine 6y ago
  > I remember how Node when it was just rising in popularity was usually demonstrated by writing a primitive HTTP server that served a "hello world" HTTP page. That is still possible in the exact same way. But a toy is just a toy. All websites should encrypt their content with TLS. In fact, all protocols should encrypt their communications. The result? Sure, it is a binary stream of random-looking bits. Yet to me, what matters about text protocols is not the ASCII encoding. It is the ability to read and edit the raw representation. As long as your protocol has an unambiguous one-to-one textual representation with two-way conversion, I can inspect it and modify it with no headache. An outstanding example of that is WASM, which converts to and from WAT: https://en.wikipedia.org/wiki/WebAssembly#Code_representation https://en.wikipedia.org/wiki/WebAssembly#Code_representatio...

jacques_chester 6y ago

Plaintext protocols are textbook case of a negative externality. Programmers who work on them capture value, but impose higher costs (vs a well-suited binary protocol) for parsing, serialising, transmission, storage, computation etc onto everyone else.

andrewflnr 6y ago
Interesting perspective, but considering how much everyone seems to want software, I think we have to say that the commons also captures a lot of the same value from plain text that the programmers do. That might work out to less of a negative externality than just a trade-off for everyone, especially when you consider the positive effects on the commons of the network effects that text makes easier.
- jacques_chester 6y ago
  Interesting counterpoint. It would come down to how things net out. And it won't be stable over time. I'm still of the view that programmers systematically overvalue their convenience, because it's a value/cost tradeoff that they directly experience.
  - andrewflnr 6y ago
    Fair. I do think the long term goal should be a compact binary format with equal or better tooling as plain text. Goodness knows there are enough formats and structural editors out there, so in principle we only need to standardize on one, but it seems none of them are actually quite good enough yet.
    - jacques_chester 6y ago
      I am generally of the view that Avro is Good Enough for most things that plaintext is used for and is pretty well-supported. Arrow looks very promising for cases where fast raw data shipment is the goal.

jpalomaki 6y ago

These look easy, but I don’t think they always are. Think about CSV for example. Looks simple to create and parse. In reality these simple implementations will give you lot of headache when they don’t handle edge cases (escaping, linefeeds etc).

u801e 6y ago
If they only used the ASCII record separator instead of a comma.
- edflsafoiewq 6y ago
  Then you couldn't type it in a text editor.
  - u801e 6y ago
    I guess it depends on a text editor. Mapping a key to insert that character is one possible solution.
  - jeffbee 6y ago
    Standard unix-style input accepts ctrl-^ as ASCII RS and ctrl-_ as ASCII US. If you want your terminal to accept an ASCII US literally — so that you can use it as the -t argument to sort, for example — you would use ctrl-v ctrl-_ to give it the literal character. $ hd ^^^_ 00000000 1e 1f 0a 00000003 $ sort -t "^_" -k 2,2n a^_42 z^_5 n^_7 z5 n7 a42
- leejoramo 6y ago
  I always preferred tabs to CVS, but how I wish our industry had made use of the ASCII record separator character. How many hours would I and my teams have saved in the last 20 years?
- SahAssar 6y ago
  You would still need to be able to handle escaping, right? Otherwise you couldn't have a string with a record separator within a CSV column.

kstenerud 6y ago

Plain text protocols are: - human readable - good for quick protyping - good for inspection while debugging They are also: - complicated and slow to parse - more bloated than binary We benefited from text protocols because we had plenty of headroom: memory was cheap enough, storage was cheap enough, network was cheap enough, power was cheap enough for our use cases. But that's not quite so true anymore when you have to scale to support many connected systems and handle lots of data. The honeymoon's almost over. These are some of the reasons why I'm building https://concise-encoding.org https://concise-encoding.org

slezyr 6y ago
> They are also: - Get funny when you need to pass binary data.
- bombcar 6y ago
  They also all eventually encode control data as text, which then causes errors with parsing some data that coincidentally has those same control characters in it. Just look at the garble URLs you see sometimes, more percent signs than a Macy's sale.
bullen 6y ago
- complicated and slow to parse - more bloated than binary Not neccessarily, you can write fast compact protocols with text... sending integer and float data as text is not the bottleneck in any system: See my root comment!
- kstenerud 6y ago
  Floating point numbers are INCREDIBLY complicated to scan (and print). https://code.woboq.org/userspace/glibc/stdio-common/vfscanf-internal.c.html#1903 https://code.woboq.org/userspace/glibc/stdio-common/vfscanf-... Compare that with reading 4 bytes directly into an ieee754 float32. If your messages are short, the benefits of fast codecs are outweighed by the inefficiencies in the communication system (most of your time, processing power and bandwidth are taken up by setting up and tearing down the communication medium). If it takes 7 "administration" packets to send 1 data packet, your codec won't be your bottleneck (in which case you probably don't care about efficiency anyway, and this discussion is not for you).

u801e 6y ago

POP, IMAP and NNTP are also plain text protocols. What's interesting about SMTP as well as NNTP is that the data phase in the former as well as the post phase in the latter allow for all ascii characters to be transmitted as is without any issue other than NUL. The period needs to be escaped in certain cases such that a CRLF.CRLF doesn't prematurely end the data phase or article post. Clients actually emply "dot-stuffing" to address that case, meaning that any line that starts with a period is modified such that it starts with 2 periods before being transmitted to the server. When a client receives the email or article, it will remove the extra period so that the lines start with a single period.

Pinus 6y ago

The obvious risk with plain-text protocols is that you don't write a rigorous spec, and don't write a strict parser, but some hack up least-effort thing with a few string.split() and whatever. This means there is a lot of slack in what is actually accepted, and unless you are in full control of both ends of the protocol, that slack will be taken advantage of, and unless you are more powerful than whoever is at the other end (which you aren't, if they're your clients and you are not Google or Facebook), you have to support it forever. So write plain-text protocols if you like, but make sure to have a rigorous spec, and a parser with the persicketyness of... I don't know.

rini17 6y ago
That works until next time someone considers the strictness of the existing protocol too overwhelming or complicated and invents a new "simpler" one.
- GoblinSlayer 6y ago
  TCP is strict. If you want something rich, you just run another protocol on top of the strict one. There can be many layers: TCP -> TLS -> HTTP -> JSON -> Base64 -> AES -> PNG -> Deflate -> Picture.
mumblemumble 6y ago
I'm thinking here of "standards" like csv and mbox that are almost impossible to handle with 100% reliability if you don't control all the programs that are producing them. It can get even worse with some niche products. I used to work with a piece of legal software that defined its own text format, and had a nasty habit of exporting files that it couldn't import. There was a defined spec, but it was riddled with ambiguities. I'm coming to think that, when it comes to text formats, it's LL(k) | GTFO.
- deleted 6y ago
  [deleted]
wtetzner 6y ago

superkuh 6y ago

Plain text protocols are in serious danger as the (confused) desire for TLS-only everywhere spreads with the best intentions. The problem is that the security TLS-only brings to protocols like HTTP(s) also brings with it massive centralization in cert authorities which provide single points of technical, social, and political failure. If the TLS-everywhere people succeed in their misguided cargo-cult efforts to kill of HTTP and other plain text connections everywhere, if browsers make HTTPS only default, then the web will lose even more of it's distributed nature. But it's not only the web (HTTP) that is under attack from the centralization of TLS, even SMTP with SMTPS might fall to it eventually. Right now you can self sign on your mailserver and it works just fine. But I imagine "privacy" advocates will want to exchange distributed security of centralized privacy there soon too. TLS is great. I love it and it has real, important uses. But TLS only is terrible for the internet. We need plain text protocols too. HTTP+HTTPS for life.

minitoar 6y ago
I sort of thought TLS everywhere was more about encryption than authentication.
- bombcar 6y ago
  If it was only authentication then they'd be perfectly fine with unsigned certs. But they're not.
jl6 6y ago
Gemini uses TLS and it is common practice for Gemini clients to use self-signed certificates and TOFU. No dependency on centralized CAs.
- elric 6y ago
  TOFU seems to work pretty well for SSH. AFAIK not many people actively verify host fingerprints on first use. It doesn't protect against MITM attacks on the first connection, but I wonder if that's not a case of better being the enemy of good to some extent?

bullen 6y ago

HTTP is unbeatable when you remove optional headers, not because of bandwidth; but because there are robust servers that can multi-thread joint-memory access with non-blocking IO and atomic concurrency. I use comet-stream for real-time 3D Action MMO data, so I have my own text based protocol wrapped in 2x sockets HTTP: F.ex. message = "move|<session>|<x>,<y>,<z>|<x>,<y>,<z>,<w>|walk": ### push (client -> server): "GET /push?data=" + message + " HTTP/1.1\r\nHost: my.host.name", then you get back "200 OK\r\nContent-Length: <length>\r\n\r\n<content>". In this case "0\r\n\r\n". ### pull (server -> client): Just one request pulls infinite response chunks: "GET /pull HTTP/1.1\r\nHost: my.host.name\r\nAccept: text/event-stream" then you get back "200 OK\r\nTransfer-Encoding: chunked\r\n\r\n". while(true) { <hex_length> + "\r\n" "data:" + message + "\n\n\r\n\r\n" } Simple and efficient! Text- + Event- based protocols over HTTP way outscale Binary- + Tick- based ones for compressed (as in averaged, not zipped) real-time data.

mattlondon 6y ago

Don't forget Gopher! Orignal: https://tools.ietf.org/html/rfc1436 https://tools.ietf.org/html/rfc1436 Gopher+: https://github.com/gopher-protocol/gopher-plus https://github.com/gopher-protocol/gopher-plus I feel like there is a lot of potential to "rejuvenate" Gopher somewhat in today's internet. No javascript, no cookies, no ads, no auto-play videos etc. There are some nice new modern GUIs like https://gophie.org https://gophie.org that are cross platform and modern. Fun fact: Redis (yes - that Redis) just added Gopher protocol support (https://redis.io/topics/gopher https://redis.io/topics/gopher)

jl6 6y ago
I hope you are aware of Gemini which aims to do exactly that?
hliyan 6y ago
I sometimes wonder if the web would have been a better place had CSS and everything but the most basic HTML tags didn't exist.
- berniemadoff69 6y ago
  yes
tdeck 6y ago
But also no hyperlinks, no mixing text with images, and no unicode support in the menus.
- mattlondon 6y ago
  I think there is probably a lot of mileage that can be gained serving markdown over gopher with embedded gopher:// hyperlinks and images and utf8 and everything else markdown supports via gopher itself. Gopher0 already has sort-of support for HTML file types so this would not be such a wild divergence from the original design. Not serving HTML provides some basic guarantees (no JavaScript, no tracking pixelsnetc) Gopher+ allows for quite flexible (albeit clunky) attributes for menu items so I can imagine an attribute on a menu directing compatible browsers to the markdown version of the menu, but allowing old clients to just view the traditional menu. This kinda relegates the gopher menu to a sort of old school directory listing type things we used to see in HTTP, but there is room for some fanciness via gopher+ to style menus themselves if browsers support that too! All of this is possible in Gopher+ if clients support it (...and there is an appetite for it). Perhaps we need some sort of "agreement"/Python-PIP-style thing to define sets of common Gopher+ attributes for all of this sort of thing.

guerrilla 6y ago

I think the author would like TreeNotation [1] https://faq.treenotation.org/ https://faq.treenotation.org/

andrewmcwatters 6y ago

What's sort of interesting is that there aren't too many overwhelming reasons why someone couldn't come up with a piece of software that autodetected a binary format and translated it to something readable in a GUI. I mean we know what the binary layout is of things, so I never understood (outside of the time that it would take to build such a utility) why I've never been able to find something that says, "Oh yeah, that binary string contains three doubles, separated by a NULL character, with an int, followed by a UTF-8 compatible string." Such a tool would be incredibly useful for reverse engineering proprietary formats, and yet I don't know of a good one, so if it exists it's at least obscure enough for it to have escaped my knowledge for well over a decade.

nograpes 6y ago
There is a command-line program called "file" that attempts to determine the file type (format). It uses a series of known formats and returns the first matching one. I have found it useful to reverse engineer proprietary formats.
- andrewmcwatters 6y ago
  Yeah, but that's for known formats. If I said I have a buffer of 512 bytes and piped it through to some cli, that would be fine if it could tell me how many ints, chars, floats, doubles, compressed bits of data, CRC32s, UTF-8 strings, etc. it contained, but there's few utilities out there that will do that.
  - magmastonealex 6y ago
    I'm curious how you'd propose doing that. If I give you a buffer of 5 bytes: [0x68 0x65 0x6c 0x6c 0x6f] there are a ton of ways to interpret that. - The ascii string "hello" - 5 single-byte integers - 2 two-byte integers and 0x6c as a delimiter - 1 four-byte integer and ending in the char "o" - 1 32-bit float, and one single-byte integer etc. Or are you hoping for something that will provide you with all the possible combinations? That would produce pages of output for any decently-sized binary blob.

nly 6y ago

Plain text protocols, unless they can be expressed in, say, 3 grammar rules are almost always more pain than they're worth. These days just go and use a flexible binary option like protobufs, flatbuffers, avro, etc.

evmar 6y ago

My favorite plain text protocol is HTML server sent events, within HTTP. It's really trivial to make a server produce these -- it's just some simple newline-delimited printfs() to the socket -- and they manifest client-side as simple event objects. https://html.spec.whatwg.org/multipage/server-sent-events.html#parsing-an-event-stream https://html.spec.whatwg.org/multipage/server-sent-events.ht...

jeffbee 6y ago
Don't you have to deal with the problem of having a double newline in the field? Any time the value has a newline you have to restart with the field name, colon, and space. So it's not quite trivial to produce.
jacques_chester 6y ago
They are simple, but it's very slow, in my experience.
- remram 6y ago
  Slower than websockets or WebRTC?
  - jacques_chester 6y ago
    The usage I've seen is to send log lines. In that case it's a lot slower than bundling stuff up into single requests/responses over HTTP.
    - remram 6y ago
      You mean, using server-sent events for things that are not events but data, one row at a time? Of course that'd be slow, but that's equivalent to sending one websocket frame for each row, or one file per row over HTTP multipart. Hardly talks to the speed of either protocol.
      - 6y ago

unnouinceput 6y ago

None mention JSON? Alright, then I'll do it.

zby 6y ago

OK - but what is Plain Text? Is it ASCII or maybe should it also include UTF8 or other Unicode encodings? What is the difference between the bits that form HTTP/1 and the bits that form a binary protocol like HTTP/2?

InfiniteRand 6y ago

It would be good to have a standard checklist of edge cases to handle with plaintext protocol design. Anyone know of one? I'm thinking along the lines of: 1. Control characters 2. Whitespace normalization 3. Newline normalization 4. Options for compression 5. Escaping characters significant to the protocol 6. Encoding characters outside of the normal character range 7. Dealing with ambiguous characters (not really an issue with strict ASCII) 8. Internationalization (which is intertwined with the previous items) 9. Dealing with invalid characters I'm not saying plain text doesn't have its advantages, I'm just saying there are issues you need to consider.

chubot 6y ago
Some comments on those issues here: https://www.arp242.net/the-art-of-unix-programming/#id2907428 https://www.arp242.net/the-art-of-unix-programming/#id290742... https://lobste.rs/s/rzhxyk/plain_text_protocols#c_5vx4ez https://lobste.rs/s/rzhxyk/plain_text_protocols#c_5vx4ez
deleted 6y ago
[deleted]

psim1 6y ago

SIP and its cousin SDP are wonderfully readable plaintext protocols used for VoIP. If you don’t think so, have a look at SIP’s predecessor H.323.

leastsquare 6y ago

I also wonder if letsencrypt will be dropped at some point so the SSL mafia can start squeezing everyone who has been shamed into using SSL.

IshKebab 6y ago

Please avoid plain text protocols! > * Simple to implement. > * Fast to parse. > * Human readable. I'll give him one out of three. They are human readable. The are clearly not fast to parse - binary protocols are obviously faster. The biggest problem is "simple to implement". That's just nonsense. Using text introduces so many ambiguities and tricky edge cases around quoting, whitespace, case, line endings, character encodings, escaping, etc. etc. etc. Soooo much more complicated than a simple binary system (e.g. Protobuf). There was a link here only recently about how Go might introduce security issues by lowercasing HTTP headers. Would a binary protocol have had that issue? I seriously doubt it. Don't use plain text protocols!!

friend-monoid 6y ago
I’ve written hundreds of parsers at this point, for many kinds of protocols - http, mqtt, canbus, midi, various strange binary stuff, cli’s, just lots and lots of parsing. I’ve found that text protocols are usually easier to parse, depending on the situation. Some binary protocols are easy because you can just cast the read buffer to a struct, but even that can fail when doing a partial read (which is the norm more then the exception in embedded spaces). And even in the best of cases you usually need to do a lot of post-processing to make sure that the data you read is correct. In the worst of cases you’re bitshifting and you have conditionals everywhere and you’re just not going to have a fun time. Canbus and midi are a lot easier to write parsers for then e.g. protobuf and mqtt. For text, stuff like http, redis, and those types of protocols are easy, but the text parsing world has json, which is the absolute worst of pains.
- IshKebab 6y ago
  Canbus is binary isn't it? And Protobuf is very simple. In any case the problem is that it's easy to write a parser for text data that works for the data you have. It's extremely difficult to write a parser that exactly matches the spec and gets all the quoting/charset/etc. stuff right.
  - friend-monoid 6y ago

AceJohnny2 6y ago

Tangentially, I'm a bit surprised that we've completely dropped the ASCII separator control characters: 28/0x1C FS (File Separator), 29/0x1D GS (Group Separator), 30/0x1E RS (Record Separator), 21/0x1F US (Unit Separator). It's a pity, because I usually see Space (32/0x20) as the record separator, which I suppose is convenient because it works well in a standard text editor, but it does mean we've built up decades of habit/trauma-avoidance about avoiding spaces in names, replacing them with underscores (_) or dashes (-)...

AceJohnny2 6y ago
BTW, at least in a Unix terminal you can input the separator characters using Ctl with another char (because terminals inherit from the time when modifier keys just set/unset bits in the input), so: - 28/0x1C FS = Ctl-\ - 29/0x1D GS = Ctl-] - 30/0x1E RS = Ctl-^ - 31/0x1F US = Ctrl-_

jchw 6y ago

Honestly, I dislike plaintext formats a lot. It is more accessible because it’s human readable. But, this only extends to humans who happen to speak the language the protocol uses for keywords. While it’s not a huge ask, I still suggest this is mostly not that interesting of a benefit. Parsing and emitting plaintext formats, meanwhile, is a rabbit hole. It’s human readable which makes you tempted to make it human writable. Should you accept extraneous whitespace? Tabs vs spaces? Terminating new line? Unix or DOS line endings? Case sensitivity? Unicode normalized forms? Etc. Binary data may seem less accessible, but I blame the libraries. There’s tons of easy (if inefficient) ways to parse text. You can use string.split, atoi and scanf in your language of choice. What is there for binary? In Go, the encoding/binary package actually implements something really cool. A simple reflection-based mechanism that can read and write binary data into a structure in a defined and simple way. lunixbochs extended this to struc[1], which adds additional tags for advanced reading and writing of binary structures, including variable length structures. I went further and maybe a bit off into the deep end with Restruct[2], a similar concept but with a lot more features, designed specifically so I could handle advanced structures quickly. The end result is that I can define some Go structs with integers, strings, byte arrays and corresponding tags, and be able to serialize and deserialize from those structures to their corresponding binary representation. For an overdone demo of what you could do with Restruct for example, see this (incomplete) PNG demo: https://github.com/go-restruct/restruct/blob/master/formats/png/png.go https://github.com/go-restruct/restruct/blob/master/formats/... (It is mainly incomplete because I had moved focus to develop a codegen for restruct, to improve runtime performance, although such work has since stalled.) Most of my inspiration for newer advanced features like expressions comes from Kaitai Struct[3], an excellent project I have also contributed to a bit. I learned about Kaitai after having written Restruct thanks to a friend, though its lack of write capabilities has lead me to continue using Restruct for many hacks, such as some tools I wrote that modified FL Studio Project files. [1]: https://pkg.go.dev/github.com/lunixbochs/struc https://pkg.go.dev/github.com/lunixbochs/struc [2]: https://pkg.go.dev/github.com/go-restruct/restruct https://pkg.go.dev/github.com/go-restruct/restruct [3]: https://kaitai.io https://kaitai.io

LinuxBender 6y ago

[Edit] Ignore me, I thought we were talking plain text vs encrypted, not binary. :-)

remram 6y ago
I'm not sure how the protocol being text rather than binary helps the MITM? Are you misunderstanding "plain text" to mean "unencrypted" rather than "not binary"?
- LinuxBender 6y ago
  Oh yes, I guess I missed the point they were making. In that case, I don't have an opinion. Binary in most protocols has been a solved problem for a very long time.

deleted 6y ago

[deleted]

doggodaddo78 6y ago

STOMP POP3 SMTP NNTP

Animats 6y ago

This holds up until you need to escape something.

DorothyOns80 6y ago

Local Moms Need Аnonуmous SЕХ! Just mеssаge аnd аsk to fuск! - https://xlink.cc/sXE1m https://xlink.cc/sXE1m