3 ms·
Simplicity is such an overloaded concept. It can mean: reducing the complexity of interface, reducing the complexity of operations, reducing the understanding r
by sophacles 14y ago
Simplicity is such an overloaded concept. It can mean: reducing the complexity of interface, reducing the complexity of operations, reducing the understanding required for use, reducing work to do the unanticipated, keeping common metaphors (as opposed to introducing new ones), and on and on. Ugh.
For example, I deal with a lot of network code. I regularly have to make various softwares work with different protocols, sometimes in async systems, sometime in threaded systems, sometimes in simple blocking systems. Further, networking is an inherently layered space - (eg. the JSON app specific protocol sits on http sits on tcp, which sits on ip, which sits on ethernet, id like to bea able to say is encapsulated in rather than sits on, but that breaks at various layers :/). So anyway I get the order to implement support for FooProto in a few systems. Great, I've heard about FooProto and wanted to play a bit.
A bit of research turns up that there are a few major and complete implementations. One is the reference implementation and it is of course big, bloated and slow. It does everything but there is no cohesion to the API or whatever. That one is probably out.
So the others have mixed reviews but a few are very popular and "simple".
Option 1: exposes 4 methods, provides my code with a well designed data structure, and looks fantastic. Problem is, it does blocking I/O and isn't thread-safe. It will work for some situations, but not all the ones I have. Worse, the transport is pretty heavily tied to the socket code and various system calls. But it is very simple.
Option 2: optimized for non-blocking. Has lots of callback hooks. Still tied in with the socket/syscall layers, but not in a way that makes extraction impossible. But it would take a few hundred lines of wrapper to make it look synchronous. And forget thread safety.
Option 3: Really thread safe, decent api, layers split out, but in a weird metaphor that requires some odd wrapper code would really require some rework to translate, or would require reworking of the hosting code to deal with the metaphor.
So which of these is simpler? Arguably each of them. One solution of course would be to use each as needed. Then of course I have to track multiple APIs, there are integration and bug compatability concerns. Not really simple.
Odds are I'll end up choosing the reference implementation or make my own, just because in the small it is more complex, but in the large, it makes my life simpler - one code base to handle multiple situations. Also a simpler choice, albeit on a different access.
So my point is, simplicity is not just hard, but sometime incompatible with itself. Simple on one axis may complicate things on another, and in irreconcilable ways.
Aside: To anyone implementing protocols -- _Please_ keep your parsers, state machines and transports as loosely coupled as possible. I, and many people I know, will love you for it. Transports change, encapsulation is common, and networks seem to be mostly edge-case. Feel free to add a layer tying them all together in a nice common-use api, this is a good thing too, but keep in mind there are always other use cases. Keep in mind that you are always just feeding your parser a stream of bytes, and feeding your state machine a series of tokens. Some protocols have a few layers of this built in. Due to the nature of the beast tho, it always can be modeled that way.
- haberman 14y ago> Aside: To anyone implementing protocols -- _Please_ keep your parsers, state machines and transports as loosely coupled as possible. I, and many people I know, will love you for it. Ok, I have to curb my excitement here a bit, because what I'm about to tell you is that I'm solving exactly your problem with an obsessive focus on achieving exactly what you desire. Though in saying so I'll have to disagree with you slightly when you say: > Simple on one axis may complicate things on another, and in irreconcilable ways. Basically I'm striving for (and believe I am on the verge of achieving) a design that achieves the best possible simplicity for network parsers and serializers in all dimensions. If you're wondering what the catch is, the catch is that it's taken me 2.5 years and counting to refine the design to the point where it can achieve this. My library performs no I/O (it's buffers in, buffers out), is completely resumable (so you can suspend the state machine at any point and resume it later), is completely composable (so you can create pipelines that feed the output of one parser to the input of another), loosely coupled, small (<5k loc in the core library), easy to bind to high-level languages, minimal enough that it can run inside the Linux kernel, and FAST (can parse protocol buffers at almost 1.5GB/second). And it's open-source. Don't be misled by the fact that it's advertised as a Protocol Buffer library; that's how it started, but I'm working on generalizing it to be capable of arbitrary parsing and serializing. Though Protocol Buffers schemas are used to specify the structure of any structured data that a parser wants to produce or a serializer wants to consume. https://github.com/haberman/upb/wiki https://github.com/haberman/upb/wiki
- spc476 14y agoI did that when doing writing a DNS resolution library (https://github.com/spc476/SPCDNS https://github.com/spc476/SPCDNS). The encoding and decoding is one library, leaving the network layer primarily up to you (I provide a very simple network interface that's probably okay for very simple uses). It's an approach I now use for any protocols I work on.