3 ms·
> And let's not forget: JSON and XML have some significant advantages over most other serialization formats. They're ubiquitous. Practically every programming l
by rdtsc 11y ago
> And let's not forget: JSON and XML have some significant advantages over most other serialization formats. They're ubiquitous. Practically every programming language has libraries to parse and generate them.
One of the interesting things about Erlang is it has binary pattern matching. Matching any binary in it is just as trivial or even more trivial than parsing json in other langauges.
Here is what an IP packet parsing might look like:
<<4:4, HeaderLength:4, _DiffServ:8, _Length:16, _Identification:16,
_Flags:3, _FragOffset:13, _TTL:8, Protocol:8, _HeaderChecksum:16,
SrcAddr:32/bits, DstAddr:32/bits, OptionsAndData/bytes>>
It looks almost like casting a binary blob to a C struct, but you can do even crazier things like take Length and match it later in the pattern, if it specifies a length prefixed binary, for example.
> the difference in data usage would scarcely be measurable
It depends. See, Joe's background is from Ericsson. Their stuff controls access of 50% of world's smartphone <-> internet traffic. When people say "but who uses functional languages in the industry?" the answer is if functional languages won't work, chances are high, you won't see picture of cats on your smartphone.
But the idea is that mobile bandwith is still a precious resourse. Even if our CPU and hard drives are getting bigger. That is one shared resource that is expensive. Maybe if a 1M clients use JSON vs a binary protocol doesn't make a difference much for some startup, when we talk about billions of connected devices talking over radio waves, those things add up. Well, also, game servers use UDP and binary often because of latency. Facebook knows a thing or two about chatting with lots of users, so they use Flatbuffers https://code.facebook.com/posts/872547912839369/improving-facebook-s-performance-on-android-with-flatbuffers/ https://code.facebook.com/posts/872547912839369/improving-fa... that's binary as well.
- viperscape 11y agoI think it's also important to note that json isn't exactly something that can be streamed while parsed.
- viraptor 11y agoThere are multiple libraries doing just that. For example https://sites.google.com/site/gson/streaming https://sites.google.com/site/gson/streaming
- ferrari8608 11y agoFrom personal experience, I can tell you that is can. The i3 window manager in Linux comes with a status bar program called i3bar. It also includes a program called i3status which constantly feeds i3bar information to display via JSON through a standard pipeline. I wrote an i3status replacement that does exactly this. It sends JSON to STDOUT at a set time interval. i3bar takes the JSON as it comes through STDIN and changes the look of the bar as it's read.
- ggreer 11y agoWhile very informative and interesting, I don't think your comment really engages with my point. I never mentioned Erlang or functional programming. I'm just talking about data serialization. Namely, that sending JSON or XML over the air isn't "totally crazy". Again, the vast majority of mobile data isn't JSON or XML. If mobile bandwidth was a Californian drought, video would be alfalfa farming. JSON would be taking a long shower: maybe you feel a little bad, but it doesn't matter. All the short showers in the world can't make up for the huge volume of water (bandwidth) used by alfalfa (video). Obviously, if you have a billion users, trade-offs change and it can make sense to use esoteric or bespoke protocols. But even for Facebook, using JSON isn't crazy. It's just inefficient. And Facebook didn't switch to FlatBuffers for the bandwidth savings. Their primary goal was to improve loading times for their local cache. Had FlatBuffers required 10% more network traffic than JSON, they likely still would have made the switch.
- rdtsc 11y ago> I don't think your comment really engages with my point. I never mentioned Erlang or functional programming [...] Sorry. It was mainly a response to "JSON and XML are only 'wasteful' if you don't count developer time. Once you do, it suddenly makes a lot of sense to "waste" hardware resources. After all, programmers are expensive. Hardware is cheap." And pointing out that it really depends on what languages or type or programmers you use and what "hardware" is. In some languages binary parsing is really easy and just as cheap and elegant as using a json parser. And well, Joe is Erlang's father, so point out Erlang was obvious choice. As for hardware, bandwidth is not hardware. It really is "something else" -- a shared resource usually. And if there are enough devices using this even small changes start to make a difference. Think a bit like a nested loop -- a small optimization in the inner loop might have large visible benefits. Or say, a huge database with 10B records, choosing an 32bit int vs an 16bit one for a column will make a for a good difference in size. That is basically where Joe was coming from. Really valuing bandiwidth as a shared resource. (He also likes to talk about protocols as well quite a bit). > Again, the vast majority of mobile data isn't JSON or XML The important data is JSON though, the control message, the pages that load (and they are slow), stuff that stops, start, manages everything all that is JSON. > If mobile bandwidth was a Californian drought, video would be alfalfa farming Hmm, video is 50% of all traffic on smart phones. (http://www.ericsson.com/res/docs/2015/ericsson-mobility-report-june-2015.pdf http://www.ericsson.com/res/docs/2015/ericsson-mobility-repo... says 45% in 2014). Not that the rest is all JSON but, I wouldn't say it is all just "taking showers" type of equivalency, it is more like grapes, almonds, strawberries, and walnuts.
- rco8786 11y agoFwiw it seems like Fb moving away from Json had little to nothing to do with size of the data.
- deleted 11y ago[deleted]