3 ms·
The compression soon mounts up in savings. It’s quicker to send over the network, disks cost money and on the cloud even low terrabyte sized data starts to moun
by tiew9Vii 7y ago
The compression soon mounts up in savings. It’s quicker to send over the network, disks cost money and on the cloud even low terrabyte sized data starts to mount up in cost.
JSON is unstructured. Every one who parses the data needs to write a parser. As it’s unstructured how do you manage schema changes?
Using a binary format like avro you get strong typing (basic validation), schema change modes, no need to write clients. You can publish your avro schemas to a registry and you then get a catalog of data available anyone can browse to see what data is available and how it looks.
> This just smells like tech debt
I’d say the opposite. Using JSON is tech debt. Everyone writing parsers/serialisers, not knowing what fields are available, what values the fields can accept, how to handle new fields / deprecating old fields.
Debugging is simple, AVRO can be inspected unlike over formats with its Generic record format that can also be represented and dumped as JSON.
people should use binary strongly typed formats more. It makes things so much easier for the developer,data evolution and data processing. You also get benefits that your application is no longer running in debug mode.