3 ms·
It's not quite clear to me why you'd use this over something more established such as protobuf, thrift, flatbuffers, cap n proto etc.
by dtech 7mo ago
It's not quite clear to me why you'd use this over something more established such as protobuf, thrift, flatbuffers, cap n proto etc.
- maxmcd 7mo agoThose care about quickly sending compact messages over the network, but most of them do not create a sparse in-memory representation that you can read on the fly. Especially in javascript. This lib keeps the compact representation at runtime and lets you read it without putting all the entities on the heap. Cool!
- IshKebab 7mo agoAmazon Ion has some support for this - items are length-prefixed so you can skip over them easily. It falls down if you have e.g. an array of 1 million small items, because you still need to skip over 999999 items to get to the last one. It looks like RX adds some support for indexes to improve that. I was in this situation where we needed to sparsely read huge JSON files. In the end we just switched to SQLite which handles all that perfectly. I'd probably still use it over RX, even though there's a somewhat awkward impedance mismatch between SQL and structs.
- creationix 7mo agoI did seriously consider SQLite, but my existing datasets don't map easily to relational database tables. This is essentially no-sql for sqlite.
- creationix 7mo agoExactly. Low heap allocations when reading values is one of the main driving factors in this design!
- konart 7mo agoWhat if you are reading from a service which already have an established API? It's not like you can just tell them to move to protobuf.
- SV_BubbleTime 7mo agoWhat about CBOR that can retain JSON compatibility? If you are working with an end you don’t control, this “newer better” format isn’t in your cards either.
- creationix 7mo agoHow does CBOR retain JSON compatibility more than RX? RX can represent any value JSON can represent. It doesn't even lose key order like some random-access formats do. In fact, RX is closer to JSON than CBOR. Take decimals as an example: JSON numbers are arbitrary precision numbers written in decimal. This means it can technically represent any decimal number to full precision. CBOR stores numbers as binary floats which are appriximations of decimal numbers. This is why they needed to add Decimal Fractions (Tag 4) RX already stores as decimal base and decimal power of 10. So out of the box, it matches JSON