5 ms·
Tny: A simple data serializer in C
- coherentpony 13y agoHave you considered const-correctness[1]? [1] https://en.wikipedia.org/wiki/Const-correctness https://en.wikipedia.org/wiki/Const-correctness
- kernelcurry 13y agothis is cool! i can't wait to play with it in one of my own applications! thanks
- deletes 13y agoI have no experience with Web programing, but i know C and low level stuff pretty well, would anyone mind to elaborate what does Tny do?
- deleted 13y ago[deleted]
- prutschman 13y agoFollowing the link yields a description of the project: "Tny is a simple library to serialize data in C. It can be seen as a kind of binary JSON but unlike BSON it also supports arrays as root elements."
- hellcow 13y agoJSON is a key/value data structure for JavaScript, but it's useful across languages due to the number of libraries. It typically looks like: { "hello": "world" } Tny converts this JSON into a (usually, but not always) smaller form that's faster to traverse and encode/decode. Tny's alternative, BSON, is being used by MongoDB [1], so that might be a good place to learn more. [1] http://www.mongodb.org/ http://www.mongodb.org/
- ChuckMcM 13y agoSo I wonder if the author read the XDR code. XDR, which is a lame acronym for eXternal Data Representation was what Sun's remote procedure call library used to encode (or 'marshall' in the vernacular) arbitrary data into something you could send over a network and reliably unmarshal on the other side and get back the same data structure. The positives (like tny) were that the code was very straightforward and easy to use. And one of the negatives were that it wasn't "self describing" (which JSON is to some extent). Of course the ultimate in binary self describing data is ASN.1 syntax :-) but lets not go there.
- huhtenberg 13y agoSerialization formats are dime a dozen. Cooking up your own variation of TLV (+ TV for fixed-size T's) is dead easy and can typically be done a spot by anyone without any prior experience. As such I doubt "the author read the XDR code", though perhaps you weren't wondering that at all, but rather wanted to mention XDR and ASN.1, in which case you should've really just done that. As it's written, it makes "the author" look like either an ignorant imbecile who doesn't know such basics as XDR or as someone who piggy-backed on XDR without acknowledging it.
- lsb 13y agoAlso interesting: MessagePack. It's more concise than BSON. http://msgpack.org http://msgpack.org
- stevelaz 13y agoGlad to see someone mention this! I've used msgpack in Lua apps and it's been great so far.
- codehero 13y agoI think it's interesting and I get that it's a simple data parser, but if I am using C I prefer performance over simplicity. The Tny structs and their string values are individually malloc'ed, worsening the performance, so allocation and deletion is a slow process. Of course he's not the only one to do this, yajl2 also makes liberal use of malloc and free when building its node tree. One good thing to say is at least his encoding format distinguishes floats from integers and accounts for NaN and Infinity values. A first step to improving this library would be to add a SAX style parser so I could build my own representation.
- Demiurge 13y agoHow much difference does sporadic malloc make these days? What is the loss in performance compared to the overhead of virtual memory, randomization and whatnot, assuming OS is also trying to optimize?
- codehero 13y agoHelp me get on the same wavelength as you. Maybe you can tell me what is "sporadic" about Tny's malloc usage? I don't understand your second question at all. Optimizing the usage of malloc and free is no different than optimizing the run time of any algorithm. The only difference is people believe they can just abstract allocation away and rely on somebody else's code to handle it.
- Demiurge 13y agoWhen you say "structs and their string values are individually malloc'ed, worsening the performance", I call this "sporadic". Maybe it's the wrong word, if I was to use a single word. My second question is, can you actually avoid relying on somebody else's code when you call malloc? Due to virtual memory being comprised of pages, address randomization, and other things, heap is not a continuous memory space in hardware ready to be used at the moment program runs. So, what is the cost/benefit of "individually called malloc" for each struct, as opposed to pre-allocating some space? I could be totally out of my depth, in which case ignore me ;)
- zenocon 13y agoI've had great success using protobuf-c https://code.google.com/p/protobuf-c/ https://code.google.com/p/protobuf-c/
- mpetrov 13y agoI also used Protocol Buffers in multiple projects and languages. One huge advantage was that I could serialize data in Objective-C on the iPad, stream it to a C++ server on a desktop, and then forward some of it to a Ruby plugin in a different application. The performance was great and the libraries in each language made it seamless to use the same data structures across the board.
- aidenn0 13y agothere seem to be a million things like this; what makes it better than bson or tnetstrings?
- vikas0380 13y agoanybody know any similar library or project for c++?
- akandiah 13y agoGoogle's Protocol Buffers? http://code.google.com/p/protobuf/ http://code.google.com/p/protobuf/
- gpsarakis 13y agoAre there any speed & size comparisons between Tny and JSON?
- huhtenberg 13y agoJSON is not a TLV format, so you can't skip over records of an unknown type without reading them in full. So JSON is generally slower.
- waffenklang 13y agodidnt read the whole comments; so sorry, if duplicate. After a short review of the add function for handling of complex structs i found a possible bug as they miss to perform a deep copy of objects containing references to other objects. they would only copy the value of the pointers of a struct like that: struct { obj* ptr; } and not the content of ptr.