creationix

7 ms·

31.
by creationix 7mo ago
Yep. I did try binary formats first. I tried existing ones like CBOR, I tried making my own like Nibs. The text encoding is an operational concern, not a technical one. This is the same reason I've been advocating for JSONL at work.
32.
by creationix 7mo ago
Exactly. Low heap allocations when reading values is one of the main driving factors in this design!
33.
by creationix 7mo ago
Serialized just means encoded as a stream of bytes so that it can be transferred between systems. There are absolutely cases where you want to be able to query a value directly like a database instead of parsing the entire thing to memory
34.
by creationix 7mo ago
Thanks. I had this for older versions, but forgot to write it up again for the latest version. One old version that is meant to be more human readable/writable is jsonito https://github.com/creationix/jsonito I&#
35.
by creationix 7mo ago
oh, sorry about that. I forgot to include the description of the format with examples. I did add some small examples to the repo. https://github.com/creationix/rx/blob/main/samples/quest-log... The
36.
by creationix 7mo ago
Thanks for testing it out! Yes, the website could use some love to make everything more discoverable. The primary use case is not compression, it's just a nice side effect of the deduplication. This will never beat something like zst
37.
by creationix 7mo ago
It's a gradient. I did design several binary formats first, but for my use cases, this is actually better. There is nuance to various use cases. > None of the space savings/efficiency of binary For string heavy datasets, it&#
38.
by creationix 7mo ago
Exactly. My for use cases, this format is amazing. I have very few floats, but lots and lots of objects, arrays and strings with moderate levels of duplication and substring duplication. My data is produced in a build and then read in th
39.
by creationix 7mo ago
if you data is lots and lots of arrays of floats, this is likely not the format for you. Use float arrays. Also note it stores decimal in a very compact encoding (two varints for base and power of 10) That said, while this is a text format
40.
by creationix 7mo ago
Or in this case, just do `rx file.rx` It has jq like queries built in and supports inputs with either rx or json. Also if you prefer jq, you can do `rx file.rx | jq`
41.
by creationix 7mo ago
As with most things in engineering, it depends. There are real logistical costs to using binary formats. This format is almost compact as a binary format while still retaining all the nice qualities of being an ASCII friendly encoding (yo
42.
by creationix 7mo ago
- this encodes to ASCII text (unless your strings contain unicode themselves) - that means you can copy-paste it (good luck doing that with compressed JSON or CBOR or SQLite - there is a scale where JSON isn't human readable anymore.
43.
by creationix 7mo ago
True, the particular use case where this really shines is large datasets where typical usage is to read a tiny part of it. Also there is no reason you couldn't write an rx parser that creates normal mutable objects. It could even be
44.
RX – a new random-access JSON alternative(github.com)
146 points by creationix 7mo ago|108 comments
45.
by creationix 7mo ago
A new random-access JSON alternative from the creator of nvm.sh, luvit.io, and js-git.
46.
by creationix 4y ago
Yep, I've got lots of experience in this area. It works very well for read-only use cases and is interesting for read-write like tedit. Also it's amazing for build systems like the one built into tedit. I wish the webdev world
47.
by creationix 10y ago
Exactly, you don't know if that's true. From my personal experience talking to actual mozilla employees and reading blog posts by mozilla employees, that statement looks pretty true. Even LGBT employees supported him through the o
48.
by creationix 10y ago
Wait, you're yelling "huge false equivalence" and then compare supporting what was a popular political viewpoint (Even Obama and the majority of California voters supported it back when Eich made the donation) to being as obj
49.
by creationix 11y ago
Why do you constantly assume that he voted to take away rights? If you've been paying attention to what he did do in relation to same-sex couples you'd know what he has been working for years to get them equal rights. But throug
50.
by creationix 11y ago
"Many religions are going to have to change" Pause and think about this for a moment. If you believed that there really was a God in heaven who spoke with you and has revealed to you the purposes of live and the purpose of marriag
51.
by creationix 12y ago
I'll be running a workshop on April 8th in San Francisco if anyone wants to learn more about luvit in person. https://www.eventbrite.com/e/questing-with-luvit-a-libuv-asy...
52.
by creationix 12y ago
Thanks for posting this. If anyone has questions, I'll try to answer them here, on twitter or in the comments on the blog post.
53.
by creationix 12y ago
Actually dukluv is the other way. It's a re-implementation of of a node-like environment using duktape instead of V8. I think the original question here was to write a node addon that allowed you to run arbitrary code in a duktape sa
54.
by creationix 12y ago
And thank you! I've wanted this for years, but haven't yet gotten up the courage to actually tackle it. ES5.1 is big and ES6 is massive.
55.
by creationix 12y ago
A lot of these one-man projects aren't because nobody else in the world can work on them. It's usually because everyone else is busy working on something else. If I ended up having to maintain duktape I'm confident I could u
56.
by creationix 12y ago
http://dukluv.io/
57.
by creationix 12y ago
It's worlds smaller and easier to work with. Integrating duktape into my project was literally just including the C file. It added just a couple hundred KB of overhead. Compare that to SM or V8 that require massive custom build system
58.
by creationix 12y ago
It a couple orders of magnatude slower, but smaller than V8. I love the low overhead and have been using duktape for some time.
59.
by creationix 12y ago
It could be node compatible. I'm just not done. Personally though I prefer the libuv APIs over the node APIs.
60.
DukLuv – Evented I/O for DukTape(github.com)
14 points by creationix 12y ago|1 comments

More ›