4 ms·
Looks very much like a format that should just have been gzipped JSON. Don't use binary formats when it isn't absolutely needed.
by silvestrov 8mo ago
Looks very much like a format that should just have been gzipped JSON.
Don't use binary formats when it isn't absolutely needed.
- mort96 8mo agoUh. You want to store assets in JSON? Why? You generally want asset packs to be seekable so that you can extract just one asset, and why would you want to add the overhead of parsing potentially gigabytes of JSON and then, per asset, decoding potentially tens of megabytes of base64?
- silvestrov 8mo agoKeep all the meta info in JSON and then the big binary files in a zip file. Much easier to parse.
- mort96 8mo agoEasier for the developer or easier for the computer? Computers need to do it a bunch for every program launch for every single user of macOS for decades. The developer just needed to write a generator and a parser for the format once. Would it have been a bit easier to write a parser for a format that's based around a zip file with a manifest.json? I don't know, maybe. You'd end up with some pretty huge dependencies (a zip file reader, a JSON parser), but maybe it'd be slightly easier. Is it worth it?
- embedding-shape 8mo ago> You want to store assets in JSON? Why? Why not have both options? .gltf and .glb being possible for assets been more than helpful to me more than once, having the option gives you the best of both worlds :)
- deleted 8mo ago[deleted]
- mort96 8mo agoWhat's Apple's incentive for having two different asset pack formats? It seems like more work to support parsing and generating both, and Apple expects you to use their tools to generate asset packs. Working with binary files really isn't that hard. If Apple documented the .car format, writing your own parser wouldn't be difficult. It's not like it's a complicated format. Still, Apple clearly doesn't intend for people to make their own .car generators; to them, ease of reverse engineering is a bug, not a feature.
- pjmlp 8mo agoOnly because they were originally designed for Web 3D.
- ape4 8mo agoSomeday JSON will be out of fashion - like XML is now.
- yrro 8mo ago... and that is why all 'modern' software is incredibly memory and CPU intensive...
- high_na_euv 8mo agoBut when things go wrong, you can usually find some random json file and adjust it :)
- lpcvoid 8mo agoStrong disagree. I like binary formats because I can just fopen(), fseek() and fread() stuff. I don't need json parser dependency, and don't need to deal with compression. Binary formats are simple, fast and I need a way smaller buffer to read/write them normally. I don't like wasting resources.
- high_na_euv 8mo agoBinary formats are painful to deal with from user perspective Sane for 3rd party devs
- adolph 8mo agoHazard a guess that serializing demented data structures into something text encoded like json or yaml or XML/SOAP is no less painful than a straight binary representation aside from unfamiliarity of tooling to reason about and arbitrarily query the structure, like jq, lq, etc.
- taneq 8mo agoBinary formats (that you can just fopen(), fseek(), fread() etc.) are generally super platform dependent. You can read in an array of bytes and then manually deserialise it (and I’ve done this, for sure!) but that’s basically one step away from parsing anyway.
- deleted 8mo ago[deleted]
- mike_hearn 8mo agoIt's not new. BOMStore is a format inherited from NeXTStep. JSON didn't exist back then. Also, it's a format designed to hold binary data. JSON can't do that without hacks like base64 encoding. Binary file stores like this are very common in highly optimized software, which operating systems tend to be, especially if you go looking at the older parts. Windows has a similar format embedded in EXE/DLL files. Same concept: a kind of pseudo-filesystem used to hold app icons and other resources.
- MrBuddyCasino 8mo agoYou don't have to put the files in the json, you can just put them inside the gzip file. But yeah obviously these are different eras.
- trashb 8mo agoEvery format is binary in the end, you are just swapping out the separators. I personally subscribe to the Unix philosophy. There are really two options, binary or plain text (readable binary due to a agreed standard formatting). All other formats are a variation of the two. Additional a binary format suits makes sense when the format is to be used for mobile devices which may not have much storage or bandwidth.
- peterspath 8mo agoI choose binary formats over JSON almost every time I can. JSON sucks big time.
- drob518 8mo agoJSON is… adequate. I like binary formats, too, when doing low-level programming, but you often want something readable, and JSON is a lot better and easier to parse than many other things, say XML.
- FrankBooth 8mo ago[flagged]
- jasode 8mo ago>Looks very much like a format that should just have been gzipped JSON. For application file formats that require storing binary blob data such as images, bitmaps, etc , many in the industry have settled on "SQLite db as a file format": (https://www.sqlite.org/appfileformat.html https://www.sqlite.org/appfileformat.html) Examples include Mozilla Firefox using sqlite db for favicons, Apple iOS using sqlite to store camera photos, Kodi media player uses sqlite for binary metadata, Microsoft Visual C++ IDE stores source code browsing data in sqlite, etc. Sqlite db would usually be a better choice rather than binary blobs encoded as Base64 and being stuffed into json.gzip files. One of the areas where the less efficient gzipped JSON might be better than Sqlite is web-server-to-web-browser data transfers because the client's Javascript engine has builtin gzip decompress and JSON manipulation functions.
- OskarS 8mo agoMy favorite example of this is Audacity, which stores their entire project file, including all the audio data, as a SQLite database. It's really amazing, you can stream audio data from many input sources into a SQLite database at once, it wont break a sweat, it's flexible and extremely safe from data loss due to crashes or power outages. As well as trivially mutable: many of these kinds of formats the "export" is a heavy-duty operation which serializes everything to JSON or whatever. But in SQLite, it's just a question of updating the database with normal INSERTs, very cheap and simple operations. We've put a similar system into production, and it's incredible how well it works.
- Cthulhu_ 8mo agoJSON is for humans to read and maintain, not machines to read/write - and machines read/write millions of times more than humans do. Using JSON / HTTP for backend communication (i.e. microservices) is madness when you consider how much overhead there is and how little value having it be human-readable delivers.
- streetfighter64 8mo ago> Don't use binary formats when it isn't absolutely needed. JSON in particular isn't very good [0] but I'd also argue that text formats in general aren't great. Isn't it a lot better that an int32_t always takes exactly 4 bytes instead of anywhere between one and eleven bytes (-2147483648)? [0] https://news.ycombinator.com/item?id=40555431 https://news.ycombinator.com/item?id=40555431
- coderatlarge 8mo agoone use-case is if you need to be able to stream the data.
- streetfighter64 8mo agoOne use-case of what? Whether a format is "streamable" is entirely orthogonal to whether it is text or binary. Streamability depends on other things such as if it has a header or checksum at the end, et cetera.
- zzo38computer 8mo agoI would agree that binary formats can be much better. Sometimes you can use fixed fields, but there are also structured formats such as DER (which is generally better than JSON in my opinion). JSON has many problems, including that it does not have a non-Unicode type (so you must use base64 encoding or hex encoding instead and that isn't very good), and other problems. Parsing it will also mean that you must handle escaping.