8 ms·
My favorite is simply using a tool to create a C file with your binary data: static uint8_t mydata[] = {0xDE, 0xAD, 0xBE, 0xEF, ... }; The advantage is that
by hikarudo 7y ago
My favorite is simply using a tool to create a C file with your binary data:
static uint8_t mydata[] = {0xDE, 0xAD, 0xBE, 0xEF, ... };
The advantage is that it works anywhere, with any compiler.
The disadvantage is that it can increase compile time. I would limit each .c file to 10MB; it seems there's a quadratic increase in build time with file size, at least with gcc.
Also, instead of "0x%02x" I use decimal notation and no spaces in order to decrease the .c file size:
static uint8_t mydata[] = {
42,7,105,0,0,...,
8,0,1,20,...,
...,
};
- zamadatix 7y agoI wonder what causes it to be nonlinear?
- saurik 7y agoThe article specifically leads with this as the default technique and then wanted to describe a fun alternative.
- gpvos 7y agoIf you click through from the article to the mailing list post that inspired it, you'll see that that's what they originally used, but it created a memory problem.
- KMnO4 7y agoFor those wondering, the easiest way to do this is with: xxd -i filename.bin
- dasyatidprime 7y agoYes, so long as you're willing to take a build-dep on vim. An od+awk+sed+sh combo will get you there from a POSIX base—and you have to add some wrapper text around the output of xxd anyway…
- anthk 7y agoxxd exists in standalone form. Compile it. https://github.com/ConorOG/xxd https://github.com/ConorOG/xxd
- dasyatidprime 7y agoI mean, so long as you're willing to take a build-dep on that, in that case.
- anthk 6y agoJust a c compiler.
- dasyatidprime 6y agoNo. So long as you're willing to take a build-dep on that code. Either you're expecting the code from that repository to exist somewhere in the compilation-host system already, or you are vendoring the code by copying it into your own repository and taking on any potential maintenance, unportability, any necessary configuration integration to make sure, for instance, that you're compiling for the host rather than the target in a cross-compile setup (did you make sure?)— Either way, if you use that code as part of your build process, you are taking a build-dep on that code. The fact that its only build-dep is a C compiler may play into making this a sensible choice as compared with something that has even more transitive dependencies, but it does not make it go away.
- rurban 7y agoToo bad that xxd doesn't include the const before the array decl. With const it saves a lot of memory.
- saagarjha 7y agoI don't see how const would save memory, unless you have a separate ROM region that keeps that out of the main memory.
- rurban 7y agoVMM. const memory can just be evicted and be swapped back in with zero cost. rw memory must be backed somehow, which us much more expensive. I submitted a patch
- AshamedCaptain 7y agoNot true unless you actively modify it (or the page it was in). "RW" mapped memory that is not dirty is also discarded on memory pressure.
- je42 7y agoif you do that you could also do base64 ;)
- klodolph 7y agoBut you can’t? Base64 won’t be recognized by the C compiler, you’ll end up having to decode it at runtime.
- je42 7y agoyep. That's a small piece of c-code you need to invest. See https://stackoverflow.com/questions/342409/how-do-i-base64-encode-decode-in-c https://stackoverflow.com/questions/342409/how-do-i-base64-e...
- klodolph 6y agoSo, by encoding in base64, you’ve made a more complicated solution which is also slower and harder to use? This seems like it is worse in every conceivable way. The only reason I can see for embedding base64 is if you need the source code to be compact, which makes sense for e.g. JavaScript but makes no sense for C. I’ll take the simple, fast, and easy solution, thank you.
- sheidavanesa 7y agonice salu2
- unwind 7y agoTypically such data can be 'const', too. Also note that for bytes, hex literals are longer than decimal ones a lot of the time and never shorter due to the "0x" prefix.
- KMnO4 7y agoI've never noticed the "quadratic" build time increase you mention, so I did a test[0]. Files in size from 1mb to 50mb, 3 trials each. These are the results, and they look absolutely linear[1]. [0]: https://pastebin.com/Z9329xkc https://pastebin.com/Z9329xkc [1]: https://i.imgur.com/I3XSBDg.png https://i.imgur.com/I3XSBDg.png
- RealityVoid 7y agoNice, I love it when people actually try it out. In the end, I much prefer the method outlined in the article, less processing overall, no need to transform it into an array nor compile the array into an object afterwards. You just skip 2 steps and bring it to an object directly!
- hikarudo 7y agoI stand corrected! I shouldn't have guessed "quadratic" without measuring. Large files will make gcc use more memory, and if you're building on a low-RAW machine, the OS will start to swap. That's probably the effect I saw. Nothing to do with "quadratic" compilation time.
- rwj 7y agoI also did quite a bit of benchmarking, and never saw any quadratic behaviour [0]. However, there is a floor for RSS and and elapsed time, but you would need to test assets less than ~1MB to see it. For assets over a limited range, the graph might look quadratic, but if you cover a large enough range you will see there are two regimes of behaviour. [0]: http://stonecode.ca/posts/binc_benchmarks/ http://stonecode.ca/posts/binc_benchmarks/