12 ms·

Aren't there many tools out there that can already do this? I thought even objcopy(1) can turn an arbitrary file into a .o file containing a single symbol holdi

Aren't there many tools out there that can already do this? I thought even objcopy(1) can turn an arbitrary file into a .o file containing a single symbol holding the data.

planteen 8y ago
Yes and that is the better way to do it. Compiling an enormous C file is slow. Many compilers have low limits (around 50 MB or so) but gcc has a huge limit. But even on gcc, a large file slows down the build process significantly.
- jandrese 8y ago
  More importantly gcc's memory requirements balloon quickly as the size of the file grows. There are projects that can't be built on platforms like a Raspberry Pi simply because some of the source files (typically heavily templated C++ code) blow out the memory. A good example is Firefox.
emmelaich 8y ago
What is the clang or macos equivalent to objcopy?
- planteen 8y ago
  You are just dealing with ELF files, so objcopy works fine in those scenarios as long as the architecture matches for the linker. This trick works many other compilers that use ELF such as clang, MSVC, Green Hills, etc.
  - tetromino_ 8y ago
    Except that MacOS does not use ELF, it uses Mach-O.
    - planteen 8y ago
      Clang handles intermediate ELF object files fine and can link into a Mach-O final executable. Windows under MSVC doesn't use ELF either (it uses PE for the .EXE) and it links in intermediate ELF object files fine. I've made an EXE that was a mix of files compiled by MSVC and GCC.
- comex 8y ago
  Pass -sectcreate to the linker, e.g. -sectcreate __DATA __foo foo.txt. Then you can reference the data like this: extern char foo_start[] asm("section$start$__DATA$__foo"); extern char foo_end[] asm("section$end$__DATA$__foo");
jzwinck 8y ago
I like objcopy for this. Here is an example command: objcopy --input binary --output elf64-x86-64 myfile.txt myfile.o
dvhh 8y ago
you could get a look at https://csl.name/post/embedding-binary-data/ https://csl.name/post/embedding-binary-data/

Aren't there many tools out there that can already do this? I thought even objcopy(1) can turn an arbitrary file into a .o file containing a single symbol holding the data.

planteen 8y ago

Yes and that is the better way to do it. Compiling an enormous C file is slow. Many compilers have low limits (around 50 MB or so) but gcc has a huge limit. But even on gcc, a large file slows down the build process significantly.

jandrese 8y ago
More importantly gcc's memory requirements balloon quickly as the size of the file grows. There are projects that can't be built on platforms like a Raspberry Pi simply because some of the source files (typically heavily templated C++ code) blow out the memory. A good example is Firefox.

emmelaich 8y ago

What is the clang or macos equivalent to objcopy?

planteen 8y ago
You are just dealing with ELF files, so objcopy works fine in those scenarios as long as the architecture matches for the linker. This trick works many other compilers that use ELF such as clang, MSVC, Green Hills, etc.
- tetromino_ 8y ago
  Except that MacOS does not use ELF, it uses Mach-O.
  - planteen 8y ago
    Clang handles intermediate ELF object files fine and can link into a Mach-O final executable. Windows under MSVC doesn't use ELF either (it uses PE for the .EXE) and it links in intermediate ELF object files fine. I've made an EXE that was a mix of files compiled by MSVC and GCC.
comex 8y ago
Pass -sectcreate to the linker, e.g. -sectcreate __DATA __foo foo.txt. Then you can reference the data like this: extern char foo_start[] asm("section$start$__DATA$__foo"); extern char foo_end[] asm("section$end$__DATA$__foo");

jzwinck 8y ago

I like objcopy for this. Here is an example command: objcopy --input binary --output elf64-x86-64 myfile.txt myfile.o

dvhh 8y ago

you could get a look at https://csl.name/post/embedding-binary-data/ https://csl.name/post/embedding-binary-data/