5 ms·

> Actually, no. There is no portable "incbin" in C/C++ I kinda hate this kind of argument. I mean, it's true, as .incbin is a GNU assembler directive (FWIW: b

by ajross 3y ago

> Actually, no. There is no portable "incbin" in C/C++ I kinda hate this kind of argument. I mean, it's true, as .incbin is a GNU assembler directive (FWIW: binutils/llvm objcopy is a better mechanism still for this sort of thing in most contexts, as it doesn't involve source compilation of any kind). But it leads ridiculous design decisions, like "I'm going to write 8MB of source code instead of doing the portability work on my own to turn that data into a linkable symbol". There's a huge gulf in the space between "non-portable" and "impossible". In this case, the problem is trivially solved on every platform ever. Trivial problems should employ trivial solutions, even where they have to involve some per-platform engineering.

duped 3y ago
> But it leads ridiculous design decisions, like "I'm going to write 8MB of source code instead of doing the portability work on my own to turn that data into a linkable symbol". people use scripts to do this kind of thing and never think about it again, and it's still more portable than writing a custom build step.
iainmerrick 3y ago
But it leads ridiculous design decisions, like "I'm going to write 8MB of source code instead of doing the portability work on my own to turn that data into a linkable symbol". Why is that ridiculous? It strikes me as not necessarily the best, but the most obvious approach, the most portable, and possibly the quickest to implement. Your toolchain might have a special way to import binary blobs, but a) you’ll have to dig through the docs to find it, b) you’ll probably need to solve the problem again when porting to a different platform, and c) who knows if it actually works, or if there are hidden gotchas? Sure, if there’s a known tool or option that does the job, you should go ahead and use it. But in general, writing a little script to generate a bunch of boilerplate code is perfectly workable.
- ajross 3y ago
  > Your toolchain might have a special way to import binary blobs, but a) you’ll have to dig through the docs to find it This is a corrollary: "I don't want to learn my tools, so I'll learn the language standard instead" is fundamentally exactly the problem I'm talking about. Straight up: C linkage is a 1970's paradigm full of tools that had to run on a PDP/11, and it's vastly simpler than learning C++ or Rust. It's just not "modern" and no one taught it to you, so it looks weird and mysterious. That's the problem!
  - maskros 3y ago
    It's not "I don't want to learn my tools." It's "I don't want to learn and debug _everybody else who may possible want to build this otherwise portable C program_'s tools." When those tools change how they do this unportable thing every couple of years in subtle and incompatible ways, which require #ifdef's to handle the different ways those linked against symbols can be accessed, multiplied by dozens of different platforms, then yes, I'm going to compile an 8MB literal.
  - josephg 3y ago
    I go back and forth on this argument when it comes to codegen. Like, you could make the same argument that protobuf shouldn't output C code. It should output an object file that you can link into whatever compiled language you want. C, fortran, C++, rust, who cares. As you say, the linking model is simple and works well. Why do we generate big C/C++ strings instead, and compile those? Because object files have lots of compiler/platform/architecture specific stuff in them. Outputting C (or C++) then compiling it is a much more convenient way to generate those object files, because it works on every OS, compiler and architecture. Even systems that you don't know about, or that don't exist yet. I hear what you're saying and I'm torn about it. I mean, aren't binary blobs just a simpler version of the problem protobuf faces? C code is already the most portable way to make object files. Why wouldn't we use it?
    - ajross 3y ago
      > Like, you could make the same argument that protobuf shouldn't output C code. The case at hand is an 8MB static array of integers. Obviously yes, of course, absolutely: you choose the correct/simple/obvious/trivialest implementation strategy for the problem. That's exactly what I'm saying! In the case of protobuf (static generation of an otherwise arbitrarily complicated data structure with reasonably bounded size), code generation makes a ton of sense.
      - Joker_vD 3y ago
        And the simple/obvious/trivialest solution is to write an array literal with 4 millions integers in it, while fighting with Microsoft's link.exe is anything but. Even using rc.exe and loading that data from the resource section is a non-trivial amount of additional work.
DSMan195276 3y ago
> FWIW: binutils/llvm objcopy is a better mechanism still for this sort of thing in most contexts, as it doesn't involve source compilation of any kind I used to agree with that, but honestly a simple `xxd` to get a C array avoids so many issues with `objcopy` that I'd rather just use that now. With `objcopy` even just getting the names of the produced symbols to be consistent is a pain, and you have to specify the output target and architecture which is just another thing you have to update for more platforms (and if someone's using a cross-compiler, they have to override that setting too). In contrast if you just produce a C array then it compiles like normal C code and links like normal C code, problem solved and all the complexity is gone.

by ajross 3y ago

duped 3y ago

> But it leads ridiculous design decisions, like "I'm going to write 8MB of source code instead of doing the portability work on my own to turn that data into a linkable symbol". people use scripts to do this kind of thing and never think about it again, and it's still more portable than writing a custom build step.

iainmerrick 3y ago

But it leads ridiculous design decisions, like "I'm going to write 8MB of source code instead of doing the portability work on my own to turn that data into a linkable symbol". Why is that ridiculous? It strikes me as not necessarily the best, but the most obvious approach, the most portable, and possibly the quickest to implement. Your toolchain might have a special way to import binary blobs, but a) you’ll have to dig through the docs to find it, b) you’ll probably need to solve the problem again when porting to a different platform, and c) who knows if it actually works, or if there are hidden gotchas? Sure, if there’s a known tool or option that does the job, you should go ahead and use it. But in general, writing a little script to generate a bunch of boilerplate code is perfectly workable.

ajross 3y ago
> Your toolchain might have a special way to import binary blobs, but a) you’ll have to dig through the docs to find it This is a corrollary: "I don't want to learn my tools, so I'll learn the language standard instead" is fundamentally exactly the problem I'm talking about. Straight up: C linkage is a 1970's paradigm full of tools that had to run on a PDP/11, and it's vastly simpler than learning C++ or Rust. It's just not "modern" and no one taught it to you, so it looks weird and mysterious. That's the problem!
- maskros 3y ago
  It's not "I don't want to learn my tools." It's "I don't want to learn and debug _everybody else who may possible want to build this otherwise portable C program_'s tools." When those tools change how they do this unportable thing every couple of years in subtle and incompatible ways, which require #ifdef's to handle the different ways those linked against symbols can be accessed, multiplied by dozens of different platforms, then yes, I'm going to compile an 8MB literal.
- josephg 3y ago
  I go back and forth on this argument when it comes to codegen. Like, you could make the same argument that protobuf shouldn't output C code. It should output an object file that you can link into whatever compiled language you want. C, fortran, C++, rust, who cares. As you say, the linking model is simple and works well. Why do we generate big C/C++ strings instead, and compile those? Because object files have lots of compiler/platform/architecture specific stuff in them. Outputting C (or C++) then compiling it is a much more convenient way to generate those object files, because it works on every OS, compiler and architecture. Even systems that you don't know about, or that don't exist yet. I hear what you're saying and I'm torn about it. I mean, aren't binary blobs just a simpler version of the problem protobuf faces? C code is already the most portable way to make object files. Why wouldn't we use it?
  - ajross 3y ago
    > Like, you could make the same argument that protobuf shouldn't output C code. The case at hand is an 8MB static array of integers. Obviously yes, of course, absolutely: you choose the correct/simple/obvious/trivialest implementation strategy for the problem. That's exactly what I'm saying! In the case of protobuf (static generation of an otherwise arbitrarily complicated data structure with reasonably bounded size), code generation makes a ton of sense.
    - Joker_vD 3y ago
      And the simple/obvious/trivialest solution is to write an array literal with 4 millions integers in it, while fighting with Microsoft's link.exe is anything but. Even using rc.exe and loading that data from the resource section is a non-trivial amount of additional work.

DSMan195276 3y ago

> FWIW: binutils/llvm objcopy is a better mechanism still for this sort of thing in most contexts, as it doesn't involve source compilation of any kind I used to agree with that, but honestly a simple `xxd` to get a C array avoids so many issues with `objcopy` that I'd rather just use that now. With `objcopy` even just getting the names of the produced symbols to be consistent is a pain, and you have to specify the output target and architecture which is just another thing you have to update for more platforms (and if someone's using a cross-compiler, they have to override that setting too). In contrast if you just produce a C array then it compiles like normal C code and links like normal C code, problem solved and all the complexity is gone.