4 ms·

> Opus was unable to implement a 16-bit x86 code generator needed to boot into 16-bit real mode. While the compiler can output correct 16-bit x86 via the 66/67

by shakna 9mo ago

> Opus was unable to implement a 16-bit x86 code generator needed to boot into 16-bit real mode. While the compiler can output correct 16-bit x86 via the 66/67 opcode prefixes, the resulting compiled output is over 60kb, far exceeding the 32k code limit enforced by Linux. Instead, Claude simply cheats here and calls out to GCC for this phase Does it really boot...?

ndesaulniers 9mo ago
> Does it really boot...? They don't need 16b x86 support for the RISCV or ARM ports, so yes, but depends on what 'it' we're talking about here. Also, FWIW, GCC doesn't directly assemble to machine code either; it shells out to GAS (GNU Assembler). This blog post calls it "GCC assembler and linker" but to be more precise the author should edit this to "GNU binutils assembler and linker." Even then GNU binutils contains two linkers (BFD and GOLD), or did they excise GOLD already (IIRC, there was some discussion a few years ago about it)?
- shakna 9mo ago
  Yeah, didn't mention gas or ld, for similar reasons. I agree that a compiler doesn't necessarily "need" those. I don't agree that all the claims are backed up by their own comments, which means that there's probably other places where it falls down. Its... Misrepresentation. Like Chicken is a Scheme compiler. But they're very up front that it depends on a C compiler. Here, they wrote a C compiler that is at least sometimes reliant on having a different C compiler around. So is the project at 50%? 75%? Even if its 99%, thats not the same story as they tried to write. And if they wrote that tale instead, it would be more impressive, rather than "There's some holes. How many?"
  - Philpax 9mo ago
    Their C compiler is not reliant on having another C compiler around. Compiling the 16-bit real mode bootstrap for the Linux kernel on x86(-64) requires another C compiler; you certainly don't need another compiler to compile the kernel for another architecture, or to compile another piece of software not subject to the 32k constraint. The compiler itself is entirely functional; it just can't generate code optimal enough to fit within the constraints for that very specific (tiny!) part of the system, so another compiler is required to do that step.
    - shakna 9mo ago
      It also generates the wrong relocations for link time. And so cannot boot, even with help. > The “compiles the kernel” claim needs a footnote. CCC compiles all the C source files, but the final binary cannot be produced because CCC generates incorrect relocations for kernel data structures (__jump_table, __ksymtab).
TheCondor 9mo ago
The assembler seems like nearly the easiest part. Slurp arch manuals and knock it out, it’s fixed and complete.
- shakna 9mo ago
  Huh. A second person mentioning the assembler. Don't think I ever referred to one...?
- jakewins 9mo ago
  I am surprised by the number of comments that say the assembler is trivial - it is admittedly perhaps simpler than some other parts of the compiler chain, but it’s not trivial. What you are doing is kinda serialising a self-referential graph structure of machine code entries that reference each others addresses, but you don’t know the addresses because the (x86) instructions are variable-length, so you can’t know them until you generate the machine code, chicken-and-egg problem. Personally I find writing parsers much much simpler than writing assemblers.
  - nicebyte 9mo ago
    assembler is far from trivial at least for x86 where there are many possible encodings for a given instruction. emitting the most optimal encoding that does the correct thing depends on surrounding context, and you'd have to do multiple passes over the input.
    - jmalicki 9mo ago
      What is a single example where the optimal encoding depends on context? (I am assuming you're just doing an assembler where registers have already been chosen, vs. a compiler that can choose sse vs. scalar and do register allocation etc.)?
      - chris_swenson 9mo ago
        “mov rcx, 0”. At least one assembler (the Go assembler) would at one point blindly (and arguably, incorrectly) rewrite this to “xor rcx, rcx”, which is smaller but modifies flags, which “mov” does not. I believe Go fixed this later, possibly by looking at surrounding instructions to see if the flags were being used, for instance by an “adc” later, to know if the assembler needs to pick the larger “mov” encoding. Whether that logic should belong in a compiler or an assembler is a separate issue, but it definitely was in the assembler there.

by shakna 9mo ago

ndesaulniers 9mo ago

> Does it really boot...? They don't need 16b x86 support for the RISCV or ARM ports, so yes, but depends on what 'it' we're talking about here. Also, FWIW, GCC doesn't directly assemble to machine code either; it shells out to GAS (GNU Assembler). This blog post calls it "GCC assembler and linker" but to be more precise the author should edit this to "GNU binutils assembler and linker." Even then GNU binutils contains two linkers (BFD and GOLD), or did they excise GOLD already (IIRC, there was some discussion a few years ago about it)?

shakna 9mo ago
Yeah, didn't mention gas or ld, for similar reasons. I agree that a compiler doesn't necessarily "need" those. I don't agree that all the claims are backed up by their own comments, which means that there's probably other places where it falls down. Its... Misrepresentation. Like Chicken is a Scheme compiler. But they're very up front that it depends on a C compiler. Here, they wrote a C compiler that is at least sometimes reliant on having a different C compiler around. So is the project at 50%? 75%? Even if its 99%, thats not the same story as they tried to write. And if they wrote that tale instead, it would be more impressive, rather than "There's some holes. How many?"
- Philpax 9mo ago
  Their C compiler is not reliant on having another C compiler around. Compiling the 16-bit real mode bootstrap for the Linux kernel on x86(-64) requires another C compiler; you certainly don't need another compiler to compile the kernel for another architecture, or to compile another piece of software not subject to the 32k constraint. The compiler itself is entirely functional; it just can't generate code optimal enough to fit within the constraints for that very specific (tiny!) part of the system, so another compiler is required to do that step.
  - shakna 9mo ago
    It also generates the wrong relocations for link time. And so cannot boot, even with help. > The “compiles the kernel” claim needs a footnote. CCC compiles all the C source files, but the final binary cannot be produced because CCC generates incorrect relocations for kernel data structures (__jump_table, __ksymtab).

TheCondor 9mo ago

The assembler seems like nearly the easiest part. Slurp arch manuals and knock it out, it’s fixed and complete.

shakna 9mo ago
Huh. A second person mentioning the assembler. Don't think I ever referred to one...?
jakewins 9mo ago
I am surprised by the number of comments that say the assembler is trivial - it is admittedly perhaps simpler than some other parts of the compiler chain, but it’s not trivial. What you are doing is kinda serialising a self-referential graph structure of machine code entries that reference each others addresses, but you don’t know the addresses because the (x86) instructions are variable-length, so you can’t know them until you generate the machine code, chicken-and-egg problem. Personally I find writing parsers much much simpler than writing assemblers.
- nicebyte 9mo ago
  assembler is far from trivial at least for x86 where there are many possible encodings for a given instruction. emitting the most optimal encoding that does the correct thing depends on surrounding context, and you'd have to do multiple passes over the input.
  - jmalicki 9mo ago
    What is a single example where the optimal encoding depends on context? (I am assuming you're just doing an assembler where registers have already been chosen, vs. a compiler that can choose sse vs. scalar and do register allocation etc.)?
    - chris_swenson 9mo ago
      “mov rcx, 0”. At least one assembler (the Go assembler) would at one point blindly (and arguably, incorrectly) rewrite this to “xor rcx, rcx”, which is smaller but modifies flags, which “mov” does not. I believe Go fixed this later, possibly by looking at surrounding instructions to see if the flags were being used, for instance by an “adc” later, to know if the assembler needs to pick the larger “mov” encoding. Whether that logic should belong in a compiler or an assembler is a separate issue, but it definitely was in the assembler there.