5 ms·

> What if `size` was enormous and the first character was negative? One extra byte of lookahead might SIGSEGV! So it must gingerly step one byte at a time. Pag

> What if `size` was enormous and the first character was negative? One extra byte of lookahead might SIGSEGV! So it must gingerly step one byte at a time. Pages aren't 1 byte each, they're 4KiB. You can't just segfault at by crossing 1 byte at any arbitrary address. Some alignment checks to handle edge cases should let the compiler at least read things 8-byte aligned in the steady state and only fall back on edge cases.

kevingadd 6y ago
If you're at byte 4095 and you advance by 1 byte and the next page is a guard page, you're dead. You can minimize the risk of this by requiring that input addresses be aligned (and many ABIs/APIs do require this) but it's realistic to say that an extra byte of lookahead can and will SIGSEGV in unfortunate cases.
- mehrdadn 6y ago
  What I'm saying is that the compiler can generate code that examines the memory address and looks ahead only when it's safe to do so.
  - saurik 6y ago
    (And for those who think this is strange, memcpy implementations usually do this, so there is precedent that this is very sane.)
    - BeeOnRope 6y ago
      memcpy doesn't have this same problem because it is known-length without any early exit.
ridiculous_fish 6y ago
The page size is a runtime property - for example recent iOS devices have 16KB pages while older ones are 4k. There may be ways to make it safe, but it seems to be firmly in the "compiler heroics" camp.
- mehrdadn 6y ago
  Page sizes are easy to lower bound when you're compiling. You know the target system you're compiling for. This most definitely is not in the "compiler heroics" camp.
  - ridiculous_fish 6y ago
    IMO this is one of those fragile optimizations. If the optimization is important, it should be explicit (hand-rolled asm) or else you risk breaking it. If it's not important, then nobody cares. Emitting OoB reads dependent on optimization level? Think about ASAN, debugger watchpoints...can't be worth it! clang has a cool feature that highlights missed vectorization opportunities: > clang++ -O3 -c test.cpp -Rpass-analysis=loop-vectorize test.cpp:5:4: remark: loop not vectorized: loop control flow is not understood by vectorizer [-Rpass-analysis=loop-vectorize] love it!
    - BeeOnRope 6y ago
      In practice this type of over-read is not uncommon in library code (even for standard routines), but I haven't seen compilers generate it. False positives with memory-sanitizers is definitely a problem: the hand-rolled library uses can be whitelisted, but that's not feasible once the compiler starts generating it in arbitrary code.
  - cesarb 6y ago
    > Page sizes are easy to lower bound when you're compiling. You know the target system you're compiling for. Do you really know the target system? Most of the time, you want the compiled code to work not only on the current system, but also in future systems of the same lineage. What prevents a future version of the architecture from for instance lowering the translation granule from 4096 bytes to 512 bytes, or even lower?
    - BeeOnRope 6y ago
      This would be a backwards incompatible and massively breaking change on most systems that define a minimum page size. These minimum page sizes are generally treated as or are explicitly architecturally guaranteed.

kevingadd 6y ago

If you're at byte 4095 and you advance by 1 byte and the next page is a guard page, you're dead. You can minimize the risk of this by requiring that input addresses be aligned (and many ABIs/APIs do require this) but it's realistic to say that an extra byte of lookahead can and will SIGSEGV in unfortunate cases.

mehrdadn 6y ago
What I'm saying is that the compiler can generate code that examines the memory address and looks ahead only when it's safe to do so.
- saurik 6y ago
  (And for those who think this is strange, memcpy implementations usually do this, so there is precedent that this is very sane.)
  - BeeOnRope 6y ago
    memcpy doesn't have this same problem because it is known-length without any early exit.

ridiculous_fish 6y ago

The page size is a runtime property - for example recent iOS devices have 16KB pages while older ones are 4k. There may be ways to make it safe, but it seems to be firmly in the "compiler heroics" camp.

mehrdadn 6y ago
Page sizes are easy to lower bound when you're compiling. You know the target system you're compiling for. This most definitely is not in the "compiler heroics" camp.
- ridiculous_fish 6y ago
  IMO this is one of those fragile optimizations. If the optimization is important, it should be explicit (hand-rolled asm) or else you risk breaking it. If it's not important, then nobody cares. Emitting OoB reads dependent on optimization level? Think about ASAN, debugger watchpoints...can't be worth it! clang has a cool feature that highlights missed vectorization opportunities: > clang++ -O3 -c test.cpp -Rpass-analysis=loop-vectorize test.cpp:5:4: remark: loop not vectorized: loop control flow is not understood by vectorizer [-Rpass-analysis=loop-vectorize] love it!
  - BeeOnRope 6y ago
    In practice this type of over-read is not uncommon in library code (even for standard routines), but I haven't seen compilers generate it. False positives with memory-sanitizers is definitely a problem: the hand-rolled library uses can be whitelisted, but that's not feasible once the compiler starts generating it in arbitrary code.
- cesarb 6y ago
  > Page sizes are easy to lower bound when you're compiling. You know the target system you're compiling for. Do you really know the target system? Most of the time, you want the compiled code to work not only on the current system, but also in future systems of the same lineage. What prevents a future version of the architecture from for instance lowering the translation granule from 4096 bytes to 512 bytes, or even lower?
  - BeeOnRope 6y ago
    This would be a backwards incompatible and massively breaking change on most systems that define a minimum page size. These minimum page sizes are generally treated as or are explicitly architecturally guaranteed.