7 ms·

Apple must know of some not-too-obscure software that did rely on those flags, curious if anyone's found what it might be.

by mkj 4y ago

userbinator 4y ago
There's this somewhat well-known sequence that uses AF: cmp al, 10 sbb al, 69h das
irdc 4y ago
It could very well be that they cannot reliably prove those flags are unused in enough cases to optimise out their (expensive) calculation.
- Someone 4y ago
  The flags get saved and restored on task switches and interrupts. I would think deciding whether those bits actually get used across that is infeasible, but yet, https://dougallj.wordpress.com/2022/11/09/why-is-rosetta-2-fast/ https://dougallj.wordpress.com/2022/11/09/why-is-rosetta-2-f... says: “This almost entirely prevents inter-instruction optimisations. There are two known exceptions. The first is an “unused-flags” optimisation, which avoids calculating x86 flags value if they are not used before being overwritten on every path from a flag-setting instruction.” Does this mean the emulator is buggy (if I write a loop that sets and clears flags, but doesn’t do anything with them, an interrupt handler that samples the flag values would need to see both values), but in a way that no sane code would notice?
  - atq2119 4y ago
    It does technically sound like there is a gap in the emulation, yes. Though, since interrupts are normally transparent to user space software, you'd have to use something like ptrace or be messing with the signal delivery mechanism to even have a chance at noticing the gap. And you'd have to do quite obscure and absurd things even by those APIs' standards.
  - erincandescent 4y ago
    Rosetta 2 doesn't emulate interrupts (they're not visible to userspace components) It does have to emulate signals but its moderately easy to handle those by deferring them to "safe" points where all architectural state is known
mrweasel 4y ago
Yeah that seems odd. The article says: "While almost no modern applications read these AF and PF bits" It can't really be legacy software, as most AMD64 software on the Mac is fairly recent (2006) and I suspect that we're talking software older than that for the AF and PF to be in regular use. So it also have to be something fairly important like virtualization or something used in image processing or compression. It seems like overkill for something that would be a problem for an end-user application that would transition to Apple Silicon anyway.
- rockdoe 4y ago
  Isn't it more like that computing these flags is basically free in hardware? So might as well add support for it.
  - mrweasel 4y ago
    Sure, but designing those flags into the hardware has to come at a cost. Maybe it's simple and cheap enough that it doesn't matter.
    - bni 4y ago
      Could it have been a test case "to see if we can" do these kinds of things. And then it got left in with the more important stuff
mrspuratic 4y ago
AF is more commonly known as half-carry flag, used in BCD, it's uncommon that anything other than BCD opcodes would use it, in which case a correct implementation of DAA/AAA (6 ops in total) would suffice (though the state of the previous calculation has to go somewhere). Since flags can be load/stored or push/pop-ed I'd speculate avoiding the hit in correctly handling every flags access was deemed to be worth it. Based on experience from a long time ago (8-bit micros, z80 in particular, and implementing a 6800 emulator, DAA was more code than any other): unusual opcodes and edge case behaviours are common in optimisations, and anti-debug/disassembly. Games perhaps? In any case, I find the idea of a correctly behaving, deterministic binary translation more appealing than a bunch of beartr^W on-the-fly software fixups.
- WorldMaker 4y ago
  I've learned more than once in my education and career to never underestimate how common (and weirdly useful sometimes) BCD is. Lots of random hardware still uses BCD for all sorts of strange timing tasks. The classic 7-segment display is easiest to program against in BCD as one bit of still somewhat ubiquitous types of hardware displays (though slowly disappearing as other screen formats have gotten cheaper). There's a reason that BCD is in the middle of EBCDIC, IBM used BCD math so heavily in its decades of history that even their text encoding and punch cards were built around it. I've seen Enterprise software that still needs to read/write and sometimes even do math in BCD for compatibility with old Mainframe apps and software still written in COBOL.
- Mindwipe 4y ago
  > Games perhaps? It's very hard to conclude Apple care much about breaking a bunch of games on the Mac when they cut off 80% of the Macs gaming library a year earlier by removing 32bit execution, presumably mostly to ease the use of Rosetta 2. And, y'know, literally every other action Apple has taken for twenty years.
  - andy_ppp 4y ago
    I think the issue is probably games devs don’t want to use Xcode so they don’t get simple upgrades. If you use Xcode almost every change Apple make there is a path to be supported by these changes.
dougall 4y ago
They might know of some, but I'm of the opinion that it's worth it to support the specification. It only takes one previously undiscovered application to rely on that specified behaviour, and then you need to fix it. If it were solely a game emulator I wouldn't expect this - performance probably comes first... But for running professional software, I'd choose correctness any day. I don't know of any applications the use AF, or the PF result in question, but I haven't looked into it. I'd maybe check some of the big professional things with a lot of legacy history like Photoshop or Excel. As I understand it Rosetta 2 also supports 32-bit x86, not for native macOS applications (where 32-bit isn't supported), but to allow Wine/Crossovers to work, so a whole bunch of ancient 32-bit Windows games might be in scope too.
Someone 4y ago
My money (but not a lot of it) is on Microsoft Excel. Its number arithmetic is, let’s say, special. Microsoft claims they use IEEE 754 floats (https://learn.microsoft.com/en-us/office/troubleshoot/excel/floating-point-arithmetic-inaccurate-result https://learn.microsoft.com/en-us/office/troubleshoot/excel/...), but they don’t completely do that. For example, do A1: 0.3 A2: 0.2 A3: 0.1 A4: =A1-A2-A3 A5: =(A1-A2-A3) B4: =A4=0 B5: =A5=0 A4 will show as “0”, A5 as “-2.77556E-17”, and B4 and B5 show these aren’t purely display issues. B4 shows “TRUE”, B5 shows “FALSE”. My hunch is that they sometimes use BCD arithmetic. More info at http://people.eecs.berkeley.edu/~wkahan/ARITH_17.pdf http://people.eecs.berkeley.edu/~wkahan/ARITH_17.pdf.
- jonhohle 4y ago
  Is `B5: =_A6_=0` intentional or should it be A5?
  - Someone 4y ago
    Typo. Sorry. Fixed. Thanks.
- lobochrome 4y ago
  You are shaking my foundations here. I frequently, when writing complicated formulas, leave in a set of parentheses too much, just because when writing the formula, I wasn't sure yet whether I'd need them. Bill - j'accuse!
- ezfe 4y ago
  iWork/Numbers does this properly, and repost 0/0/TRUE/TRUE in each case
  - ilyt 4y ago
    define "properly". It should be false/false for the IEEE floats...
    - wmf 4y ago

Apple must know of some not-too-obscure software that did rely on those flags, curious if anyone's found what it might be.

by mkj 4y ago

Apple must know of some not-too-obscure software that did rely on those flags, curious if anyone's found what it might be.

userbinator 4y ago

There's this somewhat well-known sequence that uses AF: cmp al, 10 sbb al, 69h das

irdc 4y ago

It could very well be that they cannot reliably prove those flags are unused in enough cases to optimise out their (expensive) calculation.

Someone 4y ago
The flags get saved and restored on task switches and interrupts. I would think deciding whether those bits actually get used across that is infeasible, but yet, https://dougallj.wordpress.com/2022/11/09/why-is-rosetta-2-fast/ https://dougallj.wordpress.com/2022/11/09/why-is-rosetta-2-f... says: “This almost entirely prevents inter-instruction optimisations. There are two known exceptions. The first is an “unused-flags” optimisation, which avoids calculating x86 flags value if they are not used before being overwritten on every path from a flag-setting instruction.” Does this mean the emulator is buggy (if I write a loop that sets and clears flags, but doesn’t do anything with them, an interrupt handler that samples the flag values would need to see both values), but in a way that no sane code would notice?
- atq2119 4y ago
  It does technically sound like there is a gap in the emulation, yes. Though, since interrupts are normally transparent to user space software, you'd have to use something like ptrace or be messing with the signal delivery mechanism to even have a chance at noticing the gap. And you'd have to do quite obscure and absurd things even by those APIs' standards.
- erincandescent 4y ago
  Rosetta 2 doesn't emulate interrupts (they're not visible to userspace components) It does have to emulate signals but its moderately easy to handle those by deferring them to "safe" points where all architectural state is known

mrweasel 4y ago

Yeah that seems odd. The article says: "While almost no modern applications read these AF and PF bits" It can't really be legacy software, as most AMD64 software on the Mac is fairly recent (2006) and I suspect that we're talking software older than that for the AF and PF to be in regular use. So it also have to be something fairly important like virtualization or something used in image processing or compression. It seems like overkill for something that would be a problem for an end-user application that would transition to Apple Silicon anyway.

rockdoe 4y ago
Isn't it more like that computing these flags is basically free in hardware? So might as well add support for it.
- mrweasel 4y ago
  Sure, but designing those flags into the hardware has to come at a cost. Maybe it's simple and cheap enough that it doesn't matter.
  - bni 4y ago
    Could it have been a test case "to see if we can" do these kinds of things. And then it got left in with the more important stuff

mrspuratic 4y ago

AF is more commonly known as half-carry flag, used in BCD, it's uncommon that anything other than BCD opcodes would use it, in which case a correct implementation of DAA/AAA (6 ops in total) would suffice (though the state of the previous calculation has to go somewhere). Since flags can be load/stored or push/pop-ed I'd speculate avoiding the hit in correctly handling every flags access was deemed to be worth it. Based on experience from a long time ago (8-bit micros, z80 in particular, and implementing a 6800 emulator, DAA was more code than any other): unusual opcodes and edge case behaviours are common in optimisations, and anti-debug/disassembly. Games perhaps? In any case, I find the idea of a correctly behaving, deterministic binary translation more appealing than a bunch of beartr^W on-the-fly software fixups.

WorldMaker 4y ago
I've learned more than once in my education and career to never underestimate how common (and weirdly useful sometimes) BCD is. Lots of random hardware still uses BCD for all sorts of strange timing tasks. The classic 7-segment display is easiest to program against in BCD as one bit of still somewhat ubiquitous types of hardware displays (though slowly disappearing as other screen formats have gotten cheaper). There's a reason that BCD is in the middle of EBCDIC, IBM used BCD math so heavily in its decades of history that even their text encoding and punch cards were built around it. I've seen Enterprise software that still needs to read/write and sometimes even do math in BCD for compatibility with old Mainframe apps and software still written in COBOL.
Mindwipe 4y ago
> Games perhaps? It's very hard to conclude Apple care much about breaking a bunch of games on the Mac when they cut off 80% of the Macs gaming library a year earlier by removing 32bit execution, presumably mostly to ease the use of Rosetta 2. And, y'know, literally every other action Apple has taken for twenty years.
- andy_ppp 4y ago
  I think the issue is probably games devs don’t want to use Xcode so they don’t get simple upgrades. If you use Xcode almost every change Apple make there is a path to be supported by these changes.

dougall 4y ago

They might know of some, but I'm of the opinion that it's worth it to support the specification. It only takes one previously undiscovered application to rely on that specified behaviour, and then you need to fix it. If it were solely a game emulator I wouldn't expect this - performance probably comes first... But for running professional software, I'd choose correctness any day. I don't know of any applications the use AF, or the PF result in question, but I haven't looked into it. I'd maybe check some of the big professional things with a lot of legacy history like Photoshop or Excel. As I understand it Rosetta 2 also supports 32-bit x86, not for native macOS applications (where 32-bit isn't supported), but to allow Wine/Crossovers to work, so a whole bunch of ancient 32-bit Windows games might be in scope too.

Someone 4y ago

My money (but not a lot of it) is on Microsoft Excel. Its number arithmetic is, let’s say, special. Microsoft claims they use IEEE 754 floats (https://learn.microsoft.com/en-us/office/troubleshoot/excel/floating-point-arithmetic-inaccurate-result https://learn.microsoft.com/en-us/office/troubleshoot/excel/...), but they don’t completely do that. For example, do A1: 0.3 A2: 0.2 A3: 0.1 A4: =A1-A2-A3 A5: =(A1-A2-A3) B4: =A4=0 B5: =A5=0 A4 will show as “0”, A5 as “-2.77556E-17”, and B4 and B5 show these aren’t purely display issues. B4 shows “TRUE”, B5 shows “FALSE”. My hunch is that they sometimes use BCD arithmetic. More info at http://people.eecs.berkeley.edu/~wkahan/ARITH_17.pdf http://people.eecs.berkeley.edu/~wkahan/ARITH_17.pdf.

jonhohle 4y ago
Is `B5: =_A6_=0` intentional or should it be A5?
- Someone 4y ago
  Typo. Sorry. Fixed. Thanks.
lobochrome 4y ago
You are shaking my foundations here. I frequently, when writing complicated formulas, leave in a set of parentheses too much, just because when writing the formula, I wasn't sure yet whether I'd need them. Bill - j'accuse!
ezfe 4y ago
iWork/Numbers does this properly, and repost 0/0/TRUE/TRUE in each case
- ilyt 4y ago
  define "properly". It should be false/false for the IEEE floats...
  - wmf 4y ago