5 ms·

> HPE was notified by a Solid State Drive (SSD) manufacturer [...] That's a curious bit of context. It seems to imply they're shifting some of the blame onto

> HPE was notified by a Solid State Drive (SSD) manufacturer [...] That's a curious bit of context. It seems to imply they're shifting some of the blame onto their manufacturer? I makes me wonder if this firmware is 100% HPE specific, or if there a 2^16 hours bug about to bite a bunch of other pipelines.

wtallis 7y ago
This is a 2^15 hours bug, not a 2^16 hours bug. Odds are, somewhere in the SSD firmware source code, there's a missing "unsigned". And in the Makefile, probably a missing "-Wextra".
- richthegeek 7y ago
  Presumably the bug would still be considered a bug if it occured at 65536 hours? The incorrectly-signed bit only makes it appear sooner, but it's not the bug.
  - pfortuny 7y ago
    Someone says below that rollover may work OK but not negative times, for whatever reasons.
    - air7 7y ago
      Oh! So in that case owners just need to wait 32,768 hours and they'll be fine. /s
  - moonbug 7y ago
    That's 7.4 years, and well outside warranty.
    - eb0la 7y ago
      If you purchase a Carepaq, can actually extend the service beyond that? 3,4,and 5-Year 24x7 Carepaqs are available on purchase for the _whole_ chassis and parts... but I wonder if I can keep purchasing 1-year carepaq extensions beyond the 6th year.
      - HorstG 7y ago
        Yes, its possible. However, prices go up with age.
- garaetjjte 7y ago
  2^16 would be still rather too small, 65536 hours is not impossible duration.
- coldpie 7y ago
  > Odds are, somewhere in the SSD firmware source code, there's a missing "unsigned". And in the Makefile, probably a missing "-Wextra". Our industry really sucks. We need languages where this can't happen, and we need testing procedures where these things are caught. I wonder if software is the industry with the lowest quality:importance ratio.
- rocqua 7y ago
  Using signed integers for values that are always positive isn't necessarily a mistake. Most notably because for signed integers (in C and C++) overflow is undefined behavior. This allows for more aggressive optimizations by the compiler. Some advice I've read is to only use unsigned integers if you want to explicitly opt-in to having overflow be defined behavior.
  - cesarb 7y ago
    > Some advice I've read is to only use unsigned integers if you want to explicitly opt-in to having overflow be defined behavior. I read somewhere that the true reason for that advice is that it allows the compiler to silently store an "int" loop counter in a 64-bit register, without having to care about 32-bit overflow. If you use size_t for the loop counter, that's no longer an issue.
    - jdsully 7y ago
      At least on X86 that truncation behavior is free. 32-bit EAX is just first 32-bits of RAX.
    - nullc 7y ago
      I haven't seen an example where that's the case. It commonly factors in loop analysis around unrolling and vectorization, e.g. the loop will run exactly 16*n times OR an integer will overflow and it'll run some other number of times. Use of signed precludes the overflow and the exact bound enables efficient vectorization.
  - jschwartzi 7y ago
    Undefined behavior is platform-specific behavior. In this case it means that rollover’s effect on the value depends on how the register stores the integer and on how the particular instruction is documented to behave.
    - rocqua 7y ago
      No, undefined behavior is specified by the standard. There is also 'implementation defined' which tends to be both architecture and compiler specific. The difference is that a program which invokes 'implementation defined' behavior can be well defined, whereas a program that invokes undefined behavior is literally free to do anything.
icegreentea2 7y ago
I probably wouldn't even call it blame. Certainly if HPE isn't doing a full firmware audit (which I don't expect them to do), there's no way to run into this issue until it shows up in life testing. The manufacturer/supplier would be in the best position to encounter these types of issues first.
q3k 7y ago
Of course HPE doesn't write their own firmware from scratch. It's likely just whitelabeled by the drive manufacturer. HPE is just a reseller of existing OEM drives, like all other enterprise server manufacturers are.
- tyingq 7y ago
  It is possible, though, that this firmware was written specifically for HPE by the OEM.
  - q3k 7y ago
    There might have been some small changes, but the bulk of the code is likely copy pasted from a common codebase that is shared across models or even families. There's no way they're rewriting the entire controller code just for one customer, it doesn't make sense technically or from a business perspective.
  - HorstG 7y ago
    The HP specific part is just to make sure that the HP disk vendor lockin works: HP controllers only talk to HP branded disks...
SkyPuncher 7y ago
I'm always baffled when companies try to pass these issues off onto one of their sub-vendors - especially for critical issues like this. I didn't choose YOUR sub-vendor. You did. It's your responsibility to ensure that sub-vendor is operating at your standards. Passing blame to a sub-vendor indicates an unwillingness to take accountability.
- olyjohn 7y ago
  > an unwillingness to take accountability. I mean, that's no shock coming from HP.

wtallis 7y ago

This is a 2^15 hours bug, not a 2^16 hours bug. Odds are, somewhere in the SSD firmware source code, there's a missing "unsigned". And in the Makefile, probably a missing "-Wextra".

richthegeek 7y ago
Presumably the bug would still be considered a bug if it occured at 65536 hours? The incorrectly-signed bit only makes it appear sooner, but it's not the bug.
- pfortuny 7y ago
  Someone says below that rollover may work OK but not negative times, for whatever reasons.
  - air7 7y ago
    Oh! So in that case owners just need to wait 32,768 hours and they'll be fine. /s
- moonbug 7y ago
  That's 7.4 years, and well outside warranty.
  - eb0la 7y ago
    If you purchase a Carepaq, can actually extend the service beyond that? 3,4,and 5-Year 24x7 Carepaqs are available on purchase for the _whole_ chassis and parts... but I wonder if I can keep purchasing 1-year carepaq extensions beyond the 6th year.
    - HorstG 7y ago
      Yes, its possible. However, prices go up with age.
garaetjjte 7y ago
2^16 would be still rather too small, 65536 hours is not impossible duration.
coldpie 7y ago
> Odds are, somewhere in the SSD firmware source code, there's a missing "unsigned". And in the Makefile, probably a missing "-Wextra". Our industry really sucks. We need languages where this can't happen, and we need testing procedures where these things are caught. I wonder if software is the industry with the lowest quality:importance ratio.
rocqua 7y ago
Using signed integers for values that are always positive isn't necessarily a mistake. Most notably because for signed integers (in C and C++) overflow is undefined behavior. This allows for more aggressive optimizations by the compiler. Some advice I've read is to only use unsigned integers if you want to explicitly opt-in to having overflow be defined behavior.
- cesarb 7y ago
  > Some advice I've read is to only use unsigned integers if you want to explicitly opt-in to having overflow be defined behavior. I read somewhere that the true reason for that advice is that it allows the compiler to silently store an "int" loop counter in a 64-bit register, without having to care about 32-bit overflow. If you use size_t for the loop counter, that's no longer an issue.
  - jdsully 7y ago
    At least on X86 that truncation behavior is free. 32-bit EAX is just first 32-bits of RAX.
  - nullc 7y ago
    I haven't seen an example where that's the case. It commonly factors in loop analysis around unrolling and vectorization, e.g. the loop will run exactly 16*n times OR an integer will overflow and it'll run some other number of times. Use of signed precludes the overflow and the exact bound enables efficient vectorization.
- jschwartzi 7y ago
  Undefined behavior is platform-specific behavior. In this case it means that rollover’s effect on the value depends on how the register stores the integer and on how the particular instruction is documented to behave.
  - rocqua 7y ago
    No, undefined behavior is specified by the standard. There is also 'implementation defined' which tends to be both architecture and compiler specific. The difference is that a program which invokes 'implementation defined' behavior can be well defined, whereas a program that invokes undefined behavior is literally free to do anything.

icegreentea2 7y ago

I probably wouldn't even call it blame. Certainly if HPE isn't doing a full firmware audit (which I don't expect them to do), there's no way to run into this issue until it shows up in life testing. The manufacturer/supplier would be in the best position to encounter these types of issues first.

q3k 7y ago

Of course HPE doesn't write their own firmware from scratch. It's likely just whitelabeled by the drive manufacturer. HPE is just a reseller of existing OEM drives, like all other enterprise server manufacturers are.

tyingq 7y ago
It is possible, though, that this firmware was written specifically for HPE by the OEM.
- q3k 7y ago
  There might have been some small changes, but the bulk of the code is likely copy pasted from a common codebase that is shared across models or even families. There's no way they're rewriting the entire controller code just for one customer, it doesn't make sense technically or from a business perspective.
- HorstG 7y ago
  The HP specific part is just to make sure that the HP disk vendor lockin works: HP controllers only talk to HP branded disks...

SkyPuncher 7y ago

I'm always baffled when companies try to pass these issues off onto one of their sub-vendors - especially for critical issues like this. I didn't choose YOUR sub-vendor. You did. It's your responsibility to ensure that sub-vendor is operating at your standards. Passing blame to a sub-vendor indicates an unwillingness to take accountability.

olyjohn 7y ago
> an unwillingness to take accountability. I mean, that's no shock coming from HP.