6 ms·

Fundamentally Understanding and Solving RowHammer

52 points by g0xA52A2A 4y ago

myself248 4y ago
Clarify something for me: If a DRAM chip is operated within its specifications, and suffers even a single bit-flip from this effect, is it not defective, either by manufacture or by design? Memory has a profoundly simple job, to return the data that was stored. Returning something other than what was stored is pretty fundamentally wrong -- either it needed more refreshes, or the specification should've prohibited repeated access patterns, or the memory controller allowed access patterns that were prohibited by the specification. Are we all pretending this is mysterious and novel just because it would be uncomfortable to admit so much has been so wrong for so long?
- pfortuny 4y ago
  If the engine in your car stutters once in a while… is it defective? If your LCD monitor has a dead pixel, is it defective? That is a question of quality. Rowhammer showed that what previously was acceptable quality is no longer so.
  - mort96 4y ago
    > If your LCD monitor has a dead pixel, is it defective? Yes.
    - yunohn 4y ago
      Dead pixels happen, it doesn’t mean the whole panel is defective.
      - 1letterunixname 4y ago
        It's the stipulative definition of "defective" as a term. One could assess it with functional and nonfunctional criteria. Consumer A demands perfection for full brightness video in darkness while customer B requires serviceability only to see a large letter X or O.
    - DougMerritt 4y ago
      So you always buy Class I monitors, guaranteed zero dead pixels, not the affordable Class II monitors with up to 15 dead pixels? [1] [1] https://en.wikipedia.org/wiki/ISO_13406-2#Pixel_Fault_Classes https://en.wikipedia.org/wiki/ISO_13406-2#Pixel_Fault_Classe...
      - mort96 4y ago
        I must admit, I don't pay attention to that. But my monitors don't have dead pixels, and if they had dead pixels, I would absolutely consider that a defective monitor. The class presumably just determines whether I can get the manufacturer to fix it for free or not.
        DougMerritt 4y ago
        Yes, but they're also choosing the monitors that pass QA best. It may be that all of their monitors have zero defects, but that is statistically unlikely. Look at chip manufacturing. They really care about high yields, but there are always defects, 20 years ago and today as well. It's an issue with manufacturing in general. Perfection is typically impossible to achieve. Maybe not always, but typically. Back to the main issue, since modern monitors are quite high resolution, you and I might both have a few dead pixels, or more likely a few that have pixels that are half-dead (dim) in one channel (of red/blue/green), and we just don't notice on our desktops -- and it would usually take careful examination to spot them even on a pure white screen. Your mileage may vary, of course.
        mort96 4y ago
        Dead pixels which are just dimmer than usual is okay, some variance is expected. But if a monitor has that kind of dead pixel that's stuck displaying a bright white or green or something even when the rest of the screen is black, the screen is 100% defective, whatever the manufacturer's QA standards might say.
        sroussey 4y ago
        As an EE, I’m shocked computers work at all, and so reliably. If you knew how the sausage was made you would be horrified.
    - pfortuny 4y ago
      From Dell: A display with 1 to 5 fixed subpixel is considered normal and within industry standards https://www.dell.com/support/kbdoc/es-es/000126004/dell-display-pixel-guidelines?lang=en https://www.dell.com/support/kbdoc/es-es/000126004/dell-disp...
      - mort96 4y ago
        Yeah, so Dell sells defective monitors and doesn't serve them under warranty. Doesn't mean they're not defective.
  - myself248 4y ago
    If the engine in my car runs fine on whatever gas for 10 years, but when I put the same gas in the tank 50 tank-fuls in a row and suddenly the engine turns into a pink flamingo, yes it is defective. If my LCD monitor displays any image I want, but when I try to show an image of a checkerboard it loses sync with the incoming signal, yes it is defective. (There were a number of similar problems in the telephone network around the introduction of T1 signaling, and it's why the 1000Hz milliwatt test tone was changed to 1004Hz.) These things were never acceptable.
    - orbz 4y ago
      The difference in this case is there’s a malicious party acting against the system too. It would be like one gas station you go to throwing some sugar in the tank just to mess with you.
- orbz 4y ago
  The cost of increased performance and efficiency is unfortunately met with an increase in overall fragility of the system. As gate sizes shrink down they’re more susceptible to fluctuations in voltage, temperature and electromagnetic induction. So yes, you could have a very robust system, but you’d have to pay more for less RAM, which market forces have chosen against.
  - mort96 4y ago
    Fundamentally irrational market forces, we might add. No person compares spec sheets and chooses the chip with a 10% efficiency advantage but with a rowhammer style vulnerability instead of the 10% less efficient chip without a rowhammer style vulnerability. At least I've never seen "is susceptible to rowhammer" in a spec sheet. So as consumers, we don't really have the choice to select the more expensive or less efficient but more robust chip. Hell, ECC isn't even an option for most classes of CPU.
    - _3u10 4y ago
      Why not? There are CPUs being designed that do floating point math “wrong” but faster… I’m sure many people would use RAM that’s cheaper and faster but row hammer vulnerable. Just as many people drive cars with engines that will break if driven at redline for long periods of time. It’s not like row hammer is a realistic use case. No algorithm depends on row hammer access patterns. Just like people don’t buy cars to bounce off the rev limiter.
      - mort96 4y ago
        "Why not" what? Why we don't get the option to buy rowhammer-resistent RAM? I don't know, ask manufacturers and resellers I suppose? I don't make the spec sheets.
        sroussey 4y ago
        If you run a server farm, you absolutely know these things. And you are using ECC based RAM.
- dboreham 4y ago
  Yes. If I could find the original rowhammer thread here, I believe I said so at the time.
29athrowaway 4y ago
Even ECC modules are vulnerable.
- someweirdperson 4y ago
  Takes longer though.
  - bell-cot 4y ago
    IIR - So long as the time it takes is longer than the periodic DRAM refresh interval, the ECC can effectively stop rowhammer.
    - j16sdiz 4y ago
      https://ieeexplore.ieee.org/document/8835222 https://ieeexplore.ieee.org/document/8835222 > . In addition, we show that, despite the non-trivial constraints imposed by ECC, ECCploit can still be powerful in practice and mimic the behavior of prior Rowhammer exploits
      - sroussey 4y ago
        Just like with non-ecc dram, quality levels vary per dimm, and you can avoid this with good quality ecc dram.
      - toast0 4y ago
        I didn't read the full paper, which is very interesting, but on systems with proper ECC configuration[1], the scanning portion of the process would generate a huge number of machine check exceptions. Having run a large number of ECC enabled machines, high volume ECC correctable errors are disruptive to normal processing, and the machine goes out of service for repair quickly. If I were running a multi-tenant cloud, knowing that RowHammer is a thing, I would migrate the VMs on a machine throwing correctable errors, and also track the customers. VM migration during scanning is going to result in inconsistent data, and successful exploitation seems unlikely. Tracking the customers, allows one to prevent repeat rowhammerers from accessing your systems. That said, it's still very interesting and many systems have improper ECC configuration. ECC without reporting isn't going to mitigate RowHammer; nor will it help to have ECC with reporting, if the reporting is ignored, etc. [1] I didn't make it to their system configurations, but I did see their description that only one system under test properly responded to errors, by logging errors, and crashing on uncorrectable errors.
deleted 4y ago
[deleted]
ackbar03 4y ago
For me, rowhammer has always been the most badass exploit I've ever heard of. I mean, flipping bits in memory by forcing an electrical current down a pathway over and over again? That's completely mind boggling. If I was hacked using rowhammer, I'd be so impressed I'd give the hacker whatever data they want and maybe even throw in a credit card
- 1letterunixname 4y ago
  It's not magic. Slower but cheaper than SRAM cells, DRAM-derived logic needs to refresh periodically and partially/fully when read. It's likely an inability to model worst-case current needs or clocking accurately enough to match reality when a large number of cells reinforce a draining property.

52 points by g0xA52A2A 4y ago

Clarify something for me: If a DRAM chip is operated within its specifications, and suffers even a single bit-flip from this effect, is it not defective, either by manufacture or by design? Memory has a profoundly simple job, to return the data that was stored. Returning something other than what was stored is pretty fundamentally wrong -- either it needed more refreshes, or the specification should've prohibited repeated access patterns, or the memory controller allowed access patterns that were prohibited by the specification. Are we all pretending this is mysterious and novel just because it would be uncomfortable to admit so much has been so wrong for so long?

pfortuny 4y ago
If the engine in your car stutters once in a while… is it defective? If your LCD monitor has a dead pixel, is it defective? That is a question of quality. Rowhammer showed that what previously was acceptable quality is no longer so.
- mort96 4y ago
  > If your LCD monitor has a dead pixel, is it defective? Yes.
  - yunohn 4y ago
    Dead pixels happen, it doesn’t mean the whole panel is defective.
    - 1letterunixname 4y ago
      It's the stipulative definition of "defective" as a term. One could assess it with functional and nonfunctional criteria. Consumer A demands perfection for full brightness video in darkness while customer B requires serviceability only to see a large letter X or O.
  - DougMerritt 4y ago
    So you always buy Class I monitors, guaranteed zero dead pixels, not the affordable Class II monitors with up to 15 dead pixels? [1] [1] https://en.wikipedia.org/wiki/ISO_13406-2#Pixel_Fault_Classes https://en.wikipedia.org/wiki/ISO_13406-2#Pixel_Fault_Classe...
    - mort96 4y ago
      I must admit, I don't pay attention to that. But my monitors don't have dead pixels, and if they had dead pixels, I would absolutely consider that a defective monitor. The class presumably just determines whether I can get the manufacturer to fix it for free or not.
      - DougMerritt 4y ago
        Yes, but they're also choosing the monitors that pass QA best. It may be that all of their monitors have zero defects, but that is statistically unlikely. Look at chip manufacturing. They really care about high yields, but there are always defects, 20 years ago and today as well. It's an issue with manufacturing in general. Perfection is typically impossible to achieve. Maybe not always, but typically. Back to the main issue, since modern monitors are quite high resolution, you and I might both have a few dead pixels, or more likely a few that have pixels that are half-dead (dim) in one channel (of red/blue/green), and we just don't notice on our desktops -- and it would usually take careful examination to spot them even on a pure white screen. Your mileage may vary, of course.
        mort96 4y ago
        Dead pixels which are just dimmer than usual is okay, some variance is expected. But if a monitor has that kind of dead pixel that's stuck displaying a bright white or green or something even when the rest of the screen is black, the screen is 100% defective, whatever the manufacturer's QA standards might say.
      - sroussey 4y ago
        As an EE, I’m shocked computers work at all, and so reliably. If you knew how the sausage was made you would be horrified.
  - pfortuny 4y ago
    From Dell: A display with 1 to 5 fixed subpixel is considered normal and within industry standards https://www.dell.com/support/kbdoc/es-es/000126004/dell-display-pixel-guidelines?lang=en https://www.dell.com/support/kbdoc/es-es/000126004/dell-disp...
    - mort96 4y ago
      Yeah, so Dell sells defective monitors and doesn't serve them under warranty. Doesn't mean they're not defective.
- myself248 4y ago
  If the engine in my car runs fine on whatever gas for 10 years, but when I put the same gas in the tank 50 tank-fuls in a row and suddenly the engine turns into a pink flamingo, yes it is defective. If my LCD monitor displays any image I want, but when I try to show an image of a checkerboard it loses sync with the incoming signal, yes it is defective. (There were a number of similar problems in the telephone network around the introduction of T1 signaling, and it's why the 1000Hz milliwatt test tone was changed to 1004Hz.) These things were never acceptable.
  - orbz 4y ago
    The difference in this case is there’s a malicious party acting against the system too. It would be like one gas station you go to throwing some sugar in the tank just to mess with you.
orbz 4y ago
The cost of increased performance and efficiency is unfortunately met with an increase in overall fragility of the system. As gate sizes shrink down they’re more susceptible to fluctuations in voltage, temperature and electromagnetic induction. So yes, you could have a very robust system, but you’d have to pay more for less RAM, which market forces have chosen against.
- mort96 4y ago
  Fundamentally irrational market forces, we might add. No person compares spec sheets and chooses the chip with a 10% efficiency advantage but with a rowhammer style vulnerability instead of the 10% less efficient chip without a rowhammer style vulnerability. At least I've never seen "is susceptible to rowhammer" in a spec sheet. So as consumers, we don't really have the choice to select the more expensive or less efficient but more robust chip. Hell, ECC isn't even an option for most classes of CPU.
  - _3u10 4y ago
    Why not? There are CPUs being designed that do floating point math “wrong” but faster… I’m sure many people would use RAM that’s cheaper and faster but row hammer vulnerable. Just as many people drive cars with engines that will break if driven at redline for long periods of time. It’s not like row hammer is a realistic use case. No algorithm depends on row hammer access patterns. Just like people don’t buy cars to bounce off the rev limiter.
    - mort96 4y ago
      "Why not" what? Why we don't get the option to buy rowhammer-resistent RAM? I don't know, ask manufacturers and resellers I suppose? I don't make the spec sheets.
      - sroussey 4y ago
        If you run a server farm, you absolutely know these things. And you are using ECC based RAM.
dboreham 4y ago
Yes. If I could find the original rowhammer thread here, I believe I said so at the time.

29athrowaway 4y ago

Even ECC modules are vulnerable.

someweirdperson 4y ago
Takes longer though.
- bell-cot 4y ago
  IIR - So long as the time it takes is longer than the periodic DRAM refresh interval, the ECC can effectively stop rowhammer.
  - j16sdiz 4y ago
    https://ieeexplore.ieee.org/document/8835222 https://ieeexplore.ieee.org/document/8835222 > . In addition, we show that, despite the non-trivial constraints imposed by ECC, ECCploit can still be powerful in practice and mimic the behavior of prior Rowhammer exploits
    - sroussey 4y ago
      Just like with non-ecc dram, quality levels vary per dimm, and you can avoid this with good quality ecc dram.
    - toast0 4y ago
      I didn't read the full paper, which is very interesting, but on systems with proper ECC configuration[1], the scanning portion of the process would generate a huge number of machine check exceptions. Having run a large number of ECC enabled machines, high volume ECC correctable errors are disruptive to normal processing, and the machine goes out of service for repair quickly. If I were running a multi-tenant cloud, knowing that RowHammer is a thing, I would migrate the VMs on a machine throwing correctable errors, and also track the customers. VM migration during scanning is going to result in inconsistent data, and successful exploitation seems unlikely. Tracking the customers, allows one to prevent repeat rowhammerers from accessing your systems. That said, it's still very interesting and many systems have improper ECC configuration. ECC without reporting isn't going to mitigate RowHammer; nor will it help to have ECC with reporting, if the reporting is ignored, etc. [1] I didn't make it to their system configurations, but I did see their description that only one system under test properly responded to errors, by logging errors, and crashing on uncorrectable errors.

deleted 4y ago

[deleted]

ackbar03 4y ago

For me, rowhammer has always been the most badass exploit I've ever heard of. I mean, flipping bits in memory by forcing an electrical current down a pathway over and over again? That's completely mind boggling. If I was hacked using rowhammer, I'd be so impressed I'd give the hacker whatever data they want and maybe even throw in a credit card

1letterunixname 4y ago
It's not magic. Slower but cheaper than SRAM cells, DRAM-derived logic needs to refresh periodically and partially/fully when read. It's likely an inability to model worst-case current needs or clocking accurately enough to match reality when a large number of cells reinforce a draining property.