5 ms·

Putting on my Paranoia hat: what if some aggressor indeed was able to introduce code into the Nvidia drivers, which - if put on enough systems - would cripple t

Putting on my Paranoia hat: what if some aggressor indeed was able to introduce code into the Nvidia drivers, which - if put on enough systems - would cripple the ability to (re-)train Ai systems which might be used in military defense systems. What if - even worse - people decided to use Nvidia hardware in the inference systems as well… Putting down the paranoia hat. Happy weekend.

ethbr0 5y ago
> would cripple the ability to (re-)train Ai systems which might be used in military defense systems Not sure you're familiar with defense update and release schedules. As long as this gets fixed sometime in the next 5+ years, everything will be fine.
Melio 5y ago
That's just a very very weird though. Sry but no one just hacks into Nvidias driver dev department and injects complex code to cripple ml training. It's just nothing someone can just do. And there is also nothing which will prevent Nvidia to debug the ml issue and revert the change.
- pjerem 5y ago
  AI aside, hacking into the driver’s build process to inject hidden backdoors into the drivers could be a realist attack.
  - Melio 5y ago
    Is it realistic though? Hacking into Nvidias corp network, infiltrating their git server, disabling security scans and then injecting a backdoor undetected in complex code? In a process which is highly controlled due to it being a very central peace of software. Very unrealistic. It's easier to find or buy zero days in the wild for the same goal
    - nimrody 5y ago
      Well.. that's exactly what happened to Solarwinds last year, didn't it? Actually smarter than that - they got into the build system and added the malicious code in the build process so you couldn't see it in the repository. Do you think it's that difficult for a state sponsored body to infiltrate into a commercial company?
      - Melio 5y ago
        The effort my big software company does on regards of requirements of releasing software, I would say yes. Big companies like Nvidia have background checks, independent security teams etc. Impossible? No. But easier and cheaper is still other means.
- prettyStandard 5y ago
  Didn't a bunch of Linux distro s get infected with a "Ken Thompson Hack" a while back? https://softwareengineering.stackexchange.com/questions/184874/is-ken-thompsons-compiler-hack-still-a-threat https://softwareengineering.stackexchange.com/questions/1848...
  - prettyStandard 5y ago
    Ok I think it was Delphi now, but my brain remembered debian. lol. There is a double cross compilation method to detect if you are infected. https://wiki.c2.com/?TheKenThompsonHack https://wiki.c2.com/?TheKenThompsonHack
ganzuul 5y ago
Picking up said hat, we can ask why they would duplicate functionality already in the hardware if they could just steal the keys. It's not a very good hat, honestly.
AlexAltea 5y ago
> would cripple the ability to (re-)train Ai systems which might be used in military defense systems. Crippling use-cases is quite difficult: how could you distinguish at hardware/firmware-level object detection for fighter jets vs object detection for cars. Under the hood everything is just a bunch of compute units with extremely wide ALUs. I would even say, it's next to impossible to cripple "AI" without crippling graphics engines and most GPGPU kernels. EDIT: Ah, you meant drivers. Yeah, that's perhaps more doable (since the OS can provide context on the calling application), also more detectable by the end-users: many people diff drivers to find patched vulnerabilities, security researchers would eventually notice it.
pinephoneguy 5y ago
This has always been a problem. Third party closed source OS components are a massive security risk. Te people of the next century will look back on us as barbarians.

ethbr0 5y ago

> would cripple the ability to (re-)train Ai systems which might be used in military defense systems Not sure you're familiar with defense update and release schedules. As long as this gets fixed sometime in the next 5+ years, everything will be fine.

Melio 5y ago

That's just a very very weird though. Sry but no one just hacks into Nvidias driver dev department and injects complex code to cripple ml training. It's just nothing someone can just do. And there is also nothing which will prevent Nvidia to debug the ml issue and revert the change.

pjerem 5y ago
AI aside, hacking into the driver’s build process to inject hidden backdoors into the drivers could be a realist attack.
- Melio 5y ago
  Is it realistic though? Hacking into Nvidias corp network, infiltrating their git server, disabling security scans and then injecting a backdoor undetected in complex code? In a process which is highly controlled due to it being a very central peace of software. Very unrealistic. It's easier to find or buy zero days in the wild for the same goal
  - nimrody 5y ago
    Well.. that's exactly what happened to Solarwinds last year, didn't it? Actually smarter than that - they got into the build system and added the malicious code in the build process so you couldn't see it in the repository. Do you think it's that difficult for a state sponsored body to infiltrate into a commercial company?
    - Melio 5y ago
      The effort my big software company does on regards of requirements of releasing software, I would say yes. Big companies like Nvidia have background checks, independent security teams etc. Impossible? No. But easier and cheaper is still other means.
prettyStandard 5y ago
Didn't a bunch of Linux distro s get infected with a "Ken Thompson Hack" a while back? https://softwareengineering.stackexchange.com/questions/184874/is-ken-thompsons-compiler-hack-still-a-threat https://softwareengineering.stackexchange.com/questions/1848...
- prettyStandard 5y ago
  Ok I think it was Delphi now, but my brain remembered debian. lol. There is a double cross compilation method to detect if you are infected. https://wiki.c2.com/?TheKenThompsonHack https://wiki.c2.com/?TheKenThompsonHack

ganzuul 5y ago

Picking up said hat, we can ask why they would duplicate functionality already in the hardware if they could just steal the keys. It's not a very good hat, honestly.

AlexAltea 5y ago

> would cripple the ability to (re-)train Ai systems which might be used in military defense systems. Crippling use-cases is quite difficult: how could you distinguish at hardware/firmware-level object detection for fighter jets vs object detection for cars. Under the hood everything is just a bunch of compute units with extremely wide ALUs. I would even say, it's next to impossible to cripple "AI" without crippling graphics engines and most GPGPU kernels. EDIT: Ah, you meant drivers. Yeah, that's perhaps more doable (since the OS can provide context on the calling application), also more detectable by the end-users: many people diff drivers to find patched vulnerabilities, security researchers would eventually notice it.

pinephoneguy 5y ago

This has always been a problem. Third party closed source OS components are a massive security risk. Te people of the next century will look back on us as barbarians.