7 ms·

Backdooring the Lottery [pdf]

99 points by enricotal 9y ago

dghughes 9y ago
Didn't read the pdf but I used to work for a lottery Corp and from what I heard security was nuts. One of the server room IT guys left for Blue Cross and later said Blue Cross was much less strict than the lottery. I also recall one of the compliance guys telling me about the balls used in one of those bingo ball tumbling machines. He said they had to wash, dry and weigh each ball before every draw. Of course it was fun to yell to him saying on draw day "hey it's Friday did you wash and dry your balls?"
- dfc 9y ago
  When did Blue Cross become the standard bearer for infosec?
  - sillysaurus3 9y ago
    People who haven't worked in infosec think companies like that would naturally be good at security. In reality they're some of the worst. Breaches are just very rare.
    - FatAmericanDev 9y ago
      Reported breaches
  - Justin_K 9y ago
    It's just a comparison. Relax.
madez 9y ago
I wonder why they didn't go for an easy technique to defend against backdoors: use two independent sources for random numbers that don't know of each other and xor their output together. The increase in cost is small and the win for integrity and security is huge.
- PhantomGremlin 9y ago
  Yes, XOR. But what's also needed is simplicity. As another poster here said, "simply use getrandom()". Whether or not you agree with that exact function, the idea behind "simply" is exactly that, simplicity. It's a lot easier to audit code that does: getrandom() read hardware RNG XOR those two together done! btw ... keep a permanent log of these random numbers But, as the article said, things were much more complicated. Who allowed something into production that had an opportunity to check for Wednesdays and Saturdays?
  - tekacs 9y ago
    > Who allowed something into production that had an opportunity to check for Wednesdays and Saturdays? From the analysis in the article it sounds like the function that you're describing was added to the dylib binary after compilation, so it was 'slipped' rather than 'allowed' into production. That said, auditing the binary and bindiff as suggested could perhaps mitigate the problem, modulo 'Reflections on trusting trust' and its mitigations.
  - deleted 9y ago
    [deleted]
  - deleted 9y ago
    [deleted]
  - gizmo686 9y ago
    >getrandom() >read hardware RNG Not secure against a mallicious hardware RNG. In principle, the hardware RNG could access the memory and corralate its output with that of getrandom. For that matter, getrandom might be able to do something to muck with the hardware RNG, or with the code following. Or any code not part of your RNG scheme could try and muck with the results. >Who allowed something into production that had an opportunity to check for Wednesdays and Saturdays? The code was audited and passed because there was no such check. The machine code that made it onto the machine did not match the source code submitted, but instead had an extra function tacked onto the end. How do you defend against this [0]. What if someone sneaks the backdoor in the firmware of the harddrive, that only provides the backdoored code to the CPU occasionally. At some point, you need to trust the people making the lottery machines. [0] Hint, reproducible builds and signed code; but this still devolves into a trusting trust situation.
gwern 9y ago
Anyone getting SSL errors trying to load the slides? Nothing seems to work, including proxying over Tor.
- buro9 9y ago
  Yup. Not working on multiple browsers, curl, etc. File listing via Google Cache shows it was accessible: https://webcache.googleusercontent.com/search?q=cache:uatZ4ufzez4J:https://media.defcon.org/DEF%2520CON%252025/DEF%2520CON%252025%2520presentations/&num=1&hl=en&gl=us&strip=1&vwsrc=0 https://webcache.googleusercontent.com/search?q=cache:uatZ4u... Wayback Machine doesn't have it. Ah... Google cache has it: https://webcache.googleusercontent.com/search?safe=off&output=googleabout&ie=UTF-8&q=cache%3Ahttps%3A%2F%2Fmedia.defcon.org%2FDEF%2520CON%252025%2FDEF%2520CON%252025%2520presentations%2FDEFCON-25-Gus-Frischie-and-Evan-Teitelman-Backdooring-the-Lottery.pdf https://webcache.googleusercontent.com/search?safe=off&outpu...
- alberts00 9y ago
  Seems to work now.
- jlgaddis 9y ago
  I was, but it's working now.
- jis 9y ago
  I put a copy in IPFS: https://ipfs.io/ipfs/QmcgWw89GFo3Z1w9ad1H6Gom4DV4t5VRBu5zhkW6s96NP2 https://ipfs.io/ipfs/QmcgWw89GFo3Z1w9ad1H6Gom4DV4t5VRBu5zhkW...
thephyber 9y ago
I remember reading about the MUSL RNG fraud recently (1-2 years ago). Interesting that they are tying state lottery frauds to that malicious insider from as far back as 2005 (or so the slides suggest).
- dfc 9y ago
  For anyone else that reads this and thinks the comment is referring to the musl libc fear not. Apparently MUSL is also the "Multi state Lottery":
mrb 9y ago
As an infosec pro, If I was in charge of a lottery RNG, I would require the use of a recent unmodified Linux or OpenBSD distro and simply use getrandom(2). These custom RNG introduce 100% unnecessary complexity and risks. I am baffled none of them seem to do that and they need thirdparty certification procedures (who miss RNG flaws anyway.)
- PhantomGremlin 9y ago
  I'd take the output of getrandom() and XOR it with something like a quantum RNG. E.g. these guys claim their hardware is used by Lotteries and Gaming: http://www.idquantique.com/random-number-generation/quantis-random-number-generator/ http://www.idquantique.com/random-number-generation/quantis-... But, as Agent Mulder knew, "trust no one". So the above is necessary but not sufficient. Above all else, there must be constant vigilance.
  - jk2323 9y ago
    "TRUE RANDOM NUMBER GENERATOR EXPLOITING THE RANDOMNESS OF QUANTUM PHYSICS" not sure what they mean by this. Likely it is just "avalanche noise". This is not very difficult. In fact, I build an hardware RNG that is based on avalanche noise (not my development): https://emergent.unpythonic.net/01257868826 https://emergent.unpythonic.net/01257868826
    - PhantomGremlin 9y ago
      Evaluating the veracity of their claims is way above my pay grade. They do have a PDF on their site: http://marketing.idquantique.com/acton/attachment/11868/f-004b/1/-/-/-/-/Quantum%20RNG%20White%20Paper.pdf http://marketing.idquantique.com/acton/attachment/11868/f-00... Basically, emit photons at a semi-transparent mirror. 50% go straight, 50% reflect 90 degrees. Have single-photon detectors at both directions. Quoting from their PDF: Figure 2 shows the block diagram of the Quantis random number generator. It consists of three subsystems. The first one is the core of the generator and contains the optical elements that are used to implement the random process and produce the random outcomes. It comprises a light emitting diode producing the photons, a transmission element, where the random process takes place, and two single-photon detectors – detectors with single-photon resolution – to record the outcomes. The optical subsystem is controlled by a synchronization and acquisition electronic circuit. This subsystem comprises a clock and triggering electronics for the photon source, as well the acquisition electronics for the single-photon detectors. The processing and interfacing subsystem perform statistical and hardware checks, as well as unbiasing of the sequence. Edit: the ID Quantique hardware is a bit pricey for hobbyists, starting at about 1000 euros. If I were building something for myself, I'd look here for inspiration: https://www.fourmilab.ch/hotbits/ https://www.fourmilab.ch/hotbits/ That hardware detects beta decay events. That's preferable to me than "avalanche noise". Not that there's anything fundamentally wrong with thermal noise or shot noise; it's that in real implementations the quality of results is very dependent on things such as quality of power supply. To me it's much simpler to correctly detect intervals between consecutive radioactive decay events. The downside to a cheap decay detector is that the number of events is much lower than what the commercial hardware provides. A few hundred bits per second versus a few million bits per second.
callesgg 9y ago
Is there a video of their talk?
- teitelman 9y ago
  The video should be up within a few months.
anon4728 9y ago
Sad to see no mention of Voltaire and Condamine, until now, of course.
matt_wulfeck 9y ago
The idea of a public/random mixing source is always fun. Everybody has a fun pet idea but they all suffer from one issue or another. Here's some of mine: 1. Move the mouse around 2. Force someone to dance and read the output from a webcam 3. Record various RF signals 4. Speak a joke into a microphone. The best solution is probably just a plain, non-network computer with an open-hardware TRNG to mix the CSPRNG (I believe BSD is switching to Fortuna). Read from the CSPRNG all day loooong.
teitelman 9y ago
Hey I'm the guy who gave the talk (Evan Teitelman). Let me know if you have any questions.

99 points by enricotal 9y ago

Didn't read the pdf but I used to work for a lottery Corp and from what I heard security was nuts. One of the server room IT guys left for Blue Cross and later said Blue Cross was much less strict than the lottery. I also recall one of the compliance guys telling me about the balls used in one of those bingo ball tumbling machines. He said they had to wash, dry and weigh each ball before every draw. Of course it was fun to yell to him saying on draw day "hey it's Friday did you wash and dry your balls?"

dfc 9y ago
When did Blue Cross become the standard bearer for infosec?
- sillysaurus3 9y ago
  People who haven't worked in infosec think companies like that would naturally be good at security. In reality they're some of the worst. Breaches are just very rare.
  - FatAmericanDev 9y ago
    Reported breaches
- Justin_K 9y ago
  It's just a comparison. Relax.

madez 9y ago

I wonder why they didn't go for an easy technique to defend against backdoors: use two independent sources for random numbers that don't know of each other and xor their output together. The increase in cost is small and the win for integrity and security is huge.

PhantomGremlin 9y ago
Yes, XOR. But what's also needed is simplicity. As another poster here said, "simply use getrandom()". Whether or not you agree with that exact function, the idea behind "simply" is exactly that, simplicity. It's a lot easier to audit code that does: getrandom() read hardware RNG XOR those two together done! btw ... keep a permanent log of these random numbers But, as the article said, things were much more complicated. Who allowed something into production that had an opportunity to check for Wednesdays and Saturdays?
- tekacs 9y ago
  > Who allowed something into production that had an opportunity to check for Wednesdays and Saturdays? From the analysis in the article it sounds like the function that you're describing was added to the dylib binary after compilation, so it was 'slipped' rather than 'allowed' into production. That said, auditing the binary and bindiff as suggested could perhaps mitigate the problem, modulo 'Reflections on trusting trust' and its mitigations.
- deleted 9y ago
  [deleted]
- deleted 9y ago
  [deleted]
- gizmo686 9y ago
  >getrandom() >read hardware RNG Not secure against a mallicious hardware RNG. In principle, the hardware RNG could access the memory and corralate its output with that of getrandom. For that matter, getrandom might be able to do something to muck with the hardware RNG, or with the code following. Or any code not part of your RNG scheme could try and muck with the results. >Who allowed something into production that had an opportunity to check for Wednesdays and Saturdays? The code was audited and passed because there was no such check. The machine code that made it onto the machine did not match the source code submitted, but instead had an extra function tacked onto the end. How do you defend against this [0]. What if someone sneaks the backdoor in the firmware of the harddrive, that only provides the backdoored code to the CPU occasionally. At some point, you need to trust the people making the lottery machines. [0] Hint, reproducible builds and signed code; but this still devolves into a trusting trust situation.

gwern 9y ago

Anyone getting SSL errors trying to load the slides? Nothing seems to work, including proxying over Tor.

buro9 9y ago
Yup. Not working on multiple browsers, curl, etc. File listing via Google Cache shows it was accessible: https://webcache.googleusercontent.com/search?q=cache:uatZ4ufzez4J:https://media.defcon.org/DEF%2520CON%252025/DEF%2520CON%252025%2520presentations/&num=1&hl=en&gl=us&strip=1&vwsrc=0 https://webcache.googleusercontent.com/search?q=cache:uatZ4u... Wayback Machine doesn't have it. Ah... Google cache has it: https://webcache.googleusercontent.com/search?safe=off&output=googleabout&ie=UTF-8&q=cache%3Ahttps%3A%2F%2Fmedia.defcon.org%2FDEF%2520CON%252025%2FDEF%2520CON%252025%2520presentations%2FDEFCON-25-Gus-Frischie-and-Evan-Teitelman-Backdooring-the-Lottery.pdf https://webcache.googleusercontent.com/search?safe=off&outpu...
alberts00 9y ago
Seems to work now.
jlgaddis 9y ago
I was, but it's working now.
jis 9y ago
I put a copy in IPFS: https://ipfs.io/ipfs/QmcgWw89GFo3Z1w9ad1H6Gom4DV4t5VRBu5zhkW6s96NP2 https://ipfs.io/ipfs/QmcgWw89GFo3Z1w9ad1H6Gom4DV4t5VRBu5zhkW...

thephyber 9y ago

I remember reading about the MUSL RNG fraud recently (1-2 years ago). Interesting that they are tying state lottery frauds to that malicious insider from as far back as 2005 (or so the slides suggest).

dfc 9y ago
For anyone else that reads this and thinks the comment is referring to the musl libc fear not. Apparently MUSL is also the "Multi state Lottery":

mrb 9y ago

As an infosec pro, If I was in charge of a lottery RNG, I would require the use of a recent unmodified Linux or OpenBSD distro and simply use getrandom(2). These custom RNG introduce 100% unnecessary complexity and risks. I am baffled none of them seem to do that and they need thirdparty certification procedures (who miss RNG flaws anyway.)

PhantomGremlin 9y ago
I'd take the output of getrandom() and XOR it with something like a quantum RNG. E.g. these guys claim their hardware is used by Lotteries and Gaming: http://www.idquantique.com/random-number-generation/quantis-random-number-generator/ http://www.idquantique.com/random-number-generation/quantis-... But, as Agent Mulder knew, "trust no one". So the above is necessary but not sufficient. Above all else, there must be constant vigilance.
- jk2323 9y ago
  "TRUE RANDOM NUMBER GENERATOR EXPLOITING THE RANDOMNESS OF QUANTUM PHYSICS" not sure what they mean by this. Likely it is just "avalanche noise". This is not very difficult. In fact, I build an hardware RNG that is based on avalanche noise (not my development): https://emergent.unpythonic.net/01257868826 https://emergent.unpythonic.net/01257868826
  - PhantomGremlin 9y ago
    Evaluating the veracity of their claims is way above my pay grade. They do have a PDF on their site: http://marketing.idquantique.com/acton/attachment/11868/f-004b/1/-/-/-/-/Quantum%20RNG%20White%20Paper.pdf http://marketing.idquantique.com/acton/attachment/11868/f-00... Basically, emit photons at a semi-transparent mirror. 50% go straight, 50% reflect 90 degrees. Have single-photon detectors at both directions. Quoting from their PDF: Figure 2 shows the block diagram of the Quantis random number generator. It consists of three subsystems. The first one is the core of the generator and contains the optical elements that are used to implement the random process and produce the random outcomes. It comprises a light emitting diode producing the photons, a transmission element, where the random process takes place, and two single-photon detectors – detectors with single-photon resolution – to record the outcomes. The optical subsystem is controlled by a synchronization and acquisition electronic circuit. This subsystem comprises a clock and triggering electronics for the photon source, as well the acquisition electronics for the single-photon detectors. The processing and interfacing subsystem perform statistical and hardware checks, as well as unbiasing of the sequence. Edit: the ID Quantique hardware is a bit pricey for hobbyists, starting at about 1000 euros. If I were building something for myself, I'd look here for inspiration: https://www.fourmilab.ch/hotbits/ https://www.fourmilab.ch/hotbits/ That hardware detects beta decay events. That's preferable to me than "avalanche noise". Not that there's anything fundamentally wrong with thermal noise or shot noise; it's that in real implementations the quality of results is very dependent on things such as quality of power supply. To me it's much simpler to correctly detect intervals between consecutive radioactive decay events. The downside to a cheap decay detector is that the number of events is much lower than what the commercial hardware provides. A few hundred bits per second versus a few million bits per second.

callesgg 9y ago

Is there a video of their talk?

teitelman 9y ago
The video should be up within a few months.

anon4728 9y ago

Sad to see no mention of Voltaire and Condamine, until now, of course.

matt_wulfeck 9y ago

The idea of a public/random mixing source is always fun. Everybody has a fun pet idea but they all suffer from one issue or another. Here's some of mine: 1. Move the mouse around 2. Force someone to dance and read the output from a webcam 3. Record various RF signals 4. Speak a joke into a microphone. The best solution is probably just a plain, non-network computer with an open-hardware TRNG to mix the CSPRNG (I believe BSD is switching to Fortuna). Read from the CSPRNG all day loooong.

teitelman 9y ago

Hey I'm the guy who gave the talk (Evan Teitelman). Let me know if you have any questions.