8 ms·

Gene sequencer malware injection via encoded DNA (2017)

55 points by carlsborg 5y ago

Thorentis 5y ago
I immediately thought of that Bones episode where a computer is set on fire because a bone put into a scanner had some kind of malware embedded in it. The future is now.
Causality1 5y ago
Sanitize. Your. Inputs. No exceptions.
- jjtheblunt 5y ago
  that almost looks like a logical implication: sanitize your inputs ---> no exceptions!
- OminousWeapons 5y ago
  Drop, log, and alert on problematic inputs* Trying to sanitize creates it's own problem.
- tialaramex 5y ago
  The way you get bitten is that there's an opportunity to trade off security versus functionality. Imagine you've got the alleged bomber's computer, it has hundreds of thousands of images with long hexadecimal names like 384d38b0c32aaf03.jpg and you're under pressure to figure out where the missing fifth bomb is before everybody finds out the hard way. Now, on one hand you've got a simple image viewer built with WUFFS: https://github.com/google/wuffs https://github.com/google/wuffs which you are completely sure is safe for viewing the images, you could perhaps glance at a hundred images per hour, although if there's anything subtle you'd miss it. On the other hand, you've got a fancy AI-assisted tool that processes over 1000 images per minute and cross-matches them with over a hundred law enforcement databases and other sources to understand what it sees. It might be able to pick out anomalous images in time to make a difference. Still, it was written in C++ and the last time you used it the program crashed three times with mysterious errors. Which do you choose? If the bomb blows up and kills sixty elementary school kids while you're glancing at image #406 having not noticed that school is in the background of images #284, #301, #305, #313, and #326 and never guessing that the unknown interiors in images #192-198 and #350-400 are from that same school - you're going to feel like an idiot for choosing the slow path. But if the bomb blows up when your AI assisted image processor is exploited by a C++ JPEG library zero day inside image #4861 and that sends the "explode now" code to a custom web site the bomber set up for exactly this eventuality, you're going to feel like an idiot for not using WUFFS. This sort of trade-off is how Vinge's "A Fire Upon the Deep" begins. Humans (roughly) are poking around in an apparently forgotten archive in deep space. They could be slow and careful, but it's too tempting to choose quicker methods, smarter methods, letting the archive direct how to improve their AI to better understand it. By the time the humans realise their AI is doing what the archive wanted it to do, not what they wanted it to do, it's far too late for them.
- nszceta 5y ago
  Parse don't sanitize
  - XorNot 5y ago
    Except parsing is usually a pretty good way to wind up with an exploit in the first place. You need to do both.
    - TeMPOraL 5y ago
      Not if you're actually parsing, with an explicit parse step and a well-defined language. Exploits tend to happen when parsing is done by the so-called "shotgun parser" - aka. various checks and conditionals randomly scattered throughout your code, that implicitly define an input language that's different from what you think it is.
- GuB-42 5y ago
  Considering the input, sure, you need to sanitize, or you might get a virus, literally.
mgarfias 5y ago
Surely there is a Gibson meat space joke here somewhere.
jacquesm 5y ago
Burried quite far into the article: "the researchers admit, they also had to take some serious shortcuts in their proof-of-concept that verge on cheating. Rather than exploit an existing vulnerability in the fqzcomp program, as real-world hackers do, they modified the program's open-source code to insert their own flaw allowing the buffer overflow." I think they should leave out the verge. But it is an interesting thing to think about, that you can encode DNA with the specific intent to cause a buffer overflow in a vulnerable program. But not this one.
- tantalor 5y ago
  Yeah, adding `if(<my-input>) { <bug> }` to the code first is definitely cheating.
  - scintill76 5y ago
    That seems uncharitable; it was not that trivial of a cheat. They inserted a realistic type of vulnerability that was not specific to their test. > we first copied fqzcomp from https://sourceforge.net/projects/fqzcomp/and https://sourceforge.net/projects/fqzcomp/and inserted a vulnerability into version 4.6 of its source code; a function that processes and compresses DNA reads individually, using a fixed-size buffer to store the compressed data. This modification lets us perform a buffer overflow with a longer than expected DNA read in order to hijack control flow. While the use of such a fixed-size buffer is an obvious vulnerability, we note that fqzcomp already contains over two dozen static buffers. > The modified fqzcomp program contained a buffer too small for the 177 base pair read length, so it would overflow after processing the first read. https://www.usenix.org/system/files/conference/usenixsecurity17/sec17-ney.pdf https://www.usenix.org/system/files/conference/usenixsecurit...
    - busymom0 5y ago
      I think your link should be: https://sourceforge.net/projects/fqzcomp/ https://sourceforge.net/projects/fqzcomp/ needs a space before "and".
    - 5y ago
dwheeler 5y ago
I think the HN title should say (2017). It's amazing, but it also shows how poor the defense against buffer overflows is in many programs.
- jacquesm 5y ago
  Except it wasn't.
deleted 5y ago
[deleted]
nahuel0x 5y ago
If the gene sequencer is exploitable by some DNA sequence, maybe is possible for a physical virus to evolve in a way to trigger the bug so his survival probability increases by confusing the scientists looking at them :) Maybe the future will be full of unexpected interplay's between software and the biological world.
- simonh 5y ago
  At first I thought that was silly, the feedback loop from the variation in the virus to this attack having any effect on it's propagation is too slow. But then I realised in the future we might have virus fighting nanomachines with built-in gene sequencers, and suddenly this becomes much more credible.
k__ 5y ago
So, computers can now get viruses from living beings?
- simonh 5y ago
  Sure you could encode the computer virus in the gene sequence as a payload, and use this attack to compromised the sequencer and then load the code into the computer system.
disposekinetics 5y ago
A bug worthy of Peter Watts.
- jacquesm 5y ago
  The bug was inserted by the exploiters, it wasn't present in the code to begin with.
imglorp 5y ago
If you go one level meta, it's not hard to imagine a Ken Thompson "Reflections on Trusting Trust" situation where that DNA codes for some brain malware that will inject more malware whenever its host is at work writing software, such as for a DNA sequencer. https://users.ece.cmu.edu/~ganger/712.fall02/papers/p761-thompson.pdf https://users.ece.cmu.edu/~ganger/712.fall02/papers/p761-tho...
123123as1asd12 5y ago
send it to 23 and me
lmilcin 5y ago
I think mandatory XKCD is in order: https://xkcd.com/327/ https://xkcd.com/327/
- jacquesm 5y ago
  It isn't.
dekhn 5y ago
I helped build a cloud product that parsed BAM and VCF files (Google Cloud Genomics). When we first started I looked around for various open source BAM parsers so we didn't have to write our own. I showed the parsers to my engineer, who came back a day later with 10 files that caused the parsers to crash (common problem: if you put a negative length in a field that's supposed to be positive, it will reverse index into a string). We contacted all the BAM parser developers (academic) and they all said they had no interest in identifying and fixing bugs in their parsers. We wrote an industrial strength BAM parser in Go. I think a year or two later one of the major sequencing centers got hozed by a malicious BAM file and that was when all the open source BAM developers got security religion.
bronzeage 5y ago
I don't think this situation is possible because from the point of view of sequencers, all DNA is equally valid input. Because systems are usually tested in happy flows, vulnerabilities can only happen in cases the input is in some way different from the happy flow, triggering an untested behavior. Because all DNA is equally valid to these tools, it's impossible to have a vulnerability that won't get triggered and patched by the normal testing and developing routine.

55 points by carlsborg 5y ago

I immediately thought of that Bones episode where a computer is set on fire because a bone put into a scanner had some kind of malware embedded in it. The future is now.

Causality1 5y ago

Sanitize. Your. Inputs. No exceptions.

jjtheblunt 5y ago
that almost looks like a logical implication: sanitize your inputs ---> no exceptions!
OminousWeapons 5y ago
Drop, log, and alert on problematic inputs* Trying to sanitize creates it's own problem.
tialaramex 5y ago
The way you get bitten is that there's an opportunity to trade off security versus functionality. Imagine you've got the alleged bomber's computer, it has hundreds of thousands of images with long hexadecimal names like 384d38b0c32aaf03.jpg and you're under pressure to figure out where the missing fifth bomb is before everybody finds out the hard way. Now, on one hand you've got a simple image viewer built with WUFFS: https://github.com/google/wuffs https://github.com/google/wuffs which you are completely sure is safe for viewing the images, you could perhaps glance at a hundred images per hour, although if there's anything subtle you'd miss it. On the other hand, you've got a fancy AI-assisted tool that processes over 1000 images per minute and cross-matches them with over a hundred law enforcement databases and other sources to understand what it sees. It might be able to pick out anomalous images in time to make a difference. Still, it was written in C++ and the last time you used it the program crashed three times with mysterious errors. Which do you choose? If the bomb blows up and kills sixty elementary school kids while you're glancing at image #406 having not noticed that school is in the background of images #284, #301, #305, #313, and #326 and never guessing that the unknown interiors in images #192-198 and #350-400 are from that same school - you're going to feel like an idiot for choosing the slow path. But if the bomb blows up when your AI assisted image processor is exploited by a C++ JPEG library zero day inside image #4861 and that sends the "explode now" code to a custom web site the bomber set up for exactly this eventuality, you're going to feel like an idiot for not using WUFFS. This sort of trade-off is how Vinge's "A Fire Upon the Deep" begins. Humans (roughly) are poking around in an apparently forgotten archive in deep space. They could be slow and careful, but it's too tempting to choose quicker methods, smarter methods, letting the archive direct how to improve their AI to better understand it. By the time the humans realise their AI is doing what the archive wanted it to do, not what they wanted it to do, it's far too late for them.
nszceta 5y ago
Parse don't sanitize
- XorNot 5y ago
  Except parsing is usually a pretty good way to wind up with an exploit in the first place. You need to do both.
  - TeMPOraL 5y ago
    Not if you're actually parsing, with an explicit parse step and a well-defined language. Exploits tend to happen when parsing is done by the so-called "shotgun parser" - aka. various checks and conditionals randomly scattered throughout your code, that implicitly define an input language that's different from what you think it is.
GuB-42 5y ago
Considering the input, sure, you need to sanitize, or you might get a virus, literally.

mgarfias 5y ago

Surely there is a Gibson meat space joke here somewhere.

jacquesm 5y ago

Burried quite far into the article: "the researchers admit, they also had to take some serious shortcuts in their proof-of-concept that verge on cheating. Rather than exploit an existing vulnerability in the fqzcomp program, as real-world hackers do, they modified the program's open-source code to insert their own flaw allowing the buffer overflow." I think they should leave out the verge. But it is an interesting thing to think about, that you can encode DNA with the specific intent to cause a buffer overflow in a vulnerable program. But not this one.

tantalor 5y ago
Yeah, adding `if(<my-input>) { <bug> }` to the code first is definitely cheating.
- scintill76 5y ago
  That seems uncharitable; it was not that trivial of a cheat. They inserted a realistic type of vulnerability that was not specific to their test. > we first copied fqzcomp from https://sourceforge.net/projects/fqzcomp/and https://sourceforge.net/projects/fqzcomp/and inserted a vulnerability into version 4.6 of its source code; a function that processes and compresses DNA reads individually, using a fixed-size buffer to store the compressed data. This modification lets us perform a buffer overflow with a longer than expected DNA read in order to hijack control flow. While the use of such a fixed-size buffer is an obvious vulnerability, we note that fqzcomp already contains over two dozen static buffers. > The modified fqzcomp program contained a buffer too small for the 177 base pair read length, so it would overflow after processing the first read. https://www.usenix.org/system/files/conference/usenixsecurity17/sec17-ney.pdf https://www.usenix.org/system/files/conference/usenixsecurit...
  - busymom0 5y ago
    I think your link should be: https://sourceforge.net/projects/fqzcomp/ https://sourceforge.net/projects/fqzcomp/ needs a space before "and".
  - 5y ago

dwheeler 5y ago

I think the HN title should say (2017). It's amazing, but it also shows how poor the defense against buffer overflows is in many programs.

jacquesm 5y ago
Except it wasn't.

deleted 5y ago

[deleted]

nahuel0x 5y ago

If the gene sequencer is exploitable by some DNA sequence, maybe is possible for a physical virus to evolve in a way to trigger the bug so his survival probability increases by confusing the scientists looking at them :) Maybe the future will be full of unexpected interplay's between software and the biological world.

simonh 5y ago
At first I thought that was silly, the feedback loop from the variation in the virus to this attack having any effect on it's propagation is too slow. But then I realised in the future we might have virus fighting nanomachines with built-in gene sequencers, and suddenly this becomes much more credible.

k__ 5y ago

So, computers can now get viruses from living beings?

simonh 5y ago
Sure you could encode the computer virus in the gene sequence as a payload, and use this attack to compromised the sequencer and then load the code into the computer system.

disposekinetics 5y ago

A bug worthy of Peter Watts.

jacquesm 5y ago
The bug was inserted by the exploiters, it wasn't present in the code to begin with.

imglorp 5y ago

If you go one level meta, it's not hard to imagine a Ken Thompson "Reflections on Trusting Trust" situation where that DNA codes for some brain malware that will inject more malware whenever its host is at work writing software, such as for a DNA sequencer. https://users.ece.cmu.edu/~ganger/712.fall02/papers/p761-thompson.pdf https://users.ece.cmu.edu/~ganger/712.fall02/papers/p761-tho...

123123as1asd12 5y ago

send it to 23 and me

lmilcin 5y ago

I think mandatory XKCD is in order: https://xkcd.com/327/ https://xkcd.com/327/

jacquesm 5y ago
It isn't.

dekhn 5y ago

I helped build a cloud product that parsed BAM and VCF files (Google Cloud Genomics). When we first started I looked around for various open source BAM parsers so we didn't have to write our own. I showed the parsers to my engineer, who came back a day later with 10 files that caused the parsers to crash (common problem: if you put a negative length in a field that's supposed to be positive, it will reverse index into a string). We contacted all the BAM parser developers (academic) and they all said they had no interest in identifying and fixing bugs in their parsers. We wrote an industrial strength BAM parser in Go. I think a year or two later one of the major sequencing centers got hozed by a malicious BAM file and that was when all the open source BAM developers got security religion.

bronzeage 5y ago

I don't think this situation is possible because from the point of view of sequencers, all DNA is equally valid input. Because systems are usually tested in happy flows, vulnerabilities can only happen in cases the input is in some way different from the happy flow, triggering an untested behavior. Because all DNA is equally valid to these tools, it's impossible to have a vulnerability that won't get triggered and patched by the normal testing and developing routine.