9 ms·

"..Bowling said he discovered a way to abuse how ExifTool handles uploads for DjVu file format used for scanned documents to gain control over the entire underl

"..Bowling said he discovered a way to abuse how ExifTool handles uploads for DjVu file format used for scanned documents to gain control over the entire underlying GitLab web server" Ah, the good old "File upload vulnerability". File uploads remain one of the hardest problems to solve when it comes to security.

tata71 5y ago
Is that because people use hackjob dependencies to handle it more often than not?
- djbusby 5y ago
  ExifTool is hackjob? I think not. But also, file uploads should be handled in a jail or box of some type - and never let their analysis make network calls.
  - jeffbee 5y ago
    It's a perl program that evaluates untrusted strings it finds in user files. What exactly is your standard for "is hackjob"? It appears to be a complete piece of shit.
    - rndgermandude 5y ago
      ExifTool is the tool to extract and write and fix metadata, bar none. I haven't seen anything that even comes remotely close at how good it is at handling just jpeg metadata (exif, XMP, IPTC, MarkerNotes, and all the fine bugs every vendor has when creating these), let alone other formats too. ExifTool is essentially for (image) metadata what ffmpeg is for video. It isn't a hack job, but it started out as one, like so very many other things. Version 1.00 was released end of 2003, while the problematic code was added in 2008. Mind you, the problematic code does not just eval whatever it sees, it tries to make sure the input isn't dangerous first. That check failed spectacularly, defeated by a newline combined with how '$' in perl regex works without any special flags[0]. Using eval was a bad choice to begin with (but I am told something you would commonly see in perl software of the time, yes, even in 2008 still), it was the lazy choice of re-using perl to unescape C-strings instead of rolling your own unescaping code. So what do you suggest? Use libexif[1]? Exiv2[2]? Where would I run it? Can you suggest any operating system that never had a "stupid" RCE? >It appears to be a complete piece of shit. Let's see your code then. All the code you ever wrote that is possibly still in use somewhere. So if you never ever fucked up or got lazy, feel free to cast the first stone, otherwise I would suggest you dial down your rhetoric when it comes to taking massive steaming piles on other people's work. Yes, Phil Harvey had a "WTF?!"-class security bug here[4], shit happens, "goto fail", let me deRail your yaml and the Debian random number of the day is: 6. He patched it promptly compared to other vendors and projects (public release on April 13th, while April 7th was the initial bug report, to Gitlab not ExifTool, which Gitlab then passed along[3]). You can blame him for the bug, you can blame Gitlab for not running exiftool in some sandbox. But that half the gitlab instances remain unpatched some 7 months after patches became available, that you'll have to put on the people running these instances. [0] https://github.com/exiftool/exiftool/commit/cf0f4e7dcd024ca99615bfd1102a841a25dde031#diff-fa0d652d10dbcd246e6b1df16c1e992931d3bb717a7e36157596b76bdadb3800 https://github.com/exiftool/exiftool/commit/cf0f4e7dcd024ca9... [1] http://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=libexif http://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=libexif [2] http://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=exiv2 http://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=exiv2 [3] https://hackerone.com/reports/1154542 https://hackerone.com/reports/1154542 [4] I cannot be sure if he wrote it, or if somebody else contributed it, but at the very least he didn't catch it during review. Looks like he wrote it, tho.
    - djbusby 5y ago
      It's clearly not complete shit, else it wouldn't be used by literally millions of people/systems. ExifTool is so far away from shit that in fact it was chosen by a highly respected company with a very good team. A hackjob usually has less deploys than my own stuff (which, outside of Windows 2000 components, is less than a few millions)
      - jeffbee 5y ago
        By this reasoning "Baywatch" was a great TV show because lots of people saw it.
        aurelianito 5y ago
        Baywatch was a great TV show. Maybe you don't like it. Millions all over the world do.
        rsj_hn 5y ago
        I have never seen it, but am aware of the low brow reputation. If you genuinely like it, could you tell me what you like about it? I recently saw some old episodes of Knight Rider and admit that the show was fun.
      - justin_oaks 5y ago
        Just because something is useful doesn't mean it's not a hackjob. Just looks at how PHP got so popular. Clearly it was useful and thus became popular. I think it's hard to argue that it wasn't a hackjob when it first started.
        popcube 5y ago
        JavaScript is a perfect instance
        justin_oaks 5y ago
        Quite so. It was put together in a day, and we have to endure some of its quirks decades later.
  - wswope 5y ago
    I’ve had a really hard time finding good guides for hardening VMs for malware analysis or processing untrusted inputs as you suggest - any guidance on learning resources?
    - djbusby 5y ago
      I learned the hard way. But for these restricted jails I'd start by making a VM that is not allowed out at all, like DROP on iptavles OUTPUT chain.
      - yjftsjthsd-h 5y ago
        At that point, why not just boot the virtual machine with no network interface attached?
        jcpham2 5y ago
        Malware anti detection: some will not launch without a network connection. The idea is to simulate a functional network but not provide access.
        djbusby 5y ago
        Yea, the VM has a network so it looks "real" and I'm using the network interface to push the files to it (via SSH). Iptables let's related. And on the Host I'm also blocking/logging the VM network.
seph-reed 5y ago
> File uploads remain one of the hardest problems to solve when it comes to security. Why? It seems like they should have read/write but no execute. What goes wrong?
- meragrin_ 5y ago
  That doesn't help when the uploaded file is only read/write but crafted in a way to exploit the code processing the file.
- jacquesm 5y ago
  You essentially have a gateway into a very large chunk of code that was most likely not built with security in mind on the parsing side, on top of that you are guaranteed write access to some file system.
  - danudey 5y ago
    Ideally you would sandbox this with: 1. No filesystem access 2. No network access 3. Input passed on stdin (or a pre-opened fd) 4. Output passed to stdout (or a pre-opened fd) 5. A hard timeout specified before the process is killed Suddenly bam, dramatically safer. If you're looking for a tool that can do all of this for you, check out firejail: https://firejail.wordpress.com/ https://firejail.wordpress.com/ It has a ton of options, but you can do all of what I suggested and more, really easily.
    - cookiengineer 5y ago
      Note that firejail had a serious RCE in the way it parses URLs for at least emails. Personally, having read the codebase, I wouldn't put too much faith in its security, given how long it took that the meta character parsing problems were discovered. A user input facing software should always use fuzzing to uncover such bugs.
- staunch 5y ago
  > What goes wrong? Usually what goes wrong is parsing or processing the files. It's hard to get programmers to safely validate a 20 byte email address string. It takes a lot more care to safely parse a 4,000,000 byte image file in a complex format.
ignoramous 5y ago
> uploads for DjVu file format used for scanned documents to gain control over the entire underlying GitLab web server A usecase for WASM's nanoprocesses (capability-based security) perhaps? Of course, until such a time someone exploits the WASM runtime itself.
- mintplant 5y ago
  RLBox is a toolkit for doing just this: https://plsyssec.github.io/rlbox_sandboxing_api/sphinx/ https://plsyssec.github.io/rlbox_sandboxing_api/sphinx/ Used in Firefox to sandbox some libraries, including image handling IIRC.
- dmw_ng 5y ago
  Retrofitting seccomp or a custom Apparmor policy are both much lower hanging fruit. The problem is folk tend to link these things directly into their web servers
- jrockway 5y ago
  I really like the idea of using WASM for application "plugins". You can pick your implementation language, and with the right runtime, it can be speedy and secure. Seems like a win. The blockers, to me, right now are: 1) I mostly write Go, and the Go runtimes didn't seem to be particularly maintained when I last looked. So it just hasn't been worth it to me to do plugins. (I have done "provide your own code to a Go application" before -- "gojq" and "expr" got the job done. Less features than a full WASM runtime, but still pretty powerful.) 2) It's unclear to me which programming language APIs should target. You add a plugin system and you want developers to use it -- what are the popular languages that target WASM? Go and Tinygo look great here, but I have a feeling that the average programmer wants something a little more dynamic for their small plugins. AssemblyScript obviously wants to be the standard, but it's probably too different from Typescript to make it a no-brainer for Javascript developers. Some sort of Perl/Python/Ruby that compiles to WASM would be great, but I haven't seen much progress on that front. As for running untrusted code in general, I don't think WASM needs to block you. gVisor simulates the linux kernel for containers, providing stronger isolation between them, and is designed to protect you from things like this. (I think the original usecase was running ffmpeg to transcode user-provided video files?) And you can always go full VM on these things. Or take the nuclear option -- carefully audit the untrusted code and build up that trust ;)

tata71 5y ago

Is that because people use hackjob dependencies to handle it more often than not?

djbusby 5y ago
ExifTool is hackjob? I think not. But also, file uploads should be handled in a jail or box of some type - and never let their analysis make network calls.
- jeffbee 5y ago
  It's a perl program that evaluates untrusted strings it finds in user files. What exactly is your standard for "is hackjob"? It appears to be a complete piece of shit.
  - rndgermandude 5y ago
    ExifTool is the tool to extract and write and fix metadata, bar none. I haven't seen anything that even comes remotely close at how good it is at handling just jpeg metadata (exif, XMP, IPTC, MarkerNotes, and all the fine bugs every vendor has when creating these), let alone other formats too. ExifTool is essentially for (image) metadata what ffmpeg is for video. It isn't a hack job, but it started out as one, like so very many other things. Version 1.00 was released end of 2003, while the problematic code was added in 2008. Mind you, the problematic code does not just eval whatever it sees, it tries to make sure the input isn't dangerous first. That check failed spectacularly, defeated by a newline combined with how '$' in perl regex works without any special flags[0]. Using eval was a bad choice to begin with (but I am told something you would commonly see in perl software of the time, yes, even in 2008 still), it was the lazy choice of re-using perl to unescape C-strings instead of rolling your own unescaping code. So what do you suggest? Use libexif[1]? Exiv2[2]? Where would I run it? Can you suggest any operating system that never had a "stupid" RCE? >It appears to be a complete piece of shit. Let's see your code then. All the code you ever wrote that is possibly still in use somewhere. So if you never ever fucked up or got lazy, feel free to cast the first stone, otherwise I would suggest you dial down your rhetoric when it comes to taking massive steaming piles on other people's work. Yes, Phil Harvey had a "WTF?!"-class security bug here[4], shit happens, "goto fail", let me deRail your yaml and the Debian random number of the day is: 6. He patched it promptly compared to other vendors and projects (public release on April 13th, while April 7th was the initial bug report, to Gitlab not ExifTool, which Gitlab then passed along[3]). You can blame him for the bug, you can blame Gitlab for not running exiftool in some sandbox. But that half the gitlab instances remain unpatched some 7 months after patches became available, that you'll have to put on the people running these instances. [0] https://github.com/exiftool/exiftool/commit/cf0f4e7dcd024ca99615bfd1102a841a25dde031#diff-fa0d652d10dbcd246e6b1df16c1e992931d3bb717a7e36157596b76bdadb3800 https://github.com/exiftool/exiftool/commit/cf0f4e7dcd024ca9... [1] http://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=libexif http://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=libexif [2] http://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=exiv2 http://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=exiv2 [3] https://hackerone.com/reports/1154542 https://hackerone.com/reports/1154542 [4] I cannot be sure if he wrote it, or if somebody else contributed it, but at the very least he didn't catch it during review. Looks like he wrote it, tho.
  - djbusby 5y ago
    It's clearly not complete shit, else it wouldn't be used by literally millions of people/systems. ExifTool is so far away from shit that in fact it was chosen by a highly respected company with a very good team. A hackjob usually has less deploys than my own stuff (which, outside of Windows 2000 components, is less than a few millions)
    - jeffbee 5y ago
      By this reasoning "Baywatch" was a great TV show because lots of people saw it.
      - aurelianito 5y ago
        Baywatch was a great TV show. Maybe you don't like it. Millions all over the world do.
        rsj_hn 5y ago
        I have never seen it, but am aware of the low brow reputation. If you genuinely like it, could you tell me what you like about it? I recently saw some old episodes of Knight Rider and admit that the show was fun.
    - justin_oaks 5y ago
      Just because something is useful doesn't mean it's not a hackjob. Just looks at how PHP got so popular. Clearly it was useful and thus became popular. I think it's hard to argue that it wasn't a hackjob when it first started.
      - popcube 5y ago
        JavaScript is a perfect instance
        justin_oaks 5y ago
        Quite so. It was put together in a day, and we have to endure some of its quirks decades later.
- wswope 5y ago
  I’ve had a really hard time finding good guides for hardening VMs for malware analysis or processing untrusted inputs as you suggest - any guidance on learning resources?
  - djbusby 5y ago
    I learned the hard way. But for these restricted jails I'd start by making a VM that is not allowed out at all, like DROP on iptavles OUTPUT chain.
    - yjftsjthsd-h 5y ago
      At that point, why not just boot the virtual machine with no network interface attached?
      - jcpham2 5y ago
        Malware anti detection: some will not launch without a network connection. The idea is to simulate a functional network but not provide access.
        djbusby 5y ago
        Yea, the VM has a network so it looks "real" and I'm using the network interface to push the files to it (via SSH). Iptables let's related. And on the Host I'm also blocking/logging the VM network.

seph-reed 5y ago

> File uploads remain one of the hardest problems to solve when it comes to security. Why? It seems like they should have read/write but no execute. What goes wrong?

meragrin_ 5y ago
That doesn't help when the uploaded file is only read/write but crafted in a way to exploit the code processing the file.
jacquesm 5y ago
You essentially have a gateway into a very large chunk of code that was most likely not built with security in mind on the parsing side, on top of that you are guaranteed write access to some file system.
- danudey 5y ago
  Ideally you would sandbox this with: 1. No filesystem access 2. No network access 3. Input passed on stdin (or a pre-opened fd) 4. Output passed to stdout (or a pre-opened fd) 5. A hard timeout specified before the process is killed Suddenly bam, dramatically safer. If you're looking for a tool that can do all of this for you, check out firejail: https://firejail.wordpress.com/ https://firejail.wordpress.com/ It has a ton of options, but you can do all of what I suggested and more, really easily.
  - cookiengineer 5y ago
    Note that firejail had a serious RCE in the way it parses URLs for at least emails. Personally, having read the codebase, I wouldn't put too much faith in its security, given how long it took that the meta character parsing problems were discovered. A user input facing software should always use fuzzing to uncover such bugs.
staunch 5y ago
> What goes wrong? Usually what goes wrong is parsing or processing the files. It's hard to get programmers to safely validate a 20 byte email address string. It takes a lot more care to safely parse a 4,000,000 byte image file in a complex format.

ignoramous 5y ago

> uploads for DjVu file format used for scanned documents to gain control over the entire underlying GitLab web server A usecase for WASM's nanoprocesses (capability-based security) perhaps? Of course, until such a time someone exploits the WASM runtime itself.

mintplant 5y ago
RLBox is a toolkit for doing just this: https://plsyssec.github.io/rlbox_sandboxing_api/sphinx/ https://plsyssec.github.io/rlbox_sandboxing_api/sphinx/ Used in Firefox to sandbox some libraries, including image handling IIRC.
dmw_ng 5y ago
Retrofitting seccomp or a custom Apparmor policy are both much lower hanging fruit. The problem is folk tend to link these things directly into their web servers
jrockway 5y ago
I really like the idea of using WASM for application "plugins". You can pick your implementation language, and with the right runtime, it can be speedy and secure. Seems like a win. The blockers, to me, right now are: 1) I mostly write Go, and the Go runtimes didn't seem to be particularly maintained when I last looked. So it just hasn't been worth it to me to do plugins. (I have done "provide your own code to a Go application" before -- "gojq" and "expr" got the job done. Less features than a full WASM runtime, but still pretty powerful.) 2) It's unclear to me which programming language APIs should target. You add a plugin system and you want developers to use it -- what are the popular languages that target WASM? Go and Tinygo look great here, but I have a feeling that the average programmer wants something a little more dynamic for their small plugins. AssemblyScript obviously wants to be the standard, but it's probably too different from Typescript to make it a no-brainer for Javascript developers. Some sort of Perl/Python/Ruby that compiles to WASM would be great, but I haven't seen much progress on that front. As for running untrusted code in general, I don't think WASM needs to block you. gVisor simulates the linux kernel for containers, providing stronger isolation between them, and is designed to protect you from things like this. (I think the original usecase was running ffmpeg to transcode user-provided video files?) And you can always go full VM on these things. Or take the nuclear option -- carefully audit the untrusted code and build up that trust ;)