10 ms·

Backdoored images downloaded 5M times removed from Docker Hub

283 points by cel1ne 8y ago

jchw 8y ago
The worst part here is definitely the timeline. NPM is often criticized about security, perhaps rightfully so, but at least the issues are handled promptly after raised publicly.
- paulie_a 8y ago
  There is no "perhaps" about npm, it is absolutely a shit show. It's like the creators went out of their way to build an ecosystem of security nightmares.
  - outside1234 8y ago
    ok, hang on, how is npm different from maven, pypi, etc. ?
    - oneweekwonder 8y ago
      I will take a stap at comparing npm vs pypi. While this will be anecdotal I found my `pip freeze` packages a lot more manageable compared to `npm ls --parseable`. My Full Stack Flask application sits at about 64 requirements. Where last time I used expressjs my dependancies inside of node_modules inside of node_modules border-lined to insanity. I can see myself hand picking and reviewing my requirements.txt, which I did to some extent. But I just gave up with npm. Which is a bit of a personal dilemma for me, because some of my tooling needs npm.
      - ehnto 8y ago
        So many packages and authors creates an impossibly large surface area to review and secure. It is my impression that most people don't even try so the issue falls on deaf ears. But for a PCI compliant piece of software for example, you would have to had reviewed every module in node_modules. As a stack it makes it a non-starter.
      - cup-of-tea 8y ago
        I've been doing Python for over five years now. I recently installed a small command line tool using npm. I was completely stunned by the number of dependencies. At first I thought it must be something else, like maybe it's running tests? But no, hundreds of dependencies. If it were written in python it probably wouldn't even have one.
      - eeZah7Ux 8y ago
        In Python, 64 dependencies is quite a lot. Most services can be build with less that 5 or 10 well-known, trusted libraries. The numbers and ecosystem maturity make a huge difference in being able to vet dependencies.
      - inapis 8y ago
        You’re not OP but you’ve move the goal posts from security to number of dependencies. I think GP was focusing on how are other package managers different from npm when it comes to security except maybe apt and pacman? And node_modules is now a flat tree for the most part. The number of dependencies a basic JavaScript project pulls is definitely something to be concerned about though.
    - earenndil 8y ago
      It's not a technical issue, it's a cultural one.
      - paulie_a 8y ago
        It is both
    - paulie_a 8y ago
      Because I need 90,000 stupid packages on npm which can be disabled and removed at any moment. Vs python which mostly requires 20-100 if you are really making a complex system. Also the versioning on npm is incredibly pathetic vs pypi. I would never trust npm for anything serious or waste my time debugging that crap.
      - overkalix 8y ago
        Not a webdev, but from what I've seen webdevs are forced to compete, not in what they can do, but in terms of how they can do it (frameworks, packages). So, an aspect of your work process becomes a proxy for the quality of your work. In a context of permanent competition, this forces an acritical and rapid adoption of frameworks that perhaps only marginally improve some aspects of some other frameworks. Then compound this with the inflow of bootcamp-trained webdevs with poor practices whose employability relies on having being recently trained on the latest hyped framework.
        heavenlyblue 8y ago
        No, JavaScript just doesn't have a standard library. And even if you think it does - then look at what Py std has to offer.
  - jchw 8y ago
    I'm not making claims either way. I don't like NPM either, it just wasn't relevant to what I was saying.
LeoPanthera 8y ago
This bug: https://github.com/docker/hub-feedback/issues/1121 https://github.com/docker/hub-feedback/issues/1121 raised over a year ago(!) is really interesting. It seems like many of the downloads may have been malicious - the author of the malicious images was scanning for open docker api ports and then installing their own images to mine cryptocurrency. So they're essentially using docker as a dropper. Clever, in a way.
outside1234 8y ago
where are the images enumerated?
- rafaele 8y ago
  https://kromtech.com/blog/security-center/cryptojacking-invades-cloud-how-modern-containerization-trend-is-exploited-by-attackers https://kromtech.com/blog/security-center/cryptojacking-inva... search for "Figure 7"
cyphar 8y ago
It should be noted that some of the reports talk about the Docker API being publicly accessible over the internet which allowed people to run containers on their machines. This is actually not the worst thing that could have happened -- having access to the Docker API gives you root access on that machine without any authentication! (One of the ideas of rootless containers is to remove the possibility of any privileged codepath, which helps eliminate this issue.)
- avip 8y ago
  Docker api exposed over internet without TLS implies a head should be removed. That’s not the default config. Not what the docs recommend. Why??
  - djsumdog 8y ago
    I don't think that's even possible. Docker doesn't let you expose the daemon over HTTP without configuring certs. I had to write an ansible script to do that, and even then I locked down my Docker port to my VPN subnet: https://github.com/sumdog/bee2/blob/master/ansible/roles/docker/tasks/servertls.yml https://github.com/sumdog/bee2/blob/master/ansible/roles/doc...
    - toopsss 8y ago
      What the heck are you talking about? If dockerd is started in tcp mode, it is unencrypted and unauthenticated by default.
    - cyphar 8y ago
      sudo dockerd -H tcp://0.0.0.0:8080 will happily start Docker with it listening on my IP address without TLS. It will print an all-caps warning, but nothing else (you don't even need to pass a --give-the-internet-root-access flag). However, I just submitted a PR which adds the --give-the-internet-root-access flag[1] because it's pretty obvious to me that very few users do this intentionally (and with full knowledge of the consequences). [1]: https://github.com/moby/moby/pull/37299 https://github.com/moby/moby/pull/37299
pavbelshippable 8y ago
This is what we call security issues? Cloud is such a thing that anything can happen. That's the reason we are talking about shifting security to the left as much as possible, whether it is DevOps, Cryptocurrency or anything other.
etaioinshrdlu 8y ago
A startup I'm aware of (not associated with) that aims to help tame this problem a bit: https://anchore.io/ https://anchore.io/
bboreham 8y ago
In what sense is this a “backdoor”? Seems to me the code is coming through the front door, which the victims left open. DockerHub is just the delivery mechanism.
- imtringued 8y ago
  I'm scratching my head at where the /mnt mount is coming from. If you're doing "docker run -v /:/mnt <sketchy_username>/mysql" then absolutely nobody can help you.
  - b6z 8y ago
    Same for me. From Kromtech's article I deduced that this only happens when a docker daemon (or kubernetes interface) is exposed to the Internet and an attacker uses that to download and start a docker image on the victim's host. Then they can bind mount a host directory like described and attack the host computer.
fpgaminer 8y ago
Somewhat related, since this is about Docker security: I started looking at Traefik today. It's a reverse proxy that runs as a Docker container and automagically configures itself to expose your other services (that are also running in Docker containers). Neat idea. However, to accomplish this you have to mount the docker socket into Traefik's container... Which means that when a bug shows up in Traefik attackers can pivot out of the container and onto the host; access to the docker socket is equivalent to root on the host. And of course Traefik is the thing you're exposing directly to the internet. It's like giving the guards outside manning your castle's gate the skeleton key to the rest of the castle. Of course, Traefik is quickly becoming popular because of its simplicity. But to achieve this simplicity it carves a giant hole in the security of your application.
- gnur 8y ago
  The integration of Traefik with the Docker daemon should mainly just be used while developing (imho). Once you get to acceptance / production environments, you are very unlikely to run plain docker containers, if you use kubernetes you interface Traefik with the kubernetes api itself, and the service account you create for Traefik can be (and should be) completely read only. Same for Docker Swarm, Marathon, Consul and AWS ECS. So no, Traefik is not the big security problem you make it out to be. Sorry to be so harsh, but Traefik is one of the most amazin pieces of software I have come across in the last years that has seriously made my life much easier.
  - jakobegger 8y ago
    If software has an insecure mode "just for development" that absolutely shouldn't be used in production, you can be certain that a large fraction of developers will use that in production nevertheless. Security today doesn't mean that you are safe if you do everything according to best practices and follow the docs. Modern Security includes making sure that default settings are safe, and that it should be impossible or hard to set up the software in an insecure manner. If you make it easy to shoot yourself in the foot, that's what people will do.
    - 8y ago
yani 8y ago
This is not a backdoor. I myself have a miner on Docker hub. The image can be used by anyone with correct envars set. Should my image be removed if used by other users no matter what their intensions are?
- sleepychu 8y ago
  You are being facetious. If your image is called monero-miner and a bunch of people download it, of course it's not going to be considered malicious code. If your image is called apache-webserver and a bunch of people download it and you've stealth bundled a monero miner, of course it's going to be considered malicious code. EDIT: even worse than that, the images are actually back doored they open up a reverse shell to allow the remote to execute arbitrary commands.
- inapis 8y ago
  This particular case is definitely a backdoor and malicious. The images were pretending to by mysql, mssql, Apache etc.
TekMol 8y ago
I wonder how much malicious code like this is doing its work deep down in the endless pyramid of npm dependencies. And how much as-of-now clean code will turn into malicious code when bad guys take over npm repos in the future. It might be possible to tackle this issue by some intelligent trust algo that combines a trust rank similar to google-page-rank and signed messages. Say somebody pushes an update to their repo. Now the first user of it might read it and sign it with 'Looks OK /Joe'. And the next user sees the signed message by Joe in some kind of package-review-message list. Based on all the reviews and the trust of the reviewers, they then can calculate a trust score for the update.
- onion2k 8y ago
  ...or CPAN modules, composer libs, cocoa pods, etc. Anything you use to install unchecked external code is potentially dangerous. At least npm has auditing now, so checking for problems is relatively trivial. GitHub even does it automatically.
  - dvfjsdhgfv 8y ago
    Well, it's much, much easier for me to audit a CPAN module than a Docker image - the latter is practically impossible.
    - raesene9 8y ago
      eh? Assuming automated builds, you can just read the dockerfile (and it's hierarchy if necessary), it's less complex syntax than perl by a long way. Even assuming no automated build, all the information is in the manifest, and using something like portainer it's pretty easy to read.
      - palotasb 8y ago
        Don't a lot of OS images start by importing a non-transparent prebuilt tarball containing nontrivial binaries? I would hide the malware inside those.
crypt1d 8y ago
>By the time Docker Hub removed the images, they had received 5 million “pulls.” A wallet address included in many of the submissions showed it had mined almost 545 Monero digital coins, worth almost $90,000. This seems incorrect because its impossible to see wallet balances on the Monero network. So I'm assuming they just came up with the numbers based on some rough calculations.
- deleted 8y ago
  [deleted]
- ricANNArdo 8y ago
  If you go to the pool website with the address specified on the botnet you can see how much it was mined. The main article [1] linked on the news said: > The actor has been able to mine about 630 XMR to date, which at the current USD rate is more than $172,000 for just a little more than one year of activity. [1]: https://www.fortinet.com/blog/threat-research/yet-another-crypto-mining-botnet.html https://www.fortinet.com/blog/threat-research/yet-another-cr...
  - crypt1d 8y ago
    that makes more sense, thanks.
INTPenis 8y ago
This is exactly why I never liked Ansible Galaxy, and Docker Hub came into the same category. Screw the extra work, I'd rather write my own roles and Dockerfiles.
- pmlnr 8y ago
  Most of these things are plain text instruction files - yaml for ansible, docker's own thing for docker. It falls under the same category as random bash install scripts: download the text file, read it, use it, if it's safe.
  - INTPenis 8y ago
    Yes read it, use it but the next step can't be update it because then you'd have to read it again. I just don't have the time to audit someone elses yaml constantly.
- geerlingguy 8y ago
  Better yet, fork it. Most roles and Docker Hub images are pretty simple, and you should be evaluating them anyways before using them. If you’re concerned about the security but want to save the time in building and debugging, fork it, and maintain your fork, only pulling in changes from the upstream when you have time to vet them.
ccnafr 8y ago
ORiginal report: https://kromtech.com/blog/security-center/cryptojacking-invades-cloud-how-modern-containerization-trend-is-exploited-by-attackers https://kromtech.com/blog/security-center/cryptojacking-inva... The ArsTechnica article, like most AT articles, glosses over most details and focuses on a small-time cryptomining campaign
raesene9 8y ago
This is essentially a dupe of https://news.ycombinator.com/item?id=17303570 https://news.ycombinator.com/item?id=17303570 FWIW that headline isn't great. Docker hub pulls in no way correlate to innocent users pulling/using those images. It could be (and this is quite likely) just other malware which made use of those images and just used Docker hub as a repository. There are official images for the software in question and I don't think it's that likely that that many people ignored the official ones and got these ones.
zimmerfrei 8y ago
On the same topic, PyPI has recently moved to a new backend, and in the process all end-to-end PGP signatures (created by the package owner upstream, proving that no tampering happened on the online servers) have disappeared from the UI, and that is seen as a "feature": https://github.com/pypa/warehouse/issues/3356 https://github.com/pypa/warehouse/issues/3356 You can still get them through some obscure API and you still need to know the right PGP key for verification, but this really signals the lack of consensus and awareness on the path toward a secure software supply chain. EDIT: typos
- llampx 8y ago
  It is the appification of software development.
- egjerlow 8y ago
  FWIW, here is a blog post by dstufft which might help contextualize this behaviour: https://caremad.io/posts/2013/07/packaging-signing-not-holy-grail/ https://caremad.io/posts/2013/07/packaging-signing-not-holy-...
  - msl09 8y ago
    That is discussed extensively in the issues related to the OP. The problem is that package maintainers of distros actually check whether the GPG signature has changed in order to repackaged python projects for their distros.
    - x1798DE 8y ago
      They can still do that, it's just not exposed in the UI anymore.
      - msl09 8y ago
        I have tried checking the REST API[1] but I only found a has_sig parameter. Where is the actual signature? https://warehouse.readthedocs.io/api-reference/json/ https://warehouse.readthedocs.io/api-reference/json/
ex_amazon_sde 8y ago
For those who wonder why Linux distributions are "still" around, this is a reason. Some have a good vetting process for packages.
- HankB99 8y ago
  I wonder which ones have the best vetting. And if it is adequate. I also wonder about other packaging systems. CPAN, pip (pypy?) AUR and so on. It doesn't surprise me to see this happen. I wonder what other surprises might be in any of these packages. FWIW, I'm running mostly Debian and some Ubuntu. I always prefer to install packages via the package manager rather than directly from some tool specific repository because I'll get automatic updates and some level of testing/vetting.
meuk 8y ago
“For ordinary users, just pulling a Docker image from Docker Hub is like pulling arbitrary binary data from somewhere, executing it, and hoping for the best without really knowing what’s in it,” This is basically what you do every time you install something (except when it's via a walled garden like an 'app store'). Besides, I'm not sure I would even classify mining for someone else as 'malicious'. It hogs your CPU a little, but if that's malicious then visual studio should be considered malicious as well.
- w8rbt 8y ago
  Or when you visit web pages that run JS in your browser on your machine. No one seems to mind that, but they should.
- laumars 8y ago
  Maybe it's not malicious in the strictest sense of the term but it's also not the same as your Visual Studio example. In your case Visual Studio is a productivity tool that brings you value and thus you chose to install it. In the case of this docker image it's not adding you value and was installed without your concent. The JS example another commenter made is more apt however the argument there is that you still requested the site and it's content (even if you didn't really want it). Whereas many of the installs of this "dockerised" miner were remotely via exposed Docker APIs. That I think is the real crux of the potential "malice" (for want a better description) here.
ataturk 8y ago
I've been concerned about this very thing for quite awhile now. It's too easy of an attack vector. I first noticed the possibility when tools like VMWare came out on Windows and developers started playing around with images of Linux on their computers. It would be pretty easy to compromise an image, get it loaded on one of the main distribution sites and then as soon as someone starts one up inside a corporate network, you're in!
djsumdog 8y ago
I don't understand why people use other people's Docker images. Unless it comes from an official repository for the tool you're using, it's better to look at the source code/Dockerfile in the github link and just roll your own. A lot of times you're just installing the package you want with apt-get within your Dockerfile anyway; a package you can't check for normal updates for anymore since it's in a container. So now you need a tooling system around making sure your packages in your containers don't have security issues. Docker is kinda a mess.
- y4mi 8y ago
  its not really a mess for its usecase. its immutable infrastructure at its heart, so yeah, you don't do updates on containers... what you do need is periodic rebuild of your images for upgrades and each new image needs to run all integration and system tests again. it just makes this process easier than it is without docker. But it doesnt alleviate you of writing the system that actually keep everything updated in an automated way. It also doesnt help you deploy unless you're already experienced with docker. and while we'Re on the topic... no, if you know how to execute 'docker run -it --rm ubuntu bash' you still don't know shit about it. sigh sorry, i'm just remembering someone from work today...

283 points by cel1ne 8y ago

jchw 8y ago

The worst part here is definitely the timeline. NPM is often criticized about security, perhaps rightfully so, but at least the issues are handled promptly after raised publicly.

paulie_a 8y ago
There is no "perhaps" about npm, it is absolutely a shit show. It's like the creators went out of their way to build an ecosystem of security nightmares.
- outside1234 8y ago
  ok, hang on, how is npm different from maven, pypi, etc. ?
  - oneweekwonder 8y ago
    I will take a stap at comparing npm vs pypi. While this will be anecdotal I found my `pip freeze` packages a lot more manageable compared to `npm ls --parseable`. My Full Stack Flask application sits at about 64 requirements. Where last time I used expressjs my dependancies inside of node_modules inside of node_modules border-lined to insanity. I can see myself hand picking and reviewing my requirements.txt, which I did to some extent. But I just gave up with npm. Which is a bit of a personal dilemma for me, because some of my tooling needs npm.
    - ehnto 8y ago
      So many packages and authors creates an impossibly large surface area to review and secure. It is my impression that most people don't even try so the issue falls on deaf ears. But for a PCI compliant piece of software for example, you would have to had reviewed every module in node_modules. As a stack it makes it a non-starter.
    - cup-of-tea 8y ago
      I've been doing Python for over five years now. I recently installed a small command line tool using npm. I was completely stunned by the number of dependencies. At first I thought it must be something else, like maybe it's running tests? But no, hundreds of dependencies. If it were written in python it probably wouldn't even have one.
    - eeZah7Ux 8y ago
      In Python, 64 dependencies is quite a lot. Most services can be build with less that 5 or 10 well-known, trusted libraries. The numbers and ecosystem maturity make a huge difference in being able to vet dependencies.
    - inapis 8y ago
      You’re not OP but you’ve move the goal posts from security to number of dependencies. I think GP was focusing on how are other package managers different from npm when it comes to security except maybe apt and pacman? And node_modules is now a flat tree for the most part. The number of dependencies a basic JavaScript project pulls is definitely something to be concerned about though.
  - earenndil 8y ago
    It's not a technical issue, it's a cultural one.
    - paulie_a 8y ago
      It is both
  - paulie_a 8y ago
    Because I need 90,000 stupid packages on npm which can be disabled and removed at any moment. Vs python which mostly requires 20-100 if you are really making a complex system. Also the versioning on npm is incredibly pathetic vs pypi. I would never trust npm for anything serious or waste my time debugging that crap.
    - overkalix 8y ago
      Not a webdev, but from what I've seen webdevs are forced to compete, not in what they can do, but in terms of how they can do it (frameworks, packages). So, an aspect of your work process becomes a proxy for the quality of your work. In a context of permanent competition, this forces an acritical and rapid adoption of frameworks that perhaps only marginally improve some aspects of some other frameworks. Then compound this with the inflow of bootcamp-trained webdevs with poor practices whose employability relies on having being recently trained on the latest hyped framework.
      - heavenlyblue 8y ago
        No, JavaScript just doesn't have a standard library. And even if you think it does - then look at what Py std has to offer.
- jchw 8y ago
  I'm not making claims either way. I don't like NPM either, it just wasn't relevant to what I was saying.

LeoPanthera 8y ago

This bug: https://github.com/docker/hub-feedback/issues/1121 https://github.com/docker/hub-feedback/issues/1121 raised over a year ago(!) is really interesting. It seems like many of the downloads may have been malicious - the author of the malicious images was scanning for open docker api ports and then installing their own images to mine cryptocurrency. So they're essentially using docker as a dropper. Clever, in a way.

outside1234 8y ago

where are the images enumerated?

rafaele 8y ago
https://kromtech.com/blog/security-center/cryptojacking-invades-cloud-how-modern-containerization-trend-is-exploited-by-attackers https://kromtech.com/blog/security-center/cryptojacking-inva... search for "Figure 7"

cyphar 8y ago

It should be noted that some of the reports talk about the Docker API being publicly accessible over the internet which allowed people to run containers on their machines. This is actually not the worst thing that could have happened -- having access to the Docker API gives you root access on that machine without any authentication! (One of the ideas of rootless containers is to remove the possibility of any privileged codepath, which helps eliminate this issue.)

avip 8y ago
Docker api exposed over internet without TLS implies a head should be removed. That’s not the default config. Not what the docs recommend. Why??
- djsumdog 8y ago
  I don't think that's even possible. Docker doesn't let you expose the daemon over HTTP without configuring certs. I had to write an ansible script to do that, and even then I locked down my Docker port to my VPN subnet: https://github.com/sumdog/bee2/blob/master/ansible/roles/docker/tasks/servertls.yml https://github.com/sumdog/bee2/blob/master/ansible/roles/doc...
  - toopsss 8y ago
    What the heck are you talking about? If dockerd is started in tcp mode, it is unencrypted and unauthenticated by default.
  - cyphar 8y ago
    sudo dockerd -H tcp://0.0.0.0:8080 will happily start Docker with it listening on my IP address without TLS. It will print an all-caps warning, but nothing else (you don't even need to pass a --give-the-internet-root-access flag). However, I just submitted a PR which adds the --give-the-internet-root-access flag[1] because it's pretty obvious to me that very few users do this intentionally (and with full knowledge of the consequences). [1]: https://github.com/moby/moby/pull/37299 https://github.com/moby/moby/pull/37299

pavbelshippable 8y ago

This is what we call security issues? Cloud is such a thing that anything can happen. That's the reason we are talking about shifting security to the left as much as possible, whether it is DevOps, Cryptocurrency or anything other.

etaioinshrdlu 8y ago

A startup I'm aware of (not associated with) that aims to help tame this problem a bit: https://anchore.io/ https://anchore.io/

bboreham 8y ago

In what sense is this a “backdoor”? Seems to me the code is coming through the front door, which the victims left open. DockerHub is just the delivery mechanism.

imtringued 8y ago
I'm scratching my head at where the /mnt mount is coming from. If you're doing "docker run -v /:/mnt <sketchy_username>/mysql" then absolutely nobody can help you.
- b6z 8y ago
  Same for me. From Kromtech's article I deduced that this only happens when a docker daemon (or kubernetes interface) is exposed to the Internet and an attacker uses that to download and start a docker image on the victim's host. Then they can bind mount a host directory like described and attack the host computer.

fpgaminer 8y ago

Somewhat related, since this is about Docker security: I started looking at Traefik today. It's a reverse proxy that runs as a Docker container and automagically configures itself to expose your other services (that are also running in Docker containers). Neat idea. However, to accomplish this you have to mount the docker socket into Traefik's container... Which means that when a bug shows up in Traefik attackers can pivot out of the container and onto the host; access to the docker socket is equivalent to root on the host. And of course Traefik is the thing you're exposing directly to the internet. It's like giving the guards outside manning your castle's gate the skeleton key to the rest of the castle. Of course, Traefik is quickly becoming popular because of its simplicity. But to achieve this simplicity it carves a giant hole in the security of your application.

gnur 8y ago
The integration of Traefik with the Docker daemon should mainly just be used while developing (imho). Once you get to acceptance / production environments, you are very unlikely to run plain docker containers, if you use kubernetes you interface Traefik with the kubernetes api itself, and the service account you create for Traefik can be (and should be) completely read only. Same for Docker Swarm, Marathon, Consul and AWS ECS. So no, Traefik is not the big security problem you make it out to be. Sorry to be so harsh, but Traefik is one of the most amazin pieces of software I have come across in the last years that has seriously made my life much easier.
- jakobegger 8y ago
  If software has an insecure mode "just for development" that absolutely shouldn't be used in production, you can be certain that a large fraction of developers will use that in production nevertheless. Security today doesn't mean that you are safe if you do everything according to best practices and follow the docs. Modern Security includes making sure that default settings are safe, and that it should be impossible or hard to set up the software in an insecure manner. If you make it easy to shoot yourself in the foot, that's what people will do.
  - 8y ago

yani 8y ago

This is not a backdoor. I myself have a miner on Docker hub. The image can be used by anyone with correct envars set. Should my image be removed if used by other users no matter what their intensions are?

sleepychu 8y ago
You are being facetious. If your image is called monero-miner and a bunch of people download it, of course it's not going to be considered malicious code. If your image is called apache-webserver and a bunch of people download it and you've stealth bundled a monero miner, of course it's going to be considered malicious code. EDIT: even worse than that, the images are actually back doored they open up a reverse shell to allow the remote to execute arbitrary commands.
inapis 8y ago
This particular case is definitely a backdoor and malicious. The images were pretending to by mysql, mssql, Apache etc.

TekMol 8y ago

I wonder how much malicious code like this is doing its work deep down in the endless pyramid of npm dependencies. And how much as-of-now clean code will turn into malicious code when bad guys take over npm repos in the future. It might be possible to tackle this issue by some intelligent trust algo that combines a trust rank similar to google-page-rank and signed messages. Say somebody pushes an update to their repo. Now the first user of it might read it and sign it with 'Looks OK /Joe'. And the next user sees the signed message by Joe in some kind of package-review-message list. Based on all the reviews and the trust of the reviewers, they then can calculate a trust score for the update.

onion2k 8y ago
...or CPAN modules, composer libs, cocoa pods, etc. Anything you use to install unchecked external code is potentially dangerous. At least npm has auditing now, so checking for problems is relatively trivial. GitHub even does it automatically.
- dvfjsdhgfv 8y ago
  Well, it's much, much easier for me to audit a CPAN module than a Docker image - the latter is practically impossible.
  - raesene9 8y ago
    eh? Assuming automated builds, you can just read the dockerfile (and it's hierarchy if necessary), it's less complex syntax than perl by a long way. Even assuming no automated build, all the information is in the manifest, and using something like portainer it's pretty easy to read.
    - palotasb 8y ago
      Don't a lot of OS images start by importing a non-transparent prebuilt tarball containing nontrivial binaries? I would hide the malware inside those.

crypt1d 8y ago

>By the time Docker Hub removed the images, they had received 5 million “pulls.” A wallet address included in many of the submissions showed it had mined almost 545 Monero digital coins, worth almost $90,000. This seems incorrect because its impossible to see wallet balances on the Monero network. So I'm assuming they just came up with the numbers based on some rough calculations.

deleted 8y ago
[deleted]
ricANNArdo 8y ago
If you go to the pool website with the address specified on the botnet you can see how much it was mined. The main article [1] linked on the news said: > The actor has been able to mine about 630 XMR to date, which at the current USD rate is more than $172,000 for just a little more than one year of activity. [1]: https://www.fortinet.com/blog/threat-research/yet-another-crypto-mining-botnet.html https://www.fortinet.com/blog/threat-research/yet-another-cr...
- crypt1d 8y ago
  that makes more sense, thanks.

INTPenis 8y ago

This is exactly why I never liked Ansible Galaxy, and Docker Hub came into the same category. Screw the extra work, I'd rather write my own roles and Dockerfiles.

pmlnr 8y ago
Most of these things are plain text instruction files - yaml for ansible, docker's own thing for docker. It falls under the same category as random bash install scripts: download the text file, read it, use it, if it's safe.
- INTPenis 8y ago
  Yes read it, use it but the next step can't be update it because then you'd have to read it again. I just don't have the time to audit someone elses yaml constantly.
geerlingguy 8y ago
Better yet, fork it. Most roles and Docker Hub images are pretty simple, and you should be evaluating them anyways before using them. If you’re concerned about the security but want to save the time in building and debugging, fork it, and maintain your fork, only pulling in changes from the upstream when you have time to vet them.

ccnafr 8y ago

ORiginal report: https://kromtech.com/blog/security-center/cryptojacking-invades-cloud-how-modern-containerization-trend-is-exploited-by-attackers https://kromtech.com/blog/security-center/cryptojacking-inva... The ArsTechnica article, like most AT articles, glosses over most details and focuses on a small-time cryptomining campaign

raesene9 8y ago

This is essentially a dupe of https://news.ycombinator.com/item?id=17303570 https://news.ycombinator.com/item?id=17303570 FWIW that headline isn't great. Docker hub pulls in no way correlate to innocent users pulling/using those images. It could be (and this is quite likely) just other malware which made use of those images and just used Docker hub as a repository. There are official images for the software in question and I don't think it's that likely that that many people ignored the official ones and got these ones.

zimmerfrei 8y ago

On the same topic, PyPI has recently moved to a new backend, and in the process all end-to-end PGP signatures (created by the package owner upstream, proving that no tampering happened on the online servers) have disappeared from the UI, and that is seen as a "feature": https://github.com/pypa/warehouse/issues/3356 https://github.com/pypa/warehouse/issues/3356 You can still get them through some obscure API and you still need to know the right PGP key for verification, but this really signals the lack of consensus and awareness on the path toward a secure software supply chain. EDIT: typos

llampx 8y ago
It is the appification of software development.
egjerlow 8y ago
FWIW, here is a blog post by dstufft which might help contextualize this behaviour: https://caremad.io/posts/2013/07/packaging-signing-not-holy-grail/ https://caremad.io/posts/2013/07/packaging-signing-not-holy-...
- msl09 8y ago
  That is discussed extensively in the issues related to the OP. The problem is that package maintainers of distros actually check whether the GPG signature has changed in order to repackaged python projects for their distros.
  - x1798DE 8y ago
    They can still do that, it's just not exposed in the UI anymore.
    - msl09 8y ago
      I have tried checking the REST API[1] but I only found a has_sig parameter. Where is the actual signature? https://warehouse.readthedocs.io/api-reference/json/ https://warehouse.readthedocs.io/api-reference/json/

ex_amazon_sde 8y ago

For those who wonder why Linux distributions are "still" around, this is a reason. Some have a good vetting process for packages.

HankB99 8y ago
I wonder which ones have the best vetting. And if it is adequate. I also wonder about other packaging systems. CPAN, pip (pypy?) AUR and so on. It doesn't surprise me to see this happen. I wonder what other surprises might be in any of these packages. FWIW, I'm running mostly Debian and some Ubuntu. I always prefer to install packages via the package manager rather than directly from some tool specific repository because I'll get automatic updates and some level of testing/vetting.

meuk 8y ago

“For ordinary users, just pulling a Docker image from Docker Hub is like pulling arbitrary binary data from somewhere, executing it, and hoping for the best without really knowing what’s in it,” This is basically what you do every time you install something (except when it's via a walled garden like an 'app store'). Besides, I'm not sure I would even classify mining for someone else as 'malicious'. It hogs your CPU a little, but if that's malicious then visual studio should be considered malicious as well.

w8rbt 8y ago
Or when you visit web pages that run JS in your browser on your machine. No one seems to mind that, but they should.
laumars 8y ago
Maybe it's not malicious in the strictest sense of the term but it's also not the same as your Visual Studio example. In your case Visual Studio is a productivity tool that brings you value and thus you chose to install it. In the case of this docker image it's not adding you value and was installed without your concent. The JS example another commenter made is more apt however the argument there is that you still requested the site and it's content (even if you didn't really want it). Whereas many of the installs of this "dockerised" miner were remotely via exposed Docker APIs. That I think is the real crux of the potential "malice" (for want a better description) here.

ataturk 8y ago

I've been concerned about this very thing for quite awhile now. It's too easy of an attack vector. I first noticed the possibility when tools like VMWare came out on Windows and developers started playing around with images of Linux on their computers. It would be pretty easy to compromise an image, get it loaded on one of the main distribution sites and then as soon as someone starts one up inside a corporate network, you're in!

djsumdog 8y ago

I don't understand why people use other people's Docker images. Unless it comes from an official repository for the tool you're using, it's better to look at the source code/Dockerfile in the github link and just roll your own. A lot of times you're just installing the package you want with apt-get within your Dockerfile anyway; a package you can't check for normal updates for anymore since it's in a container. So now you need a tooling system around making sure your packages in your containers don't have security issues. Docker is kinda a mess.

y4mi 8y ago
its not really a mess for its usecase. its immutable infrastructure at its heart, so yeah, you don't do updates on containers... what you do need is periodic rebuild of your images for upgrades and each new image needs to run all integration and system tests again. it just makes this process easier than it is without docker. But it doesnt alleviate you of writing the system that actually keep everything updated in an automated way. It also doesnt help you deploy unless you're already experienced with docker. and while we'Re on the topic... no, if you know how to execute 'docker run -it --rm ubuntu bash' you still don't know shit about it. sigh sorry, i'm just remembering someone from work today...