7 ms·

200,000+ MikroTik routers worldwide compromised to inject cryptojacking malware

104 points by pjf 8y ago

pjf 8y ago
this is getting even bigger: 421K+ routers compromised - https://twitter.com/bad_packets/status/1050533001824595968 https://twitter.com/bad_packets/status/1050533001824595968
shakna 8y ago
I guess nobody listened [0]. But considering some of the comments on that thread, I'm not entirely shocked. > Attackers will not be able to use that, nor will they care. > If you're exposing that unfettered to the web at large, imo you deserve what you get. Did no one really think 200,000 devices was a worthwhile target? [0] https://news.ycombinator.com/item?id=18166003 https://news.ycombinator.com/item?id=18166003
have_faith 8y ago
If you have a currency that can be generated via compute power the incentive structure it creates is to take over as much compute power as you can. Does the incentive structure represent a serious design flaw in the system for widespread adoption?
- SuoDuanDao 8y ago
  I would say it's a consequence of computer power becoming commoditized - the flaw might be that using computer power in the past can be 'stored', unlike hijacking someone else's computer and renting it out as server space, there's a point in this kind of crime where one has the profits and got away with it. I just had this thought and want to get it down, apologies for the rant: I suspect that philosophically, the basic measure of our economy has been units of energy, but it's transitioning to units of power. The 'subscription model' most businesses are transitioning to seems to be an early recognition of that fact. Server space, similarly, isn't really something one can store as easily as gold or gasoline - it's only valuable while in use and degrading whether it's in use or not. Our whole concept of currency may be based on an idea of 'stored' value, analogous to swapping joules, while we should be thinking about 'sources' of value - swapping watts. I think the trouble is that renting (or hijacking) servers is accounting in terms of watts, while currency is accounting in terms of joules. The accounting of the two types of approaches may not be as easily reconcilable as it seems at firs.
  - Tae3cahN 8y ago
    I think both types of currency are important. Humans have basic needs which need to be met in the shape of atoms and joules which are quite storable. We also have luxury wants which translate better to watts. This separation might also be useful in the context of UBI schemes. And more generally I have a gut feeling, but can't quite substantiate, that many fairness problems stem from using the same currency for both, forcing the lower classes to work multiple jobs just to make basic needs while the haves compete with them just to improve their lifestyle and status.
    - SuoDuanDao 8y ago
      Fascinating! That makes excellent sense.
Tharkun 8y ago
So ... which SoHo router manufacturer can we actually trust? It seems pretty common in this industry to either not supply security updates, or to only supply them for a very short amount of time.
- TonyTheSlayer 8y ago
  Seems like the solution is still to see what router models are supported by your favorite aftermarket firmware.
- thecatspaw 8y ago
  Turris omnia. Open source (both hardware and software), lets you ssh into it directly out of the box (well, after you have set a root password in the gui) They provide regular software updates, and it is imho a good hacker/tinkerer router.
  - pgaddict 8y ago
    I agree - got one, quite happy with it. But it's targeted for home use, not sure how well it fits into larger networks (small offices). MikroTik seems like a better fit for that, not sure. One major advantage of Turris Omnia is that CZ.NIC is a non-profit, so they are not as constrained by profitability of this project, and it's a part of a larger strategy (e.g. you may allow reporting statistics back to turris: https://project.turris.cz/en/global-stats/ https://project.turris.cz/en/global-stats/).
  - anc84 8y ago
    300€ is about 5 times what I would consider a price to pay for a home router though.
    - thecatspaw 8y ago
      this depends on what you want out of it I guess. I wanted a tinker router, one which doesnt treat me like an absolute idiot, with a spf for fiber, can do gigabit easily and is expandable.
hkt 8y ago
http://pcengines.ch/ http://pcengines.ch/ APU2 plus debian. Be secure. Maybe I'll tidy up the ansible I use for this and publish it.
- chli 8y ago
  Please do !
- decasteve 8y ago
  Or *BSD. Great device.
- MikusR 8y ago
  So those don't need updates?
megous 8y ago
It's good to regularly update your router.
hkt 8y ago
Does anyone know if Ubiquiti's edgerouters are any good?
- 8fingerlouie 8y ago
  They are. They run a fork of Vyos (https://vyos.io/ https://vyos.io/), and get regular updates.
- throwaway9d0291 8y ago
  In my experience, no, they're not. Their wireless gear is great but when it comes to wired, I'd avoid them. There's a serious bug [0] on the ER-X-SFP that's been around for over a year without any serious action from Ubiquiti. Their support when I tried to get an RMA for said issue was utterly abysmal. [0]: https://community.ubnt.com/t5/EdgeRouter/EdgeRouter-X-SFP-leaking-MAC-addresses/td-p/1884107 https://community.ubnt.com/t5/EdgeRouter/EdgeRouter-X-SFP-le...
  - eltoozero 8y ago
    Agreed, I run MikroTik on the wired side and UniFi for the AP side; have many installs in my charge and this is our default deployment when “more WiFi” is requested.
- NoErx 8y ago
  I have an EdgeRouter X and I'm searching for a replacement. EdgeRouter X issues: * 3.10 kernel. This is out of LTS support. * Poor IPv6 support. The GUI has practically 0 support and you instead have to learn EdgeOS config, and it's awkward. I'm happy to use EdgeRouter X as a switch, so I'm looking for a SBC that can act as router/firewall and run vanilla Debian.
  - NoErx 8y ago
    Scratch that. Upon reading VyOS's docs for the EdgeOS config (which has relatively poorer docs), it has strong advantages: commit/rollback, single config file, and Ansible has an EdgeOS module included by default to coordinate that.
lowry 8y ago
I have several hAP ac Mikrotik routers and upgrading them is a pain. You can not just download an image from their website, flash and reboot. If you do so, your router will likely be locked in a bootloop. I managed to have consistent upgrades by using only the main package and Netinstall, but it is still a huge pain in the ass. Mikrotik makes stable routers, but they messed up the upgrade process completely.
- miahi 8y ago
  I also have several hAP ac routers and had no issue upgrading them. I only had a problem with an older routerboard (first time I tried upgrading my first ever Mikrotik), but that was because I did not know how the upgrade works and I cut off the power during the reboot (I was under the impression I have to reboot it manually). I never use the web interface though - winbox is way better.
- iofiiiiiiiii 8y ago
  What? I have updated all my MikrotikDevices by literally just dropping the new firmware image onto the device and restarting the device. That's it.
- philamonster 8y ago
  Huh? With the exception of them converting all master > slave port configs to bridges in 6.41 I believe, I have never had any issues using System > Packages > Check for updates, selecting bugfix as opposed to current branch and downloading and installing in Winbox. SwOS devices I need to download a binary and upgrade through web (no Winbox) but still have never had any issues.
  - lowry 8y ago
    Try doing it through the web interface, you'll be unpleasantly surprised. I just upgraded my last hAP ac from 6.39.2 to 6.42.9 through the web interface, entered the bootloop, then did the Netinstall of the system package only, then manually restored the configuration.
    - 8y ago
throwaway9d0291 8y ago
I'm really curious, where are these routers? The problem is that the admin interface port is exposed to the internet, which is something any competent administrator would ensure isn't accessible. So are there incompetent admins managing 200k devices or is someone distributing these to residential users?
- philamonster 8y ago
  When I first started reading about these reports over the past week I believe around 90%+ were in Brazil. If you peruse BPR timeline on Twitter they mention type of orgs using them. One of hardest hit I believe was ISP in Arizona or New Mexico, US.
hendry 8y ago
IMHO Mikrotik are being sloppy by introducing breaking changes to their stable channel. Hence ISPs are reluctant to update automatically, fearing some subtle bridge/VLAN change which is sadly set to happen again (6.43 -> 6.44!). Also doesn't help that the underlying Linux stable kernel updates more than once a week. Every Internet connected device needs some automatic update functionality by default. It's tricky for routers since you typically do not want any downtime and it's really difficult/expensive (well at least 2x the cost for hardware alone) to do a blue/green (or is it red/black?) deployment. Not to mention since Mikrotik is low end, there is no state replication functionality between routers. Here is a config for Mikrotik that updates my non-ISP hardware once a week at 3AM: https://gist.github.com/kaihendry/59a656c3883450d2df2fd52574f43b4c https://gist.github.com/kaihendry/59a656c3883450d2df2fd52574... And when the ISP opts out of the suggested automatic update default, they need to make the commitment to test and roll up updates in a timely fashion. As we all know this is hugely expensive, and I strongly believe a vendor/"computer program" should be able to provide this service. Customers with these ISPs who didn't update are probably seeing some crazy packetloss. So the future I'd like to see is Mikrotik updates being automatic, staged to some degree & ultimately non-breaking. Cons are downtime for some & I guess allowing your device to be remotely controlled by Latvians. ;)
- lowry 8y ago
  AFAIK, most Mikrotik employees speak native Russian and only passable Latvian. "Remotely controlled by Russians living in Latvia".
  - pilsetnieks 8y ago
    That's patently false, and also irrelevant.
- philamonster 8y ago
  Are these changes coming in bugfix channel as well? I'm currently in holding pattern going from 6.40.9 to 6.42.9 due to master > slave to bridge change which occurred in March I believe. Point is bugfix seems to be less affected by major changes like this (first I have experienced in 3 years).
philamonster 8y ago
Ha! https://www.zdnet.com/article/a-mysterious-grey-hat-is-patching-peoples-outdated-mikrotik-routers/ https://www.zdnet.com/article/a-mysterious-grey-hat-is-patch...

104 points by pjf 8y ago

pjf 8y ago

this is getting even bigger: 421K+ routers compromised - https://twitter.com/bad_packets/status/1050533001824595968 https://twitter.com/bad_packets/status/1050533001824595968

shakna 8y ago

I guess nobody listened [0]. But considering some of the comments on that thread, I'm not entirely shocked. > Attackers will not be able to use that, nor will they care. > If you're exposing that unfettered to the web at large, imo you deserve what you get. Did no one really think 200,000 devices was a worthwhile target? [0] https://news.ycombinator.com/item?id=18166003 https://news.ycombinator.com/item?id=18166003

have_faith 8y ago

If you have a currency that can be generated via compute power the incentive structure it creates is to take over as much compute power as you can. Does the incentive structure represent a serious design flaw in the system for widespread adoption?

SuoDuanDao 8y ago
I would say it's a consequence of computer power becoming commoditized - the flaw might be that using computer power in the past can be 'stored', unlike hijacking someone else's computer and renting it out as server space, there's a point in this kind of crime where one has the profits and got away with it. I just had this thought and want to get it down, apologies for the rant: I suspect that philosophically, the basic measure of our economy has been units of energy, but it's transitioning to units of power. The 'subscription model' most businesses are transitioning to seems to be an early recognition of that fact. Server space, similarly, isn't really something one can store as easily as gold or gasoline - it's only valuable while in use and degrading whether it's in use or not. Our whole concept of currency may be based on an idea of 'stored' value, analogous to swapping joules, while we should be thinking about 'sources' of value - swapping watts. I think the trouble is that renting (or hijacking) servers is accounting in terms of watts, while currency is accounting in terms of joules. The accounting of the two types of approaches may not be as easily reconcilable as it seems at firs.
- Tae3cahN 8y ago
  I think both types of currency are important. Humans have basic needs which need to be met in the shape of atoms and joules which are quite storable. We also have luxury wants which translate better to watts. This separation might also be useful in the context of UBI schemes. And more generally I have a gut feeling, but can't quite substantiate, that many fairness problems stem from using the same currency for both, forcing the lower classes to work multiple jobs just to make basic needs while the haves compete with them just to improve their lifestyle and status.
  - SuoDuanDao 8y ago
    Fascinating! That makes excellent sense.

Tharkun 8y ago

So ... which SoHo router manufacturer can we actually trust? It seems pretty common in this industry to either not supply security updates, or to only supply them for a very short amount of time.

TonyTheSlayer 8y ago
Seems like the solution is still to see what router models are supported by your favorite aftermarket firmware.
thecatspaw 8y ago
Turris omnia. Open source (both hardware and software), lets you ssh into it directly out of the box (well, after you have set a root password in the gui) They provide regular software updates, and it is imho a good hacker/tinkerer router.
- pgaddict 8y ago
  I agree - got one, quite happy with it. But it's targeted for home use, not sure how well it fits into larger networks (small offices). MikroTik seems like a better fit for that, not sure. One major advantage of Turris Omnia is that CZ.NIC is a non-profit, so they are not as constrained by profitability of this project, and it's a part of a larger strategy (e.g. you may allow reporting statistics back to turris: https://project.turris.cz/en/global-stats/ https://project.turris.cz/en/global-stats/).
- anc84 8y ago
  300€ is about 5 times what I would consider a price to pay for a home router though.
  - thecatspaw 8y ago
    this depends on what you want out of it I guess. I wanted a tinker router, one which doesnt treat me like an absolute idiot, with a spf for fiber, can do gigabit easily and is expandable.

hkt 8y ago

http://pcengines.ch/ http://pcengines.ch/ APU2 plus debian. Be secure. Maybe I'll tidy up the ansible I use for this and publish it.

chli 8y ago
Please do !
decasteve 8y ago
Or *BSD. Great device.
MikusR 8y ago
So those don't need updates?

megous 8y ago

It's good to regularly update your router.

hkt 8y ago

Does anyone know if Ubiquiti's edgerouters are any good?

8fingerlouie 8y ago
They are. They run a fork of Vyos (https://vyos.io/ https://vyos.io/), and get regular updates.
throwaway9d0291 8y ago
In my experience, no, they're not. Their wireless gear is great but when it comes to wired, I'd avoid them. There's a serious bug [0] on the ER-X-SFP that's been around for over a year without any serious action from Ubiquiti. Their support when I tried to get an RMA for said issue was utterly abysmal. [0]: https://community.ubnt.com/t5/EdgeRouter/EdgeRouter-X-SFP-leaking-MAC-addresses/td-p/1884107 https://community.ubnt.com/t5/EdgeRouter/EdgeRouter-X-SFP-le...
- eltoozero 8y ago
  Agreed, I run MikroTik on the wired side and UniFi for the AP side; have many installs in my charge and this is our default deployment when “more WiFi” is requested.
NoErx 8y ago
I have an EdgeRouter X and I'm searching for a replacement. EdgeRouter X issues: * 3.10 kernel. This is out of LTS support. * Poor IPv6 support. The GUI has practically 0 support and you instead have to learn EdgeOS config, and it's awkward. I'm happy to use EdgeRouter X as a switch, so I'm looking for a SBC that can act as router/firewall and run vanilla Debian.
- NoErx 8y ago
  Scratch that. Upon reading VyOS's docs for the EdgeOS config (which has relatively poorer docs), it has strong advantages: commit/rollback, single config file, and Ansible has an EdgeOS module included by default to coordinate that.

lowry 8y ago

I have several hAP ac Mikrotik routers and upgrading them is a pain. You can not just download an image from their website, flash and reboot. If you do so, your router will likely be locked in a bootloop. I managed to have consistent upgrades by using only the main package and Netinstall, but it is still a huge pain in the ass. Mikrotik makes stable routers, but they messed up the upgrade process completely.

miahi 8y ago
I also have several hAP ac routers and had no issue upgrading them. I only had a problem with an older routerboard (first time I tried upgrading my first ever Mikrotik), but that was because I did not know how the upgrade works and I cut off the power during the reboot (I was under the impression I have to reboot it manually). I never use the web interface though - winbox is way better.
iofiiiiiiiii 8y ago
What? I have updated all my MikrotikDevices by literally just dropping the new firmware image onto the device and restarting the device. That's it.
philamonster 8y ago
Huh? With the exception of them converting all master > slave port configs to bridges in 6.41 I believe, I have never had any issues using System > Packages > Check for updates, selecting bugfix as opposed to current branch and downloading and installing in Winbox. SwOS devices I need to download a binary and upgrade through web (no Winbox) but still have never had any issues.
- lowry 8y ago
  Try doing it through the web interface, you'll be unpleasantly surprised. I just upgraded my last hAP ac from 6.39.2 to 6.42.9 through the web interface, entered the bootloop, then did the Netinstall of the system package only, then manually restored the configuration.
  - 8y ago

throwaway9d0291 8y ago

I'm really curious, where are these routers? The problem is that the admin interface port is exposed to the internet, which is something any competent administrator would ensure isn't accessible. So are there incompetent admins managing 200k devices or is someone distributing these to residential users?

philamonster 8y ago
When I first started reading about these reports over the past week I believe around 90%+ were in Brazil. If you peruse BPR timeline on Twitter they mention type of orgs using them. One of hardest hit I believe was ISP in Arizona or New Mexico, US.

hendry 8y ago

IMHO Mikrotik are being sloppy by introducing breaking changes to their stable channel. Hence ISPs are reluctant to update automatically, fearing some subtle bridge/VLAN change which is sadly set to happen again (6.43 -> 6.44!). Also doesn't help that the underlying Linux stable kernel updates more than once a week. Every Internet connected device needs some automatic update functionality by default. It's tricky for routers since you typically do not want any downtime and it's really difficult/expensive (well at least 2x the cost for hardware alone) to do a blue/green (or is it red/black?) deployment. Not to mention since Mikrotik is low end, there is no state replication functionality between routers. Here is a config for Mikrotik that updates my non-ISP hardware once a week at 3AM: https://gist.github.com/kaihendry/59a656c3883450d2df2fd52574f43b4c https://gist.github.com/kaihendry/59a656c3883450d2df2fd52574... And when the ISP opts out of the suggested automatic update default, they need to make the commitment to test and roll up updates in a timely fashion. As we all know this is hugely expensive, and I strongly believe a vendor/"computer program" should be able to provide this service. Customers with these ISPs who didn't update are probably seeing some crazy packetloss. So the future I'd like to see is Mikrotik updates being automatic, staged to some degree & ultimately non-breaking. Cons are downtime for some & I guess allowing your device to be remotely controlled by Latvians. ;)

lowry 8y ago
AFAIK, most Mikrotik employees speak native Russian and only passable Latvian. "Remotely controlled by Russians living in Latvia".
- pilsetnieks 8y ago
  That's patently false, and also irrelevant.
philamonster 8y ago
Are these changes coming in bugfix channel as well? I'm currently in holding pattern going from 6.40.9 to 6.42.9 due to master > slave to bridge change which occurred in March I believe. Point is bugfix seems to be less affected by major changes like this (first I have experienced in 3 years).

philamonster 8y ago

Ha! https://www.zdnet.com/article/a-mysterious-grey-hat-is-patching-peoples-outdated-mikrotik-routers/ https://www.zdnet.com/article/a-mysterious-grey-hat-is-patch...