5 ms·

Very interested to see what the root cause ends up being - this is mind bogglingly bad. I know several people who use these devices because of their simplicity,

Very interested to see what the root cause ends up being - this is mind bogglingly bad. I know several people who use these devices because of their simplicity, and they’d be devastated. I run a FreeNAS setup at home on an HP MicroServer. From time to time, I lament the amount of time I’ve spent over the years maintaining it, and after a recent drive failure (and wishing for more performance), I started considering consumer options like Synology/QNAP. Articles like this always give me pause, and remind me why I invest the time I do to maintain my own thing. I’d rather fuck something up myself than fall victim to an issue like this, and the time spent setting it up likely pales in comparison to dealing with data recovery. I really hope these users can get their data back.

TheGuyWhoCodes 5y ago
Qnap had a lot of security issues in the past, like unpatched RCE and backdoors. I wouldn't recommend, (and never ever use qnap cloud or upnp) as it seems they don't really do a good job on software QA or penetration testing or security audits. Hardware is fine but it's much more expensive than just building it yourself.
- lazide 5y ago
  QNAp takes the ‘just ship it’ approach. They have a lot of functionality, but it’s clear if you deal with their software much that it was a rush job. Lots of random breakages, bugs, stability issues, etc. You’re wise to not trust it with anything important.
- Marsymars 5y ago
  Some of their hardware is tough to do yourself. e.g. I've got a TL-D800S hooked up to my home server. I know I could build something comparable, but once hardware involves actual "building" rather than just "assembly", it's past the level of effort I want to spend.
shric 5y ago
I've never tried FreeNAS, but am surprised it took so much maintenance. I've been using FreeBSD with an HP Microserver and ZFS for the same purpose for about 10 years. I just install the updates and everything just works. Curious what kind of maintenance is taking up your time?
- teejmya 5y ago
  I was going to ask the same thing, as someone who's run FreeNAS on the AMD-based HP Microserver for nearly 8 years now. The only fiddly bit to setup for me was LetsEncrypt auto-renewals, but once I had it automated I never needed to touch it again.
- haswell 5y ago
  For me, the issues were mostly related to the period of time I started using FreeNAS, and getting a bit too creative with plugins and jails. On the first part, there was a period of time when the project was going through some upheaval (quite a few years ago now) and I had some issues with upgrades. This goes hand in hand with the 2nd thing - I was trying all sorts of plugins and using it as a BT and newsgroup client. These did not upgrade well. I suspect that a fresh vanilla build (TrueNAS now) would probable be zero maintenance. I need to upgrade my drives (a bunch of 1TB drives from the era I built it), and I’ll probably use this opportunity to start fresh.
  - pomian 5y ago
    Those projects that start easy, and then suffer from: hmm, let's try that, then let's try these add ons, etc. And everything works, for years, and then you need to return to them, but can't remember anything! And then, it turns out, that you don't have the time you used to have. And even though you loved tinkering late into the night, 10 years later, hight has become precious. For sleeping. Or trying to sleep!
xattt 5y ago
I converted a QNAP device to Ubuntu after the updates stopped coming. It’s decent and fits perfectly where it lives in my house. Of all things, it has a weird boot-up bug where it fails to boot to Ubuntu during about a half-hour period (around :15 and :45) each hour and lands on a kernel panic within a second after grub. I have no idea where to start. Firmware is latest.
- bombcar 5y ago
  Interesting - is it reliably between those times? What is the panic?
  - xattt 5y ago
    I’m about 3/4 certain that it’s between those times, and never at the top of the hour. Getting the exact panic would unfortunately mean to have to get it down from the shelf where it lives, and plugging it into a monitor to replicate. It doesn’t make it out of low-res text mode when it does. Most of the time, I’m usually in a rush to get it back up as my family wants lights and Plex to work normally. I do my best not to reboot to keep the peace!
    - cmeacham98 5y ago
      If you get a chance to test, I'd recommend changing the time on the device and seeing if that makes a difference, should clue you in on if it's a software or environment issue.
flatiron 5y ago
My guess is someone figured out how to bounce commands off WD command server (customer reps helping people reset their devices) and they are spamming customer ids. Either that or these devices have unique ports on the WAN with a 0-day but that would take a lot more effort to exploit.
- dannyw 5y ago
  someone didnt just figure ut out, it wss dislosed as a 9.8 CVE in 2019 .
  - HotHotLava 5y ago
    That CVE only shows how to factory-reset a single MyBook to which you can connect and send HTTP requests. There's still something missing to explain how it could happen to seemingly all MyBook's at once, including those behind a NAT.
    - addingnumbers 5y ago
      > explain how it could happen to seemingly all MyBook's at once Explain what makes you think this happened to all of them at once. That's a huge leap
      - HotHotLava 5y ago
        There are comments in this thread from people who read about this in the news, checked their own device, and found it was wiped as well. This seems like it would be very unlikely to happen if this were just happening in isolated cases.
        addingnumbers 5y ago
        Now you're just advocating for the Hasty Generalization Fallacy
gruez 5y ago
>Very interested to see what the root cause ends up being The article says: >Multiple users reported that the data loss coincided with a factory reset that was performed on their devices. One person posted a log that showed unexplained behavior occurring on Wednesday: I'm leaning more towards 0day than "software bug", mainly because I can't really think of a reason why a bug would cause the factory reset to get triggered, but I can think of plenty of ways an internet connected NAS can be exploited.
- JKCalhoun 5y ago
  > internet connected NAS I am guessing this is normal? My data backups are offline. The idea of making my backups read/write on the internet seems insane.
  - klodolph 5y ago
    For certain notions of "read/write", the risk can be lower. You might make backups to storage, but not have permission to delete files. Or when you delete files, they are kept for a certain safety window (e.g. if you delete a file on June 24, the file is instead moved and can be restored any time before July 24, at which point it is purged).
  - gruez 5y ago
    >I am guessing this is normal? My data backups are offline. The idea of making my backups read/write on the internet seems insane. It's a NAS. Backup isn't the only thing you can do with it. A personal cloud (aka dropbox) is also a valid use case, and requires internet access to work.
djrogers 5y ago
> I know several people who use these devices because of their simplicity, and they’d be devastated. I can’t imagine a scenario where I’d rely on an 8-10 year old device that hadn’t received a security update in 6+ years…
- userbinator 5y ago
  There are countless systems out there which are far older and also more secure. Age is of zero relevance except to those who propagate forced obolescence, mindless consumption, and generation of massive amounts of e-waste. This "update mentality" is a cancer itself. Especially when this is supposed to be a simple device. That said, relying on the same credentials for a long time is more problematic. My bet is on credentials being bruteforced, especially if many people have left them at their default or used easily guessable ones.
  - davidcbc 5y ago
    Age is of zero relevance for hard drives?
    - bigiain 5y ago
      Also - age is of zero relevance to internet connected linux devices? I rely on many "8-10 year old device that hadn’t received a security update in 6+ years". My coffee machine. My fridge. My hifi. All my motorcycles. None of them run linux and are directly connected to the internet.
      - haswell 5y ago
        But think about this from the perspective of the average buyer/user. Hard drives are traditionally "local". Local = safe. Now, a new generation of drives are introduced with this awesome capability to get access to data remotely. "Cool", they think. They don't understand how the Internet works, and so they don't really see the issue, or if they have a basic understanding of networks they might understandably believe that "it's behind the router, so it's safe".
technion 5y ago
QNAP's recent situation was hard to believe. Having only just fixed a significant SQL injection vulnerability[0], a spate of ransomware hit. It was originally announced to be this issue that people rushed out to patch[1]. It ultimately turned out to be a backdoor account[2]. Backdoors really upset me. Every reasonable "you can't blame people for mistakes" goes out the window when it's not a mistake. Following the ransomware, they released an auto-installing malware remover. A Python script that detected one particular piece of code associated with this recent attacker. That malware remover was a python script full of vulnerable exec calls introducing multiple new RCEs. [0] https://www.qnap.com/en/security-advisory/qsa-21-11 https://www.qnap.com/en/security-advisory/qsa-21-11 [1] https://www.qnap.com/de-de/security-advisory/qsa-21-05 https://www.qnap.com/de-de/security-advisory/qsa-21-05 [2] https://www.bleepingcomputer.com/news/security/qnap-confirms-qlocker-ransomware-used-hbs-backdoor-account/ https://www.bleepingcomputer.com/news/security/qnap-confirms... [3] https://www.qnap.com/en/security-advisory/qsa-21-16 https://www.qnap.com/en/security-advisory/qsa-21-16
- nneonneo 5y ago
  Wow. Back door accounts are just totally inexcusable these days - manufacturers should be held to account if their back doors end up getting exploited by threat actors. (Granted, you could argue it’s hard to distinguish between a really terrible bug and a back door, sometimes, but “intent” should cover this difference…) Would you happen to have a write up/article about this series of events that isn’t a sanitized series of security bulletins? Sounds like it’d make for good reading.
  - technion 5y ago
    Try this: https://forum.qnap.com/viewtopic.php?f=45&t=160849&start=450#p788325 https://forum.qnap.com/viewtopic.php?f=45&t=160849&start=450...
    - rawoke083600 5y ago
      Wow !
    - 5y ago
lrem 5y ago
Have you ever found a fix to FreeNAS not knowing how to slow down the fan in the Microserver? Or is your generation free of this issue?
intrasight 5y ago
I've been using Synology products for about ten years and remain happy.

TheGuyWhoCodes 5y ago

Qnap had a lot of security issues in the past, like unpatched RCE and backdoors. I wouldn't recommend, (and never ever use qnap cloud or upnp) as it seems they don't really do a good job on software QA or penetration testing or security audits. Hardware is fine but it's much more expensive than just building it yourself.

lazide 5y ago
QNAp takes the ‘just ship it’ approach. They have a lot of functionality, but it’s clear if you deal with their software much that it was a rush job. Lots of random breakages, bugs, stability issues, etc. You’re wise to not trust it with anything important.
Marsymars 5y ago
Some of their hardware is tough to do yourself. e.g. I've got a TL-D800S hooked up to my home server. I know I could build something comparable, but once hardware involves actual "building" rather than just "assembly", it's past the level of effort I want to spend.

shric 5y ago

I've never tried FreeNAS, but am surprised it took so much maintenance. I've been using FreeBSD with an HP Microserver and ZFS for the same purpose for about 10 years. I just install the updates and everything just works. Curious what kind of maintenance is taking up your time?

teejmya 5y ago
I was going to ask the same thing, as someone who's run FreeNAS on the AMD-based HP Microserver for nearly 8 years now. The only fiddly bit to setup for me was LetsEncrypt auto-renewals, but once I had it automated I never needed to touch it again.
haswell 5y ago
For me, the issues were mostly related to the period of time I started using FreeNAS, and getting a bit too creative with plugins and jails. On the first part, there was a period of time when the project was going through some upheaval (quite a few years ago now) and I had some issues with upgrades. This goes hand in hand with the 2nd thing - I was trying all sorts of plugins and using it as a BT and newsgroup client. These did not upgrade well. I suspect that a fresh vanilla build (TrueNAS now) would probable be zero maintenance. I need to upgrade my drives (a bunch of 1TB drives from the era I built it), and I’ll probably use this opportunity to start fresh.
- pomian 5y ago
  Those projects that start easy, and then suffer from: hmm, let's try that, then let's try these add ons, etc. And everything works, for years, and then you need to return to them, but can't remember anything! And then, it turns out, that you don't have the time you used to have. And even though you loved tinkering late into the night, 10 years later, hight has become precious. For sleeping. Or trying to sleep!

xattt 5y ago

I converted a QNAP device to Ubuntu after the updates stopped coming. It’s decent and fits perfectly where it lives in my house. Of all things, it has a weird boot-up bug where it fails to boot to Ubuntu during about a half-hour period (around :15 and :45) each hour and lands on a kernel panic within a second after grub. I have no idea where to start. Firmware is latest.

bombcar 5y ago
Interesting - is it reliably between those times? What is the panic?
- xattt 5y ago
  I’m about 3/4 certain that it’s between those times, and never at the top of the hour. Getting the exact panic would unfortunately mean to have to get it down from the shelf where it lives, and plugging it into a monitor to replicate. It doesn’t make it out of low-res text mode when it does. Most of the time, I’m usually in a rush to get it back up as my family wants lights and Plex to work normally. I do my best not to reboot to keep the peace!
  - cmeacham98 5y ago
    If you get a chance to test, I'd recommend changing the time on the device and seeing if that makes a difference, should clue you in on if it's a software or environment issue.

flatiron 5y ago

My guess is someone figured out how to bounce commands off WD command server (customer reps helping people reset their devices) and they are spamming customer ids. Either that or these devices have unique ports on the WAN with a 0-day but that would take a lot more effort to exploit.

dannyw 5y ago
someone didnt just figure ut out, it wss dislosed as a 9.8 CVE in 2019 .
- HotHotLava 5y ago
  That CVE only shows how to factory-reset a single MyBook to which you can connect and send HTTP requests. There's still something missing to explain how it could happen to seemingly all MyBook's at once, including those behind a NAT.
  - addingnumbers 5y ago
    > explain how it could happen to seemingly all MyBook's at once Explain what makes you think this happened to all of them at once. That's a huge leap
    - HotHotLava 5y ago
      There are comments in this thread from people who read about this in the news, checked their own device, and found it was wiped as well. This seems like it would be very unlikely to happen if this were just happening in isolated cases.
      - addingnumbers 5y ago
        Now you're just advocating for the Hasty Generalization Fallacy

gruez 5y ago

>Very interested to see what the root cause ends up being The article says: >Multiple users reported that the data loss coincided with a factory reset that was performed on their devices. One person posted a log that showed unexplained behavior occurring on Wednesday: I'm leaning more towards 0day than "software bug", mainly because I can't really think of a reason why a bug would cause the factory reset to get triggered, but I can think of plenty of ways an internet connected NAS can be exploited.

JKCalhoun 5y ago
> internet connected NAS I am guessing this is normal? My data backups are offline. The idea of making my backups read/write on the internet seems insane.
- klodolph 5y ago
  For certain notions of "read/write", the risk can be lower. You might make backups to storage, but not have permission to delete files. Or when you delete files, they are kept for a certain safety window (e.g. if you delete a file on June 24, the file is instead moved and can be restored any time before July 24, at which point it is purged).
- gruez 5y ago
  >I am guessing this is normal? My data backups are offline. The idea of making my backups read/write on the internet seems insane. It's a NAS. Backup isn't the only thing you can do with it. A personal cloud (aka dropbox) is also a valid use case, and requires internet access to work.

djrogers 5y ago

> I know several people who use these devices because of their simplicity, and they’d be devastated. I can’t imagine a scenario where I’d rely on an 8-10 year old device that hadn’t received a security update in 6+ years…

userbinator 5y ago
There are countless systems out there which are far older and also more secure. Age is of zero relevance except to those who propagate forced obolescence, mindless consumption, and generation of massive amounts of e-waste. This "update mentality" is a cancer itself. Especially when this is supposed to be a simple device. That said, relying on the same credentials for a long time is more problematic. My bet is on credentials being bruteforced, especially if many people have left them at their default or used easily guessable ones.
- davidcbc 5y ago
  Age is of zero relevance for hard drives?
  - bigiain 5y ago
    Also - age is of zero relevance to internet connected linux devices? I rely on many "8-10 year old device that hadn’t received a security update in 6+ years". My coffee machine. My fridge. My hifi. All my motorcycles. None of them run linux and are directly connected to the internet.
    - haswell 5y ago
      But think about this from the perspective of the average buyer/user. Hard drives are traditionally "local". Local = safe. Now, a new generation of drives are introduced with this awesome capability to get access to data remotely. "Cool", they think. They don't understand how the Internet works, and so they don't really see the issue, or if they have a basic understanding of networks they might understandably believe that "it's behind the router, so it's safe".

technion 5y ago

QNAP's recent situation was hard to believe. Having only just fixed a significant SQL injection vulnerability[0], a spate of ransomware hit. It was originally announced to be this issue that people rushed out to patch[1]. It ultimately turned out to be a backdoor account[2]. Backdoors really upset me. Every reasonable "you can't blame people for mistakes" goes out the window when it's not a mistake. Following the ransomware, they released an auto-installing malware remover. A Python script that detected one particular piece of code associated with this recent attacker. That malware remover was a python script full of vulnerable exec calls introducing multiple new RCEs. [0] https://www.qnap.com/en/security-advisory/qsa-21-11 https://www.qnap.com/en/security-advisory/qsa-21-11 [1] https://www.qnap.com/de-de/security-advisory/qsa-21-05 https://www.qnap.com/de-de/security-advisory/qsa-21-05 [2] https://www.bleepingcomputer.com/news/security/qnap-confirms-qlocker-ransomware-used-hbs-backdoor-account/ https://www.bleepingcomputer.com/news/security/qnap-confirms... [3] https://www.qnap.com/en/security-advisory/qsa-21-16 https://www.qnap.com/en/security-advisory/qsa-21-16

nneonneo 5y ago
Wow. Back door accounts are just totally inexcusable these days - manufacturers should be held to account if their back doors end up getting exploited by threat actors. (Granted, you could argue it’s hard to distinguish between a really terrible bug and a back door, sometimes, but “intent” should cover this difference…) Would you happen to have a write up/article about this series of events that isn’t a sanitized series of security bulletins? Sounds like it’d make for good reading.
- technion 5y ago
  Try this: https://forum.qnap.com/viewtopic.php?f=45&t=160849&start=450#p788325 https://forum.qnap.com/viewtopic.php?f=45&t=160849&start=450...
  - rawoke083600 5y ago
    Wow !
  - 5y ago

lrem 5y ago

Have you ever found a fix to FreeNAS not knowing how to slow down the fan in the Microserver? Or is your generation free of this issue?

intrasight 5y ago

I've been using Synology products for about ten years and remain happy.