15 ms·
QuickBooks Cloud Hosting Firm iNSYNQ Hit in Ransomware Attack
- ForrestN 7y ago"After the third day of outages, customers were saying Bye Bye Bye."
- pixl97 7y agoOne company was already in the process of moving after iNSYNQ got hit earlier this year by ransomware. This isnt their first attack.
- pgrote 7y agoTextbook on how not to handle the issue. Deleted Twitter, sanitized Facebook and more than 2 days to admit the issue. There is discussion on twitter that the company said the backups were on the same network as the data. Hopefully there is an offsite backup available. https://twitter.com/ConleyU/status/1151862278909825024 https://twitter.com/ConleyU/status/1151862278909825024 https://twitter.com/MRasconCPA/status/1151894366291734533 https://twitter.com/MRasconCPA/status/1151894366291734533 https://twitter.com/hockeygirlPDX/status/1151945932935585792 https://twitter.com/hockeygirlPDX/status/1151945932935585792 Ouch. This is the sort of stuff that can kill a company. Does Quickbooks with the cloud option offer local backups?
- LeifCarrotson 7y agoAs the resident tech guy in a small shop with Quickbooks, yes, you can do local backups. We're not using iNSYNQ for cloud hosting, though.
- jacquesm 7y agoIt is interesting how often I see companies that refuse to make back-ups of their cloud hosted data on the assumption that this is now someone else's problem. I also have a - recent - case of a very large manufacturer of storage solutions that managed to fuck up a restore of a raid array to the point that a whole pile of companies lost their data. Backups are so simple, and yet the only times people seem to realize their true value is when they don't have any.
- ygjb 7y agoI am not arguing the importance of backups, but tbh, simple is not a fair statement. At the very least, if you have backups, you might have the ability to rebuild/redeploy, but for any non-trivial system, without a comprehensive DR plan that is drilled regularly, even having backups won't save the business if the tooling to get those backups into a production ready state.
- gowld 7y agoIt's also a major flaw of the cloud and SaaS providers. They don't encourage customers to maintain local (or alternate-cloud) backups, because they refuse to admit the appearance of weakness. If a Cloud/SaaS provider is going to claim to be your hassle-free one-stop shop, shouldn't they provide, as a matter of course, a feature to dump your data daily/weekly/monthly to a designated off-cloud site of your choosing? But they don't, because they want to pretend that they are impenetrable and 100% available.
- rob-olmos 7y agoWould be nice if someone could build a site like https://twofactorauth.org/ https://twofactorauth.org/ for bringing more attention to this alarming and growing issue. The goal to hopefully influence companies into providing such a fundamental ability.
- londons_explore 7y agoA data dump isnt much use if you don't have the software to use said dump. Most SaaS providers don't run open source software...
- PopeDotNinja 7y ago> Backups are so simple It gets more interesting when you're trying to backup a distributed database. When your durability strategy is to spread your risk across a lot of nodes in a cluster, and you're federating your cluster across multiple sites, traditional backups may not be a practical option. However, as I write this, I'm realizing that there's nothing simple about distributed anything, and saying "it's hard to..." may not be all that interesting.
- rob-olmos 7y agoQuickBooks Online does not have a native backup or restore, at all, and it's pretty ridiculous. QBO told me they also can't do on-request restorations with their backups. There's a third-party or two that can use the API to do a backup/restore, but also, still, not all of the QBO data has an API. Eg, IIRC, recurring transaction tasks. Just a matter of time for ransomware to replace data via APIs.
- elliekelly 7y ago> Just a matter of time for ransomware to replace data via APIs. I asked about this in a cybersecurity meeting at an investment bank a few years ago - how much security is there around Bloomberg's data feed? All of the algorithms that make million & billion dollar decisions rely on that data so I asked if someone stepped in the middle and fed the algorithms bad data or even slightly delayed data would we even know it had happened? They looked at me like I was a crazy person. I'm actually kind of surprised it hasn't happened yet. Or maybe no one has noticed yet.
- rob-olmos 7y agoThat's a fascinating scenario. Even more malicious embedding shellcode in a series of tickers and prices kicked off with a few invalid tickers/prices. I wonder how many other feeds/sources like that in which so many businesses rely on in at least some level of autonomous fashion. Reminds of the story of Google wiping out all of their CDN servers by Diskerase because of a single detected value, IIRC.
- nodesocket 7y agoMy Quickbook Self-Employed (hosted) seems unaffected (at least so far).
- pavanagrawal123 7y agothat is different from this. The affected is a 3rd party hoster, not Intuit.
- trollied 7y agoIf anyone wants a recommendation for an alternative, I can happily suggest Xero. What a mess, though. Worst part of a business to be crippled is its core - financials. A part often overlooked by techies. If you can’t invoice, you can’t pay the wages.
- pavanagrawal123 7y agoIt's not Intuit that's down... Just a 3rd party hosting provider.
- PKop 7y agoWhat are they hosting? Backups of the desktop version of Quickbooks?
- pavanagrawal123 7y agoAFAIK, there are legacy versions of a "networked" quickbooks that can be hosted by third parties. Not really sure though. Some are doing app streaming of desktop quickbooks
- npo9 7y agoI wouldn’t call the networked versions of quickbooks legacy. (Quickbooks Desktop and it’s ancillary products) That’s like saying Excel is legacy because Office 365 will let you edit spreadsheets in the cloud. Sometimes local operations are just better, or cloud versions don’t fully match features.
- pavanagrawal123 7y agoYep a more appropriate term would be "older".
- wrycoder 7y agoI think these are multi-user versions of QB Desktop, normally restricted to a LAN, that are being hosted to allow access from the internet. The hosting entity also provides backup.
- AdmiralAsshat 7y agoThis is a question I've thought alot about, so many some Sys Admins can give a good idea about how to approach it: How do you create a backup server that is reachable by production servers (so that they can back up to it) without then being vulnerable to the same kind of ransomware attacks that infect the production servers? You can't exactly make them read-only, or else they can't accept the "legitimate" writes that might occur during the normal backup process.
- computer 7y agoMake the backup servers pull?
- PowerfulWizard 7y agoThat is what I do. The backup server can access the production server, but production can't access the backup. But my setup requires the backup to go to disk on the production server and not stream out directly, so that aspect is annoying.
- donmcronald 7y agoAppend only?
- adrianmsmith 7y agoRight, that's what I use, https://www.tarsnap.com/ https://www.tarsnap.com/ has the option of allowing an access key to only be able to create a new backup but not read or alter or delete any existing backups. I blogged about my setup here: https://www.databasesandlife.com/write-only-backups/ https://www.databasesandlife.com/write-only-backups/
- deleted 7y ago[deleted]
- deleted 7y ago[deleted]
- 7y ago
- ryanmercer 7y agoI lost all interest in the topic when I read the hosting companies name, can we just talk about THAT? Like, really? Who picks that name for their company, certainly the band nsync has to be older than the company.
- benjaminbrodie 7y agoIt's certainly very search engine friendly. You essentially get a curated list of all and ONLY items of potential interest to you.
- ryanmercer 7y agoMy point was more "hey that seems like a MikeRoweSoft [1] potential legal quagmire, terrible for branding and SEO and a great way to be mocked and not taken seriously at all by potential clients". [1] https://en.wikipedia.org/wiki/Microsoft_v._MikeRoweSoft https://en.wikipedia.org/wiki/Microsoft_v._MikeRoweSoft
- julianlam 7y agoThe article seems light on details about how QuickBooks is involved. Does iNSYNQ host QB Cloud data for them, or does iNSYNQ white-label QB software?
- blendo 7y agoWhat are the odds their vulnerability was due to IT staff choosing to not patch their Windows servers, as doomed Baltimore in May? https://krebsonsecurity.com/2019/06/report-no-eternal-blue-exploit-found-in-baltimore-city-ransomware/ https://krebsonsecurity.com/2019/06/report-no-eternal-blue-e...
- belltaco 7y agoThe article does not say the attack was due to IT staff choosing to not patch their Windows servers, am I missing something?
- blendo 7y agoYou are correct, the Krebs article I cited does not say that. But it's my strong hunch. There was much discussion of the poor state of Baltimore's IT infra (see https://www.baltimorebrew.com/2019/05/21/baltimores-out-of-date-and-underfunded-it-system-was-ripe-for-a-ransomware-attack/ https://www.baltimorebrew.com/2019/05/21/baltimores-out-of-d...), and at the same time Microsoft released a patch for an RDS flaw (CVE-2019-0708, https://msrc-blog.microsoft.com/2019/05/14/prevent-a-worm-by-updating-remote-desktop-services-cve-2019-0708/ https://msrc-blog.microsoft.com/2019/05/14/prevent-a-worm-by...) which says "Vulnerable in-support systems include Windows 7, Windows Server 2008 R2, and Windows Server 2008. Downloads for in-support versions of Windows can be found in the Microsoft Security Update Guide. Customers who use an in-support version of Windows and have automatic updates enabled are automatically protected. " So unpatched servers are where I'm laying my money.
- floatingatoll 7y agoiNSYNQ is a third-party service platform operator who specializes in hosting instances of QuickBooks in the cloud. This does not affect non-iNSYNQ QuickBooks instances, such as those operated by Intuit (the creator of QuickBooks).
- basilgohar 7y agoLooked for but didn't see this when I made my shockingly similar post.
- floatingatoll 7y agoI’m not familiar with the branch of synesthesia that detects similarity as electrical impulses, but you’re welcome to email the mods and ask them to delete my post and give you the karma. I don’t know if they’ll do so or not but I certainly have no investment in the integer (as my most recent post shows).
- BinaryIdiot 7y agoI had no idea this was how QuickBooks even worked in the cloud. Could Intuit be in any way liable (like is this a sort of franchise type of thing)? Or is it closer to Word Press where different companies can install "QuickBooks Cloud" and then offer it?
- floatingatoll 7y agoThink “Microsoft Access in the cloud”. How would you do that without Microsoft’s cooperation? Virtual machines running Windows. Full service hosting, the works. This is classic ISP stuff, back when ISPs were service providers and not just Internet connections and other off the shelf no-touch products.
- tracker1 7y agoWas this service really less expensive than just using actual QB on an RDP available cloud server from Azure or any number of other services? Also, what kind of hacky backup system takes this much time to sort through to identify issues. They should have a clean image, and a clean way to backup/restore data for the application being hosted as a pull from production/active deployments. In the end, this will or maybe even should kill the company in question. Beyond this, it is an opportunity for others. For that matter, really surprised Intuit doesn't have this as a cloud service at this point.
- miles 7y ago> really surprised Intuit doesn't have this as a cloud service at this point. They do have a cloud offering, QuickBooks Online: https://quickbooks.intuit.com/online/ https://quickbooks.intuit.com/online/ But it does not have all the same features as the Desktop version, giving rise to a number of third party offerings, like Right Networks' "QuickBooks Desktop Cloud": https://www.rightnetworks.com/cloud-solutions/accounting-solutions/quickbooks-desktop-cloud/ https://www.rightnetworks.com/cloud-solutions/accounting-sol...
- not_a_cop75 7y agoThe cloud is just someone else's computer.
- Keverw 7y agoWonder if they obtained any people's financial data or social security numbers. Probably mostly self employed people and small businesses. Pretty scary how people use their SSN for everything. Seems so insecure to have a number you just openly pass around... Need to get a ID or license, credit card, bank account, your doctor, dentist asks for them, your phone company, cable company, of course when getting paid, and even police officers ask for them sometimes and write down in a notepad if your name happens to match someone else's name who has a warrant. Many other uses probably too I didn't think of off the top of my head. I was randomly one day looking at dentist new patient forms and one even wanted to know your relationship status, not sure how that's relevant if a single or married guy gets a cleaning... I know home alone when the internet went out, so called the local cable company to see if an outage and the lady wanted the social security number on the account before continuing, which I didn't know. Just insane how many things use the same number, it's like single sign on for real life. Same issue with bank account numbers. To pay someone with direct deposit, they can use the same number to withdraw from your account. I'm surprised banks haven't figured out a way to offer deposit only option... Just create a new account number but linked to another account, where deposits to account 4321 goes to account 1234 instead, but can't ever withdraw from 4321. I got a feeling Facebook's account system is probably more secure than my local bank. Pretty sad when someone's hobby blockchain project has more technology in it than banks with billions of dollars of assets under management.
- mschuster91 7y ago> I was randomly one day looking at dentist new patient forms and one even wanted to know your relationship status, not sure how that's relevant if a single or married guy gets a cleaning... This is for spousal rights - i.e. if your spouse is allowed to request access your data. > I'm surprised banks haven't figured out a way to offer deposit only option They have, some German banks assign an IBAN also for "Sparbücher" (saving plans). These cannot be withdrawn from. For withdrawal security, under SEPA rules you have 8 weeks to (instantly!) reverse a transaction. If you misuse this, you can get your account closed and criminal proceedings filed so that is a relatively effective fraud prevention.
- 7y ago
- anbop 7y agoWhy would someone use a third party to host Quickbooks? Won’t Intuit host it?
- the_svd_doctor 7y agoThis. Is it the same Quickbooks ?
- tornquist 7y agoNo, this is a desktop version hosted offsite instead of the browser-based cloud version from Intuit. See this above post for examples: https://news.ycombinator.com/item?id=20481074 https://news.ycombinator.com/item?id=20481074
- gowld 7y agoHow is "QuickBooks Desktop Hosted in the Cloud" different from "Anything you darn feel like, hosted on a remote Desktop (like Citrix)" ?
- andrewf 7y agoThe former is probably easier to sell to accountants who want QuickBooks Desktop plus the benefits of a remote managed service (accessible over the Internet, someone else takes care of backups - in theory! - etc)
- snthd 7y agoIs there a way of freezing a compromised machine, such that the contents of RAM are kept around for forensics, but the compromised system can't do any more damage to itself? SysRq? Intel ME?
- yjftsjthsd-h 7y agoIn the common case of a VM it should be easy. Bare metal, less so.
- nabilhat 7y agoIt's not impacting all of Insynq's services. I work with an Insynq customer. Their Insynq services are still running, by some generous stroke of fate. The only outage we noticed was in the middle of the day on the 16th. For about an hour, users weren't able to access the service. I called support, and was diverted to a recorded message saying that they were doing normal maintenance, would be finished shortly, and were aware of and sorry for the disruption. It's been my outspoken opinion that this was an inevitable outcome for as long as I've been familiar with their product.
- roshanravan 7y agoFor an accounting software this is disaster.
- basilgohar 7y agoIt's not immediately obvious, but iNSYNQ provides hosting for QuickBooks Desktop as a virtual desktop service (think VNC or RDP). This is distinct from Intuit's own QuickBooks Online cloud service, which I believe is unaffected by this breach. I was concerned because one of my clients' customers rely heavily on QuickBooks Online and her app integrates heavily with it.
- chx 7y agoI can confirm QBO is up (I am in Canada). Holy Batman, the chaos if that data would be gone... I did a backup, quickly. I need to ask my accountant whether they back this data up regularly, if not then I need to. I am a very small company but my invoices are international and while I have my invoices at hand, if I would need to reconstruct the exchange rates for an audit a few years back... shudders
- wildduck 7y agoALWAYS make OFFLINE backup! USB USB USB!
- WheelsAtLarge 7y agoSlowly but surely ransomware is becoming an absolute threat to all data. It seems more like not if but when it will hit. It seems to me that it's time the OS providers start providing a very easy way to restore the state of data. We all know that backups are the answer but as long as people, have to think about it, there will always be some that don't do them. And now that you can get a 1TB HD for less that $100 then it's a no-brainer. Virus protection is now automatic with Windows when will backups become automatic on all OSs?
- swiley 7y ago>It seems to me that it's time the OS providers start providing a very easy way to restore the state of data. Wat? How are you imagining this would be implemented? Right now you can have a cronjob that runs rsync which allows exactly that but doesn’t protect you from ransomeware. Are you thinking if it were part of the file system in the kernel it would be better protected? I guess that’s partially true but in practice probably will do very little (also, that already exists on Linux in the form of things like ZFS.) Once something like that infects the machine there really isn’t a whole lot the OS can do to prevent the damage, all it’s doing is handling user requests for operations on things like the file system and the malware is operating as a proxy for the user.
- normalperson 7y agoHosted QuickBooks is a horrible software product and people should stop using it.