28 ms·
Backblaze submitting names and sizes of files in B2 buckets to Facebook
- ashneo76 6y agoBackblaze lost me now
- deleted 6y ago[deleted]
- selftest 6y agoReading further down the thread, backblaze responded that they are using Facebook pixel and the concern has been forwarded to the appropriate team.
- guywhocodes 6y agoYikes, that's a lot of good will burned. Doesn't matter if it is unintentional or not.
- celsoazevedo 6y agoThe inclusion of Facebook tracking was intentional: https://twitter.com/backblaze/status/1373751015594356739 https://twitter.com/backblaze/status/1373751015594356739 Not sure if they intended to send file names and sizes to FB, but in any case this doesn't look good. I'm currently looking for alternatives.
- chunkles 6y agorsync.net was posted here on HN a few days ago. Also, tarsnap is another popular service. Neither have the special additions that makes Backblaze so popular, but could be popular alternatives.
- latchkey 6y agovorta + borg backing up to cloud disk.
- minxomat 6y agoIf you’re in the EU, Hetzner Storage boxes.
- danieldk 6y agoKeep in mind that their storage boxes only use RAID and do not provide any other kind of redundancy by default. If the machine goes up in flames, your data is gone.
- Skunkleton 6y agoRsync.net has a self-supported borg option that is a great deal.
- thayne 6y agoFrom a quick look at pricing pages it looks like rsync.net is 5x as expensive as backblaze b2, and has a minimum of 400gb per month (it also looks like you might have to preallocate vs pay on demand). And tarsnap is 10x as expensive as rsync.net. My guess the bulk of that price difference is due to economies of scale.
- jiveturkey 6y agothe rsync guy posted in another recent thread that it’s actually 3x above a certain size, which i took to be most serious usage.
- kelnos 6y agoI don't have huge backup needs (around 500GB, doesn't grow much over time), so I just use duplicity and upload to AWS S3 with the "infrequent access" setting, and have the bucket auto-replicate to another region in another country. Costs me around $12/mo just for the data storage. The AWS calculator tells me if I had to retrieve all that data it would cost under $10 to do so. Given it's all personal data that I can stand to be without for days if needed, I could probably use Glacier (or even Glacier Deep Archive) and pay less than a third of the cost (or less than a twelfth!), but the absolute dollar amount isn't high enough for me to go through the trouble of changing up my backup scripts. Sure, if you're running a business and are constantly generating lots of new data that needs to be backed up, that gets more expensive, but I also would expect a business with that much data could easily afford to spend several orders of magnitude more than I do on backups.
- monsieurbanana 6y agoI use Linode Object Storage (a S3-like service) with rclone to manage my backups. (my backups are only a few dozens of gb, I don't need something like duplicity at the moment) So far I'm happy with it, pricing is $5/month per 250gb + 1To egress. And, most importantly, none of the AWS complexity overhead. I have enough of that at work.
- viraptor 6y agoOne big difference is that with B2 you can't have append-only backups. (or maybe it's very non-obvious from the docs?) That's pretty much why I use rsync.net instead - I can configure a separate key for uploads. Otherwise what's to stop anyone lifting the B2 access key from the host with automatic backups configured and just deleting all old data?
- gpm 6y agoSo, looking at some potential things to switch to... OVH has two interesting products here: OVH cloud archive works with rsync, costs roughly half as much for storage as blackblaze, roughly the same for egress, but charges for ingress (at the same rate as egress). OVH object store is s3 compatible (like blackblaze), charges roughly the same for bandwidth, 2x for storage. https://www.ovhcloud.com/en/public-cloud/prices/#439 https://www.ovhcloud.com/en/public-cloud/prices/#439 Digitalocean has a blob store with the same pricing on bandwidth, 4x the pricing on storage, a minimum spend of $5/month on storage, but the first tb ($10) of bandwidth free. https://www.digitalocean.com/pricing#spaces-object-storage https://www.digitalocean.com/pricing#spaces-object-storage
- ocdtrekkie 6y agoAlso, you know OVH will be investing significantly in fire prevention strategies, so that will all be state of the art.
- spzb 6y agoWell, you would hope they would be but then they thought they had done so already and look how that worked out.
- ocdtrekkie 6y agoBefore you experience a major problem, you tend to guess what is good enough to mitigate it. Once you've experienced a major problem, you tend to ensure that exact failure mode never happens again. I would be surprised if OVH had another fire in the future.
- celsoazevedo 6y agoAnd that's why we should always keep backups somewhere else.
- avinassh 6y ago> OVH cloud archive seems this the cheapest of all
- tbodt 6y agoThe only reason I use Backblaze instead of tarsnap is it is 62 times cheaper for the same amount of storage. tarsnap is dramatically overpriced if you ask me.
- jlawer 6y agoI get the feeling the reason for the Tarsnap price is simply a lack of scale and bootstrap nature of it. (i.e. there isn't a VC feeding money into hosting costs to drive the costs down to grow the userbase). The cost seems fair, but noncompetitive against other vendors on most easy to find quantitative measures (Cost, features, SLAs, locations, etc). However I do think there is value in Tarsnap if security is really that important. Colin is really switched on, and I trust both the service, and if anything happened, he would deal with it quickly and professionally. If I had the kind of profile that meant I needed to protect myself against determined attackers, then tarsnap would be a no-brainer for me.
- toomuchtodo 6y agoBackblaze is also bootstrapped, no VC investment except a small initial round. They’re likely investing in the business using the profits thrown off by their unlimited desktop storage offering.
- nvarsj 6y agoIt's more of the fact that S3 is expensive for storing lots of data, isn't it? I think tarsnap charges only a small margin on top of S3.
- floatboth 6y agoAnd tarsnap possibly uses the most expensive tier of S3: > the original version, which can survive the loss of 2 datacenters, not the "reduced redundancy" version which can only survive the loss of a single datacenter These days, there's not just reduced redundancy, but also infrequent access, which seems better for backups…
- avinassh 6y agomay it's just me, I had a hard time calculating the pricing of tarsnap. Wish they had provided an easy calculator too, on their site.
- varispeed 6y agoDoes rsync.net offer end to end encryption of data?
- rsync 6y agoYes, of course. First, we only offer SSH / TCP22 so the transit is encrypted. Second, we have installed, and maintain on the server side tools like 'borg' and 'rclone'. So while you might just 'rsync' or 'sftp' your data to us (in which cases it would not be encrypted on our end) you can also use sophisticated, encrypted backup tools like (borg, duplicity, git-annex, rclone, restic, etc.) If you choose a tool like that, rsync.net does not hold the encryption keys. The data appears to be random from our viewpoint.
- varispeed 6y agoThis is great, thank you!
- teruakohatu 6y agoIncluding the filenames seems to have been unintentional, looks like they were logging analytics to Facebook, probably an even (form submission) but uploaded the form html with contents. But why they need to submit that to Facebook for paying users I don't understand. The only thing I can think of is excluding active users from advertising... But is that worth the privacy intrusion?
- robbiemitchell 6y agoThey answered in the Twitter thread: they send data about paying users to Facebook so they can build lookalike audience targeting for new user acquisition. Other major ad platforms (Google, LinkedIn) have similar features. This doesn’t look like the right data to be sending FB for that, though.
- mcny 6y agoThat’s my gripe with “if you aren’t a paying customer, you are the product” because if you pay you are still the product. Good job, backblaze. So many years of work building goodwill with transparency in hard disk reports ruined completely with one stroke.
- BeefySwain 6y agoCan someone here translate the PR/marketing speak here for us mere mortals? How does having Facebook tracking on the web front-end of existing and paying users help with lead generation?
- busymom0 6y agoThat PR/marketing tweet comes across as from someone who doesn't understand how big deal this actually is and why customers won't be comfortable with a FB pixel on their dashboards with data filenames and sizes.
- yellow_postit 6y agoAssuming it’s to build lookalike audiences [1] (find me people that act like the paying customers). [1] https://www.facebook.com/business/help/164749007013531?id=401668390442328 https://www.facebook.com/business/help/164749007013531?id=40...
- CuriousNinja 6y agoThanks for the link, I didn't know that was a thing. Having said that, this sounds like "Hey guess what? We are gonna snoop on you, and profile the hell out of you and leak all of your sensitive data all over the place (filenames can be) all because you are a paying customer" That's about the worst way to disrespect a paying customer. Is there a way to easily identify companies that does this, I can avoid them?
- manigandham 6y agoNo, they don't need to snoop on you. Just fire a single pixel saying this is a user that they would like to find more of. FB does the rest based on data in FB, not on Backblaze. The mistake was them just adding it to the online dashboard because that's the easy way to automatically include paying users. > "easily identify companies that does this" No. And this can happen for a lot of reasons as explained so even harder to check for.
- cosmie 6y ago
- busymom0 6y agoSome might find this tweet useful for the time being: > What happens if you resolve http://facebook.com http://facebook.com to 127.0.0.1 via hosts-file? (Or put it into your Pi-Hole DNS Ad-Blocker, or the like.) Does the Backblaze UI still work? > Answer: Seems to work well. You need to add the main domain and sub domains (www. in this case), btw.
- whenlambo 6y agoTake a look on https://wasabi.com/ https://wasabi.com/
- ev1 6y agoSigning up for Wasabi resulted in a deluge of unwanted marketing spam, all written to be "cheerful and your friend" Also, huge warning for Wasabi newbies so you are not surprised. This might wreck your wallet. Your egress is capped at your total amount stored. You cannot store 5TB and have 6TB of downloads against that account. The front page is covered in 'No Charges For Egress' 'no additional charges for egress or API requests'.. etc. This is not written anywhere on the main marketing pages, only behind a small link. Another important point is file deletion. You are required to pay for multiple billing cycles (months) of files. Be very cautious about this. Using it as a temporary bucket will incur significant costs, if you upload a file, YOU WILL PAY FOR 90 DAYS OF THAT FILE. It is almost certainly not cost effective to store files in Wasabi any less than long term. Deleting a file that you just uploaded will incur a deletion fee equivalent to 3 months of storing that file.
- bb101 6y agoOne thing people should bear in mind when looking at Wasabi is that they have a 90 day minimum retention period for their PAYG plans. So if you upload a large file and overwrite it tomorrow, you'll be charged the storage fee for the day plus the storage fee for the remaining 89 days, and then the storage fee for the new file, and so on. The same goes for AWS IA and Glacier classes and Google's Nearline and Coldline. Unless you're storing files for a long time, always factor in the minimum retention period before estimating costs. It'll prevent any nasty billing surprises -- speaking from experience unfortunately.
- mdoms 6y agoWow they're doubling down? Wow.
- nvarsj 6y agoWhy not just use client side encryption and be done with it? Why would anyone browse their personal files remotely on a web ui? If you care about privacy, take the proper precautions. There really isn't any competitor to B2 on price, or convenience (e.g. them sending you a NAS device for recovery).
- kardos 6y agoYep that pretty much burns the service :/
- neilv 6y agoThey could maybe salvage goodwill with genuine corporate soul-searching that ends up asserting/reasserting values -- and leads them to focus on providing trustworthy service to their users, and conspicuously away from some "tech" industry norms of selling out one's users. As a provider of a paid service, it seems like they're in a better position to take the high road than a lot of tech companies are, but they have to decide that's who they are, and be clear they mean it.
- sneak 6y agoImagine if S3 did this. I don't like Amazon, but at least they are security professionals. Ad tracking pixels in your object store dashboard is just clownshoes from a security engineering standpoint, over and above the fact that it's a slimy, dickhead move for a paid service.
- WillPostForFood 6y agoslimy, dickhead move for a paid service It is just a dumb mistake.
- alacombe 6y agoIt 100% voluntary: https://twitter.com/backblaze/status/1373751015594356739 https://twitter.com/backblaze/status/1373751015594356739
- loeg 6y agoI believe the tracking is intentional; I also believe that sending the filenames as part of that is a dumb mistake.
- busymom0 6y agoThe tweet says "I even opted out of their tracking widget thing!" and yet it continues tracking. Major mess up.
- smolder 6y agoA dumb mistake that leaks private information to a third party seems like good reason not to trust or use their service. For me, the idealogue, the fact they're sending any tracking data from the dash to Facebook is enough reason.
- magicalhippo 6y agoThe thing is, even if it was a dumb mistake, it's one of those mistakes a company like Backblaze can't afford to make. If they don't pay attention to stuff like this, then why should I trust them with anything at all? This isn't some minor oopsie, this is failing to deliver on their core product[1]: Top Backblaze B2 Use Case Solutions Backup & Archive Store securely to the cloud incl. safeguarding data on VMs, servers, NAS, and computers [1]: https://www.backblaze.com/b2/cloud-storage.html https://www.backblaze.com/b2/cloud-storage.html
- peteretep 6y agoI'd have thought the set of people who use B2 and people who haven't blocked Facebook trackers would be pretty small. Still a WTF, though.
- KingOfCoders 6y agoI'm a long term B2 customer and left FB a long time ago and block trackers. The major point to me is: If marketing at Backblaze drives decisions that influence security and tech has either no say in this or is not competent enough, it's not a company for me to trust my data with.
- alphabettsy 6y agoI doubt it’s intentional, but companies should seriously consider if the benefits of integrating things like Facebook Analytics tools outweighs the negatives. It seems like considering their audience they would not use Facebook of all things.
- busymom0 6y agoI operate quite a few apps and recently launched a website too which handles a lot of sensitive user data. I decided to make - not having any analytics, trackers and ads the selling points of my apps and sites. Get a lot of positive emails from customers thanking me for that. I was recently even wondering if Google penalizes sites for not having putting their analytics on the sites/apps from appearing on the search results. I legit don't understand why a paid storage service would put a FB pixel on their dashboard which handles user files. It's a completely foreign concept to me. This seems like a screw up but also erodes a lot of trust which is unfortunate as I had been looking at them for past 2 months actually. I even made a post just yesterday and another couple weeks ago on how BackBlaze's inability to set a specific file name, file size limit and expiry date on the pre-signed urls is preventing some of us from switching over from S3 to Backblaze for our storage of app data needs. And surprisingly, I wasn't the only one as I got a few people responding with same concern. https://news.ycombinator.com/item?id=26430959 https://news.ycombinator.com/item?id=26430959 Basically: > A limitation I ran across when using B2 was that their pre-defined url generation doesn't allow you to set file-size limits nor does it allow you to set the file name in the pre-defined url. It simply gives you a pod url to upload it to. So if you are using b2 for storage for lets say image uploads from browser, some malicious user has the ability to modify the network request with whatever file name or file size they want. Next thing you know, you have a 5gb sized image uploads happening.... This pretty much prevents me from using B2 for now. > I ran into the same limitation! IIRC, there also wasn't a way to expire a signed upload URL sooner than whatever the default was, which was hours or maybe a day. I had the exact use case you mentioned, too - image uploads bypassing my backend server. I didn't want the generation of a signed url to, say, upload a profile photo, give carte blanche to create a hidden image host when combined with the limitation that you highlighted. All sorts of bad things could come of that. I ended up just going back to S3 - costs more, but still worth it. Since this is for a site/app which lets users upload data, I am really trying to avoid S3 due to crazy costs. I might look into DigitalOcean's offerings. Anyone have any other recommendations?
- AdmiralAsshat 6y agoJust remember the old adage: Even if you're paying for the product, you're probably still the product.
- risyachka 6y agoPure speculation. In some cases you are, in some you are not. In this particular one you are not.
- withjive 6y agoYou're being down voted because in this case customers are obviously the product. Being sold to Facebook.
- Griffinsauce 6y agoDid you read anything more than the incredibly poorly worded headline? The data was harvested by the Facebook pixel as part of their audience building tech for acquisition of new customers. So you are being sold by Facebook, Backblaze just happens to have been quite careless here but they are not "selling your data". The breach of trust is BB letting third party code on their platform and especially from a particularly untrustworthy third party. That's it, it's egregious and should be taken seriously, that also means discussing it seriously. This kind of hyperbole is counter-productive, it only makes it easier to ignore your concerns as "crazy overreacting".
- C19is20 6y agoDoes anyone care about downvotes? /serious question.
- datapolitical 6y agoI get using FB tracking for your payment flow but it’s sketchy as hell to have it on every page of a paid product.
- ev1 6y agoThis should not be on anything even slightly related to payments
- yeskia 6y agoThis is not a good look. I appreciate that their Twitter account is looking into it, but feel like this should be a pretty quick fix. I'm hesitant to even log into my Backblaze account again until it's sorted.
- allendoerfer 6y agoIn a sense, that makes life easier for me. One less alternative to consider. The name "Blackblaze" is burned forever. You will never win me as a customer. I write this here, because people in similar positions at similar companies might read this. And I do not think I am alone.
- magicalhippo 6y agoYeah off my list as well. Was looking into them as my backup needs had changed somewhat and they'd finally gotten a datasenter in the EU. Shame.
- ajdude 6y agoBeen a huge fan of Backblaze for years and b2 was my plan for server backup; always seemed like a great underdog company in my eyes- they just lost me. I already canceled my Spotify for a similar —Facebook-related— reason.
- throw6119088 6y agoWhat's that with Spotify?
- foepys 6y agoProbably because Spotify is using the Facebook SDK in all their apps. As a user you cannot disable it and a few years back the Facebook SDK caused an outage where you couldn't start Spotify (and many other major apps).
- smnrchrds 6y ago> The name "Blackblaze" is burned forever. So they should be good continuing to use "Backblaze"?
- colechristensen 6y agohttps://help.backblaze.com/hc/en-us/articles/217667238-Backblaze-and-HIPAA https://help.backblaze.com/hc/en-us/articles/217667238-Backb... backblaze signs BAA agreements with companies storing personally identifiable medical information, I wouldn't believe anyone who told me that this facebook data leak was turned off for those customers; they should immediately be investigated and fined if any such breaches indeed happened. Full agreement, any interest and goodwill towards the company is now completely gone.
- Zhenya 6y agoThis is actually pretty crazy. You pay for this service, and then they share, what many consider to be pii, directly to facebook. I'm guessing if you're logged into facebook, now FB can correlate: 1) you use a backup service 2) all the metadata from the file names Whoops. This is why my network runs pi-hole / diversion with all tracking blocked network wide.
- tuwtuwtuwtuw 6y agoWho are the "many" that consider file name PII?
- ncallaway 6y agoI would generally consider a filename to be the kind of field that could easily contain PII. Names or identifiers are the kinds of things that very often end up in file names.
- ipnon 6y agoURLs, of which file names could be considered a subset, are definitely considered PII.[0] [0] https://cphs.berkeley.edu/hipaa/hipaa18.html https://cphs.berkeley.edu/hipaa/hipaa18.html
- Zhenya 6y agoJohn_smith_college_graduation.jpg
- KingOfCoders 6y agoJohn_smith_doctored_college_graduation.jpg
- Nadya 6y agosisters_graduation_2019.jpg Rebecca_Range_will_and_final_testament.pdf NadyaNayme - Resume - [Company Name].pdf divorce_process.pdf steps-to-take-after-a-car-accident.pdf personal_injury_michael_perez_west_coast_trial_lawyers.pdf embarassing_porno_filename.mp4 And this is a smalls subset of filenames that could not only provide PII but also potentially embarrassing or private information that isn't identifying but would be accompanied with files that are personally identifying.
- CA0DA 6y agoWow. That's a big deal and the person in marketing/sales who requested the Facebook Ad pixel did not realize that's a big deal.
- artellectual 6y agoI'm in engineering of a financial services. When we built our front-end UI for eKYC our marketing requested for google tag manager / facebook pixel and various other tracking features to be built. I had to fight hard as an engineer to make sure that it does not happen. We had meetings after meetings, and it took a lot of effort for me to explain the risk of data leakage. I was questioned on my "insecurity" for not "trusting" people. It was not a nice experience. I had to inform them that tracking needs to be dealt with properly, not just lazily install google tag manager because it gives marketing 'flexibility'.
- 0xy 6y agoNever ever give marketing access to deploy arbitrary JS onto your website under any circumstances. Google Tag Manager is an absolute cancer on web development. Once it's in, you'll never get rid of it.
- artellectual 6y agoAgreed!
- Natsu 6y agoThe only thing I knew about 'tag manager' before this was that it was always blocked by NoScript. Your comment made me go look up what it does and now I know that I will never unblock it. Apparently it lets people drop in random code from a bunch of different analytics platforms, so it's pretty much guaranteed to consist entirely of the sort of stuff I have NoScript enabled to block in the first place.
- 0xy 6y agoI've seen GTM take down production multiple times because of marketing shipping random JS with no approvals. Some random guy in his basement assured someone in marketing they could handle our volume? Chuck their tag in and watch their website get DDoS'ed with millions of requests per minute, which takes out our website because marketing made it fail loudly.
- forgingahead 6y agoSeems like a poor decision / dumb mistake. Fix it and move on. I can understand the impetus to need FB ads & lookalike audiences to grow the company -- it serves no one if BackBlaze goes bust. It's obviously wrong to send filenames to FB as well. If they fix this, goodwill will be burned, but it's not a dealbreaker for using them. If they don't fix it, then yes by all means leave the service.
- freitasm 6y agoThat's... terrible. Just ill-thought out.
- mgraczyk 6y agoJust an FYI in case this isn't clear to people. Even though this is a serious problem, it's super unlikely that FB is using the filenames in any way, or aware they were being sent. That part is probably just a (really awful) mistake.
- walrus01 6y agoBut you have absolutely no way of knowing for certain.
- mgraczyk 6y agoThat's not actually true, it's possible to wait and see if FB publishes a statement. If they do, you'll know.
- swiley 6y agoThis is the same company that has published iOS apps that have silently escaped the sandbox. I think pathological is probably an appropriate word for them and wouldn't trust anything remotely related to them with a single bit of personal data.
- mgraczyk 6y agoSounds unrelated
- tzmudzin 6y agoPossibly, but company culture transcends the immediate team. And if your business model is based on data siphoned off every possible channel, the concern is very justified.
- exikyut 6y ago> silently escaped the sandbox I can has reference? Sounds genuinely interesting, and that info didn't find its way through the rock I'm apparently under.
- 6y ago
- simpleguitar 6y agoI’m glad that my backups to BB obfuscates file names and sizes in addition to being encrypted.
- KingOfCoders 6y agoYes but but if marketing drives their security decisions, where else do they leak? What data do they sell? If they play lose with security, in what areas do they also play lose with your security? With your login email? Payment data? Do they sell your credit history? How do they vet their employees?
- mjthompson 6y agoCan browsers please just disable tracking pixels already? I look forward to the day of reckoning.
- ocdtrekkie 6y agoBrowsers blocking it is a stopgap, adtech companies will find new and innovative ways to track users. You need strong legislation and even stronger penalties. The only way to stop pervasive tracking is to financially ruin any company that employs it.
- fooey 6y agoAll you'll do is shift it to people hosting "proxy" services on their main domain so they can pipeline the requests through if the browser stops it. That's extra effort, but the big ones will do it
- deleted 6y ago[deleted]
- sergiotapia 6y agoWhy are they adding that stuff if this is a paid service? This is such a bummer.
- trhway 6y agoIt is just lazy engineering. They want lookalikes. Fine. Collect, clean, mask the data in house and send it to FB. Dumpling to a 3rd party data directly from page is irresponsible to say the least as such "Opsies" are bound to happen.
- jiveturkey 6y agothey can’t do that. they don’t have the original data (it is owned by fb) to create the lookalike audience. methinks you don’t understand how these tracking doohickeys work
- hnick 6y agoFB allows a custom audience to be built from identifiers like email, phone, etc. They claim it is hashed before use so they never see the raw data (not sure if they mean hashed client side or server side though). This will rely on a user's FB account having the same email as used for BB, which could be unlikely in the case a company is paying for it. But it should work well enough for retail targeting. https://www.facebook.com/business/help/341425252616329?id=2469097953376494 https://www.facebook.com/business/help/341425252616329?id=24...
- searchableguy 6y agoAll they say is they don't learn anything new about your customers. I would take that statement with a lot of caution. For one, they should know that I am a bb customer now. Having a graph of all the companies and products I use is huge PI. > They claim it is hashed before use so they never see the raw data If they can match hash data with real data, they can know more than they did before. Depending on what algorithm they use for hashing (no mention of it), they could be using a similarity hash so that will minimize changes if there are minor differences in the dataset. Let's say I find a profile through comparing hash of email to email in Facebook's database. I can then compare additional information to see if a customer has provided incorrect information to Facebook as a user. Facebook could check if my address is similar to online shopping sites I use and if not, flag the account.
- bogomipz 6y agoWow they pretty much flushed all user trust and good will down the toilet with this. I had considered using them in the past but I sure won't be considering them now. Even if this were an accident how are there not procedures in place that would have required approval and vetting?
- gullevek 6y agoI cannot confirm this. The only thing I see is googletagmanger (blocked by uBlock origin). No facebook access at all.
- macNchz 6y agoTag manager is used for managing all of the various tracking and analytics snippets for a site in a central location, so if your ad blocker stops tag manager itself, you often preempt many other things that would run. In this case I’d assume the facebook pixel was installed using tag manager.
- gullevek 6y agoNot going to try that and am happy that this is all blocked here.
- rachelbythebay 6y agoYeah, pretty sure that's the tracking pixel. What the hell is it doing there?
- zaroth 6y agoIMO if you are browsing your B2 buckets with an open browser running arbitrary JavaScript you already lost. If file names are truly sensitive you have to actually be pragmatic in how you choose to render them.
- swiley 6y agoWow that's a fantastically effective way to burn a lot of good will.
- tzs 6y agoThis only happens if you use their web interface, right? If you only interact with your B2 storage via their API, such as through a backup program like Arq, there would be no Facebook involvement?
- sreitshamer 6y agoArq doesn't write recognizable filenames to B2, just checksum-based object identifiers, so it's not an issue for Arq backup data stored at B2.
- azernik 6y agoOne easy way to avoid this kind of mistake in your own product: make a clear distinction between your publicly-facing web site ("corpweb") and your web app for logged-in users. Preferably, they should be served from separate infrastructure. Corpweb should be as static as possible, except for whatever third-party JS the marketing professionals think is necessary. It's their job, they know what's best. Your app should have zero third-party JS except for technical analytics (New Relic, Datadog, whatever). (This distinction can be fuzzier for free services, and for consumer stuff with non-sensitive data; Backblaze is neither of those.)
- walrus01 6y agoI agree with this delineation, and if necessary, it lets the sales/marketing people go absolutely wild with the tracking, analytics and CRM-system integration on the public facing marketing website, should the C-level people decide to allow them to do so.
- deleted 6y ago[deleted]
- tarsinge 6y agoI’m baffled by this mentality but I guess it explains a lot. At which point are you concerned about what other services or even the company is doing? That’s a real slippery slope for me.
- smolder 6y agoI agree with this approach but there's an issue with "it's their job, they know best". They will inevitably want to put the same tracking crap in the logged-in site as well, so they can see how visitors converted into users, how they used the site once they were users, how valuable that made them as customers, and so on. As long as FB/other analytics firm is saying "we can help you market better with additional data", marketers are going to advocate sharing it.
- azernik 6y ago
- anoncow 6y agoGiven Backblaze's good track record, I am assuming this was default FB pixel tracking behaviour. A company like BB should however know better than to use FB tracking on their paid products.
- mgraczyk 6y agoIt's not. The url param that contains the data is set by client javascript code.
- miked85 6y agoDamnit...I guess there had to be a reason they are so much cheaper. Account deleted.
- cosmie 6y agoPSA for any devs out there implementing FB Pixels: – Facebook's pixel will, by default, attach click listeners to the page and send back associated metadata. This simplifies implementation needs, but can create unintentional information leaks in privileged contexts. To disable this behavior, there's a flag[1] you can use. After which, you can manually trigger FB Pixel hits and control both when they're fired and what information is included in them. – There's a feature for the FB Pixel called Advanced Matching[2] that allows you to send hashed PII as parameters with your FB events. "Automatic Advanced Matching" can be enabled at any time via a toggle in the FB interface. I believe that setting autoConfig to false as mentioned above will similarly prevent Automatic Advanced Matching from working (since it disables the auto-creation of all those listeners to begin with). When manually triggering pixel calls like above, you can use this functionality via "Manual Advanced Matching"[3]. As a general rule, I'd strongly encourage anyone implementing a Facebook pixel to also include the autoConfig = false flag. This makes it work like most other pixels, where the base tag just instantiates an object. After which, hits only occur when explicitly defined in the site code, and include specifically what details you include in it. That way you're fully aware of the scope of data disclosure happening and any need from marketing to include sensitive (or potentially sensitive) information in these calls has to be explicitly requested (and theoretically vetted) as part of the standard dev process. [1] https://developers.facebook.com/docs/facebook-pixel/advanced#automatic-configuration https://developers.facebook.com/docs/facebook-pixel/advanced... [2] https://www.facebook.com/business/help/611774685654668 https://www.facebook.com/business/help/611774685654668 [3] https://developers.facebook.com/docs/facebook-pixel/advanced/advanced-matching https://developers.facebook.com/docs/facebook-pixel/advanced...
- miked85 6y agoThis is good info, but Backblaze shouldn't even be using FB pixels to begin with.
- KingOfCoders 6y agoExactly. When the most important thing for a backup provider is the trust people put in you, don't do anything that risks that trust.
- emptysongglass 6y agoWhat's the alternative for unlimited backups from personal machines?
- deleted 6y ago[deleted]
- throw6119088 6y agoTarsnap maybe? Not unlimited but encrypted. The author is on HN as 'cperciva'. https://en.wikipedia.org/wiki/Tarsnap https://en.wikipedia.org/wiki/Tarsnap Recent talk with the creator: https://www.youtube.com/watch?v=Rg3NQLzMBKw https://www.youtube.com/watch?v=Rg3NQLzMBKw
- noisy_boy 6y agoGreat. I finally shook off my lethargy and did some research and thought they were perfect feature and price-wise. Setup a backup script using rclone and did a backup yesterday - worked great too - now this. Had a look at rsync.net - too pricey for my puny sub-500GB data.
- rsync 6y agoEmail us ...
- noisy_boy 6y agoThanks - just did.
- AnonC 6y agoI'm curious to know what emailing you would help with. Is it a lower than published pricing tier plus no 400GB minimum order? Or some special plan based on geographical location? Something else? FWIW, I agree with you on your previous posts/thoughts and I'm all for pay for use and against flat rate/unlimited plans, as long as it fits in a budget.
- noisy_boy 6y agoI was curious about that as well. Haven't heard from them after my email so that probably got lost in the hate-Backblaze haze yesterday :)
- PhantomGremlin 6y agoA long time ago they said this about a discount. I don't know how relevant that is anymore: https://news.ycombinator.com/item?id=9186428 https://news.ycombinator.com/item?id=9186428 Just google for it and I'm sure you'll find more comments along those lines.
- KingOfCoders 6y agoSame boat here, rsync too pricy for me, though I used them in some startups (as CTO) and service was excellent from people who know what they do.
- berniemadoff69 6y agoPay careful attention to the response from Backblaze: > "Hi Brett! The pixels we use are primarily for audience building when we advertise on other platforms like Facebook for example." [...] The carefully calculated cutesy "Hi Brett!" with the exclamation point is the same reason big tech companies use infantile graphics [0]: by seeming playful, they create the illusion they are a Safe Friend you can Trust. [0] http://jollo.org/LNT/public/nursery.html http://jollo.org/LNT/public/nursery.html
- Edman274 6y agoUsing a salutation, and addressing someone by name is not a conspiracy to make people trust you. The things that you should care about are the banality of evil, and that no one believes that they've done anything wrong. I live in the Midwest and my job is to make low-impact CRUD applications for a small car insurance company. I would use the same salutation, because I have been taught that that is what I'm supposed to do. I wasn't coached in some session on how to trick people into thinking I'm their buddy - it just becomes part of the shared social vocabulary.
- Scene_Cast2 6y agoModern business vocabulary has shifted from "Dear Mr. LastName," to "Hi FirstName!". This shift happened first in more "trendy" places, although most everyone has already moved on to using informal language in customer relations. I do agree with your point about banality of evil.
- petecooper 6y agoI’m a B2 user via Arq, and this still stings.
- isoprophlex 6y agoDoes anyone have any idea how long this has been going on? And: it only happens when rendering the filenames to a browser window right? So only when I browse folder y in bucket x are my filenames for that folder shared with FB? Backblaze, I really enjoyed being a paying customer. Until now. Bunch of dorks. Edit: goes to show you have to encrypt EVERYTHING at rest, even file names...
- Griffinsauce 6y ago> Bunch of dorks. This is uncalled for and devalues the rest of your comment.
- isoprophlex 6y agoThe ad hominem stuff isn't classy at all. I agree. It was a heat of the moment thing. Mostly I'm being angry at invasive tracking being the norm when going down the 'growth hacking' path. I must lack perspective but it saddens me that contextual advertising and focus groups apparently aren't enough. edit: to make up by adding something actually useful to the discussion... I checked and I don't see any DNS requests being made for any facebook domain when browsing my B2 buckets. Maybe by now they got rid of the tracking pixel?
- breakingcups 6y agoDo you have an adblocker installed? Or do you uses Firefox with its built-in tracker protection?
- jhgb 6y ago> The ad hominem stuff isn't classy at all. I agree. I don't see any ad hominem there. Where is it?
- deadbytes 6y ago>goes to show you have to encrypt EVERYTHING at rest, even file names Or just block all third-party javascript. Facebook code shouldn't be running anywhere aside from facebook.com
- aitchnyu 6y agoI heard of a company with serious security process having Facebook like buttons on each page of their internal wiki.
- November-Echo 6y agoI don’t understand how those services always claim to be „HIPAA compliant“ and yet they do this kind of stuff. Can anybody enlighten me?
- dylan604 6y agoNot all of the various silos within evilCorp structure knows what the others are doing, if even the groups within the same silos know what is going on.
- foepys 6y agoIt's inexcusable, honestly. These are the kind of decisions that not only need department head approval but also CTO approval. If the CTO didn't see it coming, they failed. HIPAA isn't some random cert you have to satisfy a single big customer. It needs to be a priority and all business decisions have to be made around it. GDPR is another one.
- dleslie 6y agoIn what way did integrating FB Pixels spyware into their product seem like a good idea? I'm paying for Backblaze, now I'm wondering if that was a mistake.
- KingOfCoders 6y agoI'm also a long term paying customer and I'm leaving for sure. If they can have FB pixels in the admin area, they basically have no security processes working. If marketing drives their tech decisions, this is not a company to trust your data to.
- ing33k 6y agoI get why they might be doing it ( audience building ), but they should have restricted it only to their HomePage or some info pages. ( check azernik's response on how to do this correctly ! ) It's easy to attack Backblaze. Before attacking them for this, please make sure the company that you are working for or building doesn't do the same thing. ( I know for a fact that a lot of startups make heavy use of Audience building). Note : I have used their service in the past and moved on to OVH due to a different issue in the past . https://www.backblaze.com/blog/b2-503-500-server-error/ https://www.backblaze.com/blog/b2-503-500-server-error/
- KingOfCoders 6y agoWhat product from OVH are you using? Can it use rclone?
- ing33k 6y agoObject Storage - https://www.ovhcloud.com/en/public-cloud/object-storage/ https://www.ovhcloud.com/en/public-cloud/object-storage/ yes, they support rclone https://docs.ovh.com/gb/en/storage/sync-rclone-object-storage/ https://docs.ovh.com/gb/en/storage/sync-rclone-object-storag...
- KingOfCoders 6y agoThanks!
- newscracker 6y ago> It's easy to attack Backblaze. > Before attacking them for this, please make sure the company that you are working for or building doesn't do the same thing. I don't get why these two are related at all. One can do both the second and the first. By their own admission in other HN threads, Backblaze earns several million dollars a year and is proud not to have VC backing. So it doesn't seem like anybody is attacking an underdog who's struggling to change the status quo and needs to be held to lower standards.
- accountofme 6y agoSigh. I was considering backblaze for s3 storage. Not anymore.
- submeta 6y agoI was planning on deploying Backblaze for various remote backup projects. This post makes me look for alternatives.
- intricatedetail 6y agoThis is crazy. Who knows what else they send then. Time to reconsider.
- williamsmj 6y agoThe login page now redirects to https://www.backblaze.com/maintenance.html https://www.backblaze.com/maintenance.html > We're performing site maintenance. The page you're looking for will be back soon. I guess somebody is having a rough Sunday night.
- williamsmj 6y agoAnd they're back. As far as I can tell, they've removed tracking. ublock is no longer blocking anything on that page, and I do not see the cd[buttonFeatures] property in headers.
- deleted 6y ago[deleted]
- rosstex 6y agoI do hope Backblaze will address this in the morning.
- intricatedetail 6y agoIf they track users this way it means this data is used for manipulation and extracting value. This is a nasty practice and I despise companies that do it. Only metric you should be interested is how many subscriptions you have. If you think people don't click where you want are you going to use tricks and social engineering to make them? Do you do circlejerk meetings about your funnels how you lure people in? This whole marketing culture is a cancer.
- exikyut 6y agoWhat nobody seems to be pointing out here: Ben's account contains exactly what you want to hand off to Someone Else's Storage™. It's just GPG-encrypted backups. Look. Here's what the innerText says, with the `\n`s interpreted: Upload Download New Folder Delete Snapshot Selected: 0 Files: 0 bytes Name Size Uploaded dbbackup_06_23_2017.sql.gpg 10.2 GB 06/23/2017 22:24 dbbackup_06_24_2017.sql.gpg 10.2 GB 06/24/2017 22:24 dbbackup_07_02_2017.sql.gpg 10.2 GB 07/02/2017 22:25 dbbackup_07_03_2017.sql.gpg 10.2 GB 07/03/2017 22:26 dbbackup_07_04_2017.sql.gpg 10.2 GB 07/04/2017 22:25 dbbackup_07_05_2017.sql.gpg 10.2 GB 07/05/2017 22:25 key.asc 3.9 KB 02/24/2016 22:01 Once again, noise. Perfect. (For the curious: I couldn't be bothered to install https://projectnaptha.com/ https://projectnaptha.com/ (probably would've worked), I just resized a terminal to the same column width and blindly retyped the text into a shell printf statement.)
- cm2187 6y agoAnd also when you upload something to blackblaze, you are already handing out your files to a third party. There shouldn't be anything sensitive that you didn't encrypt client side.
- nvarsj 6y agoExactly... all my b2 files are client side encrypted. It would be a really bad idea not to do that.
- dbmnt 6y agoExcept... are the filenames encrypted? For many, if not most, filenames are plaintext. Backblaze's default "SSE-B2" encryption leaves the filenames in plaintext. GPG encryption typically leaves the filenames in plaintext. There's still information people consider private and didn't expect to be shared with Facebook.
- nvarsj 6y agoThat's server side encryption. My point is if you store sensitive files on a remote server not under your complete control, you should always use client side encryption. Good client side encryption will not leak filenames/directories either. A lot of them do, unfortunately.
- KingOfCoders 6y agoAmateur hour, tracking pixel in admin areas. Have been a Backblaze customer for many years, mostly because of their state of HD here on HN. Lost all confidence in Backlaze. Alternatives? Had been using rsyncnet in startups for many years, but was more expensive (now using Backblaze for storing GBs of raw images from DSLRs)
- nerdponx 6y agoSome people in the thread are mentioning OVH. I would love to know about alternatives myself, I have zero trust in Backblaze now.
- christophilus 6y agoHappy Wasabi customer here. On the other hand, I suspect Backblaze won't pull something like this again after this backlash.
- atYevP 6y agoYev from Backblaze here -> we’ve looked into and verified the issue and have pushed out a fix. We will continue to investigate and will provide updates as we have them.
- KingOfCoders 6y agoCan you fix all your security processes too? And make tech drive tech decisions not let marketing define security? Thanks.
- tobr 6y agoThanks for participating in this thread. As a longtime paying customer, I consider this a monumental security breach and I will be leaving the service. It’s clear that Backblaze have prioritized growth hacking or whatever over the security and privacy of me as a paying customer, and that your security processes are woefully inadequate.
- skinkestek 6y agoWell, for many of us this is where it starts: I might very well start using backblaze next year or maybe even next month[1], but that is depending on the outcome of this event. For a comparison: one good friend once called med to apologize that he had been laughing behind my back with some friends. Guess who I definitely trust today? The one admitted his mistake. He always was a nice bloke and I guess he will never ever do anything like that ever again. [1]: I won't start using it this week or the next however.
- tobr 6y agoWhat kind of scenario do you have in mind? I think it’s possible to turn an incident like this around, PR-wise, but I can’t see how they will explain how they can sell something as secure and trusted, when their security process was unable to discover that they had deployed spyware in production. If it was there for a few hours before it was discovered and removed, well maybe.
- ehnto 6y ago
- asiachick 6y agoI know this is a common experience on many sites but I'm a paying customer on flickr.com (have been since ~2004) and other day I visited in a new browser or in private mode and got the "we use cookies..." dialog, it list over 90 advertising companies and probably over 500 tracking options. Want to see it, go to flickr.com in a private window, it should pop up something about cookies. Pick "Manage Settings". It's insane.
- berniemadoff69 6y agowow you werent joking. heres a gif of it https://i.imgur.com/Uvei8lX.gif https://i.imgur.com/Uvei8lX.gif
- tobr 6y agoThis means Backblaze has to send out a GDPR data breach notification within 72 hours, right?
- KingOfCoders 6y agoYes.
- KingOfCoders 6y agoIf this leaks personal information that might have been in the filenames (abc_abc_xyz-street_cancer_report.pdf), this is a GDPR incident for Backlaze.
- busymom0 6y agoTheir staff says they have pushed out a fix: https://www.reddit.com/r/backblaze/comments/madqug/backblaze_submitting_names_and_sizes_of_files_in/grsbndi/ https://www.reddit.com/r/backblaze/comments/madqug/backblaze... I looked at my network requests and I no longer see the googletagmanager requests being made (which I think calls the FB pixel code).
- alkonaut 6y agoNot even sending an advertising id (or anything else) to a Facebook url when I visit a public non-Facebook page is ok. This is unbelievably far from ok.
- benja123 6y agoIt’s bad that Backblaze did not do their due diligence while integrating with Facebook pixel, but the bigger problem is the tendency of third party integrations having default settings that are overly aggressive when it comes to access of user/site data. This should be a warning to every developer, when you integrate with any third party - don’t just copy the default snippet of code they recommend, read the docs thoroughly and then test/monitor what is being sent back to the third party. And if you are writing a third party widget, be considerate and make the defaults the least aggressive possible when it comes to accessing user/site/server data.
- xorcist 6y agoDue diligence when installing a tracking script? Is this not a third party script which runs in the page context?
- qeternity 6y ago> and then test/monitor what is being sent back to the third party. Trust, but verify. Especially FB. I see a lot of comments about a paid service using tracking pixels. I understand the backlash but this is also how companies that are taking your hard earned money improve and iterate on their products. I know people don’t like tracking but usage analytics are invaluable to improving the product. There’s only so much customer engagement you can do, and often that leaves really unexpected outcomes in the dark (if you asked most customers, they’d ask for a faster horse). The issue here is that Backblaze injected untrusted code into their core product which by its very nature is extremely sensitive. I’m not really sure what they were thinking. Rolling your own tracking is a pain, but some security contexts demand it.
- skinkestek 6y ago> Trust, but verify. Especially FB. In the particular case of Facebook many of us simplify to "don't trust if not absolutely necessary" ;-)
- happymellon 6y ago
- iforgotpassword 6y agoIt's disappointing that a company dealing with sensitive data resorts to third party tracking solutions (and then Facebook of all things!) Especially a tech shop like backblaze who have engineers building amazing tech in general. But then you cheap out on implementing some basic metrics for the web UI. Do you even need all the bells and whistles Facebook offers?
- dusted 6y ago"ouch"
- Robin_Message 6y agoSurely Backblaze needs to do more than just fix the pixel and "investigate". At a minimum: - provide customers with an exact timeline of when the pixel was introduced so they can work out for themselves what data has been leaked - report themselves to relevant regulators in each jurisdiction that they operate in for leaking customers' sensitive data – under the EU GDPR there is a time limit for this. - get on the phone to Facebook and beg them to demonstrably delete the data Filenames are not the worst possible thing to leak, but they can be sensitive and it's not good enough to just go "oh, oops, we implemented it wrong, we've fixed it now".
- Groxx 6y agoAnd this is why we use adblockers. They prevent quite a broad range annoyances, accidents, negligence, and malicious intent.
- electro_blah 6y agoWe take your privacy seriously - coming soon.
- trollied 6y agoThey have pushed a fix https://twitter.com/backblaze/status/1373890710680694787?s=21 https://twitter.com/backblaze/status/1373890710680694787?s=2...
- leokennis 6y agoExhibit 9374729 in the case “use ad and tracker blocking at the DNS level”
- xenio 6y agoDumb question, can Backblaze ask Facebook to delete the submitted data?
- jacquesm 6y agoWhy is Backblaze even connecting to FB in the first place? It would appear that the two have absolutely nothing to do with each other. As a backup entity your first priority should be end-user trust, you can't just squander that in the name of growth.
- tehbeard 6y agoIs this just an issue with the B2 web app, or does it also extend into the personal backup UI?
- cameronperot 6y agoQuite disappointing. I'm glad I always encrypt my uploads to Backblaze before uploading using Restic [1:2]. [1] https://github.com/restic/restic https://github.com/restic/restic [2] https://help.backblaze.com/hc/en-us/articles/115002880514-How-to-configure-Backblaze-B2-with-Restic-on-Linux https://help.backblaze.com/hc/en-us/articles/115002880514-Ho...
- ksec 6y agoAnything Facebook brings the worst out on HN. Not a single bit of decency were given to Backblaze. Instead HN demands them to be burned to oblivion.
- christophilus 6y agoI think Backblaze will think two or three times before integrating spyware into their product in the future, so I agree that jumping ship is the wrong answer. Now, if it happens again, then yes. I think jumping ship is the right move.
- jcims 6y agoIs this just getting press because it’s Facebook? Anyone that works at a company that runs a proxy server to access the Internet should just check the referrer fields leaving the company for requests to sites like Gravatar.
- therealtbs 6y agoIt's getting press because it is siphoning off the page contents which include the names of files you have stored. It is quite a different scenario than just loading an image through an img-tag (which can be controlled through things like Referrer-Policy these days)
- jcims 6y agoIn the instance maybe, but the effect is similar over time. Both of these have effective countermeasures.
- genericstorage3 6y agoThey also don't allow you to encrypt data on first setup. So you need to first upload a few files unencrypted before you can set your key. That's why I'm not using them. Basic encryption feature is not thought well...
- tinus_hn 6y agoWhy do people even implement this? Does Facebook pay for this kind of data?
- lazyweb 6y agoI know, this shouldn't have happened. But Ublock Origin all the way, always. The days of trusting any website without blockers are gone.
- phatfish 6y agoI was just thinking the other day, was it worth using rclone to encrypt the file names in the B2 bucket I used for backup. Given that it limits you to the rclone tool to access files for recovery. A resounding yes it seems...
- imglorp 6y agoI still don't understand why anyone need third party pixels on a paid professional service. Is it really so hard to count GET's in a server log?
- deleted 6y ago[deleted]
- 2ion 6y agoGood thing I use remote cloud storage extensively but with client-side encryption of content, directory and filenames. Let this be a lesson.
- pdimitar 6y agoThis should be trivially and universally fixable everywhere: Never include the 3rd party marketing scripts on any pages where a user is authenticated. But of course that would deprive many companies of mountains of valuable data so it ain't ever happening, right? --- Also, am I missing something obvious here? If Backblaze -- or anyone else really -- wants analytics, what do they need the Facebook pixel for? There are so many good analytics services out there.
- intrasight 6y agoThe titled implied for me that Backblaze send FB a database with all their customers file metadata. I still don't understand what really happened.
- kmfrk 6y agoConsidering this, do Backblaze regularly undergo audits to make sure there aren't any security and privacy holes that go undetected? I know now all services do this, but Mullvad at least have built a nice VPN reputation on account of that. Might be worth looking into, if nothing else then for public perception reasons.
- deleted 6y ago[deleted]
- foofoo55 6y agoNote that this is for the web UI of their "B2" file-sharing service, and not general backups. Did their web UI for backups have the same issue? What meta-data and file name info is sent to Backblaze when using their end-to-end encryption option for backups?
- williesleg 6y agoIt's baked into the go language. Google tracks it at the microcode level.
- londons_explore 6y agoI hope that Backblaze: * Announces this breach to the relevant data protection authorities. They have 48 hours from learning about it to giving an initial report to the UK's ICO. * Makes a blog post apologizing, explaining how it happened, and what they've done to prevent it happening again.
- IOT_Apprentice 6y agoMy question is, WHY is Backblaze using Facebook Pixel at all? Exactly what business purpose does it serve from a customer perspective? Why are you tracking ADVERTISING on a portal with private customer data?
- betaby 6y agoWe are at the pint where it would be wise just to black-hole FB owned IP space on a production/corporate networks.
- Tepix 6y agoTalking about encrypting backups, what's the best way to do it your remote backup storage only offers rsync?
- newscracker 6y agoI'm guessing Backblaze will come back with a more detailed report on this, but I wonder if the data already given to Facebook can be deleted properly (so that Facebook doesn't have it to use for other purposes elsewhere). Is that even possible to get done for a relatively large company like Backblaze? Also, isn't this a violation of GDPR in EU, or is there a "you opted in by default when you logged in and so it's not our fault" argument in play here?