10 ms·
The unlimited storage that Google promised my university is being discontinued
- wolpoli 5y agoRecent events indicate that the G Suite team is under pressure to cut storage cost. I wonder what's next.
- kyrra 5y agoGoogler, opinions are my own. Not just G Suite. Google Photos + Pixel unlimited storage[0], along with Google Photos free storage[1]. [0] https://9to5google.com/2022/01/31/pixel-3-google-photos-backup/ https://9to5google.com/2022/01/31/pixel-3-google-photos-back... [1] https://support.google.com/photos/answer/10100180?hl=en https://support.google.com/photos/answer/10100180?hl=en
- echelon 5y agoYes, but why? Why is Google cutting back so much? I thought storage was cheap. How much have they overextended themselves?
- fomine3 5y agoEven if storage is cheap, they can charge for their service. Or possibly abusers are too aggressive, see r/DataHoarder. Also Chia faming on cloud storage is possible.
- rs999gti 5y ago> Promised Like a written contract or this University just assumed there was unlimited Google storage?
- smnrchrds 5y agoLike everything else in the software world, I am sure they always had a "we can change or cancel anything anytime we want for whatever reason or no reason at all" clause in their written contracts.
- 908B64B197 5y agoI'm surprised they weren't relying on internal HPC resources. Here Stanford has centralized almost everything to use Sherlock[0], and apparently it's quite a nice environment. [0] https://www.sherlock.stanford.edu/docs/ https://www.sherlock.stanford.edu/docs/
- d3nj4l 5y agoA follow up tweet (https://twitter.com/BWJones/status/1490802663499317250 https://twitter.com/BWJones/status/1490802663499317250) mentions that they are maintaining their own servers and don't have to migrate data.
- pmontra 5y agoDo they have a contract with penalties for non delivery?
- jakedata 5y agoStarfish is getting great traction in universities and labs. It can do amazing things for storage analysis and migration across filesystems and cloud buckets. https://www.starfishstorage.com https://www.starfishstorage.com
- d3nj4l 5y agoWhat is it with Google pulling off the bait and switch on so many products at once? Unlimited high quality photos for Pixel owners, free workspace for personal/family users and unlimited drive storage for education are all going paid over the span of ~a year. It's almost like Google's exploding from within.
- chaostheory 5y agoThis isn't the first time, nor the last. They've been doing this for years as part of their kill old products and replace them with similar new one products culture. Google Music comes to mind.
- syspec 5y agoThey probably started looking at their GCP bill
- tux3 5y agoIf that's the case, and not a deliberate aggressive strategy to acquire users ("bait and switch"), Colab users playing with GPT on free TPUs and GPUs should be by far the most worried. The cost of storage at Google's scale is low. GPUs/TPUs are another thing entirely.
- deleted 5y ago[deleted]
- echelon 5y agoThe community of https://fakeyou.com https://fakeyou.com users has been using Colab to train TTS models. Changes late last year to available free instance types forced most of our users to have to pay for the pro plan.
- morbia 5y agoStorage is cheap, but egress/ingress in and out of gcp is not. I suspect the real reason Google are doing this is a mixture of the two, to reduce network costs and because they've got enough users now.
- jsnell 5y agoThis was announced a year ago [0], not with just a couple of months of notice like the tweets claim. The announcement also included the rationale. [0] https://blog.google/outreach-initiatives/education/google-workspace-for-education/ https://blog.google/outreach-initiatives/education/google-wo..., the "new storage policy" section.
- IncRnd 5y agoThe obvious rationale, not wanting to really give unlimited storage, is exactly what their storage policy section said. This is simply a sales tactic that people keep falling for over and over, as if there is really something for nothing forever.
- mort96 5y agoMakes one wonder why it's even legal for companies to just... lie in their marketing. Why aren't they getting sued? For that matter, why aren't cell companies getting sued for "unlimited" data with data caps? Why aren't there laws requiring advertising to be truthful?
- ThunderSizzle 5y agoBecause of the asterisk. Buy a new phone and get unlimited data* * Speeds may be throttled after a certain about of data usage Technically still unlimited.
- IncRnd 5y ago> Technically still unlimited. If there is a cap of 1 byte per hour, there is absolutely a limit of 720 bytes per 30 day month. Whenever there is a cap, it is not "technically unlimited".
- ThunderSizzle 5y agoWith that logic, everything is capped by the network speed. And that's why there's typically a 2nd asterisk about speeds dependent on network conditions. Still legally advertised as unlimited, thanks to the fine print.
- tempnow987 5y agoAnytime you see free and unlimited it is most likely a lie. Absolutely with all the small players. But even the big players as we see here.
- tzfld 5y agoJust waiting for youtube to remove unlimited video uploads.
- hiptobecubic 5y agoHonestly they should. Most videos have zero views. Zero. Not even the owners watching them to make sure they uploaded correctly.
- tempnow987 5y agoThere are folks streaming security cameras onto youtube. I think you are limited to 12 hours / 250 GB per clip? The bandwidth / storage demands of video are not insignificant. Someday someone will do something - ie, insure you are paying for some kind of service.
- droopyEyelids 5y agoOne may foresee these changes by determining whether one lives in a post-scarcity society or not
- WalterBright 5y agoUnlimited is also limited by the bandwidth to transfer the data.
- kevincox 5y agoI was told by an SRE on Drive that the unlimited offer was legit because the at the rate you could upload data to drive the costs were still far bellow the bill. Of course that seemed wrong because if you are uploading for years $5 a month isn't going to keep covering all of that storage. I imagine that part of this assumption that storage costs would keep dropping and presumably this isn't happening as quickly as originally predicted.
- 300bps 5y agoI look at this two different ways. 1) There is no such thing as "unlimited" anything with technology. Either you haven't read the fine print of your provider's terms or your provider will at some point have to change them. 2) What's interesting is that storage costs are dropping precipitously while they're restricting their customer's use of storage space. I just bought an 18 TB SATA drive for my main desktop that I paid $320 for. 100 TB just isn't that much space today.
- RcouF1uZ4gsC 5y ago>I just bought an 18 TB SATA drive for my main desktop I think one of the big issues is the performance of the disk. At 200 MB/s (which is generous for a spinning disk) it would take more than 24 hours to read all the data from the disk. With those kind of performance characteristics, I suspect that it makes redundancy and checking for integrity much more complicated.
- g_p 5y agoOn point 2, you can build a 100 TB "NAS" with an 8 drive chassis and 8 of the new 20 TB drives. Pricing should be under 5 or 6k USD unless you go crazy on the server. That should give enough extra to allow for a good bit of RAID parity and filesystem overhead etc. I think it's less about storage quantity now, and more about the available bandwidth for access to that storage, if we assume 100 TB is reasonable for a whole university. If we assume parallel reads from all 8 drives at, say, 200 MB/s (which is likely optimistic and assumes one single linear workload), this will only get you 12 Gbps. Throw multiple users and a need to seek, and rapidly the cost seems to be to become random access, latency and bandwidth (SSD caching in a tiered architecture etc). This might help Google justify higher pricing for storage than the "raw" storage cost, if the market will bear it. Alternatively, maybe the solution is for Google to badge up a non Google Drive storage service in the education workspace suite for research data, based on more appropriate cloud storage offerings with glacier type tiers etc? Offer universities unlimited storage and people will find incredible things to store there you never dreamed of....
- nonbirithm 5y agoI wonder what's going to happen if or when YouTube starts to implement restrictions on how many videos you can upload. People are being conditioned year after year that they can go back to YouTube and expect to be able to upload videos at any time. There will probably be some level of strife if that were to suddenly change. That would also bring into question if "significant" content creators would have more leverage to upload more videos than the average user. Then all the people that expect to be able to upload unlimited videos would either move off to a competitor, condition themselves to think otherwise, or something else.
- rob_c 5y agolol at managers, sympathies to the IT staff dealing with incredibly poor decisions
- frizlab 5y ago
- pumaontheprowl 5y agoHow long does Google need to maintain your unlimited storage before you won't complain when they take it away? Until the sun goes out?
- pandemicsoul 5y agowhy make a promise you intend to break?
- chias 5y agoOof. Back a few years ago when I was in grad school (University of Texas) they went all in telling everyone to use Google Drive for everything because we were getting unlimited storage for life. Last week they just sent out an email saying that as of later this year our quotas are going to be 1GB. RIP huge swaths of research data I guess.
- Tijdreiziger 5y agoWow, 1GB? Even on a consumer Google account you get 15GB for free.
- chias 5y agoYeah. I guess when you promise a perpetual account to all students for life, that adds up quick. New policy is 5GB for active students faculty and staff, and 1GB for alumni, retirees, and former students. > As of November 2021, UTmail is utilizing over 6.4 PB of pooled storage. However, due to a change in the G Suite storage policy, Google will implement a 545 TB pooled storage quota. > Of the customers impacted: 70,000 are students, faculty, and staff, and 200,000 are alumni and retirees. Edit: just read the help files closer and turns out that alumni and retirees lose Google Drive entirely -- 1GB is just for email.
- Tijdreiziger 5y ago> over 6.4 PB O.o I guess I never really think about the scale these systems operate at. Actually, considering that, the fact they give you 15GB on a free consumer account must add up to at least many hundreds of PBs?! I guess storage is cheap and/or targeted ads make a lot of money!
- hiptobecubic 5y agoThe scale is really astounding. You're underestimating.
- alecco 5y ago
- smrtinsert 5y agoThe unlimited space of the cloud is a lie. I don't even trust apps that promise free X documents these days.
- chunkyks 5y agoThere are increasing numbers of days where I think google has hit the "live long enough to become the villain" stage of the story.
- sciurus 5y agoAs someone who used to work for a bioinformatics lab, I'm confused by this tweet. The storage now being limited is for Gsuite- Classroom, Meet, Gmail, Calendar, Drive, Docs, Sheets, Slides, etc. These aren't the products you'd store a lab's datasets in. They simply aren't designed for that and won't provide acceptable APIs, performance, etc. Thus although this change may stink for university IT, I don't see an alternate world where this lab had embraced Gsuite storage and was crippled by it.
- detaro 5y agoGiven that people have tried stuffing hundreds of TB of data in every cloud service ever that offered "unlimited" storage, I would not be surprised if someone actually did that.
- g_p 5y agoUnder funded university labs often have need for archival storage to "keep results for future". You name the technology, I've seen it be used to store results. If a system is centrally funded by the main university (and not the lab itself), researchers will find a way to get it to hold research data if it has an interface to ingest a file... They don't need acceptable APIs or performance - they need something as opposed to nothing, and they'll find a way to make it work! More to the point, they need basic IT infrastructure funded by something central in the university. If there was a centrally funded way to use s3 with glacier for archival, (so they don't need to spend lab money on it forever more), many would use it. In my experience, often the issue is a lack of local budget, meaning infinite ingenuity to find a way to use what is available to get the best outcome for their research. Any ability to upload and store a file for later retrieval can, with enough low-cost RA time, be made into a apparently infinite free storage system!
- dekhn 5y agounfortunately I see many scientists using Drive storage for long-term sharing. I wouldn't (I put my terabytes in blob storage), but it's very common.
- wodenokoto 5y agoI’ve backed up my master thesis and all relevant data to google drive. I’m I’ll move a copy somewhere else if I ever need to conduct new research on it, but I’m sure if I had gone on to take a phd each paper, published or not would end up in a folder on gdrive if google had promised me the space.
- tuckerman 5y agoPreviously worked at Google in infrastructure but opinions are my own. I am always frustrated/embarrassed at Google’s messaging of these things, but it seems to me like this tweet is trying to stir the pot for attention. The Pixel phone with unlimited uploads marketed at consumers is one thing, but this unlimited plan for universities assuredly contained terms that didn’t guarantee it forever and a large institution should be aware of that. I also don’t think unlimited is a synonym for eternally unlimited. It also seems like it would be a bad idea to store frequently accessed experimental data in Google Drive and based on follow up tweets the professor agrees and doesn’t use Google Drive for this… so it seems like they are mad about a hypothetical. I don’t think Google is always the most thoughtful with things like this but this seems like something to take up with their IT department and not a second Google Reader incident.
- trunnell 5y agoWhy do you say it'd be a bad idea to store frequently accessed experimental data in Google Drive?
- g_p 5y agoIndeed - given much of Google drive marketing seems to be trying to encourage it's use for routine frequently accessed data, this is an interesting question. Much of the marketing for drive seems to still be (in sentiment at least) about dumping it all into Google drive and never having to worry about storage again. I assume they mean storing huge files in Google drive, but actually that's probably a significant use case for many users of cloud storage.
- tuckerman 5y agoI think the biggest issue is provenance and access control/logging—Drive doesn’t have a lot of great tools for that. Backups as well. That said, I’m struggling to imagine how you could even efficiently process on the order of 1 PiB of data from Drive. Having that in a data warehouse or some object storage close to your compute seems required to make any good use of it.
- 5y ago
- charcircuit 5y agoIs the 500TiB compressable?
- tristanc 5y agoJust a note for those of you who may be in a similar situation. YMMV. If you have a couple of very large folders (even 1 TB or more) you care about holding onto from your unlimited drive, share it with a few (even throwaway) personal Gmail accounts. Once my HS disabled access to my account after I graduated, the school admin account somehow inherited these folders (the organization name is the new owner). I still have access to these many TB large folders many years on in the “Shared with me” section of gdrive.
- orware 5y agoCheck out my comment above here in this same thread which basically confirms exactly what you just shared here...this is the same tactic I was observing being used with fake student accounts sharing with outside external Gmail addresses which then appeared to take over the Shared Drives being created in GSuite, allowing for TBs and TBs to be added in to the drives by outside users. It seems like a big problem to me...but at the moment I don't think Google is providing good enough tools for Admins to see/eliminate these situations very quickly/easily.
- fleddr 5y agoGoogle's bait and switch approach is wrong, but the outcome of this is good. You should be paying for storage because it simply isn't free.
- kumarvvr 5y agoA smart IT administrator would put that down on paper, but not "unlimited space", just a large amount of space, per user, committed on paper. It's like decision makers don't understand business and human psychology. Firstly, when you tell someone they have "unlimited" space, it will be filled up with "unlimited" data. Its basically a law. Secondly, since the dawn of the concept of transactions between people and businesses, everyone knows there is "no free lunch". I just don't get why enterprise environments allow "unlimited" and "perpetual license" and other weasel terms into their contracts. Understand that no business will give anything away for free. Be extra careful when a business says so.
- ab_testing 5y agoI think Google is realizing that software companies like Blackboard and Peoplesoft are milking universities for millions of dollars while Google is losing potential revenue in the name of product exposure. Since universities need to provide email and storage services to their students anyway, Google is realizing that they will happily pay the bill rather than switching providers.
- Wiseacre 5y agoGoogle always knew people would pay. The question is why now? Is Google in trouble?
- holografix 5y agoOh look, for profit organisation that gave away stuff for free for ages now decided they would like to get paid. How dare they!
- yewenjie 5y agoWhat is a good way to sort and deduplicate about 700 GB of photos stored in Google Drive (not in Google Photos but as image files in Drive)? Can it be done incrementally instead of downloading the entire thing? What are the best options for deduplication and organization if I have to do it at once?
- mceachen 5y agoUnfortunately, Google doesn't support good API read access of your files: pulling them down via their API results in several bits of metadata being stripped (including GPS coordinates). (If someone knows otherwise, I'd love to be wrong!) I'd download a Google Takeout. If you need something to deduplicate your takeout, feel free to try PhotoStructure (I'm the author). https://photostructure.com/faq/takeout/ https://photostructure.com/faq/takeout/ (You'll be left with a single datestamped-based folder hierarchy: just enable "automatic organization". The folder format is customizable, too: https://forum.photostructure.com/t/how-to-change-the-naming-structure/1184/2 https://forum.photostructure.com/t/how-to-change-the-naming-... )
- vanilla_nut 5y agoI tried downloading about 200GB of data off of my university drive last night. Took a good hour or two of "zipping" on the drive site, then... hundreds of separate zips ended up dumped on my browser at some point later. And those downloads mostly failed. So now I'm taking a look at rclone to move data off of my drive, maybe slowly, through my raspberry pi.
- MauroIksem 5y agoSame thing at my uni..we have peta bytes of data saved..it's going to be impossible to get it off in time.
- about3fitty 5y agoBox did this same thing at the large, public university where my department was responsible for the implementation and maintenance of the project. It was so wildly predictable, yet every time I went to our businesspeople they claimed without irony that there would be “unlimited storage” and that “it’s not a problem”. Didn’t tolerate dissent. I eventually ended up leaving and claiming that it was a finite universe. Now they are locked in hard, and trying to figure out how to keep the number of seats they paid for by removing accounts. No policy from the beginning on that, so they are having issues. So predictable.
- oliwarner 5y agoWhenever you're dealing with a long term supplier, you have to consider how long they can sustain themselves. If you're looking at an "unlimited" plan with the intention to get a service for a deeply discounted rate, you have to use your due diligence and consider how sustainable it would be. The tweet talks about 500TB. That's $30kpa with Backblaze B2, $126kpa on S3 before any data movement. It's £10k in raw drives without any redundancy. Unless the "unlimited" plan is $50kpa there's no way that would ever be viable for a business to take on indefinitely. Yes, it's deeply irresponsible for Google to make promises (they know more than most how much free services get abused), but customers have to be aware too.
- knaik94 5y agoI am in a similar situation buy my university has 60,000 students and alumni keep their edu gmail for life. The policy is that for institutions with more than 20,000 users, additional storage can be purchased on a 50g/license type situation. The university hasn't said anything about new policies to us.
- anticristi 5y agoDevil's advocate: Why is Google to blame at all? Any org with decent IT need to think of business continuity. They need to plan for the risk of a discontinued product or a bankrupt supplier. If the (implicit or explicit) plan is "we accept the risk", then don't blame the supplier. We offer platform as a service, and are regularly asked for exit plans from our customers. In brief, if we discontinue our platform you will need to do X, which will cost you Y.
- JoeJonathan 5y agoWhile I can completely understand limiting storage for students/alumni, especially at massive state schools, this is going to be a nightmare for faculty and researchers. I'm not sure how Gmail will factor into these totals, but <5 years of email already has me at >8gb.