15 ms·
Google reverses 5M file limit in Google Drive
- bullen 4y agoIf you want many small files make more inodes: sudo mkfs.ext4 -T small I use it for my 2000 line distributed db: http://root.rupy.se http://root.rupy.se
- Proven 4y ago[dead]
- transcriptase 4y agoI wish stupid decisions that end up quickly reversed came with post-mortem write-ups the way downtime does. How could anyone think that rolling this out without warning would go smoothly?
- lazystar 4y agoend-of-the-quarter deployments are always high-risk, high-reward. you either get promoted, or you end up front page on the NYT.
- mrtksn 4y agoIn "tech" its often forgotten that they are dealing with real people, individuals. They don't realize that their decisions cause real pain to people and do decision making purely through the formula derived to optimize profits. It's likely done by some mid level manager to improve the personal numbers to get promotion. I guess, at some point the analytics noticed that large number of files is suboptimal and they ran a simulation to see what percentage of the users are affected at what number of files limit and found out that >5M files impacts smaller number of paying customer revenue than the costs generated and they said just put the limit. Negative social media attention probably changed the outcome of the formula, so they lifted the limit.
- justsomehnguy 4y ago> In "tech" its ofta forgotten that they are dealing with real people, individuals. Don't attribute to tech that could be adequately explained by a management incompetence. I've seen stupid things done by a techy people. I've seen way more way stupider things demanded by management [and implemented by tech or whoever at the bottom of the food chain].
- mrtksn 4y agoSure. However tech is obsessed with data and optimisations, if they run a hospital and the data showed that obese people are less profitable they will put a scale on the entrance and triage by BMI or someone will calculate that some drug which kills in the long run gives enough time to propel their career through saving pennies.
- robertlagrant 4y agoYou're misattributing things to "tech". This all happened long before computers were invented.
- mrtksn 4y agoI don't see your point. Did I say that tech is the only one doing it? What's special about tech, is that they have the means to take it to another level because they are very capable to collect and analyse data and quickly deploy changes and almost never face the consequences. Low touch high scale is the name of the game and they are completely disconnected from the people impacted. 10K people losing their children's baby photos is just a blimp in the analytics and its "fixable" by an apology tweet, they will never understand the pain those impacted will suffer because they don't have a contact with the customer besides some charts.
- robertlagrant 4y ago> Did I say that tech is the only one doing it? It seems so, although possibly you did it accidentally. When you say "if tech controlled hospitals then it'd do it this way" then you're definitely attributing this mindset to "tech" people, rather to any people who address customer populations in the millions.
- Cthulhu_ 4y ago> How could anyone think that rolling this out without warning would go smoothly? They had numbers on the percentage of users that would be affected by this change as well as content to tell these users what they should do to fix things on their side and someone decided the number of people affected was small enough to not bother to warn people in advance.
- vivegi 4y agoGood they came to their senses.
- asynchronous 4y agoSo not a technical problem after all.
- ggm 4y agoIf technical goes to scaling and effects related to scaling, maybe they just decided to let the people with >5m inodes incur the technical penalty, which is almost always speed? it was always a cost issue, in some sense. If they designed the entire FS model behind drive to have inode limits per entity, then there's a small technology issue bumping that number maybe. Quota systems can be quite painful, in days gone past the Robert Elz Quota model on UFS demanded reboots for some changes, which at scale in a google sized entity is a rolling surface of state change.
- vlovich123 4y agoTheir explanation of the 5M limit is because of inode limitations when they materialize the content via the companion client?
- ggm 4y ago5m is an odd number. 24 bit numberfield would get you 16m uniques. I can't see how they needed to steal bits to get to 23 or 22. So there must be some other factor at play here apart from 'we used 3 bytes in the gRPC model' Perhaps the Windows exposed API assumed somewhere around 5m objects, and they care about the compatibility greatly.
- perbu 4y agoI don't think the number is based on any technical limitation. It could be a number based on usage. Say 99.9% of the users use less than 5M files. So my pruning 0.1% of the user base they are getting rid of half the number of files, saving X millions of dollars in cost. Nobody wants to cater to the difficult users that are doing stuff like this.
- 4y ago
- geenat 4y agoFirst they came for our staplers and now this.
- xref 4y agoAnd I said nothing because I went paperless. Unfortunately that decision left me with > 5mil digital files
- imgabe 4y agoJust use your digital stapler to combine files together until there's fewer than 5 million.
- justsomehnguy 4y ago> digital stapler Perfect name for some PDF editing software.
- imgabe 4y agoBluebeam is some PDF editing software (mostly used for architectural drawings) that calls its PDF combining feature the "stapler".
- mananaysiempre 4y ago... That happens to be a Python alternative to PDFtk[1]. It’s been kind of abandoned in recent years, though. Also, in the early times of OLE 2.0 MS Office included a utility called Binder that could put several Office documents in a single file and edit them using a common interface. Nothing came of it. [1] https://github.com/hellerbarde/stapler https://github.com/hellerbarde/stapler
- jpalomaki 4y agoI would love if Node could consume dependencies from archives (even uncompressed, like tar). Those hundreds of thousands of files make certain things slow on Windows and are cause problems if you want to sync stuff betweek computers.
- js4ever 4y agoWhat's going on?? Do they start to feel the end so much that they finally start to listen to peoples? Lol
- HeavyFeather 4y agoIf you think Alphabet is nearing “the end” I’m afraid you’re too deep into the AI bubble.
- echelon 4y agoIt doesn't take the AI bubble to sense the incredible IBMification of Google. The company can't ship product to save its life, and when it does the product gets canceled or is a duplicated effort of some other internal product. The AI bubble could steal away 50% of Google's revenue stream by this time next year. Even if Google puts up a similar product, it has to retain 100% of its revenue and margin. Stuffing ads in interleaved responses seems implausible or downright dirty under an entirely new regime of high information density retrieval tools. (Granted, this is probably my weakest argument.) It remains to be seen if users will flock from "Google search" to "Bing AI" (or whatever), but we all know how shitty Google search has become as a product compared to half a decade ago.
- bagacrap 4y ago> Even if Google puts up a similar product, it has to retain 100% of its revenue and margin. I see you've been to one of Satya's seminars. Not at all inside the bubble then ;) What happens if Google search only retains 98% of its margins?
- jay_kyburz 4y agoI'm bullish on AI, but 50% of Google revenue in a year in completely hyperbolic. 10 years perhaps.
- fhd2 4y agoAgreed - even if there's a vastly superior search engine released tomorrow, so many people grew up with Google and learned how to deal with it, it's the default in most browsers - a space owned largely by Google - etc. It's even harder to get a large number of users to move if the product works differently. I rather think LLMs will ruin the web with insane amounts of garbage content, and with it most of Google's livelihood. Hopefully not by this time next year though.
- helf 4y ago[dead]
- localtoast 4y agoOneDrive has left the chat... :(
- quitit 4y agoGoogle: "Only affected a small number of users." This language really infers that they still don't understand why there was such a loud outcry and bad press. Even if it's just a small number of users, a company can't pull the rug on them without notice - for those small number of users there was no reasonable expectation that this would become a problem, and seemingly now it is no longer a problem. Also if the number is so small: engage those users directly to find a compromise first.
- nomilk 4y agoAlso just because something affects a small proportion, it worries and unsettles a large proportion; many more than it affects directly.
- ectospheno 4y agoThey knew in advance how many would be impacted. They knew who. They just don’t care. They still don’t. And you all trust them with your data.
- stalfosknight 4y agoOne of life's unwritten laws is "you get what you pay for." I totally get why someone would want something for free but to whine about this free service not giving them what they want is some combination of entitled and ignorant.
- mrguyorama 4y agoAnyone still claiming google is free is just spreading propaganda. Each and every one of us pays with our privacy, and data, and attention.
- stalfosknight 4y agoFree is the most expensive price.
- quitit 4y ago
- AraceliHarker 4y agoDoes Google secretly impose a limit on how many channels we can block from YouTube recommendations?
- mmvora 4y agoAsking the real questions! What if the amount of data it took to store the number of channels we blocked exceeds their file limit XD
- wodenokoto 4y agoCompletely off topic but what is up with the dogecoin dog instead of the Twitter logo on Twitter?
- Spare_account 4y agohttps://www.theguardian.com/technology/2023/apr/04/twitter-changes-logo-to-dogecoin-cryptocurrency-image-in-apparent-late-april-fools-day-gag https://www.theguardian.com/technology/2023/apr/04/twitter-c... >Musk has not explained the reason for the change, but some users speculated it was meant to be an April fool’s day gag that the company was unable to bring in on time on 1 April.
- ilyt 4y agoElon just getting some money out by bumping doge value
- chaos_a 4y agoit's just a crappy late april fools joke.
- Flockster 4y agoMaybe to make stories like these ungoogleable? https://gizmodo.com/elon-musk-crypto-dogecoin-tesla-1850293273 https://gizmodo.com/elon-musk-crypto-dogecoin-tesla-18502932...
- zarzavat 4y agoAh, the Boris Johnson technique.[0][1] When Boris Johnson was prime minister he had a tendency to make some strange comments, most notably about painting model buses. There was a lot of speculation that he was trying to bury undesired articles on google. [0] https://www.theguardian.com/politics/2019/jun/26/mesmerising-boris-johnsons-bizarre-model-buses-claim-raises-eyebrows https://www.theguardian.com/politics/2019/jun/26/mesmerising... [1] https://www.wired.co.uk/article/boris-johnson-model-google-news https://www.wired.co.uk/article/boris-johnson-model-google-n...
- d--b 4y agoGoogle keeps shooting itself in the foot with google drive, it's insane. When Google Docs came out, everyone thought Microsoft was screwed. They had such a headstart in cloud-based collaboration. But somehow, they managed to eff up google drive and anything offline for so long that Dropbox is still the best option for cloud storage and Microsoft had the time to rebuild its whole Office suite in javascript.
- titzer 4y agoMy biggest gripe is that the web UI for Google drive seems exclusively designed for tablet users. Enormous icons with previews, little text, no real hierarchical organization, assumption of search-based navigation of docs, it's just so difficult to organize and use. It's like every thing anyone ever knew about organizing and finding files was just thrown out the window. As someone who has used computers for nearly 30 years at this point, I find it infuriating how hard it tries to make my skills useless.
- krashidov 4y agoI couldn’t have said it better. Why are the icons so large and useless at the same time. So much padding and such low information density
- Andrew_nenakhov 4y agoI wish LibreOffice folks would drop whatever they are trying to do with their 'online suite' and just create a self-hosted web version of LibreOffice.
- pmontra 4y agoThey kind of have it at https://www.libreoffice.org/download/libreoffice-online/ https://www.libreoffice.org/download/libreoffice-online/ "Kind of" because it seems to be unsupported but I see links to binaries and docker images on that page.
- Andrew_nenakhov 4y ago
- jordinl 4y agoReading the title I thought the limit was that files had to be at most 5MB
- satoshiiii 4y agoEven in paid shared web hosting services, there is an inode limit. And it was more than a decade ago since I stumbled with this issue. Since then I monitor my file count and delete those with last TOUCHed of at least 30 days even if I have moved already to a dedicated server long time ago. I was caching complex database queries to the disk and did not bother to delete the files before. I wonder if there is still a limit with modern Linux OSes and that is why Google is limiting some of their users?
- ilyt 4y ago> I wonder if there is still a limit with modern Linux OSes Since filesystems introduced dynamic inode count it hasn't really been a problem. I remember ages ago you had to format filesystem in certain way if you wanted a lots of small files (say news or email server), but the need of that went away with better filesystems and growing storage sizes. > and that is why Google is limiting some of their users? On their side it's most likely on the number of entries in database, which usually is more than just storage cost.
- weird-eye-issue 4y agoI think Google Drive is abstracted enough that they are not running into something basic like an inode limit... But this does remind me of having to set up a cron job on an old PHP application because the sessions were saved as files and it ran into the inode limit several years in :P
- ape4 4y agoAnd, of course, 5MB isn't some power of 2 (as most limits are). Its hard to imagine where Google got the number unless its just some hardcoded constant.
- denysvitali 4y agoThey most likely hit some sort of limit in the Zanzibar implementation. Every file has a set of permissions attached to it, because you can grant fine-grained permissions to every single file in Drive, Photos, YouTube, ... I imagine that they hit some limit in that part, and tried to find a way to limit the issue...
- PedroBatista 4y agoThe level of incompetence and mediocrity inside Google when it comes to managing a product ( let alone create a new one ) is astonishing. Even someone running a lemonade stand knows that trust is the number one requirement when serving a customer/user, everything else comes based on that. Yet, the smart people at Google seem to think otherwise.
- thesuperbigfrog 4y ago>> The level of incompetence and mediocrity inside Google when it comes to managing a product ( let alone create a new one ) is astonishing. Google's product is advertising to people. Google's users are advertisers who want to put their messages in front of people. Google Drive, Gmail, YouTube are all ways of delivering that advertising and collecting information about people for targeted advertising. Why would they spend money building a trust relationship with people when they can just offer free stuff and people will gladly hand over their data and their time to view the advertising?
- xd1936 4y agoDevil's Advocate... If they don't spend time on the end user product side also, then they have nobody to advertise _to_, and their advertisers would dry up.
- thesuperbigfrog 4y agoTrue. That's why they spend a minimal amount. The free stuff has to be good enough to bring and keep people, but not so costly that it is a large expense. "Customer service" for people does not scale well and has high costs. That is one reason why they automate it as much as possible and simply do not have it where it cannot be automated.
- mrguyorama 4y agoKilling competition is actually really cheap at google scale. Most "competition" just wants to be aquihired anyway.
- 4y ago
- cubefox 4y agoThis is great!
- voisin 4y agoIt is hard to imagine a more neurotic, unstable company than Google. The inmates are running the asylum and I am so, so happy to no longer be running any of their services.