24 ms·
In the mid nineties I worked in a research institute. There was a large shared Novell drive which was always on the verge of full. Almost every day we were aske
by rgj 6y ago
In the mid nineties I worked in a research institute. There was a large shared Novell drive which was always on the verge of full. Almost every day we were asked to clean up our files as much as possible. There were no disc quota for some reason.
One day I was working with my colleague and when the fileserver was full he went to a project folder and removed a file called balloon.txt which immediately freed up a few percent of disk space.
Turned out that we had a number of people who, as soon as the disk had some free space, created large files in order to reserve that free space for themself. About half the capacity of the fileserver was taken up by balloon.txt files.
- gowld 6y agoYou had a community-driven quota system.
- manifoldgeo 6y agoSounds like they had a tragedy of the commons, too haha
- Dylan16807 6y agoPartly that, partly the opposite. It's basically reserving part of the disk for very important things only, which scares off less important uses. Like making the commons seem more polluted than it actually is to get some action taken. If those files weren't there, the space would probably fill up, but now without any emergency relief valves. It would be better if these files were a smaller fraction of space and had more oversight... but that's just a quota system. This is something halfway in between real quotas and full-on tragedy of the commons.
- agumonkey 6y agois there a model to solve these ?
- chihuahua 6y agoI am far from an expert on game theory, but it seems that the cause of the tragedy of the commons is that people can use the shared resource for free. If there was a price to be paid, and the price was dynamically adjusted depending on conditions, then the overuse could be avoided. Similarly for file storage and "reserving" it by creating huge but useless files. If everyone was charged a fee per gigabyte per day, then people would be less likely to create those placeholder files. You probably have to be careful about how you measure, otherwise you'll get automated processes that delete the placeholder files at 11:59pm and create them at 12:01am.
- agumonkey 6y agoI was wondering if not all of human society is game theory through time/space taxes and prices.
- tinus_hn 6y agoSame thing happens with floating licenses, if they are too scarce, people open the program first thing in the morning ‘just in case’ and keep a license reserved all day.
- dasudasu 6y agoThe real game starts when people run infinite while/for loops that try to check out one as soon as it's available. Or run useless operations within the licensed software just so that that the license doesn't expire and return to the pool. I'm guilty of both, sadly. In an academic environment, additional resources aren't going to fall from the sky.
- Cullinet 6y agoouch I can't stop thinking now about how much cost gets imposed on the economy by habits like this established in higher education - I built my original business with very few formally qualified people who included a large proportion of the most experienced and professionally qualified individuals including several with multinational boardroom careers in F500s. we didn't have the culture to tolerate games like holding up a floating licence (of which licence a lot of critical software used) and we weren't the generation raised with computers by a few distant, but hearing this both makes perfect sense that it might be prevalent and simultaneously is thoroughly unnerving me about how strongly I might react on encountering the same if my present venture gets going.
- eb0la 6y agoI guess that's the reason Qlik (a business intelligence software provider) started using licensing by the minute - yes, like a phone call.
- tomrod 6y agoThis is very common in F500 companies. It's also a symptom of dysfunction.
- victor9000 6y agoPerfect example of the tragedy of the commons. If individuals don't create these balloon files then they won't be able to use the file server when they need it, yet by creating these balloon files the collective action depletes the shared resource of its main function.
- njovin 6y agoThis is similar to how some government agencies retain their budgets. At the end of the budget period they've only spent 80% of their allocated budget, so they throw out a bunch of perfectly good equipment/furniture/etc. and order new stuff so that their budget doesn't get cut the following year, rather than accepting that maybe they were over-budgeted to begin with. Rinse, repeat, thus continuing the cycle of wasting X% of the budget every year.
- eurasiantiger 6y agoBunks only go up.
- duxup 6y agoGovernment? I see that as the SOP for large companies too...
- flamble 6y agoIt's possible that once a company reaches a certain size, it's inevitable. Corporations internally have the same top-down centralized organizational structure as a typical government. Market forces can't eliminate that kind of inefficiency if it invariably affects all large enterprises, and the economies of scale enjoyed by such companies outweigh the perverse incentives of sub-organizations. What strikes me as unique to government is the tendency for sufficiently powerful appendages to secure enough resources to start wagging the dog (e.g. the military industry in the US), although now that I think about it seems possible that it would happen within companies.
- 6y ago
- bane 6y agoThis is a surprisingly common hoarding behavior among humans using scarce resources. In technology you see it everywhere, virtualization infrastructure, disk storage, etc. This is actually kind of clever. How the tribal knowledge for how to "reserve space" was developed and disseminated would be pretty interesting to study.
- jachee 6y agoIn Pittsburgh, it's evolved into the parking chair. https://en.wikipedia.org/wiki/Parking_chair https://en.wikipedia.org/wiki/Parking_chair
- the-rc 6y agoAt the opposite end, I heard a story of actually full storage from the beginning of the century, when I worked at a "large medical and research institution in the Midwest". They had expensive SMB shares (NetApp?) that kept getting full all the time. So they did the sane thing in the era of Napster: they started deleting MP3 files, with or without prior warning. Pretty soon, they got an angry call that music could not be played in the operating room. Oops. Surgeons, as you can guess, were treated like royalty and didn't appreciate seeing their routines disrupted.
- tapland 6y agoD: I like getting to listen to something in the MRI though.
- twistedpair 6y agoI worked at a large company during a migration from Lotus to Outlook. We were told we'd get our current Lost email storage + 100MiB as a new email quota limit under Outlook. I made a bunch of 100MiB files of `/dev/random` noise (so they don't compress, compressed size was part of the quota) and emailed them to myself before the migration, to get a few GiB of quota buffer. My co-workers were constantly having to delete old emails in Outlook to stay under quota, but not me. I'd just delete one of my jumbo attachment emails, as needed. ;)
- ABeeSea 6y agoEmail quotas aren’t just a cost thing. It forces deletion of files/communications that aren’t relevant anymore. The last thing the legal department wants is some executive’s laptop with 10 years of undeleted email to make it’s way to discovery.
- easton 6y agoThen why not just tell Exchange to delete any emails older than 5 years (or whatever your lawyers tell you to put)?
- shard 6y agoA previous company I worked for had a one month retention window in the email server. People just ended up storing email in their local machine's Outlook folder so they can refer to old emails.
- ianmcgowan 6y agoOr for the more technical folk with access to a linux server, setup postfix/dovecot, connect outlook to it and arrange for archived emails to go to the IMAP server. The IT people get smart about looking for OST or PST files, but let's see them catch that :-)
- realaether 6y ago
- ChuckMcM 6y agoOkay, that is hilarious. I use some scripts that monitor disk space, and monitor disk usages by "subsystem" (logs, mail, services, etc) using Nagios. And as DevOps Borat says, "Disk not full unless Nagios say 'Disk is full'" :_) Although long before it is full it starts warning me. It doesn't go off very much, but it did when I had a bunch of attacks on my web server that started core dumping and that filled up disk reasonably quickly. Back in the day we actually put different things in different partitions so that we could partition failures but that seems out of favor with a lot of the distros these days.
- Cullinet 6y agoI thought it was standard to never write logs to the same machine and to worm for that matter
- kaminar 6y agoCreate empty disk partition not possible? Seems easier to manage, no?
- herewulf 6y agoI always leave some unallocated space in LVM in my machines. However, in a cloud environment it's probably easier or only possible to delete that 8 GB file.
- phkahler 6y agoThis happens at some restaurants - we'll save a table while you get the food. Half the place is people not eating be cause it's so busy.
- kmonsen 6y agoI remember in Android one year the focus was on slimming down the memory usage. Of course we found an app that shall not be named the allocated a chunk of memory on startup just in case it was going to be needed later.
- zerd 6y agoReminds me of the speedup loop https://thedailywtf.com/articles/The-Speedup-Loop https://thedailywtf.com/articles/The-Speedup-Loop
- arkh 6y agoAt school we had a 800Mb quota for each class (around 90 people). Usually the first year everyone discovered the space problem when trying to get everything done for your first project. When you cannot compile code or generate pdf because there's no space left the witch hunt starts: there's always some people with left-over files from .pdf to .tex conversions. To help some students had put in place a crawler making statistics about who was using the space for all classes. And usually once bitten you made your own space requisition script which would take any byte left when available until it hit some reasonable size.
- Out_of_Characte 6y agoThat's dire, ~8Mb per person. Its an interesting problem though, When the resource is not scarce, allocating 800Mb per class is the correct way to do things. someone who needs 9, 12 or 30Mb would be able to complete the allocation. But as soon as resource contention happens, students with the biggest allocation would need to relinquish alot of data. 800Mb is nothing over a modern connection nowadays but playing this game with petabytes would be a nightmare.
- splithalf 6y agoThis still occurs.