5 ms·
What was the iops you eventually got or needed? Since you mentioned model and cluster I’m curious as to wether or not others were also on said cluster.
by caro_douglos 7y ago
What was the iops you eventually got or needed?
Since you mentioned model and cluster I’m curious as to wether or not others were also on said cluster.
- opportune 7y agoThe way this computing cluster worked, each person had some default disk/memory space they were allocated out of a large total. I don’t know exactly how it was set up. People were running things like Hadoop jobs, Spark, Jupyter notebooks. I think the default disk space allowed was very small, like 5GB, and I had some large NLP model that was 8GB (probably the entire library and the part I needed was smaller). I needed to save the NLP library to my user account’s disk so I would only have to download it once. I think I could redownload it every job, but that would take a pretty long time, and had other issues. So it wasn’t even IOPS, it was just the disk space I was allocated on my user account. Probably a one line command by the sysadmin guy, unless he also needed to go get another hard drive and plug it in, in which case it would be 10 minutes and then a one line command.
- deleted 7y ago[deleted]