7 ms·
Wait, wait, this is a university developing something, that needs to be downloaded, they choose a commercial solution for that, and that costs them $10k per day
by klingonopera 7y ago
Wait, wait, this is a university developing something, that needs to be downloaded, they choose a commercial solution for that, and that costs them $10k per day? They don't have their own file-hosting solutions? At a university with an AI dept?
With that money ("for a couple of days"), you could buy your own server and do it yourself, for a fraction of the price. And don't tell me you can't find people who'd be capable of doing that, if you've got AI development going on.
What am I missing?
- dooglius 7y agoMy understanding is that each user runs a separate Colab instance with its own hardware, it's not the kind of thing you could replicate easily with one server and a couple GPUs.
- klingonopera 7y agoI see, thanks. That's no longer the case now, is it? I'm presently downloading the torrent, and it seems like it will then run completely local? Well, I guess that was the cost of getting it to go viral. The torrent seems quite active now.
- cepth 7y agoColab is effectively JupyterLab/JupyterHub with Google's own add-ins (like integration with Google Drive). JupyterHub is a huge PITA to manage, and Colab also offers a degree of free compute and limited access to a dated, but still free K80 GPU. From the tweet thread, it seems like there was some misunderstanding over where the files are being stored and being executed. This is a pretty common issue with Google Drive. I.e. if someone shares a file with me, and I copy it to a folder, it's just a pointer to the original file. Only after clicking "Add to My Drive" does it count against my storage allocation, and only then is it a distinct copy. My guess is that the researchers expected each user to be able to run the game in their own personal free Colab environment, not be running it against the university's compute and storage budget.
- cbarrick 7y agoIt doesn't seem to be as simple as downloading a binary. We (those who read HN) could probably download the code and run it at home, but I think the authors want non-technical users to play. To do that, they need an accessible Python runtime, so they're hosting the game in a Colab notebook. The download in question is referring to downloading the weights of the neural net into the VM running the notebook. If only redistributing Python apps wasn't so difficult.
- Wowfunhappy 7y agoCould something like PyInstaller[1] work? [1]https://www.pyinstaller.org https://www.pyinstaller.org
- andrekorol 7y agoThe game's GitHub page[1] states that you would need a "beefy" GPU ~12 GB and CUDA to play the game locally. I think that's why the author was serving the game through Colab since the majority of users probably don't have a 12GB GPU. [1]https://github.com/AIDungeon/AIDungeon/ https://github.com/AIDungeon/AIDungeon/
- klingonopera 7y agoOoof, yes, now I see where the $10K/day is coming from... As I have demonstrated, I've really not much of a clue when it comes to AI, but do users really need 12Gb GPU RAM, 100% of the time? Maybe it's possible to use one GPU for multiple users?
- sanxiyn 7y ago$10k/day is just for file transfer, not for GPU.
- klingonopera 7y agoOk, ok, let me see if I now got this right: The AI needs a VM (because users rarely have 12GB GPUs at home), and this VM then downloads about 6GB from GitHub each time a user opens up a new session instead of sourcing that from a locally cached copy? EDIT: Almost, the VM runs on Colab, which only works with Google, and Google's charging for the upload to Colab? ...more like collaborator, amirite? dodges rotten eggs
- chillfox 7y agoPolitics is what you are likely missing.
- viraptor 7y agoIf agree in theory, but I'd like to disagree with some assumptions. AI devs don't necessarily have experience with efficient network infrastructure. (And ops people with AI development) You can't just "buy servers for the same price". Where are you going to put them? How are you going to power them? How is the bandwidth for them provisioned? At some scale these are non-trivial questions - you can't just buy a rack, stick it next to your desk and plug into an extension cord. The system deployment is something you need to spend time on as well. Bare metal provisioning and deployment of GPU libraries to make things run smoothly takes time. And finally when the hype dies down in a week or two, what are you going to do with that infrastructure? Cloud services are not trivial either. But they do have some advantages.
- bausano_michael 7y agoPossibly the author of the parent comment was puzzled why does an AI lab not have a specialist who provisions the bare metal. I would think that if the hype died in a week, an AI lab would have other projects which require similar equipment to run. I agree with your post in general as intuition tell me (without further details about ops situation) that cloud is a competitive fit in this scenario.
- satanspastaroll 7y agoAt this "scale" a single 3Gbps guaranteed pipe is ~330€/mo on OVH. You could simply buy ten of them just to be sure and still be paying 1/3 a month of what that would cost you for a day in the cloud. That includes all the maintenance and power for the equipment, you simply have to know how to use the OS.
- koffiezet 7y agoAs an "SRE engineer", I don't agree. Certainly in an university/R&D environment, you cannot compete with what 'the cloud' is able to offer within the reach of a few mouse clicks. It would be absolutely moronic to set up an entire infra yourself with GPU's, ... specifically tailored to for 1 research project to then throw it away after a few months, or see the scope change every 2 weeks. One of the reasons such AI tech is able to move this fast because the cloud has become a commodity. People start to expect this kind of flexibility. It's impossible to self-host every possible thing some guy might want to try out. Once you have a stable application - sure you might want to invest in own hosting for this, but even then it's an expensive up-front investment for something that might not go anywhere. I know a few places where they have a bunch of maxed-out nVidia DGX systems catching dust for exactly this reason.
- Tepix 7y agoAt 10k per day it is absolutely worth it to hire someone to take care of a bunch of cheap servers.
- lallysingh 7y agoThat's what, $2.5MM/yr?
- deleted 7y ago[deleted]
- koffiezet 7y agoServers with GPU's are not going to be cheap :)
- blaser-waffle 7y agoServers are a CapEx purchase, and can be depreciated or re-purposed. 10k/day makes sense for a short project, but if this is going to be an on-going thing -- like 2-3 years -- then absolutely buy that hardware and have an internal team run it.
- buboard 7y ago> What am I missing? They overbudgeted the "infrastructure" section of their grants and have money to burn; and nobody's interested in setting up another datacenter.
- deleted 7y ago[deleted]
- ska 7y ago> What am I missing? There are a couple of things. First off, academic funding for things like AI-capable clusters is a whole thing, let alone the support staff for them. This is difficult for a host of reasons, some good and some bad. Secondly, this isn't about hosting files really, it's about providing a suitable environment to run the model in. You can expect a AI research lab to have figured out how to do training in a reasonable way, but what they have here is more of an inference in production problem. That's really outside the expected expertise of a research lab. In the old days you just wouldn't be able to access this simply. Today there are turn-key(ish) solutions with built in scaling; they used one - it scaled and now they are wincing at the bill. This isn't crazy, as the google->google egress fees are not obvious.