3 ms·
I realised the same thing when building Artpip and built up a database of ~25000 images and ~5000 artists via wikimedia's extremely unfriendly API. I've though
by mcjiggerlog 8y ago
I realised the same thing when building Artpip and built up a database of ~25000 images and ~5000 artists via wikimedia's extremely unfriendly API.
I've thought about creating an open dataset that is free to use/download (everything is in the public domain anyway). The thing is, I have around 250GB of images (all images available in 4k) and I'm put off by the cost of hosting. If I were to just serve 2K images then the set would only be around 20GB.
Any suggestions for how to serve such a big file from a website?
I could just make it available as a dat (https://datproject.org https://datproject.org), but it's not exactly a widely used technology yet. Maybe I should just make a torrent? Ideally I would be able to update the dataset in the future.
- archagon 8y agoIf you ever get around to doing that, it would be super great! That's some excellent cataloguing work! (And DAT would be really neat to try out — I've been reading a lot about it recently and it seems like a very exciting new technology.)
- smsx 8y agoCan you distribute a torrent file?
- rimliu 8y agoI would love love love to have this available. I have an idea (not very original on interesting) for an iOS app and this would help me tremendously.
- kalleboo 8y agoI wonder if it would be appropriate to submit to archive.org