5 ms·
The Hugging Face Datasets Server is now open-source
- minimaxir 4y agoIt's worth noting that the reason this works well is that Datasets is backed by Arrow, which allows streaming remote data. More info in the base library docs: https://huggingface.co/docs/datasets/stream https://huggingface.co/docs/datasets/stream
- AnhTho_FR 4y agoSo much progress since the tamagotchi!!!
- joshxyz 4y agoLMAO welcome to the future!!
- jbverschoor 4y agoNote that you probably need DocArray for this as the core building block
- arthurcolle 4y agoI saw the DocArray post earlier, is it really used in this?
- jbverschoor 4y agoNah, I'm just kidding. I don't know, probably not :) I never heard about the hugging face before that post.
- arthurcolle 4y agoHuggingface hosts the stable diffusion stuff, seems hard to miss over the last 2 months ;)
- jbverschoor 4y agoAhh.. well yeah SD is on every few hours. I didn't look into any further other than the fun examples.
- MarcoZavala 4y ago
- f38zf5vdt 4y agoSurprised to see it's all Python.
- anaganisk 4y agoProves over optimisation(rust, kube etc) vs coded in a language easy to pick, understand and perfect.
- stoplying1 4y agoThe only thing proved is that the authors chose python. It likely had more to do with necessary library/functionality availability than anything.
- anaganisk 4y agoExactly my point, you can achieve results with the language in hand, by choosing premature/over optimisation. Hugginface is not a small service and if its able the do it good with python, it means “python is too slow for serious work” argument fails.
- fluoridation 4y agoOr perhaps it proves that Python and its libraries are fast enough for a service like Huggingface.
- prophesi 4y agoPython's already the main contender for AI/ML, so I don't think anyone's arguing that python is too slow for serious work. But it still makes you wonder if their AI/ML would be more performant in Lisp. And of course, you never know if something is "too slow" until your traffic increases exponentially and grinds your service to a halt, and now you have to figure out a solution in your language to address that with your source of income in peril.
- sangnoir 4y ago
- j-sizz 4y agoWhy just the first few rows
- surajs 4y agowhat at time to be alive!