17 ms·
There's indeed plenty of tools out there, so the next few years are very interesting in terms of seeing which approach gets adopted by the wider audience. Feedb
by davidbuniat 4y ago
There's indeed plenty of tools out there, so the next few years are very interesting in terms of seeing which approach gets adopted by the wider audience. Feedback from the community (inc. feedback from you!) is instrumental in designing a tool that works well and addresses all the needs, so thanks a lot for your input re: CLI and branding. :)
Good observation, we've made sure that datasets can evolve better as you go. Specifically for your use case, you can query subsets of data and materialize it on the fly to be streamed, and then go back to a specific dataset "view" (i.e. saved query), as needed.
See how this works at around 5:50 here - https://youtu.be/SxsofpSIw3k https://youtu.be/SxsofpSIw3k
As for your last question, when the jpeg is appended using its file path, the compressed bytes get stored in the dataset without decompression/recompression. When the data is accessed as a numpy array, then the jpeg bytes are decompressed.
For researchers in Academia, our Growth plan is free. Since you work at a startup, the trial for Growth plan is for two weeks. If you want access, hit us up in the Community slack (slack.activeloop.ai - or you can just test the querying on public activeloop datasets!)
- davidbuniat 4y agoOne more thing regarding the compression -> you can both deeplake.link() to your raw data lake without touching it, or use deeplake.read() which preservers the compression as long as it matches with tensor default compression. https://docs.deeplake.ai/en/latest/deeplake.html?highlight=link#deeplake.link https://docs.deeplake.ai/en/latest/deeplake.html?highlight=l... https://docs.deeplake.ai/en/latest/deeplake.html#deeplake.read https://docs.deeplake.ai/en/latest/deeplake.html#deeplake.re...
- dimatura 4y agothanks for the pointer, I'll definitely give deeplake a closer look
- davidbuniat 4y agomy pleasure, let us know if you have any feedback, would love to see you succeed wtih Deep Lake. :)