5 ms·

From what I understand the basis for their model are these two described in these papers: https://arxiv.org/abs/2107.07651 https://arxiv.org/abs/2107.07651 http

From what I understand the basis for their model are these two described in these papers: https://arxiv.org/abs/2107.07651 https://arxiv.org/abs/2107.07651 https://arxiv.org/abs/2208.13628 https://arxiv.org/abs/2208.13628 Lot of tricks put together for a great final result it seems

ashvardanian 4y ago
Thank you! Founder here :) You are right, those are the base papers, but we have extended the set of objectives quite significantly, tapping into modalities that haven’t been publicly CLIP-ed :) It is probably worth writing a paper about, but we are just too busy building tons of open-source stuff. Check out the GitHub org here: https://github.com/unum-cloud https://github.com/unum-cloud It is not just about the tranformers, but also about databases, networking, and improving the modern data stack for very large scale retrieval-based AI. A lot of the pieces may be pre-production, but I believe the amazing HN community may still enjoy the ways we use io_uring, SIMD, and a few other less then popular technologies.
- cosmojg 4y ago
  Are the pretraining and training pipelines available anywhere under a FOSS license? I'd love to take a swing at training a mid-fusion model on data other than text and images (e.g., sound, neuron spike trains, etc.)
  - ashvardanian 4y ago
    Not yet, but you can ping our team on Discord or Twitter. They are soft like marshmallows, a couple of compliments and they will be leaking scripts left and right :)
- debdut 4y ago
  man I just looked at ukv, it looks to good to be true, 30x RocksDB, wtf! Hoping it's true
  - ashvardanian 4y ago
    He-hey! Yes we are fast, but I don’t think we ever claimed 30x. We are faster in almost every workload (loose range scans for some reason), but at best by 7x (batch reads) and 5x (batch writes). Still, this should be plenty for all intents and purposes! I can post some updates on that tomorrow :)
    - ashvardanian 4y ago
      If you are curious about how it works, here is a pretty good explanation: https://youtube.com/watch?v=ybWeUf_hC7o https://youtube.com/watch?v=ybWeUf_hC7o For some reason the conference hasn’t made the last years talks public or searchable, but you should be able to access it with a link
- eternalban 4y ago
  where is the udisk? Repo is just a readme on configuration.
  - ashvardanian 4y ago
    Yes, we decided to keep UDisk closed source for now. That repo is just a tiny description for the expected configuration files. At this point UDisk powers our soon-to-be-public cloud offering and is piloting in a few FAANG scale companies. Our human resources are very limited for now, but we can probably run a couple more such pilots concurrently. Reach out to info [at] unum.cloud or join our Discord if you are from one of those large companies and want to battle-test our secret sauce on a few Petabytes of your data :)
    - deleted 4y ago
      [deleted]
- intelVISA 4y ago
  > may still enjoy the ways we use io_uring, SIMD, and a few other less then popular technologies Fairly standard for any perf-minded shop but good to see more people discovering them.

ashvardanian 4y ago

Thank you! Founder here :) You are right, those are the base papers, but we have extended the set of objectives quite significantly, tapping into modalities that haven’t been publicly CLIP-ed :) It is probably worth writing a paper about, but we are just too busy building tons of open-source stuff. Check out the GitHub org here: https://github.com/unum-cloud https://github.com/unum-cloud It is not just about the tranformers, but also about databases, networking, and improving the modern data stack for very large scale retrieval-based AI. A lot of the pieces may be pre-production, but I believe the amazing HN community may still enjoy the ways we use io_uring, SIMD, and a few other less then popular technologies.

cosmojg 4y ago
Are the pretraining and training pipelines available anywhere under a FOSS license? I'd love to take a swing at training a mid-fusion model on data other than text and images (e.g., sound, neuron spike trains, etc.)
- ashvardanian 4y ago
  Not yet, but you can ping our team on Discord or Twitter. They are soft like marshmallows, a couple of compliments and they will be leaking scripts left and right :)
debdut 4y ago
man I just looked at ukv, it looks to good to be true, 30x RocksDB, wtf! Hoping it's true
- ashvardanian 4y ago
  He-hey! Yes we are fast, but I don’t think we ever claimed 30x. We are faster in almost every workload (loose range scans for some reason), but at best by 7x (batch reads) and 5x (batch writes). Still, this should be plenty for all intents and purposes! I can post some updates on that tomorrow :)
  - ashvardanian 4y ago
    If you are curious about how it works, here is a pretty good explanation: https://youtube.com/watch?v=ybWeUf_hC7o https://youtube.com/watch?v=ybWeUf_hC7o For some reason the conference hasn’t made the last years talks public or searchable, but you should be able to access it with a link
eternalban 4y ago
where is the udisk? Repo is just a readme on configuration.
- ashvardanian 4y ago
  Yes, we decided to keep UDisk closed source for now. That repo is just a tiny description for the expected configuration files. At this point UDisk powers our soon-to-be-public cloud offering and is piloting in a few FAANG scale companies. Our human resources are very limited for now, but we can probably run a couple more such pilots concurrently. Reach out to info [at] unum.cloud or join our Discord if you are from one of those large companies and want to battle-test our secret sauce on a few Petabytes of your data :)
  - deleted 4y ago
    [deleted]
intelVISA 4y ago
> may still enjoy the ways we use io_uring, SIMD, and a few other less then popular technologies Fairly standard for any perf-minded shop but good to see more people discovering them.