4 ms·
Looking at the rest of your code, it looks like you are doing a lot of unnecessary processing on these embeddings. They even seem to be doubly json encoded at
by lovasoa 3y ago
Looking at the rest of your code, it looks like you are doing a lot of unnecessary processing on these embeddings.
They even seem to be doubly json encoded at some point:
https://github.com/ente-io/clip-ggml/blob/main/lib/clip_ggml.dart#L67 https://github.com/ente-io/clip-ggml/blob/main/lib/clip_ggml...
In my opinion, they should probably just be a memory buffer representing the raw floats all the way down: from the output of the model to the database. They should never be encoded, neither in json, nor as a dart List<double>.
- vishnumohandas 3y agoType-conversions with FFI turned out to be non-trivial, so `{"embedding":[...]}` was a way out at the cost of a small performance hit (when compared to the time spent on inference). We'll take another look.
- lovasoa 3y agoThe result of clip.[...].call(...) is already a buffer, isn't it? You just have not to touch it at all. And on the cpp side, remove the json encoding and just return a raw buffer.