21 ms·

hae - like grep but with natural language queries

• andy99 3y ago
  This is interesting, I'd like to see how fast it runs. A challenge even with fast implementations of models is there is always some overhead starting up. I also want to mention this is exactly the kind of use case I had in mind for Ferrite, a dependency free Fortran implementation of sentence transformers: https://github.com/rbitr/ferrite https://github.com/rbitr/ferrite The idea is a very simple script to run transformer inference on a CPU that can be incorporated into a project without pulling in a million dependencies. I am happy to see though that this is a C++ project and appears not to require pytorch and HF transformers to use!
  • dmezzetti 3y ago
    It's definitely interesting. Models exported to onnx, uses tokenizers-cpp bindings. All packaged as a single binary.
    • deleted 3y ago
      [deleted]