4 ms·
It doesn’t break text into chunks at all. These models can handle sequences of arbitrary length.
by stephantul 1y ago
It doesn’t break text into chunks at all. These models can handle sequences of arbitrary length.
- jasonjmcghee 1y agoI believe parent is referring to: https://github.com/MinishLab/model2vec-rs/blob/480ec988d7f4a41b333df92fbdea0e570e723844/src/main.rs#L36 https://github.com/MinishLab/model2vec-rs/blob/480ec988d7f4a... https://github.com/MinishLab/model2vec-rs/blob/480ec988d7f4a41b333df92fbdea0e570e723844/src/model.rs#L158 https://github.com/MinishLab/model2vec-rs/blob/480ec988d7f4a...