3 ms·
Just use your embeddings model of choice that works with your language, I believe Ada from openai is multilingual but I don’t know what languages it works well
by dbish 3y ago
Just use your embeddings model of choice that works with your language, I believe Ada from openai is multilingual but I don’t know what languages it works well on, there are many embedding models out there, huggingface is your friend in this search. The output is just a vector and the rest of the system can basically stay the same. The only other thing that may need to change depending on language is any text preprocessing you need to do like word or sentence breaking for languages with compound words (German), agglutination (Turkish), etc.