mortimerp9

4 ms·

1.
Gaia2 and Are: Empowering the Community to Evaluate Agents(huggingface.co)
5 points by mortimerp9 1y ago|1 comments
2.
by mortimerp9 1y ago
Meta AI is releasing two new resources for AI agents research: - GAIA 2 Benchmark: An updated approach to agents evaluation • 800 dynamic scenarios across ten realistic universes • Tests adaptability, robustness to failure, and time sensiti
3.
by mortimerp9 3y ago
Hello, I work on seamless. > It runs but any audio input (you will need to provide wav not mp3's) I tried (tried 20s/40s/300s) I get just one short sentence returned in target language that seems not related at all to my a
4.
by mortimerp9 3y ago
For audiobooks, it's already a reality: https://marhamilresearch4.blob.core.windows.net/gutenberg-pu...
5.
by mortimerp9 3y ago
I work on seamless and you can find sample code here: https://github.com/fairinternal/seamless_communication or in the HuggingFace space.
6.
by mortimerp9 3y ago
Meta employee here. The system is not perfect, or it would not "hallucinate", while it's pretty good, it does sometime make errors (not just hallucination, maybe just some mistranslation due to noise in the training data). Wh
7.
by mortimerp9 3y ago
it only kicks-in if the output is more "toxic" than the input. If the input has a lot of swear words and the output has the same amount, then it will be left alone.
8.
by mortimerp9 3y ago
Sorry if I wasn't clear, internally we've been talking about it a lot, but I forgot that it doesn't have such a solid definition outside of our work. Thankfully, we try to define it in section 7.3 of the NLLB paper: https:&#
9.
by mortimerp9 3y ago
Our goal is to have a good recall, sometimes to the detriment of precision, so for words with multiple meanings, it might consider them toxic when in the actual context they are used in, they are not. The toxicity mitigation algorithm will
10.
by mortimerp9 3y ago
Hi, I work on seamless. What this refers to is added toxicity mitigation. We try to detect the level of toxicity in the input and make sure that the output toxicity level is not higher. This protects the model from doing egregious errors in