2 ms·
> To mitigate such errors, we propose designating functions as unique functional tokens. I just skimmed the paper but this seems to be the crux of it. They map
by gardnr 3y ago
> To mitigate such errors, we propose designating functions as unique functional tokens.
I just skimmed the paper but this seems to be the crux of it. They map functions to a single token and can then fine-tune models to use the token instead of the function name. This increases accuracy of smaller LLMs and reduces total number of tokens required for prompts and for generations, which is where they get their speed gains from.
The paper is worth a look just to see "Figure (2)"
- alwa 3y agoFigure 2 is incredible. With only passing familiarity with the norms in this kind of work, the accuracy rates of all models on this benchmark suite seem suspiciously (and uniformly) high. Is choosing the right intention among “20 vehicle functions” or “20 Android APIs” consistent with an ordinary level of ambition in this kind of research these days?
- deleted 3y ago[deleted]
- jerpint 3y agoThat’s pretty clever, encoding atomic concepts as a token