3 ms·

My 250M parameter model runs in 50ms/token ;) Releasing April 26th when Chrome 113 hits stable. Open source NPM library you can add to any project. Preview he

My 250M parameter model runs in 50ms/token ;) Releasing April 26th when Chrome 113 hits stable. Open source NPM library you can add to any project. Preview here: https://twitter.com/fleetwood___/status/1646608499126816799?s=20 https://twitter.com/fleetwood___/status/1646608499126816799?...

lxe 4y ago
That's pretty impressive. What model are you using?
- FL33TW00D 4y ago
  FLAN-T5 Base currently, 780M parameter variant coming shortly!
  - VierScar 4y ago
    What kind of model? Question-Answering? I imagine it must be quite specialised at <1b Params when many are 7b, 13b, or more?