4 ms·
It's probably still too big to be helpful with these model sizes, but if someone helpful runs the same training on `small.en` (and smaller) we might have someth
by regularfry 3y ago
It's probably still too big to be helpful with these model sizes, but if someone helpful runs the same training on `small.en` (and smaller) we might have something.
Yes, this is me praying to the benevolent HN gods that someone will pick this up and run with it. I don't have a GPU anywhere close to capable...
- FL33TW00D 3y agoYou'd be surprised how capable old GPUs are! I've had great success with people running Whisper-Turbo in the browser on really old hardware: https://whisper-turbo.com/ https://whisper-turbo.com/
- kkielhofner 3y agoWe have benchmarks[0] for Willow Inference Server using Whisper + ctranslate2 + some of our own optimizations. TLD a six year old ~$100 used GTX 1070 is roughly 5x faster than a Threadripper PRO 5955WX at a fraction of the cost and power. [0] - https://heywillow.io/components/willow-inference-server/#benchmarks https://heywillow.io/components/willow-inference-server/#ben...
- RecycledEle 3y ago> TLD a six year old ~$100 used GTX 1070 is roughly 5x faster Did you mean TIL?
- regularfry 3y agoIt's not the inference, it's the training. They say in the paper: "We train with a batch size of 256 for a total of 80,000 optimisation steps, which amounts to eight epochs of training." That's a fair chunk of time. Mind you, `small.en` has smaller decoder layers than `medium.en`...