3 ms·
> Yes, people are not running weather forecasting on their home computers. But they're playing video, synthesizing audio, doing speech recognition, etc. all the
by acdha 4y ago
> Yes, people are not running weather forecasting on their home computers. But they're playing video, synthesizing audio, doing speech recognition, etc. all the time. And more of this ever day.
>
> And if there's any chance of rescuing the uses of ML from a hellish privacy violating landscape worth of a Butlerian Jihad... it would by making sure most of this type of computation was done locally on-device.
I think SIMD support is useful too but most of the examples you gave are things which are handled on-device now using dedicated hardware. Video playback has been hardware accelerated for ages, ML acceleration is multiple hardware generations in, etc.
- adgjlsfhk1 4y agoVideo playback only has dedicated hardware for the formats that exist when your CPU was made. A cpu with AVX-512 will do a lot better on AV1 decode/encode than one without until someone actually makes a cpu with hardware for it. Funnily enough, the best ML acceleration hardware on CPUs is AVX-512. For small neural nets, a well tuned CPU implementation will do way better than GPUs.
- acdha 4y agoI'm not saying that there's no value in it, but rather that while you're right that it can produce a fair improvement over a traditional CPU implementation things like video are not the best examples because they're _so_ common that they get dedicated hardware which is usually considerably faster / more power-efficient so there's a narrow window for a couple of years where that SIMD implementation is most valuable. ML is similar – many people are running these apps now but, for example, many millions of them are running Apple ML models on Apple hardware with acceleration features so again, while SIMD is unquestionably useful, I'm not surprised that the average working programmer doesn't feel a huge need to dive in rather than using something like a library which will pick from multiple backends.
- cmrdporcupine 4y agoThis is not the case for synthesizing audio. Vectorization with GPU is essentially still useless there due to memory latency considerations, at least when I was looking at this most recently. EDIT: there was a period in the late 90s/early 00s when dedicated DSPs were a useful but costly add-on to digital audio PC systems. SIMD moving into the CPU put an end to that.
- acdha 4y agoThanks - I said “most” because there are definitely plenty of cases like that, but I think there’s an interesting phenomenon where SIMD is useful for its flexibility and low latency but we’re seeing faster cycles adding hardware support for the highest-value acceleration targets than we used to.