4 ms·
I wish there was some reference of what can I actually do with 4 or 8 TOPS and 4 or 8 or whatever GiB of accelerator memory. Can I run speech recognition? Run a
by mynegation 3y ago
I wish there was some reference of what can I actually do with 4 or 8 TOPS and 4 or 8 or whatever GiB of accelerator memory. Can I run speech recognition? Run a model for object detection in images? In video? LLMs - probably not. Stable Diffusion also seems out of question. But really - what can I run?
- deleted 3y ago[deleted]
- yjftsjthsd-h 3y agoAgreed! What I really want to know from any hardware accelerator is: * What can it accelerate? * What software supports it (and what's the state of driver support)? * How much faster or more power efficient is it vs CPU/GPU? * How exactly do I use it? (Okay, I buy the thing, it arrives in a box, I plug it in... what exact steps do I take to get to running real workloads on it?)
- randunel 3y agoThere's about two 240p object detection demo videos with source code, one picture object detection demo, a bunch of forum posts with incomplete code about some audio processing, and they're all about a different SBC. Software wise, there's not much to it, you're on your own.
- jdietrich 3y ago4 TOPS is last-gen smartphone territory - you aren't going to have a good time running big generative models, but it's a useful amount of performance for typical edge workloads like machine vision and robotics.
- sciencesama 3y agogoogle coral tpu is 4Tops btw
- eternityforest 3y agoEfficientdet gives almost acceptable performance for security cams on an i5 if you only decode keyframes, I wonder how many NVR channels it could handle?
- robcohen 3y agoOne thing that comes to mind is AI detection in Frigate https://docs.frigate.video https://docs.frigate.video. I’ve not played very much with AI but I’ve dabbled with Coral this past month and it would definitely be nice to have a fully integrated machine that could be dedicated to my security video feeds.
- bombela 3y agoI was given a coral USB by a friend, and it is handling detection for 6 cameras without issues on frigate. I have configured the cameras (had to go through an activX on IE6 via a MS Windows VM, yay) to serve two streams. 480x360 5fps for the AI detection, and 2592x1944 25fps for recording. Only the low quality stream is decoded on the computer, for a very light load.
- juitpykyk 3y agoFor scale, Microsoft requires 40 TOPS to certify a (consumer) computer as Copilot capable. https://www.techpowerup.com/320933/microsoft-copilot-to-run-locally-on-ai-pcs-with-at-least-40-tops-of-npu-performance https://www.techpowerup.com/320933/microsoft-copilot-to-run-...
- deleted 3y ago[deleted]
- gigatexal 3y agoProbably good enough to run openCV and turn it into a people and object recognizing brain for cameras but not sure.
- explorigin 3y ago4 TOPS is the same as the Coral Accelerator. Look on YT for how people have used it. Another poster mentioned video object recognition with Frigate.
- sheepybloke 3y agoThe bigger question for me is if it's easy to uses or add the accelerator memory to your application. For something like a virtual assistant or something with some basic image recognition, it can be really confusing about how to take advantage of the different hardware tools that are there. This is especially true in the maker space, when you're trying to leverage open source tool kits and don't necessarily have the time or knowledge to write a full, lower-level driver for open source tools kits.