5 ms·
Great, let me know when those models can run on-server and process/analyze streams of ID images with less than 100ms of latency. You’ll need to make sure you ha
by sebmellen 4mo ago
Great, let me know when those models can run on-server and process/analyze streams of ID images with less than 100ms of latency. You’ll need to make sure you have a massive set of training data including all manner of slightly blurred and slightly distorted ID cards
- _the_inflator 4mo agoExactly, and all on an embedded system with quite restrictive settings and no overclocked Intel lastest generation combined with NVIDIA's 10k graphic cards.
- charcircuit 4mo agoEmbedded systems can make network calls to powerful, GPU equipped servers.
- Chu4eeno 4mo agoThey really shouldn't, though.
- charcircuit 4mo agoIt can offer a ton of user value. There is a whole industry built upon this idea, Internet of Things.
- ceejayoz 4mo agoIoT wasn't not built on "send all the data off to a hosted GenAI". It predated them by quite a few years.
- charcircuit 4mo agoThe GPUs were doing video transcoding instead of GenAI.
- ceejayoz 4mo agoYou can run OpenCV on a GPU-less Raspberry Pi or other IoT device just fine. And most IoT devices aren't doing video transcoding at all. You're making some very odd assertions in this thread.
- charcircuit 4mo ago>And most IoT devices aren't doing video transcoding at all. The data gets streamed to the cloud where servers with GPUs transcode it. I'm pointing out that IoT devices historically have reached out to servers with GPUs even before GenAI.
- ceejayoz 4mo agoMost IoT devices have no camera and communicate with servers that have no need for a GPU at all.
- ceejayoz 4mo agoSure. Claude does that. "Cogitated for 1m 50s" doesn't work for real-time applications.
- charcircuit 4mo agoYou can submit many queries in parallel to increase throughout. Smaller models and faster hardware can reduce the time per query too.