3 ms·

You can submit many queries in parallel to increase throughout. Smaller models and faster hardware can reduce the time per query too.

• sebmellen 5mo ago
  Ok. Claude will not work for this use case because none of the sample data (weirdly blurry ID images) is in the training data.
• ceejayoz 5mo ago
  None of that gets you the 100ms response time the parent poster talked about, for something like "who is at my doorbell?" real-time uses.