4 ms·

I am curious the need for 70 t/sec?

• Aeolun 1y ago
  Waiting minutes for your call to succeed is too frustrating?
  • ekianjo 1y ago
    Depends entirely on the use case. Not every LLM workflow is a chatbot
    • jbellis 1y ago
      no, but if you're not latency sensitive you should probably be using DeepSeek v3 (cheaper than flash, significantly smarter)
      • lostmsu 1y ago
        What makes you believe DeepSeek is smarter than Flash 2.5? It is lower on all leaderboards.
        • jbellis 1y ago
          you're right, I should clarify that I'm talking about no thinking mode, otherwise flash goes from "a bit more expensive than dsv3" to "10x more expensive"
• cootsnuck 1y ago
  High concurrency voice AI systems.
  • grepfru_it 1y ago
    Why are you self hosting that?