3 ms·

AI inference costs dropped up to 10x on Nvidia's Blackwell

• deleted 8mo ago
  [deleted]
• simianwords 8mo ago
  > Sully.ai cut healthcare AI inference costs by 90% (a 10x reduction) while improving response times 65% by switching from proprietary models to open-source models running on Baseten's Blackwell-powered platform, according to Nvidia. The company returned over 30 million minutes to physicians by automating medical coding and note-taking tasks that previously required manual data entry. Are the margins that low that it would make sense to give up on quality of output and use open source models?
  • mountainriver 8mo ago
    Interviewed at Sully, absolutely insane company, I’m sure they aren’t making a rational decision here
    • hn12 8mo ago
      You have me curious, mountainriver. While I don't understand what you've written, I want to know more. Are you saying that open-source models can't be trusted as well as (some of the?) proprietary ones, and therefore aren't fit for "mission-critical" medical applications?