3 ms·

nope

• cubefox 2y ago
  I assume even this one won't run on an RTX 5090 due to constrained memory size: https://news.ycombinator.com/item?id=43270843 https://news.ycombinator.com/item?id=43270843
  • whimsicalism 2y ago
    sure on consumer GPUs but that is not what is constraining the model inference in most actual industry setups. technically even then, you are CPU-GPU memory bandwidth bound more than just GPU memory, although that is maybe splitting hairs
    • cubefox 2y ago
      Why are industry setups considered actual while others are not?