3 ms·
What I can't wrap my head around is the fact NVDA almost is a representation of "AI infrastructure" in the narrative. It is weird now that everyone is saying th
by mrbungie 2y ago
What I can't wrap my head around is the fact NVDA almost is a representation of "AI infrastructure" in the narrative. It is weird now that everyone is saying the focus is be in inference / test time scaling compute: inference should be easier to offload to chips from other manufacturers.
Maybe I'm too dense to get it, but that doesn't make sense to me.
- somanyphotons 2y agoI think there is a money to be made in making huge slow inference chips It needs, * Does the very best model fit in ram * Does it eventually produce an answer I'm completely ok with waiting a few minutes for an inference answer if I can avoid spending $250k for an 8x
- mrbungie 2y agoMe too. Just as most problems won't need "phd-level intelligence" (whatever they might want you to think, 10x devs don't require phd skills nor research abilities / test time compute for being good at their job), most problems won't need subsecond latency or response time. I'm fine waiting, not sure if others really need the best now or ever.