2 ms·

I think this whole premise is ignoring the point that most decently sized companies want to and eventually will be running their own LLM workloads. Currently us

I think this whole premise is ignoring the point that most decently sized companies want to and eventually will be running their own LLM workloads. Currently use cases are limited by scope and imagination, and predominantly focused on cost efficiency. Once a use case becomes a top line revenue driver budgets will become essentially only limited by ROI. There are some inference workloads that are unacceptable to send to the frontier labs for privacy reasons. Most of it will come from firms who want to productionalize their own fine-tunes. In any case, the market for inference is less than 1% of what it will be in 5-10 years. This whole notion that GPUs will be sitting idle en masse is ridiculous. People will just start running GPU databases if it becomes cost efficient.

iwontberude 2mo ago
[dead]
Reviving1514 2mo ago
TIL about GPU databases. I did a bit if research but only found this one: https://github.com/bakks/virginian https://github.com/bakks/virginian What are the advantages of a GPU database?
- yunnpp 2mo ago
  TIL too, but probably what GPUs are good at: optimizing for throughput while hiding latency. Probably better suited for OLAP than OLTP.
- nr378 2mo ago
  [dead]
reticulates 2mo ago
but this is just an absolute fantasy, what, exactly, will this compute be doing? Our lives are already deeply entwined with technology and use barely any compute. How could our lives become 100x more dependent on compute? You can be thrilled by this exciting technology and all the possibilities it brings without thinking it is going to require this huge capital investment in GPUs. You could radically change billions of lives with a few dozen GPUs.
- dgellow 2mo ago
  Push everybody to buy hardware they don’t need to constantly run agents for pretty basic tasks, such as processing your daily emails and sending you little summaries notifications. So, the most inefficient software tools ever produced, but that keeps the whole industry alive. That’s pretty much the vision Jensen Huang is selling to ensure NVIDIA continues to grow
  - michelb 2mo ago
    I hope we can quickly escape these tasks and the focus on coding. It’s all very superficial. There are a ton of serious AI workloads imaginable and I haven’t seen anyone pushing those frontiers, at least not publicly. Nvidia had a shiny keynote about their digital twin of the world for solving large problems but i guess that one died already.
- b1gOhbuddy 2mo ago
  Ditch the servers. Do and sync over client to client. Keep data local again. Only use cloud for backups of local client encrypted blobs of vectors:data If you get rid of a lot of the suspect semantics hallucinated up over decades of software development it's not hard to see the geometry of an electronic snowflake. All the language just obfuscates the elegance. Crude meat suit grunts and clicks. Streamline it all to management of geometric states and access control and put the semantics on the presentation layer. What if we don't need python and go and ruby anymore? Made sense in a pre-gpu everywhere reality. Could just be high school stats classes to generate sets of values. Let go of the obscure linguistic chants. The data centers are just to serve surveillance purposes. Obfuscated behind politically correct memes of creating jobs. Chip away at the monolith and atomize the topology
mikae1 2mo ago
> most decently sized companies want to and eventually will be running their own LLM workloads And probably some decently sized states too. Not only commercial actors are up to the job.
SahAssar 2mo ago
Is anyone really running GPU databases in prod at scale? Or are you assuming that the GPU compute from a crash will become so cheap that it makes this niche scale?
- deleted 2mo ago
  [deleted]
babymetal 2mo ago
Is it just me or is this comment extremely difficult to parse? Besides the lack of any links/evidence the assertions are full of jargon "decently sized", "scope and imagination", "productionalize their own fine-tunes", "This whole notion... is ridiculous". It is currently at the top of HN and I hope that is just because it's a Sunday afternoon in many places.

iwontberude 2mo ago

[dead]

Reviving1514 2mo ago

TIL about GPU databases. I did a bit if research but only found this one: https://github.com/bakks/virginian https://github.com/bakks/virginian What are the advantages of a GPU database?

yunnpp 2mo ago
TIL too, but probably what GPUs are good at: optimizing for throughput while hiding latency. Probably better suited for OLAP than OLTP.
nr378 2mo ago
[dead]

reticulates 2mo ago

but this is just an absolute fantasy, what, exactly, will this compute be doing? Our lives are already deeply entwined with technology and use barely any compute. How could our lives become 100x more dependent on compute? You can be thrilled by this exciting technology and all the possibilities it brings without thinking it is going to require this huge capital investment in GPUs. You could radically change billions of lives with a few dozen GPUs.

dgellow 2mo ago
Push everybody to buy hardware they don’t need to constantly run agents for pretty basic tasks, such as processing your daily emails and sending you little summaries notifications. So, the most inefficient software tools ever produced, but that keeps the whole industry alive. That’s pretty much the vision Jensen Huang is selling to ensure NVIDIA continues to grow
- michelb 2mo ago
  I hope we can quickly escape these tasks and the focus on coding. It’s all very superficial. There are a ton of serious AI workloads imaginable and I haven’t seen anyone pushing those frontiers, at least not publicly. Nvidia had a shiny keynote about their digital twin of the world for solving large problems but i guess that one died already.
b1gOhbuddy 2mo ago
Ditch the servers. Do and sync over client to client. Keep data local again. Only use cloud for backups of local client encrypted blobs of vectors:data If you get rid of a lot of the suspect semantics hallucinated up over decades of software development it's not hard to see the geometry of an electronic snowflake. All the language just obfuscates the elegance. Crude meat suit grunts and clicks. Streamline it all to management of geometric states and access control and put the semantics on the presentation layer. What if we don't need python and go and ruby anymore? Made sense in a pre-gpu everywhere reality. Could just be high school stats classes to generate sets of values. Let go of the obscure linguistic chants. The data centers are just to serve surveillance purposes. Obfuscated behind politically correct memes of creating jobs. Chip away at the monolith and atomize the topology

mikae1 2mo ago

> most decently sized companies want to and eventually will be running their own LLM workloads And probably some decently sized states too. Not only commercial actors are up to the job.

SahAssar 2mo ago

Is anyone really running GPU databases in prod at scale? Or are you assuming that the GPU compute from a crash will become so cheap that it makes this niche scale?

deleted 2mo ago
[deleted]

babymetal 2mo ago

Is it just me or is this comment extremely difficult to parse? Besides the lack of any links/evidence the assertions are full of jargon "decently sized", "scope and imagination", "productionalize their own fine-tunes", "This whole notion... is ridiculous". It is currently at the top of HN and I hope that is just because it's a Sunday afternoon in many places.