7 ms·

Doubtful that GPUs purchased today would be in use for a similar time scale. Govt investment would also drive the cost of GPUs up a great deal. Not sure why a

Doubtful that GPUs purchased today would be in use for a similar time scale. Govt investment would also drive the cost of GPUs up a great deal. Not sure why a publicly accessible GPU cluster would be a better solution than the current system of research grants.

JumpCrisscross 2y ago
> Doubtful that GPUs purchased today would be in use for a similar time scale Totally agree. That doesn't mean it can't generate massive ROI. > Govt investment would also drive the cost of GPUs up a great deal Difficult to say this ex ante. On its own, yes. But it would displace some demand. And it could help boost chip production in the long run. > Not sure why a publicly accessible GPU cluster would be a better solution than the current system of research grants Those receiving the grants have to pay a private owner of the GPUs. That gatekeeping might be both problematic, if there is a conflict of interests, and inefficient. (Consider why the government runs its own supercomputers versus contracting everything to Oracle and IBM.)
- rvnx 2y ago
  It would be better that the government removes IP on such technology for public use, like drugs got generics. This way the government pays 2'500 USD per card, not 40'000 USD or whatever absurd.
  - JumpCrisscross 2y ago
    > better that the government removes IP on such technology for public use, like drugs got generics You want to punish NVIDIA for calling its shots correctly? You don't see the many ways that backfires?
    - gpm 2y ago
      No. But I do want to limit the amount we reward NVIDIA for calling the shots correctly to maximize the benefit to society. For instance by reducing the duration of the government granted monopolies on chip technology that is obsolete well before the default duration of 20 years is over. That said, it strikes me that the actual limiting factor is fab capacity not nvidia's designs and we probably need to lift the monopolies preventing competition there if we want to reduce prices.
      - whimsicalism 2y ago
        there is no such thing as a lump-sum transfer, this will shift expectations and incentives going forward and make future large capital projects an increasingly uphill battle
      - JumpCrisscross 2y ago
        > reducing the duration of the government granted monopolies on chip technology that is obsolete well before the default duration of 20 years is over Why do you think these private entities are willing to invest the massive capital it takes to keep the frontier advancing at that rate? > I do want to limit the amount we reward NVIDIA for calling the shots correctly to maximize the benefit to society Why wouldn't NVIDIA be a solid steward of that capital given their track record?
        gpm 2y ago
        > Why do you think these private entities are willing to invest the massive capital it takes to keep the frontier advancing at that rate? Because whether they make 100x or 200x they make a shitload of money. > Why wouldn't NVIDIA be a solid steward of that capital given their track record? The problem isn't who is the steward of the capital. The problem is that economically efficient thing to do for a single company is (given sufficient fab capacity, and a monopoly) to raise prices to extract a greater share of the pie at the expense of shrinking the size of the pie. I'm not worried about who takes the profit, I'm worried about the size of the pie.
        whimsicalism 2y ago
        > Because whether they make 100x or 200x they make a shitload of money. It's not a certainty that they 'make a shitload of money'. Reducing the right tail payoffs absolutely reduces the capital allocated to solve problems - many of which are risky bets. Your solution absolutely decreases capital investment at the margin, this is indisputable and basic economics. Even worse when the taking is not due to some pre-existing law, so companies have to deal with the additional uncertainty of whether & when future people will decide in retrospect that they got too large a payoff and arbitrarily decide to take it from them.
    - Teever 2y ago
      There was a post[0] on here recently about how the US went from producing woefully insufficient numbers of aircraft to producing 300k by the end of world war 2. One of the things that the post mentioned was the meager profit margin that the companies made during this time. But the thing is that this set the America auto and aviation industry up to rule the world for decades. A government going to a company and saying 'we need you to produce this product for us at a lower margin thab you'd like to' isn't the end of the world. I don't know if this is one of those scenarios but they exist. [0] https://www.construction-physics.com/p/how-to-build-300000-airplanes-in https://www.construction-physics.com/p/how-to-build-300000-a...
      - rvnx 2y ago
        In the case of NVIDIA it's even more sneaky. They are an intellectual property company holding the rights on plans to make graphic cards, not even a company actually making graphic cards. The government could launch an initiative "OpenGPU" or "OpenAI Accelerator", where the government orders GPUs from TSMC directly, without the middleman. It may require some tweaking in the law to allow exception to intellectual property for "public interest".
        whimsicalism 2y ago
        y'all really don't understand how these actions would seriously harm capital markets and make it difficult for private capital formation to produce innovations going forward.
        freeone3000 2y ago
        If we have public capital formation, we don’t necessarily need private capital. Private innovation in weather modelling isn’t outpacing government work by leaps and bounds, for instance.
        whimsicalism 2y ago
        because it is extremely challenging to capture the additional value that is being produced by better weather forecasts and generally the forecasts we have right now are pretty good. private capital is absolutely the driving force for the vast majority of innovations since the beginning of the 20th century. public capital may be involved, but it is dwarfed by private capital markets.
    - panarky 2y ago
      To the extent these are incremental units that wouldn't have been sold absent the government program, it's difficult to see how NVIDIA is "harmed".
    - nickpsecurity 2y ago
      They said remove legally-enforced monopolies on what they produce. Many of these big firms made their tech with millions to billions of taxpayer dollars at various points in time. If we’ve given them millions, shouldn’t we at least get to make independent implementations of the tech we already paid for?
  - kube-system 2y ago
    > It would be better that the government removes IP on such technology for public use, like drugs got generics. 20-25 year old drugs are a lot more useful than 20-25 year old GPUs, and the manufacturing supply chain is not a bottleneck. There's no generics for the latest and greatest drugs, and a fancy gene therapy might run a lot more than $40k.
- latchkey 2y ago
  > Those receiving the grants have to pay a private owner of the GPUs. Along similar lines, I'm trying to build a developer credits program where I get whomever (AMD/Dell) to purchase credits on my super computers, that we then give away to developers to build solutions, which drives more demand for our hardware, and we commit to re-invest those credits back into more hardware. The idea is to create a win-win-win (us, them, you) developer flywheel ecosystem. It isn't a new idea at all, Nvidia and hyperscalers have been doing this for ages.
ygjb 2y ago
Of course they won't. The investment in the Heavy Press Program was the initial build, and just citing one example, the Alcoa 50,000 ton forging press was built in 1955, operated until 2008, and needed ~$100M to get it operational again in 2012. The investment was made to build the press, which created significant jobs and capital investment. The press, and others like it, were subsequently operated by and then sold to a private operator, which in turn enabled the massive expansion of both military manufacturing, and commercial aviation and other manufacturing. The Heavy Press Program was a strategic investment that paid dividends by both advancing the state of the art in manufacturing at the time it was built, and improving manufacturing capacity. A GPU cluster might not be the correct investment, but a strategic investment in increasing, for example, the availability of training data, or interoperability of tools, or ease of use for building, training, and distributing models would probably pay big dividends.
- JumpCrisscross 2y ago
  > A GPU cluster might not be the correct investment, but a strategic investment in increasing, for example, the availability of training data, or interoperability of tools, or ease of use for building, training, and distributing models would probably pay big dividends Would you mind expanding on these options? Universal training data sounds intriguing.
  - ygjb 2y ago
    Sure, just on the training front, building and maintaining a broad corpus of properly managed training data with metadata that provides attribution (for example, content that is known to be human generated instead of model generated, what the source of data is for datasets such as weather data, census data, etc), and that also captures any licensing encumbrance so that consumers of the training data can be confident in their ability to use it without risk of legal challenge. Much of this is already available to private sector entities, but having a publicly funded organization responsible for curating and publishing this would enable new entrants to quickly and easily get a foundation without having to scrape the internet again, especially given how rapidly model generated content is being published.
jvanderbot 2y ago
A much better investment would be to (somehow) revolutionize production of chips for AI so that it's all cheaper, more reliable, and faster to stand up new generations of software and hardware codesign. This is probably much closer to the program mentioned in the top level comment: It wasn't to produce one type of thing, but to allow better production of any large thing from lighter alloys.
photonthug 2y ago
> Not sure why a publicly accessible GPU cluster would be a better solution than the current system of research grants. You mean a better solution than different teams paying AWS over and over, potentially spending 10x on rent rather than using all that cash as a down payment on actually owning hardware? I can't really speak for the total costs of depreciation/hardware maintenance but renting forever isn't usually a great alternative to buying.
- CardenB 2y ago
  Do you have some information to share to support your bias against leasing especially with a depreciating asset?
  - manux 2y ago
    In Canada, all three major AI research centers use clusters created with public money. These clusters receive regular additional hardware as new generations of GPUs become available. Considering how these institutions work, I'm pretty confident they've considered the alternatives (renting, AWS, etc). So that's one data point.
  - photonthug 2y ago
    sure, I’ll hand it over after you spend your own time first to show that everything everywhere that’s owned instead of leased is a poor financial decision.
- vasili111 2y ago
  AWS is not only hardware but also software, documentation, support and more.

JumpCrisscross 2y ago

> Doubtful that GPUs purchased today would be in use for a similar time scale Totally agree. That doesn't mean it can't generate massive ROI. > Govt investment would also drive the cost of GPUs up a great deal Difficult to say this ex ante. On its own, yes. But it would displace some demand. And it could help boost chip production in the long run. > Not sure why a publicly accessible GPU cluster would be a better solution than the current system of research grants Those receiving the grants have to pay a private owner of the GPUs. That gatekeeping might be both problematic, if there is a conflict of interests, and inefficient. (Consider why the government runs its own supercomputers versus contracting everything to Oracle and IBM.)

rvnx 2y ago
It would be better that the government removes IP on such technology for public use, like drugs got generics. This way the government pays 2'500 USD per card, not 40'000 USD or whatever absurd.
- JumpCrisscross 2y ago
  > better that the government removes IP on such technology for public use, like drugs got generics You want to punish NVIDIA for calling its shots correctly? You don't see the many ways that backfires?
  - gpm 2y ago
    No. But I do want to limit the amount we reward NVIDIA for calling the shots correctly to maximize the benefit to society. For instance by reducing the duration of the government granted monopolies on chip technology that is obsolete well before the default duration of 20 years is over. That said, it strikes me that the actual limiting factor is fab capacity not nvidia's designs and we probably need to lift the monopolies preventing competition there if we want to reduce prices.
    - whimsicalism 2y ago
      there is no such thing as a lump-sum transfer, this will shift expectations and incentives going forward and make future large capital projects an increasingly uphill battle
    - JumpCrisscross 2y ago
      > reducing the duration of the government granted monopolies on chip technology that is obsolete well before the default duration of 20 years is over Why do you think these private entities are willing to invest the massive capital it takes to keep the frontier advancing at that rate? > I do want to limit the amount we reward NVIDIA for calling the shots correctly to maximize the benefit to society Why wouldn't NVIDIA be a solid steward of that capital given their track record?
      - gpm 2y ago
        > Why do you think these private entities are willing to invest the massive capital it takes to keep the frontier advancing at that rate? Because whether they make 100x or 200x they make a shitload of money. > Why wouldn't NVIDIA be a solid steward of that capital given their track record? The problem isn't who is the steward of the capital. The problem is that economically efficient thing to do for a single company is (given sufficient fab capacity, and a monopoly) to raise prices to extract a greater share of the pie at the expense of shrinking the size of the pie. I'm not worried about who takes the profit, I'm worried about the size of the pie.
        whimsicalism 2y ago
        > Because whether they make 100x or 200x they make a shitload of money. It's not a certainty that they 'make a shitload of money'. Reducing the right tail payoffs absolutely reduces the capital allocated to solve problems - many of which are risky bets. Your solution absolutely decreases capital investment at the margin, this is indisputable and basic economics. Even worse when the taking is not due to some pre-existing law, so companies have to deal with the additional uncertainty of whether & when future people will decide in retrospect that they got too large a payoff and arbitrarily decide to take it from them.
  - Teever 2y ago
    There was a post[0] on here recently about how the US went from producing woefully insufficient numbers of aircraft to producing 300k by the end of world war 2. One of the things that the post mentioned was the meager profit margin that the companies made during this time. But the thing is that this set the America auto and aviation industry up to rule the world for decades. A government going to a company and saying 'we need you to produce this product for us at a lower margin thab you'd like to' isn't the end of the world. I don't know if this is one of those scenarios but they exist. [0] https://www.construction-physics.com/p/how-to-build-300000-airplanes-in https://www.construction-physics.com/p/how-to-build-300000-a...
    - rvnx 2y ago
      In the case of NVIDIA it's even more sneaky. They are an intellectual property company holding the rights on plans to make graphic cards, not even a company actually making graphic cards. The government could launch an initiative "OpenGPU" or "OpenAI Accelerator", where the government orders GPUs from TSMC directly, without the middleman. It may require some tweaking in the law to allow exception to intellectual property for "public interest".
      - whimsicalism 2y ago
        y'all really don't understand how these actions would seriously harm capital markets and make it difficult for private capital formation to produce innovations going forward.
        freeone3000 2y ago
        If we have public capital formation, we don’t necessarily need private capital. Private innovation in weather modelling isn’t outpacing government work by leaps and bounds, for instance.
        whimsicalism 2y ago
        because it is extremely challenging to capture the additional value that is being produced by better weather forecasts and generally the forecasts we have right now are pretty good. private capital is absolutely the driving force for the vast majority of innovations since the beginning of the 20th century. public capital may be involved, but it is dwarfed by private capital markets.
  - panarky 2y ago
    To the extent these are incremental units that wouldn't have been sold absent the government program, it's difficult to see how NVIDIA is "harmed".
  - nickpsecurity 2y ago
    They said remove legally-enforced monopolies on what they produce. Many of these big firms made their tech with millions to billions of taxpayer dollars at various points in time. If we’ve given them millions, shouldn’t we at least get to make independent implementations of the tech we already paid for?
- kube-system 2y ago
  > It would be better that the government removes IP on such technology for public use, like drugs got generics. 20-25 year old drugs are a lot more useful than 20-25 year old GPUs, and the manufacturing supply chain is not a bottleneck. There's no generics for the latest and greatest drugs, and a fancy gene therapy might run a lot more than $40k.
latchkey 2y ago
> Those receiving the grants have to pay a private owner of the GPUs. Along similar lines, I'm trying to build a developer credits program where I get whomever (AMD/Dell) to purchase credits on my super computers, that we then give away to developers to build solutions, which drives more demand for our hardware, and we commit to re-invest those credits back into more hardware. The idea is to create a win-win-win (us, them, you) developer flywheel ecosystem. It isn't a new idea at all, Nvidia and hyperscalers have been doing this for ages.

ygjb 2y ago

Of course they won't. The investment in the Heavy Press Program was the initial build, and just citing one example, the Alcoa 50,000 ton forging press was built in 1955, operated until 2008, and needed ~$100M to get it operational again in 2012. The investment was made to build the press, which created significant jobs and capital investment. The press, and others like it, were subsequently operated by and then sold to a private operator, which in turn enabled the massive expansion of both military manufacturing, and commercial aviation and other manufacturing. The Heavy Press Program was a strategic investment that paid dividends by both advancing the state of the art in manufacturing at the time it was built, and improving manufacturing capacity. A GPU cluster might not be the correct investment, but a strategic investment in increasing, for example, the availability of training data, or interoperability of tools, or ease of use for building, training, and distributing models would probably pay big dividends.

JumpCrisscross 2y ago
> A GPU cluster might not be the correct investment, but a strategic investment in increasing, for example, the availability of training data, or interoperability of tools, or ease of use for building, training, and distributing models would probably pay big dividends Would you mind expanding on these options? Universal training data sounds intriguing.
- ygjb 2y ago
  Sure, just on the training front, building and maintaining a broad corpus of properly managed training data with metadata that provides attribution (for example, content that is known to be human generated instead of model generated, what the source of data is for datasets such as weather data, census data, etc), and that also captures any licensing encumbrance so that consumers of the training data can be confident in their ability to use it without risk of legal challenge. Much of this is already available to private sector entities, but having a publicly funded organization responsible for curating and publishing this would enable new entrants to quickly and easily get a foundation without having to scrape the internet again, especially given how rapidly model generated content is being published.

jvanderbot 2y ago

A much better investment would be to (somehow) revolutionize production of chips for AI so that it's all cheaper, more reliable, and faster to stand up new generations of software and hardware codesign. This is probably much closer to the program mentioned in the top level comment: It wasn't to produce one type of thing, but to allow better production of any large thing from lighter alloys.

photonthug 2y ago

> Not sure why a publicly accessible GPU cluster would be a better solution than the current system of research grants. You mean a better solution than different teams paying AWS over and over, potentially spending 10x on rent rather than using all that cash as a down payment on actually owning hardware? I can't really speak for the total costs of depreciation/hardware maintenance but renting forever isn't usually a great alternative to buying.

CardenB 2y ago
Do you have some information to share to support your bias against leasing especially with a depreciating asset?
- manux 2y ago
  In Canada, all three major AI research centers use clusters created with public money. These clusters receive regular additional hardware as new generations of GPUs become available. Considering how these institutions work, I'm pretty confident they've considered the alternatives (renting, AWS, etc). So that's one data point.
- photonthug 2y ago
  sure, I’ll hand it over after you spend your own time first to show that everything everywhere that’s owned instead of leased is a poor financial decision.
vasili111 2y ago
AWS is not only hardware but also software, documentation, support and more.