8 ms·

I think this is an important step, but it skips over that 'fault tolerant routing architecture' means you're spending die space on routes vs transistors. This i

I think this is an important step, but it skips over that 'fault tolerant routing architecture' means you're spending die space on routes vs transistors. This is exactly analogous to using bits in your storage for error correcting vs storing data. That said, I think they do a great job of exploiting this technique to create a "larger"[1] chip. And like storage it benefits from every core is the same and you don't need to get to every core directly (pin limiting). In the early 2000's I was looking at a wafer scale startup that had the same idea but they were applying it to an FPGA architecture rather than a set of tensor units for LLMs. Nearly the exact same pitch, "we don't have to have all of our GLUs[2] work because the built in routing only uses the ones that are qualified." Xilinx was still aggressively suing people who put SERDES ports on FPGAs so they were pin limited overall but the idea is sound. While I continue to believe that many people are going to collectively lose trillions of dollars ultimately pursuing "AI" at this stage. I appreciate the the amount of money people are willing to put at risk here allow for folks to try these "out of the box" kinds of ideas. [1] It is physically more cores on a single die but the overall system is likely smaller, given the integration here. [2] "Generic Logic Unit" which was kind of an extended LUT with some block RAM and register support.

__Joker 2y ago
"While I continue to believe that many people are going to collectively lose trillions of dollars ultimately pursuing "AI" at this stage" Can you please explain more why you think so ? Thank you.
- mschuster91 2y ago
  It's a hype cycle with many of the hypers and deciders having zero idea about what AI actually is and how it works. ChatGPT, while amazing, is at its core a token predictor, it cannot ever get to an AGI level that you'd assume to be competitive to a human, even most animals. And just as every other hype cycle, this one will crash down hard. The crypto crashes were bad enough but at least gamers got some very cheap GPUs out of all the failed crypto farms back then, but this time so much more money, particularly institutional money, is flowing around AI that we're looking at a repeat of Lehman's once people wake up and realize they've been scammed.
  - KronisLV 2y ago
    > And just as every other hype cycle, this one will crash down hard. Isn't that an inherent problem with pretty much everything nowadays: crypto, blockchain, AI, even the likes of serverless and Kubernetes, or cloud and microservices in general. There's always some hype cycle where the people who are early benefit and a lot of people chasing the hype later lose when the reality of the actual limitations and the real non-inflated utility of each technology hits. And then, a while later, it all settles down. I don't think the current "AI" is special in any way, it's just that everyone tries to get rich (or benefit in other ways, as in the microservices example, where you still very much had a hype cycle) quick without caring about the actual details.
    - anon373839 2y ago
      > I don't think the current "AI" is special in any way As someone who loves to pour ice water on AI hype, I have to say: you can't be serious. The current AI tech has opened up paths to develop applications that were impossible just a few years ago. Even if the tech freezes in place, I think it will yield substantial economic value in the coming years. It's very different from crypto, the main use case for which appears to be money laundering.
      - carlmr 2y ago
        >It's very different from crypto, the main use case for which appears to be money laundering. Which has substantial economic value (for certain groups of people).
        lazide 2y ago
        According to this random estimate, black market economy alone in just the US is worth ~ $2 trillion/yr. [https://www.investopedia.com/terms/u/underground-economy.asp https://www.investopedia.com/terms/u/underground-economy.asp] Roughly 11-12% of GDP. In many countries, black+grey market is larger than the ‘white’ market. The US is notoriously ‘clean’ compared to most (probably top 10). Even in the US, if you suddenly stopped 10-12% of GDP we’re talking ‘great depression’ levels of economic pain. Honestly, the only reason Crypto isn’t bigger IMO is because there is such a large and established set of folks doing laundering in the ‘normal’ system, and those work well enough there is not nearly as much demand as you’d expect.
- ChuckMcM 2y ago
  I would guess you're not asking a serious question here but if you were feel free to contact me, it's why I put my email address in my profile.
  - bigdict 2y ago
    Why are you assuming bad faith?
    - ChuckMcM 2y ago
      What gave you the impression I was assuming bad faith? It's off topic to the discussion (which is fine) but can be annoying in the middle of an HN thread.
      - bigdict 2y ago
        > What gave you the impression I was assuming bad faith? You said "I would guess you're not asking a serious question here"
      - ripped_britches 2y ago
        It was a direct quote from your original comment
      - bruce343434 2y ago
        You brought it up...
      - ossopite 2y ago
        Without offering any opinion on its merits, if you think justifying this controversial claim is off topic, then so is the claim and you shouldn't have written it.
      - kragen 2y ago
        You said, "I would guess you're not asking a serious question here," which is to say, you were guessing that the question was asked in bad faith. Or, at any rate, you would, if for some reason the question came up, for example in deciding how to answer it. Which is what you were doing. That is to say, you did guess that it was asked in bad faith. Given the minimal amount of evidence available (12 words and a nickname "__Joker") I think it's reasonable to describe that guess as an assumption. Ergo, you were assuming bad faith.
enragedcacti 2y ago
Any thoughts on why they are disabling so many cores in their current product? I did some quick noodling based on the 46/970000 number and the only way I ended up close to 900,000 was by assuming that an entire row or column would be disabled if any core within it was faulty. But doing that gave me a ~6% yield as most trials had active core counts in the high 800,000s
- projektfu 2y ago
  They did mention that they stash extra cores to enable the re-routing. Those extra cores are presumably unused when not routed in.
  - enragedcacti 2y ago
    That was my first thought but based on the rerouting graphic it seems like the extra cores would be one or two rows and columns around the border which would only account for ~4000 cores.
    - projektfu 2y ago
      If the system were broken down into more subdivisions internally, there would be more cores dedicated to replacement. It seems like it could be more difficult to reroute an entire row or column of cores on a wafer than a small block. Perhaps, also, they are building in heavy redundancy for POC and in the future will optimize the number of cores they expect to lose.
- ChuckMcM 2y ago
  I could guess that it helps with heat dissipation/management. But I don't know. That guess is from looking at the list of patents[1] they have. [1] https://patents.justia.com/assignee/cerebras-systems-inc https://patents.justia.com/assignee/cerebras-systems-inc
girvo 2y ago
> Xilinx was still aggressively suing people who put SERDES ports on FPGAs This so isn't important to your overall point, but where would I begin to look into this? Sounds fascinating!
- nroize 2y ago
  Not OP but I was curious too. Here's all I could find that seemed related: https://www.businesswire.com/news/home/20200121005582/en/Xilinx-Files-Patent-Infringement-Lawsuit-Against-Analog-Devices https://www.businesswire.com/news/home/20200121005582/en/Xil...
- ChuckMcM 2y ago
  Well this was the patent they were threatening with as I recall (https://patents.google.com/patent/US20030023912A1/en https://patents.google.com/patent/US20030023912A1/en) and there was this one too: https://patents.google.com/patent/US5576554A/en https://patents.google.com/patent/US5576554A/en Basically the "secret sauce" of the startup recruiting me was that they were going to do wafer scale FPGAs that could be tiled together to build arbitrarily complex systems like military phased array radars and such. All very hush hush but apparently they had recruited some key talent from Xilinx which was annoying Xilinx.
deleted 2y ago
[deleted]
dogcomplex 2y ago
Of course many people are going to collectively lose trillions, AI's a very highly hyped industry with people racing into it without an intellectual edge and any temporary achievement by any one company will be quickly replicated and undercut by another using the same tools. Economic success of the individuals swarming on a new technology is not a guarantee whatsoever, nor is it an indicator of the impact of the technology. Just like the dotcom bubble, AI is gonna hit, make a few companies stinking rich, and make the vast majority (of both AI-chasing and legacy) companies bankrupt. And it's gonna rewire the way everything else operates too.
- Melomomololo 2y ago
  [dead]
- idiotsecant 2y ago
  >it's gonna rewire the way everything else operates too. This is the part that I think a lot of very tech literate people don't seem to get. I see people all the time essentially saying 'AI is just autocomplete' or pointing out that some vaporware ai company is a scam so surely everyone is. A lot of it is scams and flash in the pan. But a few of them are going to transform our lives in ways we probably don't even anticipate yet, for good and bad.
  - Retric 2y ago
    I’m not so sure it’s going to even do that much. People are currently happy to use LLM’s, but the outputs aren’t accurate and don’t seem to be improving quickly. A YouTuber watch regularly includes questions they asked Chat GPT and very single time there’s a detailed response in the comments showing how the output is wildly wrong from multiple mistakes. I suspect the backlash from disgruntled users is going to hit the industry hard and these models are still extremely expensive to keep updated.
    - Thews 2y ago
      Using function calls for correct answer lookup already practically eliminates this, it's not wide spread yet, but the ease of doing it is already practical for many. New models aren't being trained specifically on single answers which will only help. The expense for the larger models is something to be concerned about. Small models with function calls is already great, especially if you narrow down what they are being used for. Not seeing their utility is just a lack of imagination.

__Joker 2y ago

"While I continue to believe that many people are going to collectively lose trillions of dollars ultimately pursuing "AI" at this stage" Can you please explain more why you think so ? Thank you.

mschuster91 2y ago
It's a hype cycle with many of the hypers and deciders having zero idea about what AI actually is and how it works. ChatGPT, while amazing, is at its core a token predictor, it cannot ever get to an AGI level that you'd assume to be competitive to a human, even most animals. And just as every other hype cycle, this one will crash down hard. The crypto crashes were bad enough but at least gamers got some very cheap GPUs out of all the failed crypto farms back then, but this time so much more money, particularly institutional money, is flowing around AI that we're looking at a repeat of Lehman's once people wake up and realize they've been scammed.
- KronisLV 2y ago
  > And just as every other hype cycle, this one will crash down hard. Isn't that an inherent problem with pretty much everything nowadays: crypto, blockchain, AI, even the likes of serverless and Kubernetes, or cloud and microservices in general. There's always some hype cycle where the people who are early benefit and a lot of people chasing the hype later lose when the reality of the actual limitations and the real non-inflated utility of each technology hits. And then, a while later, it all settles down. I don't think the current "AI" is special in any way, it's just that everyone tries to get rich (or benefit in other ways, as in the microservices example, where you still very much had a hype cycle) quick without caring about the actual details.
  - anon373839 2y ago
    > I don't think the current "AI" is special in any way As someone who loves to pour ice water on AI hype, I have to say: you can't be serious. The current AI tech has opened up paths to develop applications that were impossible just a few years ago. Even if the tech freezes in place, I think it will yield substantial economic value in the coming years. It's very different from crypto, the main use case for which appears to be money laundering.
    - carlmr 2y ago
      >It's very different from crypto, the main use case for which appears to be money laundering. Which has substantial economic value (for certain groups of people).
      - lazide 2y ago
        According to this random estimate, black market economy alone in just the US is worth ~ $2 trillion/yr. [https://www.investopedia.com/terms/u/underground-economy.asp https://www.investopedia.com/terms/u/underground-economy.asp] Roughly 11-12% of GDP. In many countries, black+grey market is larger than the ‘white’ market. The US is notoriously ‘clean’ compared to most (probably top 10). Even in the US, if you suddenly stopped 10-12% of GDP we’re talking ‘great depression’ levels of economic pain. Honestly, the only reason Crypto isn’t bigger IMO is because there is such a large and established set of folks doing laundering in the ‘normal’ system, and those work well enough there is not nearly as much demand as you’d expect.
ChuckMcM 2y ago
I would guess you're not asking a serious question here but if you were feel free to contact me, it's why I put my email address in my profile.
- bigdict 2y ago
  Why are you assuming bad faith?
  - ChuckMcM 2y ago
    What gave you the impression I was assuming bad faith? It's off topic to the discussion (which is fine) but can be annoying in the middle of an HN thread.
    - bigdict 2y ago
      > What gave you the impression I was assuming bad faith? You said "I would guess you're not asking a serious question here"
    - ripped_britches 2y ago
      It was a direct quote from your original comment
    - bruce343434 2y ago
      You brought it up...
    - ossopite 2y ago
      Without offering any opinion on its merits, if you think justifying this controversial claim is off topic, then so is the claim and you shouldn't have written it.
    - kragen 2y ago
      You said, "I would guess you're not asking a serious question here," which is to say, you were guessing that the question was asked in bad faith. Or, at any rate, you would, if for some reason the question came up, for example in deciding how to answer it. Which is what you were doing. That is to say, you did guess that it was asked in bad faith. Given the minimal amount of evidence available (12 words and a nickname "__Joker") I think it's reasonable to describe that guess as an assumption. Ergo, you were assuming bad faith.

enragedcacti 2y ago

Any thoughts on why they are disabling so many cores in their current product? I did some quick noodling based on the 46/970000 number and the only way I ended up close to 900,000 was by assuming that an entire row or column would be disabled if any core within it was faulty. But doing that gave me a ~6% yield as most trials had active core counts in the high 800,000s

projektfu 2y ago
They did mention that they stash extra cores to enable the re-routing. Those extra cores are presumably unused when not routed in.
- enragedcacti 2y ago
  That was my first thought but based on the rerouting graphic it seems like the extra cores would be one or two rows and columns around the border which would only account for ~4000 cores.
  - projektfu 2y ago
    If the system were broken down into more subdivisions internally, there would be more cores dedicated to replacement. It seems like it could be more difficult to reroute an entire row or column of cores on a wafer than a small block. Perhaps, also, they are building in heavy redundancy for POC and in the future will optimize the number of cores they expect to lose.
ChuckMcM 2y ago
I could guess that it helps with heat dissipation/management. But I don't know. That guess is from looking at the list of patents[1] they have. [1] https://patents.justia.com/assignee/cerebras-systems-inc https://patents.justia.com/assignee/cerebras-systems-inc

girvo 2y ago

> Xilinx was still aggressively suing people who put SERDES ports on FPGAs This so isn't important to your overall point, but where would I begin to look into this? Sounds fascinating!

nroize 2y ago
Not OP but I was curious too. Here's all I could find that seemed related: https://www.businesswire.com/news/home/20200121005582/en/Xilinx-Files-Patent-Infringement-Lawsuit-Against-Analog-Devices https://www.businesswire.com/news/home/20200121005582/en/Xil...
ChuckMcM 2y ago
Well this was the patent they were threatening with as I recall (https://patents.google.com/patent/US20030023912A1/en https://patents.google.com/patent/US20030023912A1/en) and there was this one too: https://patents.google.com/patent/US5576554A/en https://patents.google.com/patent/US5576554A/en Basically the "secret sauce" of the startup recruiting me was that they were going to do wafer scale FPGAs that could be tiled together to build arbitrarily complex systems like military phased array radars and such. All very hush hush but apparently they had recruited some key talent from Xilinx which was annoying Xilinx.

deleted 2y ago

[deleted]

dogcomplex 2y ago

Of course many people are going to collectively lose trillions, AI's a very highly hyped industry with people racing into it without an intellectual edge and any temporary achievement by any one company will be quickly replicated and undercut by another using the same tools. Economic success of the individuals swarming on a new technology is not a guarantee whatsoever, nor is it an indicator of the impact of the technology. Just like the dotcom bubble, AI is gonna hit, make a few companies stinking rich, and make the vast majority (of both AI-chasing and legacy) companies bankrupt. And it's gonna rewire the way everything else operates too.

Melomomololo 2y ago
[dead]
idiotsecant 2y ago
>it's gonna rewire the way everything else operates too. This is the part that I think a lot of very tech literate people don't seem to get. I see people all the time essentially saying 'AI is just autocomplete' or pointing out that some vaporware ai company is a scam so surely everyone is. A lot of it is scams and flash in the pan. But a few of them are going to transform our lives in ways we probably don't even anticipate yet, for good and bad.
- Retric 2y ago
  I’m not so sure it’s going to even do that much. People are currently happy to use LLM’s, but the outputs aren’t accurate and don’t seem to be improving quickly. A YouTuber watch regularly includes questions they asked Chat GPT and very single time there’s a detailed response in the comments showing how the output is wildly wrong from multiple mistakes. I suspect the backlash from disgruntled users is going to hit the industry hard and these models are still extremely expensive to keep updated.
  - Thews 2y ago
    Using function calls for correct answer lookup already practically eliminates this, it's not wide spread yet, but the ease of doing it is already practical for many. New models aren't being trained specifically on single answers which will only help. The expense for the larger models is something to be concerned about. Small models with function calls is already great, especially if you narrow down what they are being used for. Not seeing their utility is just a lack of imagination.