7 ms·

Silicon innovation and custom ASICs at Amazon [pdf]

97 points by mad44 4y ago

rob-olmos 4y ago
What's the next circle going to be? Mainframes become even more specialized & powerful, and then the cloud builds new more specialized silicon to match it?
sokoloff 4y ago
The slides are interesting; I found the text commentary a good supplement: https://perspectives.mvdirona.com/2022/11/hpts-2022/ https://perspectives.mvdirona.com/2022/11/hpts-2022/
amelius 4y ago
What does "Hello World" look like in the silicon world these days? (Looking for a complete example containing everything necessary to go from code to tapeout, I know multiple answers are possible).
- reportingsjr 4y ago
  This is a pretty recent thing, but for a beginner in the VLSI world I'd say this is a good "hello world": https://tinytapeout.com/ https://tinytapeout.com/
- lnsru 4y ago
  The free asic program sponsored by big search engine: https://efabless.com/open_shuttle_program https://efabless.com/open_shuttle_program However it might be much more complex than simple “hello world”.
- zerohp 4y ago
  I don't think it exists for advanced nodes like the ones Amazon is using. What does "Hello World" look like for a skyscraper?
  - fragmede 4y ago
    More of an MVP than "hello world", but a 6 story building that utilizes girders and concrete with an elevator, using materials and construction techniques that would scale to a 110+ story building.
evancox100 4y ago
For those wondering, the definition of HPTS: “High Performance Transactions System (HPTS) is a invitational conference held once every two years at the Asilomar Conference Center near Monterey California”
jeffbee 4y ago
I like the concrete numbers in this deck. Over 20 million nitros installed, over 12GW power capacity. Gives us a chance to compare scale with the other bigs.
- nsteel 4y ago
  Genuinely curious, why do you want to directly compare? Does this make them more attractive to work for?
  - jeffbee 4y ago
    I think it's interesting to see how big these clouds are in absolute terms because it gives us an idea of when the cloud has finished eating the world. We have global IT equipment energy consumption estimates, and we have scattered data points on cloud energy consumption. Looking at the two, you can gauge the overall process. Also as an investor I like to have a general idea of Amazon vs. Google in terms of overall size, to combine with their revenue figures, because that helps me understand how much of Google is being sold as GCP and how much is being used by Google itself.
pwarner 4y ago
The cloud is the new mainframe. The difference vs the old mainframe is it's so much more accessible to anyone. Barrier to entry to build one will be high, but to consume is very low. I'm trying to decide if lock in problem becomes bigger, but I think where people follow modern software engineering best practice, they can move if needed.
- ausudhz 4y ago
  Without multi-year contacts. I think the only problem reside in applications that require longevity. Many apps nowadays get rewritten every now and then (especially front ends and Middleware) The problem is different for database and core systems.
  - pwarner 4y ago
    I think it still comes down to proper software engineering. If you have good interfaces, abstractions, and automated tests, you can move to new systems. I've seen teams struggle to move from DB version x to x+1, taking many many months, but it's because they have no idea if it works after they upgrade. On the flip side you have people like Snowflake who are building a database that runs across multiple clouds. From the outside it appears both portable and optimized for each platform. Thoughtful software engineering, with the right abstractions and test automation are a big deal...
    - ausudhz 4y ago
      Some databases and enterprise application have penalizing contract agreements that prevent cloud migrations from happening.
- js8 4y ago
  > The difference vs the old mainframe is it's so much more accessible to anyone. As someone who works on zSeries mainframes, I am not sure I agree. For developers, this is true, no doubt. But for organizations, in the cloud your data are locked away in ways they are not on the on-prem mainframe. Vendor lock-in seems to be similar - you use some middleware, you're locked in. With mainframe, there is more control over the infrastructure than in cloud. I see our management (I work for MF utilities vendor) constantly wrestle for control over our customer's environment that would be easily given in cloud (e.g. telemetry). What's also interesting (but slowly changing to the worse), the mainframe infrastructure (z/OS for instance) is quite open to custom modification, IMHO more than cloud (but it depends on type, IaaS/PaaS/SaaS).
ancharm 4y ago
Is there any video of this posted somewhere?
mk_stjames 4y ago
Whenever I see these modern cloud centers with racks and racks of GPU servers, or 64-core custom ARM CPU blade servers with terabytes of RAM... I can't help but wonder how many years it will be until I can just pick one rack up off Ebay for a few hundo and play with it, like I used to do with old 80s and 90s obsolete surplus. The way things are going... probably never.
- arbuge 4y ago
  > The way things are going... probably never. Why not? What way are things going then?
  - UncleOxidant 4y ago
    I'm wondering the same. Maybe they're implying that things are going to get so bad that Amazon will just never be able to afford to replace them and they'll stick around like an IRS mainframe?
  - sn0wf1re 4y ago
    It seems the hyperscalers are more in favour of shreding their hardware rather than putting it for sale on ebay. But I'm not sure if this holds true, I have seen old Google servers on /r/homelab.
    - count 4y ago
      They have data sanitization requirements that become difficult to manage at scale if they do anything else. Are you SURE there was no customer data stored in a recoverable-by-modern-physics manner on that machine you sold? Would you stake billions of dollars on it?
      - dafelst 4y ago
        Pretty much everything is encrypted at rest in the big datacenters these days
latchkey 4y ago
AWS uses 12,000MW of power.
brooksbp 4y ago
> Where Have I Been? 2012 to 2022 around the world in a small boat. Worked full time at AWS. Only in North America 3 to 4 times/year. Great to be back! WFH is over folks. Joking aside, that's awesome, and I hope some flexibility remains for all. Especially for those with little kids and two working parents.

97 points by mad44 4y ago

What's the next circle going to be? Mainframes become even more specialized & powerful, and then the cloud builds new more specialized silicon to match it?

sokoloff 4y ago

The slides are interesting; I found the text commentary a good supplement: https://perspectives.mvdirona.com/2022/11/hpts-2022/ https://perspectives.mvdirona.com/2022/11/hpts-2022/

amelius 4y ago

What does "Hello World" look like in the silicon world these days? (Looking for a complete example containing everything necessary to go from code to tapeout, I know multiple answers are possible).

reportingsjr 4y ago
This is a pretty recent thing, but for a beginner in the VLSI world I'd say this is a good "hello world": https://tinytapeout.com/ https://tinytapeout.com/
lnsru 4y ago
The free asic program sponsored by big search engine: https://efabless.com/open_shuttle_program https://efabless.com/open_shuttle_program However it might be much more complex than simple “hello world”.
zerohp 4y ago
I don't think it exists for advanced nodes like the ones Amazon is using. What does "Hello World" look like for a skyscraper?
- fragmede 4y ago
  More of an MVP than "hello world", but a 6 story building that utilizes girders and concrete with an elevator, using materials and construction techniques that would scale to a 110+ story building.

evancox100 4y ago

For those wondering, the definition of HPTS: “High Performance Transactions System (HPTS) is a invitational conference held once every two years at the Asilomar Conference Center near Monterey California”

jeffbee 4y ago

I like the concrete numbers in this deck. Over 20 million nitros installed, over 12GW power capacity. Gives us a chance to compare scale with the other bigs.

nsteel 4y ago
Genuinely curious, why do you want to directly compare? Does this make them more attractive to work for?
- jeffbee 4y ago
  I think it's interesting to see how big these clouds are in absolute terms because it gives us an idea of when the cloud has finished eating the world. We have global IT equipment energy consumption estimates, and we have scattered data points on cloud energy consumption. Looking at the two, you can gauge the overall process. Also as an investor I like to have a general idea of Amazon vs. Google in terms of overall size, to combine with their revenue figures, because that helps me understand how much of Google is being sold as GCP and how much is being used by Google itself.

pwarner 4y ago

The cloud is the new mainframe. The difference vs the old mainframe is it's so much more accessible to anyone. Barrier to entry to build one will be high, but to consume is very low. I'm trying to decide if lock in problem becomes bigger, but I think where people follow modern software engineering best practice, they can move if needed.

ausudhz 4y ago
Without multi-year contacts. I think the only problem reside in applications that require longevity. Many apps nowadays get rewritten every now and then (especially front ends and Middleware) The problem is different for database and core systems.
- pwarner 4y ago
  I think it still comes down to proper software engineering. If you have good interfaces, abstractions, and automated tests, you can move to new systems. I've seen teams struggle to move from DB version x to x+1, taking many many months, but it's because they have no idea if it works after they upgrade. On the flip side you have people like Snowflake who are building a database that runs across multiple clouds. From the outside it appears both portable and optimized for each platform. Thoughtful software engineering, with the right abstractions and test automation are a big deal...
  - ausudhz 4y ago
    Some databases and enterprise application have penalizing contract agreements that prevent cloud migrations from happening.
js8 4y ago
> The difference vs the old mainframe is it's so much more accessible to anyone. As someone who works on zSeries mainframes, I am not sure I agree. For developers, this is true, no doubt. But for organizations, in the cloud your data are locked away in ways they are not on the on-prem mainframe. Vendor lock-in seems to be similar - you use some middleware, you're locked in. With mainframe, there is more control over the infrastructure than in cloud. I see our management (I work for MF utilities vendor) constantly wrestle for control over our customer's environment that would be easily given in cloud (e.g. telemetry). What's also interesting (but slowly changing to the worse), the mainframe infrastructure (z/OS for instance) is quite open to custom modification, IMHO more than cloud (but it depends on type, IaaS/PaaS/SaaS).

ancharm 4y ago

Is there any video of this posted somewhere?

mk_stjames 4y ago

Whenever I see these modern cloud centers with racks and racks of GPU servers, or 64-core custom ARM CPU blade servers with terabytes of RAM... I can't help but wonder how many years it will be until I can just pick one rack up off Ebay for a few hundo and play with it, like I used to do with old 80s and 90s obsolete surplus. The way things are going... probably never.

arbuge 4y ago
> The way things are going... probably never. Why not? What way are things going then?
- UncleOxidant 4y ago
  I'm wondering the same. Maybe they're implying that things are going to get so bad that Amazon will just never be able to afford to replace them and they'll stick around like an IRS mainframe?
- sn0wf1re 4y ago
  It seems the hyperscalers are more in favour of shreding their hardware rather than putting it for sale on ebay. But I'm not sure if this holds true, I have seen old Google servers on /r/homelab.
  - count 4y ago
    They have data sanitization requirements that become difficult to manage at scale if they do anything else. Are you SURE there was no customer data stored in a recoverable-by-modern-physics manner on that machine you sold? Would you stake billions of dollars on it?
    - dafelst 4y ago
      Pretty much everything is encrypted at rest in the big datacenters these days

latchkey 4y ago

AWS uses 12,000MW of power.

brooksbp 4y ago

> Where Have I Been? 2012 to 2022 around the world in a small boat. Worked full time at AWS. Only in North America 3 to 4 times/year. Great to be back! WFH is over folks. Joking aside, that's awesome, and I hope some flexibility remains for all. Especially for those with little kids and two working parents.