You should look into the Cerebras architecture https://medium.com/@cerebras/cerebras-architecture-deep-dive-first-look-inside-the-hw-sw-co-design-for-deep-lear

You should look into the Cerebras architecture https://medium.com/@cerebras/cerebras-architecture-deep-dive-first-look-inside-the-hw-sw-co-design-for-deep-learning-b38d0d0c5e53 https://medium.com/@cerebras/cerebras-architecture-deep-dive... It's a lot closer to what you're thinking, and you can use it for inference today with an API key. https://cloud.cerebras.ai/?utm_source=homepage https://cloud.cerebras.ai/?utm_source=homepage