3 ms·
It's a pretty mad architecture tbh. Compile times must be absolutely insane. Also Tesla's Dojo also uses a manufacturing technique that has basically obsoleted
by IshKebab 4y ago
It's a pretty mad architecture tbh. Compile times must be absolutely insane. Also Tesla's Dojo also uses a manufacturing technique that has basically obsoleted their WSI design already.
- anon291 4y agoCompile times are not a whole lot different than any other large model build. It's a kernel based compilation pipeline and the kernels are simply tiled over a 'core' area in the weight streaming architecture.
- IshKebab 4y agoI seriously doubt that. What's your source? I used to work for a competitor with a more flexible architecture and even our compile times were bad (significant fractions of a day in some cases). And we didn't have to do place and route! I just googled it and it's apparently bad enough that they had to implement incremental place and route.
- anon291 4y agoI used to work at Cerebras. They don't do place and route anymore. That's the old pipeline mode. They've shifted to weight streaming