3 ms·
So I've read through the blog posts, etc... no actual details on how it works. I suspect it's a huge matrix multiply/accumulate chip with registers for all the
by mikewarot 2y ago
So I've read through the blog posts, etc... no actual details on how it works. I suspect it's a huge matrix multiply/accumulate chip with registers for all the weights and biases that get loaded once per section of the model, so that you only spend I/O on getting the tokens in and out, and doing the softmax, etc.. on a CPU.
I've considered similar thoughts with my toy BitGrid model... except I'd actually take the weights and compile them to boolean logic, since it would improve utilization. Program the chip, throw parameters in one side, get them out (later) on the other.