6 ms·

Wouldn't a layer per MCU be heavily bandwidth constrained?

• monocasa 3mo ago
  Depending on where you slice the model up, it can be not a whole lot of data. For instance each transformer block outputs a single vector in an embedding space. I can see that being cheaper to bitbang with PIO than to actually compute. There's certainly some latency stack up, but throughput should be remarkably good.