3 ms·
My thought was whether we could use architecture or programming ideas from the Connection Machine for LLM training or inference today. Also, any of the massivel
by nickpsecurity 2y ago
My thought was whether we could use architecture or programming ideas from the Connection Machine for LLM training or inference today. Also, any of the massively-parallel, distributed, shared memory systems of old times. The patents should be expired on many of them.
That many use cases today are both highly-parallel and pipelined make me think the old designs might work. Also, the hundreds of thousands of execution units in Cerebras’ hardware are a similar concept.