3 ms·
I was really impressed with Microsoft's paper on llm architectures, where instead of using giant chips with giant memory busses, they pipe together a bunch of c
by brucethemoose2 3y ago
I was really impressed with Microsoft's paper on llm architectures, where instead of using giant chips with giant memory busses, they pipe together a bunch of cheap, monolithic, SRAM heavy SoCs.
This would work really well for something like Llama, where the layers can be easily split up into SRAM sized chunks, and the whole thing can be pipelined as long as many users are making requests.
- marshray 3y ago> I was really impressed with Microsoft's paper on llm architectures Link? Please :-)
- brucethemoose2 3y agohttps://www.nextplatform.com/2023/07/12/microsofts-chiplet-cloud-to-bring-the-cost-of-llms-way-down/ https://www.nextplatform.com/2023/07/12/microsofts-chiplet-c... https://arxiv.org/pdf/2307.02666.pdf https://arxiv.org/pdf/2307.02666.pdf