There will also be hardware improvements (as always) and ASIC chips specifically designed for running this kind of model. For example, see this "Optical Transfo

There will also be hardware improvements (as always) and ASIC chips specifically designed for running this kind of model. For example, see this "Optical Transformers" paper [0] and its HN discussion [1] from last month. [0] https://arxiv.org/abs/2302.10360 https://arxiv.org/abs/2302.10360 [1] https://news.ycombinator.com/item?id=34905210 https://news.ycombinator.com/item?id=34905210