Current LLMs are very compute inefficient. I also think retrieval transformers can bring a few orders of magnitude in efficiency improvements. Combined with arc

Current LLMs are very compute inefficient. I also think retrieval transformers can bring a few orders of magnitude in efficiency improvements. Combined with architectural improvements I think we can get there.