2 ms·
Yes. The path dependence for current attention based LLMs is enormous.
by UltraSane 11mo ago
Yes. The path dependence for current attention based LLMs is enormous.
- patapong 11mo agoAt the same time, there is now a ton of data for training models to act as useful assistants, and benchmarks to compare different assistant models. The wide availability and ease of obtaining new RLHF training data will make it more feasible to build models on new architectures I think.