3 ms·
Diffusion LLMs also follow scaling laws - https://proceedings.neurips.cc/paper_files/paper/2023/file/35b5c175e139bff5f22a5361270fce87-Paper-Conference.pdf https
by gdiamos 2y ago
Diffusion LLMs also follow scaling laws - https://proceedings.neurips.cc/paper_files/paper/2023/file/35b5c175e139bff5f22a5361270fce87-Paper-Conference.pdf https://proceedings.neurips.cc/paper_files/paper/2023/file/3...
- esperent 2y agoIs it possible that combining multiple AIs will be able to somewhat bypass scaling laws, in a similar way that multicore CPUs can somewhat bypass the limitations of a single CPU core?
- gdiamos 2y agoI’m sure there are ways of bypassing scaling laws, but I think we need more research to discover and validate them
- impossiblefork 2y agoThose aren't the modern type with discrete masking based diffusion though. Of course, these too will have scaling laws.