Any company as big as Meta have teams working on optimisation (eg optimised kernels), usually with direct engagement with NVIDIA engineers. These kind of thing

Any company as big as Meta have teams working on optimisation (eg optimised kernels), usually with direct engagement with NVIDIA engineers. These kind of things are usually only selectively shared.