2 ms·

Well they do of course. There are export restrictions on supercomputers now, including many NVIDIA GPUs. I contend that doesn't matter. There is sufficient co

by nl 4y ago

Well they do of course. There are export restrictions on supercomputers now, including many NVIDIA GPUs. I contend that doesn't matter. There is sufficient compute available now at consumer levels to make it too late to stop training LLMs. If cloud A100s became unavailable tomorrow it'd be awkward, but there is enough progress being made on training on lower RAM cards to show it is possible.

dwaltrip 4y ago
> I contend that doesn't matter. Slowing things down is a real effect that will impact how circumstances unfold. Do I trust that a "slow down / pause" would be done robustly and sensibly? I wish I was more optimistic about that. At the very least, we should definitely continue having this conversation.