2 ms·
Not this one, but Google's PaLM (which is 4x OPT3) trains semi-deterministically. These kinds of large transformers can be relatively reproduceable in results
by screye 4y ago
Not this one, but Google's PaLM (which is 4x OPT3) trains semi-deterministically.
These kinds of large transformers can be relatively reproduceable in results and benchmarks. However, making them converge to the exact same parameter set might not be a reasonable expectation.