3 ms·
The number of people who can actually pre-train a true LLM is very small. It remains a major feat with many tweaks and tricks. Case in point: the 114 pages of
by whiplash451 3y ago
The number of people who can actually pre-train a true LLM is very small.
It remains a major feat with many tweaks and tricks. Case in point: the 114 pages of OPT175B logbook [1]
[1] https://github.com/facebookresearch/metaseq/blob/main/projects/OPT/chronicles/OPT175B_Logbook.pdf https://github.com/facebookresearch/metaseq/blob/main/projec...