3 ms·

I don't at all believe the people able to build models are that few. Aren't tons of people doing open source models already? I believe the number of people who

I don't at all believe the people able to build models are that few. Aren't tons of people doing open source models already? I believe the number of people who have already built models to be low, but the number of people with the capability to build models to be very large. It's not magic.

whiplash451 3y ago
The number of people who can actually pre-train a true LLM is very small. It remains a major feat with many tweaks and tricks. Case in point: the 114 pages of OPT175B logbook [1] [1] https://github.com/facebookresearch/metaseq/blob/main/projects/OPT/chronicles/OPT175B_Logbook.pdf https://github.com/facebookresearch/metaseq/blob/main/projec...