3 ms·
Good for them, Europe should compete against US, China, etc. It must be nerve-wracking being in charge of huge multiple month training cycles. In the 1980s, we
by mark_l_watson 3y ago
Good for them, Europe should compete against US, China, etc.
It must be nerve-wracking being in charge of huge multiple month training cycles. In the 1980s, we built our own hardware to speed up back propagation learning and recall and runs could still take a day or two. It was always so disappointing when a long training run went bad. I can imagine how tense the responsibility must be to manage these huge training cycles.
- Jeff_Brown 3y agoAre there no incremental ways to verify that it's working?
- olalonde 3y agoYes, you can. After every training batch, the network's loss function is updated and you can track it over time to verify that it's going down. Sometimes, the loss can get stuck on a plateau, or even go up, and it's not clear whether it's worthwhile to keep training or not.
- brucethemoose2 3y agoTraditionally, training runs can "explode" and fail, but there are methods to incrementally back them up and resume when that happens, see https://www.mosaicml.com/blog/mpt-7b https://www.mosaicml.com/blog/mpt-7b
- mark_l_watson 3y agoOf course, you can measure loss on a dev set, etc. However, LLMs are so complex that until they are done training and many people evaluate them, there will be misses.
- senectus1 3y agoWhat political lean will a LLM have... we've had some really bad politics come out of Europe throughout history. Is this the beginning of a LLLM? (Lenin LLM) /s
- WanderPanda 3y agoIt will for sure be well versed in government overreach
- deleted 3y ago[deleted]