3 ms·
YaFSDP: a sharded data parallelism framework, faster for pre-training LLMs
- ilrwbwrkhv 2y ago[flagged]
- loopdoend 2y agoBulgarians are just as smart - it's an EU country and the language is easier to learn than Russian. Здравейте! Many other benefits to hiring Bulgarians also.
- deleted 2y ago[deleted]
- maxdo 2y ago[flagged]
- josfredo 2y agoThis is true. Some of them could also be good persons, just like absolutely anywhere else in the world. Generalization is the great cow’s middle state.
- giancarlostoro 2y agoIf the code is good and open source why should I care about their personal life? Ship it.
- dayeye2006 2y agoAny idea on what are the main tricks used to achieve gains over fsdp?
- albertzeyer 2y agoThe blog post seems to contain more details and the core ideas: https://medium.com/yandex/yafsdp-a-tool-for-faster-llm-training-and-optimized-gpu-utilization-is-no-632b7539f5b3 https://medium.com/yandex/yafsdp-a-tool-for-faster-llm-train...
- az226 2y agoOdd that they don’t expand on this: In Yandex’s pre-trainings, the implementation of YaFSDP along with other memory optimization strategies resulted in a speed gain of 45%.
- codetrotter 2y agoI was surprised to see that the Ya part meant “Yet another”. I mean, I’ve seen it before in many acronyms. But it’s pretty tongue in cheek of them to do that here since one would expect it was just because it was made by Yandex.
- mikrl 2y agoI was expecting it to be a Russian acronym starting with the letter Я which is pronounced Ya. It acquired its backward R glyph when it was changed from an old Slavic letter I cannot draw.
- deaddodo 2y agoWhat do you mean that you "cannot draw" them? This is a digital medium and both (well one, the other is half supported) variants are valid Unicode glyphs: ꙗ / Ѧ Or do you mean you literally can't draw them?
- Tade0 2y agoIt's an idiomatic expression in slavic languages, indicating that the shape is particularly complex.
- deaddodo 2y agoI'm pretty sure that it's an idiomatic phrase in any language for that.
- mikrl 2y agoI could not reproduce it by hand without a reference nor do I have a keyboard installed which offers it as a symbol, nor was I going to look it up to add the spice to a shower thought tier HN comment. I see you have provided it, making it more accessible for my future use, at least on the timeframe of this thread being in my recent HN activity.
- shadow28 2y ago