2 ms·
Training an LLM (basically Transformers) is different workflow from Nuclear Simulations (basically Monte Carlo simulations) There are a lot of intricates, but
by alephnerd 2y ago
Training an LLM (basically Transformers) is different workflow from Nuclear Simulations (basically Monte Carlo simulations)
There are a lot of intricates, but at a high level they require different compute approaches.
- handfuloflight 2y agoCan you expand on why the operations per second is not an apt comparison?
- pertymcpert 2y agoWhen you're doing scientific simulations, you're generally a lot more sensitive to FP precision than ML training which is very, very tolerant of reduced precision. So while FP8 might be fine for transformer networks, it would likely be unacceptably inaccurate/unusable for simulations.
- pama 2y agoAbsolutely. Though the performance of El Capitain is only measured by a linpack benchmark not the actual application.
- pertymcpert 2y agoI thought modern supercomputers use benchmarks like HPCG instead of LINPACK?
- fancyfredbot 2y agoThe top 500 includes both. There is no HPCG result for El Capitan yet: https://top500.org/lists/hpcg/2024/11/ https://top500.org/lists/hpcg/2024/11/
- Koshkin 2y agoThis is about the raw compute, no matter the workflow.
- alephnerd 2y agoIt isn't. I recommend reading u/pertymcpert's response.