4 ms·
I wonder if the author is doing anything to overclock the HBM here or if this is within the ratings of the Samsung HBM stacks. It's nice to be able to do this
by pclmulqdq 2y ago
I wonder if the author is doing anything to overclock the HBM here or if this is within the ratings of the Samsung HBM stacks. It's nice to be able to do this when you have a few cards, but if you are working with hundreds, it may not be practical to push the HBM this far without overvolting them a bit.
- latchkey 2y agoI automated the tuning of 150k gpus that were being used to mine ethereum. The trick was that as a whole, you knew the limits of the hardware. You know how to set the knobs to max performance. Due to the silicon lottery, cards that can't perform at max end up crashing. So what I did was kind of the opposite of what everyone else was doing. I first set everything at max, watched for a crash, then tuned the knobs to be a bit lower. All of this was done with an automated piece of software that I built. The cards we used essentially had 3 knobs to twist, which resulted in hundreds of combinations. Eventually, the cards stop crashing, so you're at the right settings, for that individual piece of hardware. We were running in seasonal climates too... so each winter/summer, I'd reset things and let it auto tune back again. Heat plays a huge factor on stability. Of course, each workload has different settings too... so that plays into it, but if everything else is static, this ended up being a great way to do things.
- rowanG077 2y agoThat seems great if a failure always results in a crash. There are a ton of failure modes where your result will just spuriously be wrong.
- latchkey 2y agoTo my knowledge, HPC rarely tunes cards for max performance. My MI300x are stock settings and I doubt I'll ever modify them.
- pclmulqdq 2y agoInteresting, I generally assumed Eth miners would undervolt their GPUs to get more life out of them rather than overclocking them for absolute max performance.
- latchkey 2y agoUndervolt / overclock / memory timings
- Wolf9466 2y agoAuthor here. I did overclock it - that was one of the points of the writeup: when you modify the memory clock, you should change the timings along with, because they are often specified in tCK (ticks of the memory clock), and as such, they will change when the clock changes. I have reliable information from folks with several thousand of these FPGAs that they reliably clock to 1100Mhz - 1150Mhz on the HBM2 at stock voltage (or a bit less.) This falls in line with my personal experiences - I have seven XCVU35P FPGAs, and they range from doing only 1100Mhz to 1150Mhz, to some handling 1200Mhz. Samsung's documentation specifies this HBM2 for 1000Mhz to 1100Mhz, based on binning - this is why I was annoyed that Xilinx limited it to 900Mhz, and worked to learn how to change the PLL settings.
- Wolf9466 2y agoCorrection - 1000Mhz or 1200Mhz, depending on binning.
- pclmulqdq 2y agoI am also aware the Xilinx sets their own clock specs annoyingly conservatively, and I think they do it to preserve device lifetime or something similar. However, I did want to clarify whether you were overvolting these things or just raising the clock frequency. I have run into issues where you do get a dud FPGA that is just a lot slower than other FPGAs of its speed bin (it must have come from the edge of the wafer or something), and debugging that is pretty annoying.
- Wolf9466 2y agoAs I said explicitly, stock voltage (or less.)