3 ms·
If you can buy consumer grade GPU i'll for sure prefer that as you can buy 4x rtx 3090 in same price as one A6000 and for almost all use cases outside extremely
by machinekob 4y ago
If you can buy consumer grade GPU i'll for sure prefer that as you can buy 4x rtx 3090 in same price as one A6000 and for almost all use cases outside extremely big models it should be a lot better solution. (I have one PC with 4x rtx 3090 for DL and it is enough but sometimes as i need to run very big model i just turn on colab with TPU [inference] or rent A100 server as it'll be still cheaper then any other solution)
For CPU I cant say much cause we are always limited by PCIE speed in our use case and Xeons are idle most of the time :P
- uniqueuid 4y agoThanks, that's a good data point. I'm wondering especially what the tangent for future developments will be. Most of the large language models are out of my league anyways (e.g. the new yandex russian-english model was trained on 800 A100s and needs 200GB of GPU ram to fine tune). So maybe it would be more effective to go for high speed instead of large capacity. But then you probably end up with a custom chassis and PSU since 4x 400 watts are not something that you can use on most off-the-shelf workstations.