3 ms·
I have a couple of M40 with 24 gb on a desktop computer, i had to tape the pci-e to "scale it down to 1x". It's ok for interference and play around with small t
by _davide_ 2y ago
I have a couple of M40 with 24 gb on a desktop computer, i had to tape the pci-e to "scale it down to 1x".
It's ok for interference and play around with small trainings, but I can barely infer a quantized version of 70b parameters. Training anything bigger than 3b parameters in human timescale is impossible.
Either you scale it down or ask for a sponsor. It's frustrating because IT had always been approachable, until now...