I'm not super knowledgeable in machine learning but given that system RAM (not VRAM) in the TB range is really not that rare these days for high end workstation

I'm not super knowledgeable in machine learning but given that system RAM (not VRAM) in the TB range is really not that rare these days for high end workstations I assumed that the poster meant that you'd have 1TB of working RAM per GPU for the learning job (which would be paged in and out of VRAM as necessary). I guess the lesson to be learned here is that if you want to use an implausibly huge amount of RAM to make a point, a TB is not safe anymore. Go for Exabytes instead, that should be unambiguous for a couple of years.