Regarding the follow-up, I can at least tell you my current candidate pick: The Lenovo P620 has new Threadripper Pros with their high single-core performance,

Regarding the follow-up, I can at least tell you my current candidate pick: The Lenovo P620 has new Threadripper Pros with their high single-core performance, can take ordinary (i.e. non-blower-style active) GPUs, take 2 PCIe4 NVME ssds plus ~8 PICe3 SSDs via lane splitting and have space for ~4-5 HDDs. Putting one of these in a rackmount kit would be a pretty good choice balancing performance. Plus I can liberally use regular hardware that's not gold-plated. This should open some cheap upgrade options in the future, i.e. a new Hopper GPU. The only downsides are (1) RAM limitations (although the max capacity is 1TB with the new 128GB RDIMMS) and (2) power, since it only has a 1000W PSU.