Sure if you are using tiny net(by today's standard) and small batch sizes, A100 is just twice as fast compared to 3090 due to overhead of framework and memory m

Sure if you are using tiny net(by today's standard) and small batch sizes, A100 is just twice as fast compared to 3090 due to overhead of framework and memory movement. A100 has 312 FP16 TFlops[1], compared to 40 and 82.6 of 3090 Ti and 4090 respectively[2], and has 3x memory. Also it has 10x inter GPU communication bandwidth. [1]: https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/a100/pdf/nvidia-a100-datasheet-us-nvidia-1758950-r4-web.pdf https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Cent... [2]: https://images.nvidia.com/aem-dam/Solutions/Data-Center/l4/nvidia-ada-gpu-architecture-whitepaper-v2.1.pdf https://images.nvidia.com/aem-dam/Solutions/Data-Center/l4/n...