AWS GPUs (or ones offered by any of the big cloud providers for that matter) are not really all that cost effective. They also have to use datacenter GPUs, whic

AWS GPUs (or ones offered by any of the big cloud providers for that matter) are not really all that cost effective. They also have to use datacenter GPUs, which are a lot more expensive than consumer ones. For cost effectiveness, one route (mostly in the ML community) is to rent people's machines via services like vast.ai, who can get around NVIDIA preventing use of consumer cards in data centers by making it so they're only selling/buying time on people's computers. These end up being a lot more affordable (IIRC ~$20/day for a 2x 3090 system) but in exchange have the unreliability that comes with renting time on someone else's computer, like tasks randomly getting killed (which is pretty frustrating to wake up to, as you get charged for time rather than usage), some people having poor upload/download speeds or other gpu performance issues.