3 ms·

I have a 10kW 4090 training cluster in my garage with water cooling. It paid itself back in 3 months if I'd used AWS services.

• alanfranz 2y ago
  Not something that requires HA, backups, scaling, auditing, logs, security, and such, though. Would you run your prod inference cluster the same way?
  • llm_trw 2y ago
    It's little brother is the prod inference cluster. It's had an uptime of 100% over the last two years. People seriously don't realize just how _big_ computers have gotten. The AWS mentality is still stuck in 2005 when a t2.micro was a decent computer.
    • Dunedan 2y ago
      > The AWS mentality is still stuck in 2005 when a t2.micro was a decent computer. The T2 EC2 instance family got announced in 2014: https://aws.amazon.com/blogs/startups/announcing-amazon-ec2-t2/ https://aws.amazon.com/blogs/startups/announcing-amazon-ec2-...
      • llm_trw 2y ago
        The first instance was the m1.small it's specs are marginally better than those of t2.micro, it's not been generally available since 2022 if memory serves. https://aws.amazon.com/blogs/aws/ec2-instance-history/ https://aws.amazon.com/blogs/aws/ec2-instance-history/ https://instances.vantage.sh/aws/ec2/t2.micro https://instances.vantage.sh/aws/ec2/t2.micro https://instances.vantage.sh/aws/ec2/m1.small https://instances.vantage.sh/aws/ec2/m1.small Please nitpick better. Nothing is more embarrassing then being pedantic and wrong.
      • rcxdude 2y ago
        And t2.micro was not a decent computer at that time.
  • apitman 2y ago
    You don't need most of that until you're making enough money to hire someone to worry about it.
• sulandor 2y ago
  is that enough to heat your house in the winter?