10 ms·

Unconditioned Texas Garage Lab – 1 year later

52 points by monstermunch 6y ago

grailed 6y ago
What's the normal lifespan of this equipment? I'd imagine this environment, particularly with the dust, would cause failure in the 5-10 year range rather than the 1-2 year range.
- kitotik 6y ago
  My initial instinct was that the extreme temps of summer/winter would be the bigger lifespan shortener, but I suppose dust would start killing fans and clogging air intakes compounding the temperature problems.
  - monstermunch 6y ago
    The thing is though, there are no temperature problems All equipment is within the operating specs
  - phil21 6y ago
    This is pretty much it. Constant temps are more important than lower temps. Within reason of course. Modern equipment is pretty tolerant to high intake temps, due to lots of very poor datacenter designs. It's pretty common to see servers in older co-location facilities at the top of racks seeing 95F intake temps if the on-site staff don't really care about rack blanks and the like. These can go unnoticed for years and simply chug away with their fans at high speed.
apearson 6y ago
This is the default last mile environment for at least a couple ISPs that I've worked with. I've seen switches and other networking equipment in non air conditioned closets in the Orlando heat. The equipment doesn't seem to mind the higher temp that much.
- dkdk8283 6y ago
  Corrosion is a concern in the higher humidity states. Cell towers need the same consideration.
  - ip26 6y ago
    However, come to think of it, if it's 100F outside and 140F inside the closet, doesn't that mean the Rh is considerably lower?
kbenson 6y ago
If you aren't using all the ports on the switch, you aren't really exercising the switch much. That a 48 port witch with ~10% of the ports in use doesn't overheat and isn't even spinning its fans at max when in a garage in Texas in the summer isn't really surprising. It isn't surprising in the same way that a modern desktop or notebook system in the same environment not overheating in if all you de is power it on and let it idle is is not surprising. Start using a significant portion of the switching backplane capacity and see how well it does then though. That would be a much more interesting result, whether the device succeeds or fails.
- shiftpgdn 6y ago
  I ran a bitcoin mining rig in my garage in Houston, Texas (6x Radeon cards across two workstations) for two years in 2010-2012. No issues, no loss of hardware. Ambient temperature went from 40F to 140F.
  - tshoaib 6y ago
    Wow those were the early years of mining, must have been quite profitable if you kept the yield for a couple of years till the price skyrocketed.
    - shiftpgdn 6y ago
      I sold coins along the way to cover initial investment and then exited the majority of my holding at $500-1500. I sold my final bitcoin in 2017 when prices were sky high. If I had sold everything at peak I would have been extremely wealthy. Maybe next time. :)
  - kbenson 6y ago
    That is interesting. Do you know if the Radeon cards throttle automatically when they get too hot? Did you have any custom cooling? I can't imagine they would perform all that great at 140F if they throttles themselves, but them doing so may have kept them functional and error free at the expense of hiding poor performance, unless you have data showing otherwise (and you totally might, it seems like the person setting up that system might also check to ensure it's performing to spec :) )
mrkstu 6y ago
I live in similar conditions, not quite as humid. Tempted to move my mini rack out to the garage now. Very tempting to save on the air conditioning necessitated by the equipment in my rack. May even put my virtualization server out there for awhile and see what the temps get up to.
latchkey 6y ago
I oversee a largish datacenter (>2MW) with only air cooling (a lot of it). Ambient temps range from -23c to +43c. Equipment gets dusty. We have some fan and psu failures, but as long as the air keeps moving, everything keeps working just fine. You'd be surprised at how well plain old airflow works.
dkdk8283 6y ago
One year isn’t enough to test reduced MTBF through increased ambient temperatures. Dust and the consequence of insulating electronic components can take years to surface problems.
- monstermunch 6y ago
  Sadly I can't give a longer timeframe, as I've only had this 1 year But, everyone told me it wouldn't last a year
abetusk 6y ago
I'm no expert at this whatsoever but from what I've gleaned looking around, a good heuristic is that removing hot air is way better than pushing cold air in. As a kind of hand-waivy argument, I think of the difference between averaging cool air with how hot it gets ("pushing" air in) vs. removing and replacing with ambient temperature air. In the case of 'pushing' air in, the average of something like 40deg F with 300deg F is 170deg F. Even if ambient is 70deg F, by replacing air with ambient, that's a 2x better cooling, not to mention the energy saving from just using a fan compared with cooling with an air conditioner or some other equipment.
- latchkey 6y ago
  I manage a large datacenter of air cooled fanless (radiator) GPUs housed in cases where the case fans are on the hot side and pulling the air out. Hot air expands, which fills the space (more pressure) and thus prevents cooler air from getting in (you have to push harder to get it in). The faster you can get rid of hot air from the enclosure, the better.
h2odragon 6y ago
Others point out these arent terrible conditions, all things considered. I'd like to see reports of this quality for stuff in wood shops, chicken barns, cement plants, etc. I've seen a few examples of things like that and they were entertaining in a "wow it kept on going anyway" sense.
- Meph504 6y ago
  I can tell you that in a steel mill, its a whole different story, largely because of the ferrous dust, the heat as you can imagine is also a problem, everything has to be sealed and fed through a filtered vents, or it will die
m0zg 6y ago
Lots of surprised people in the comments. Folks, those parts inside your computer are rated for anywhere from 85C to 200C depending on the part. What this means is that those parts can work at this temperature indefinitely. Perhaps the only part whose lifetime depends on the temperature a lot are electrolytic capacitors, but even they are routinely rated at 105C. Your CPU is likely rated to 105C die temperature, and even then it will only throttle, not die outright. Moreover, letting energy dense parts warm up to a decent temperature improves the efficiency of cooling, because thermal gradient is much steeper. That's why Apple doesn't cool things until they have to. DIYers are obsessed with getting to the point where their CPU runs at ridiculously low temps, but the CPU itself is designed to tolerate much more. GPUs begin to throttle at something like 83C, and NVIDIA deliberately lets them to get to that temperature before fans kick in in earnest.
- musingsole 6y ago
  Electronics tend to be far more robust than people think. As long as it's not literally in an oven and there's nothing conductive getting involved (as in the steel mill example pointed out in another comment) most electronics will chug along just fine. Recognizing this is what led to me abandoning a case for my desktop PC. It's now just slapped on a piece of plywood on my wall because it literally doesn't matter. This setup even fell (during wall mounting attempts) ~8 feet to hardwood floors...twice! And still going just fine running Half-Life: Alyx :)
InvaderFizz 6y ago
This is unsurprising to me, but that is because I did a few years in Afghanistan servicing edge compute and communications stacks. About half of the hundred or so different sites I was responsible for had zero air conditioning. Dust was a major issue, every unit rotation (~18months) equipment would go to our depot for a full refurbishment. That generally consisted of a new set of 30m cables/ends for the satellite portion, and a complete strip down and cleaning of everything else. For field service(weight matters when you have to hike a few miles with gear), I adapted a stupid high velocity 120v leaf blower down to a 1 inch nozzle. The vast, vast majority of damage was due to operator error or neglect. It's always fun to arrive on site for a malfunctioning terminal, notice the damage, and ask what happened. Then the answer is things like, "so we have this thing called the hundred days of wind, and the terminal on the roof wasn't anchored or using ballast..."
belval 6y ago
One "fun" anecdote that I like to tell everyone. I was given a 1U server by a company I was working at. Some old ~2010 machine with 32GB of RAM and 2 CPUs. I anticipated noise but not quite that much, so I decided to move it to the exterior shed that had one outlet. This was in July in Canada so peeks of 33-34 celsius are expected but not much more than that. The shed was in the sun and would get pretty hot for humans, but I thought it would be fun to try it and worst case scenario it wouldn't last long. Turns out we had some intense heat for the entire week and the server was dead about 8 days after putting it in the shed. I opened it to see what had happened and several capacitors had popped. TLDR: Intense heat isn't good for 1U servers, but I bet you knew that already.

52 points by monstermunch 6y ago

What's the normal lifespan of this equipment? I'd imagine this environment, particularly with the dust, would cause failure in the 5-10 year range rather than the 1-2 year range.

kitotik 6y ago
My initial instinct was that the extreme temps of summer/winter would be the bigger lifespan shortener, but I suppose dust would start killing fans and clogging air intakes compounding the temperature problems.
- monstermunch 6y ago
  The thing is though, there are no temperature problems All equipment is within the operating specs
- phil21 6y ago
  This is pretty much it. Constant temps are more important than lower temps. Within reason of course. Modern equipment is pretty tolerant to high intake temps, due to lots of very poor datacenter designs. It's pretty common to see servers in older co-location facilities at the top of racks seeing 95F intake temps if the on-site staff don't really care about rack blanks and the like. These can go unnoticed for years and simply chug away with their fans at high speed.

apearson 6y ago

This is the default last mile environment for at least a couple ISPs that I've worked with. I've seen switches and other networking equipment in non air conditioned closets in the Orlando heat. The equipment doesn't seem to mind the higher temp that much.

dkdk8283 6y ago
Corrosion is a concern in the higher humidity states. Cell towers need the same consideration.
- ip26 6y ago
  However, come to think of it, if it's 100F outside and 140F inside the closet, doesn't that mean the Rh is considerably lower?

kbenson 6y ago

If you aren't using all the ports on the switch, you aren't really exercising the switch much. That a 48 port witch with ~10% of the ports in use doesn't overheat and isn't even spinning its fans at max when in a garage in Texas in the summer isn't really surprising. It isn't surprising in the same way that a modern desktop or notebook system in the same environment not overheating in if all you de is power it on and let it idle is is not surprising. Start using a significant portion of the switching backplane capacity and see how well it does then though. That would be a much more interesting result, whether the device succeeds or fails.

shiftpgdn 6y ago
I ran a bitcoin mining rig in my garage in Houston, Texas (6x Radeon cards across two workstations) for two years in 2010-2012. No issues, no loss of hardware. Ambient temperature went from 40F to 140F.
- tshoaib 6y ago
  Wow those were the early years of mining, must have been quite profitable if you kept the yield for a couple of years till the price skyrocketed.
  - shiftpgdn 6y ago
    I sold coins along the way to cover initial investment and then exited the majority of my holding at $500-1500. I sold my final bitcoin in 2017 when prices were sky high. If I had sold everything at peak I would have been extremely wealthy. Maybe next time. :)
- kbenson 6y ago
  That is interesting. Do you know if the Radeon cards throttle automatically when they get too hot? Did you have any custom cooling? I can't imagine they would perform all that great at 140F if they throttles themselves, but them doing so may have kept them functional and error free at the expense of hiding poor performance, unless you have data showing otherwise (and you totally might, it seems like the person setting up that system might also check to ensure it's performing to spec :) )

mrkstu 6y ago

I live in similar conditions, not quite as humid. Tempted to move my mini rack out to the garage now. Very tempting to save on the air conditioning necessitated by the equipment in my rack. May even put my virtualization server out there for awhile and see what the temps get up to.

latchkey 6y ago

I oversee a largish datacenter (>2MW) with only air cooling (a lot of it). Ambient temps range from -23c to +43c. Equipment gets dusty. We have some fan and psu failures, but as long as the air keeps moving, everything keeps working just fine. You'd be surprised at how well plain old airflow works.

dkdk8283 6y ago

One year isn’t enough to test reduced MTBF through increased ambient temperatures. Dust and the consequence of insulating electronic components can take years to surface problems.

monstermunch 6y ago
Sadly I can't give a longer timeframe, as I've only had this 1 year But, everyone told me it wouldn't last a year

abetusk 6y ago

I'm no expert at this whatsoever but from what I've gleaned looking around, a good heuristic is that removing hot air is way better than pushing cold air in. As a kind of hand-waivy argument, I think of the difference between averaging cool air with how hot it gets ("pushing" air in) vs. removing and replacing with ambient temperature air. In the case of 'pushing' air in, the average of something like 40deg F with 300deg F is 170deg F. Even if ambient is 70deg F, by replacing air with ambient, that's a 2x better cooling, not to mention the energy saving from just using a fan compared with cooling with an air conditioner or some other equipment.

latchkey 6y ago
I manage a large datacenter of air cooled fanless (radiator) GPUs housed in cases where the case fans are on the hot side and pulling the air out. Hot air expands, which fills the space (more pressure) and thus prevents cooler air from getting in (you have to push harder to get it in). The faster you can get rid of hot air from the enclosure, the better.

h2odragon 6y ago

Others point out these arent terrible conditions, all things considered. I'd like to see reports of this quality for stuff in wood shops, chicken barns, cement plants, etc. I've seen a few examples of things like that and they were entertaining in a "wow it kept on going anyway" sense.

Meph504 6y ago
I can tell you that in a steel mill, its a whole different story, largely because of the ferrous dust, the heat as you can imagine is also a problem, everything has to be sealed and fed through a filtered vents, or it will die

m0zg 6y ago

Lots of surprised people in the comments. Folks, those parts inside your computer are rated for anywhere from 85C to 200C depending on the part. What this means is that those parts can work at this temperature indefinitely. Perhaps the only part whose lifetime depends on the temperature a lot are electrolytic capacitors, but even they are routinely rated at 105C. Your CPU is likely rated to 105C die temperature, and even then it will only throttle, not die outright. Moreover, letting energy dense parts warm up to a decent temperature improves the efficiency of cooling, because thermal gradient is much steeper. That's why Apple doesn't cool things until they have to. DIYers are obsessed with getting to the point where their CPU runs at ridiculously low temps, but the CPU itself is designed to tolerate much more. GPUs begin to throttle at something like 83C, and NVIDIA deliberately lets them to get to that temperature before fans kick in in earnest.

musingsole 6y ago
Electronics tend to be far more robust than people think. As long as it's not literally in an oven and there's nothing conductive getting involved (as in the steel mill example pointed out in another comment) most electronics will chug along just fine. Recognizing this is what led to me abandoning a case for my desktop PC. It's now just slapped on a piece of plywood on my wall because it literally doesn't matter. This setup even fell (during wall mounting attempts) ~8 feet to hardwood floors...twice! And still going just fine running Half-Life: Alyx :)

InvaderFizz 6y ago

This is unsurprising to me, but that is because I did a few years in Afghanistan servicing edge compute and communications stacks. About half of the hundred or so different sites I was responsible for had zero air conditioning. Dust was a major issue, every unit rotation (~18months) equipment would go to our depot for a full refurbishment. That generally consisted of a new set of 30m cables/ends for the satellite portion, and a complete strip down and cleaning of everything else. For field service(weight matters when you have to hike a few miles with gear), I adapted a stupid high velocity 120v leaf blower down to a 1 inch nozzle. The vast, vast majority of damage was due to operator error or neglect. It's always fun to arrive on site for a malfunctioning terminal, notice the damage, and ask what happened. Then the answer is things like, "so we have this thing called the hundred days of wind, and the terminal on the roof wasn't anchored or using ballast..."

belval 6y ago

One "fun" anecdote that I like to tell everyone. I was given a 1U server by a company I was working at. Some old ~2010 machine with 32GB of RAM and 2 CPUs. I anticipated noise but not quite that much, so I decided to move it to the exterior shed that had one outlet. This was in July in Canada so peeks of 33-34 celsius are expected but not much more than that. The shed was in the sun and would get pretty hot for humans, but I thought it would be fun to try it and worst case scenario it wouldn't last long. Turns out we had some intense heat for the entire week and the server was dead about 8 days after putting it in the shed. I opened it to see what had happened and several capacitors had popped. TLDR: Intense heat isn't good for 1U servers, but I bet you knew that already.