9 ms·

If your engineering team had to wait for a product failure to realize an item that's going to be outdoors in the sun is going to get hot, what on earth else are

If your engineering team had to wait for a product failure to realize an item that's going to be outdoors in the sun is going to get hot, what on earth else are they missing?

tyingq 5y ago
They put a non-automotive grade LCD in Teslas. I assume because they thought they knew better. I suspect hubris is at play more than full-on incompetence.
- willhinsa 5y ago
  Just curious, what are the differences between normal LCD's and automotive-grade LCD's?
  - aaronmdjones 5y ago
    Automotive-grade LCDs don't burn out and melt at the temperatures commonly found in ... well, automotive environments. Like your car sitting in a tarmac car park all day in sunny weather with the windows closed.
    - s0rce 5y ago
      Tell that to my 2012 Suzuki SX4 radio LCD, it won't work when its warm out.
      - jagger27 5y ago
        I've seen my 2017 Mazda LCD go into protection mode a few times. It also dims before that stage.
  - tyingq 5y ago
    Certified operating temperature range would be one.
  - kosievdmerwe 5y ago
    One thing that was shocking to me when I looked this up myself is the temperature range of a car dashboard where the LCD is mounted. It can get ludicrously hot in the summer if the car is parked under the sun[1], like 80-90 C (180-200 F) hot. [1] https://www.clickorlando.com/news/2019/09/26/heres-how-hot-temperatures-can-get-in-your-car/ https://www.clickorlando.com/news/2019/09/26/heres-how-hot-t...
  - deleted 5y ago
    [deleted]
- driverdan 5y ago
  SpaceX and Tesla are separate companies with different employees. "They" did not put anything in Teslas.
  - tyingq 5y ago
    Separate companies with the same CEO, (some) shared board members, joint projects, cross sales of services and equipment, loans of employees, etc. Surely there's significant culture bleed.
    - nine_k 5y ago
      I wish more of the SpaceX culture bled into Tesla then.
      - rpmisms 5y ago
        Move faster, break more things? Tesla delivers far more refined products than SpaceX
        nine_k 5y ago
        Yes, break more things on testing grounds. Keep iterating until the thing works well. SpaceX, paradoxically, has easier time experimenting and partially failing, because they mostly haul machines. Every Tesla car, OTOH, has a human pilot.
- zaroth 5y ago
  The put a non-automotive grade LCD in the original Model S because it was that LCD or nothing. The automotive screens didn’t exist in the size they needed and they weren’t at a stage in their company where they could design their own. They took a risk to achieve the product vision that they were determined to bring into the world given the technical constraints of the age. I’d say overall it paid off. IMO it wasn’t arrogance.
  - jsight 5y ago
    Yeah, and their typical failure mode, yellowing around the edges, still looked better than a lot of other vehicles from its era. I don't like their handling of it from a warranty perspective, but I'm also not sure that their initial decision was actually the wrong one.
  - da_chicken 5y ago
    I agree, it wasn't arrogance. Deliberately going forward with something in spite of knowing for certain that it's not fit for purpose is called engineering malfeasance, not arrogance.
    - eganist 5y ago
      > Deliberately going forward with something in spite of knowing for certain that it's not fit for purpose is called engineering malfeasance, not arrogance. An argument can be made for both. Malfeasance in that it was a significant and potentially catastrophic engineering tradeoff due to unknown failure modes, and arrogant in that the decision valued company vision over potential consumer safety risks as well as over the expertise of the component manufacturers.
      - MobiusHorizons 5y ago
        +1. This is something software engineers (of which I am one) often miss-understand about other engineering disciplines. You can totally ship software that you know will run into problems down the line, because you can reasonably expect to solve those challenges and ship updates before they come into play. But when you sell someone a durable good that is expected to last 10-20 years (eg a car) there are a whole host of things that go into designing that thing that are not ok to fix later. The result is that these engineering disciplines have practices that are much more rigorous. If you think about it, a recall is essentially a hardware patch, and they are typically highly embarrassing and costly. There is also the whole separate aspect of professional licensure which certain engineers get. Licensed engineers are expected to hold their work to certain standards regardless of the business needs of the company they work for. Now i'm not saying what Tesla did falls in this category, but it certainly looks like designing a product below the standard expected for that product. (what most people would call shoddy engineering). In software we often get away with this, because we can put in the work later on, and also because people often have insanely low expectations of how well software will work.
- tzs 5y ago
  I wonder if they are doing that for other components too, or just the LCD?
  - rtkwe 5y ago
    IIRC there were issues with the flash memory in the media unit (afaik that means the whole giant center screen) where it would fail to boot after a while because the flash chips were failing. They actually issued a recall for that one at least.
- 0xTJ 5y ago
  I don't know, so this is a genuine question, but is that a safety-critical part in that case? If it's not it's still not ideal, you want to make sure that your parts will happily survive the car's environment, but it's not as bad.
  - shakna 5y ago
    > Safety concerns revolve around cars losing their display for rear-facing cameras (now required by law) and consumers complaining it’s impossible to defog windshields with the center console inoperable. [0] > Another complaint said the failure disabled safety monitors associated with Tesla’s driver-assistance system Autopilot. [1] It looks like losing access to the touchscreen does actually effect safety components, and driver's abilities to use them. (Though I'm not clear on to what extent - driver's might just need to crack out the manual to see how else to defog, for example.) [0] https://www.thetruthaboutcars.com/2020/06/tesla-investigated-over-touchscreen-failure-complaints/ https://www.thetruthaboutcars.com/2020/06/tesla-investigated... [1] https://www.reuters.com/article/us-tesla-safety/u-s-probes-touchscreen-failures-in-tesla-model-s-cars-idUSKBN23V1VG https://www.reuters.com/article/us-tesla-safety/u-s-probes-t...
  - brewdad 5y ago
    Safety critical? Who knows, but probably depending on how you want to define "safety critical." Tesla runs practically every user control through the touch screen. Climate control, lighting, wipers. Did they design with reasonable fail-safes? I wouldn't want to bet my life on it.
jrockway 5y ago
I doubt that everyone on the team missed the fact that electronics get hot outdoors in the summer. Rather, they probably chose to launch a product that breaks when it gets hot, because it's an easy problem to fix later. Now that something is out in the field, they can debug their network, satellites, RF hardware, installation instructions, management applications, and figure out what kind of customers want to buy this thing -- all in parallel while some tiny EE team redoes the BOM to use better-spec'd components. Everyone at the company now has a ton of work to do to produce the next version, taking into account real-world feedback as they do it. Would it be worth blocking all those projects another 3 months just so that some guy in Arizona gets more Internet uptime on the hottest day of the year? Probably not. It sucks to invest in an early product and have it break on you, but it's also probably not possible to have a company that releases a totally flawless product on day 1. If you demand perfection, what you are really demanding is for the product to never exist. (It's also completely possible that the parts simply aren't available. You may have noticed that there's been a pandemic going on for the last year or so, and that's hurt the availability of many electronic components, among other things.)
- kortex 5y ago
  This is exactly the whole "agile" vs traditional/waterfall mentality, scooters before motorcars, etc., put into practice. Hardware is hard. > It's also completely possible that the parts simply aren't available Fun fact: they don't make (commercially available) GPUs, NICs, pretty much any PC components rated below 0°F. If you need to design a system which needs to sit in an aircraft in Deadhorse, Alaska, you either pay out the nose for some serious engineering, or you just roll the dice. We rolled, and it turned out fine. That won't always be the case. What Spacex is doing is smart, not sloppy.
  - MichaelApproved 5y ago
    > We rolled, and it turned out fine. Who is “we”?
    - kortex 5y ago
alisonkisk 5y ago
There is an interesting article about this. https://arstechnica.com/information-technology/2021/06/starlink-dish-overheats-in-arizona-sun-knocking-user-offline-for-7-hours/ https://arstechnica.com/information-technology/2021/06/starl... > Officially, SpaceX has said that "Dishy McFlatface" is certified to operate from 22° below zero up to 104° Fahrenheit. Temperatures reached about 120° yesterday in Martin's town of Topock, > the Starlink app provided an error message saying, "Offline: Thermal shutdown." The dish "overheated" and "Starlink will reconnect after cooling down," the error message said.
- shaklee3 5y ago
  This is a problem when Arizona was above 100 degrees for 144 days out of the year last year.
washadjeffmad 5y ago
I mean, the units didn't fail randomly, they shutdown. They're beta, so it could be that the thermistor was placed poorly in a redesign, that they intended 122°C instead of °F, that they shipped with a dummy value used in an older fw during testing, etc. All in all, a little embarrassing but not that significant, assuming this is certifiably not a scam.
- froh 5y ago
  I dont get the °C vs F one? with 122C youre covered for 122F. 122C = 251.6F and 122F = 50C
  - nine_k 5y ago
    The device detects overheating and shuts down orderly. If they planned to shut it down at 122C (which is indeed close to melting the solder) but by mistake made it shut down at 122F (not that much for most electronics), it would be a completely understandable blooper. It doesn't mean that the hardware isn't capable of operating at 122F.
    - nitrogen 5y ago
      122F is exactly 50C, so it's probably just a nice round number that was chosen for the shutdown temperature.
    - deleted 5y ago
      [deleted]
- rtkwe 5y ago
  The stated temperature range tops out at 104F so I doubt it's a units issue, it's just not designed to stand up to temperatures a good portion of the US reaches over the summer.
Meph504 5y ago
Your missing optimization engineers who come behind the original engineers and optimize design with a focus on production cost over performance. This often leads to some of the most spectacular failures, and unforseen issues. Or someone fucked up
only_as_i_fall 5y ago
According to the article the dish literally reported to the user that it had overheated, and then it worked once it had cooled down. Maybe not the greatest behavior, but clearly they already accounted for this.
matheweis 5y ago
Do aircraft engineers get a pass? > the Bombardier CRJ aircraft used on some shorter routes have a maximum operating temperature of 118 degrees. https://www.npr.org/sections/thetwo-way/2017/06/20/533662790/its-too-hot-for-some-planes-to-fly-in-phoenix https://www.npr.org/sections/thetwo-way/2017/06/20/533662790...
- FartyMcFarter 5y ago
  Is that plane temperature, or air temperature in the shade? Probably the latter? In the case of Starlink's dish, it's not clear which temperature they mean.
- t0mas88 5y ago
  You may think this is a smart example, but you're mostly illustrating that you know nothing about aviation engineering. The temperature limit for takeoff performance has nothing to do with the maximum temperature for electronics or other components. It's about air density decreasing which reduces the engine thrust as well as the maximum lift from the wings. So the warmer it gets the lower the maximum weight you can take off with and the higher the speed required before lift-off. At some point that's not safe anymore so the manufacturers only publish verified performance data up to a maximum temperature. You can't legally fly above that.
  - matheweis 5y ago
    I think this particular issue might not be related to the safe flight envelope, although that was the hot take at the time... I should have added more context. About a year or so after the article I linked above was published, Bombardier obtained ISA+40C certification for the CRJ700 and CRJ900 from both the FAA and TC. For the flights in and out of Phoenix, AZ, this meant that those aircraft have since been able to legally fly in temps of up to 123.8F. I don’t know the details of the certification upgrade, but my understanding is that there were no changes to either the aircraft or weight restrictions. I don’t personally know anyone at Bombardier to know exactly why they didn’t certify the aircraft for ISA+40C to begin with, but given that there were apparently no changes to either the aircraft or the weight restrictions, this seemed very much a case of “your engineering team had to wait ... to realize an item that's going to be outdoors in the sun is going to get hot”.
robertlagrant 5y ago
They didn't wait. That's why it shuts itself down.

If your engineering team had to wait for a product failure to realize an item that's going to be outdoors in the sun is going to get hot, what on earth else are they missing?

tyingq 5y ago

They put a non-automotive grade LCD in Teslas. I assume because they thought they knew better. I suspect hubris is at play more than full-on incompetence.

willhinsa 5y ago
Just curious, what are the differences between normal LCD's and automotive-grade LCD's?
- aaronmdjones 5y ago
  Automotive-grade LCDs don't burn out and melt at the temperatures commonly found in ... well, automotive environments. Like your car sitting in a tarmac car park all day in sunny weather with the windows closed.
  - s0rce 5y ago
    Tell that to my 2012 Suzuki SX4 radio LCD, it won't work when its warm out.
    - jagger27 5y ago
      I've seen my 2017 Mazda LCD go into protection mode a few times. It also dims before that stage.
- tyingq 5y ago
  Certified operating temperature range would be one.
- kosievdmerwe 5y ago
  One thing that was shocking to me when I looked this up myself is the temperature range of a car dashboard where the LCD is mounted. It can get ludicrously hot in the summer if the car is parked under the sun[1], like 80-90 C (180-200 F) hot. [1] https://www.clickorlando.com/news/2019/09/26/heres-how-hot-temperatures-can-get-in-your-car/ https://www.clickorlando.com/news/2019/09/26/heres-how-hot-t...
- deleted 5y ago
  [deleted]
driverdan 5y ago
SpaceX and Tesla are separate companies with different employees. "They" did not put anything in Teslas.
- tyingq 5y ago
  Separate companies with the same CEO, (some) shared board members, joint projects, cross sales of services and equipment, loans of employees, etc. Surely there's significant culture bleed.
  - nine_k 5y ago
    I wish more of the SpaceX culture bled into Tesla then.
    - rpmisms 5y ago
      Move faster, break more things? Tesla delivers far more refined products than SpaceX
      - nine_k 5y ago
        Yes, break more things on testing grounds. Keep iterating until the thing works well. SpaceX, paradoxically, has easier time experimenting and partially failing, because they mostly haul machines. Every Tesla car, OTOH, has a human pilot.
zaroth 5y ago
The put a non-automotive grade LCD in the original Model S because it was that LCD or nothing. The automotive screens didn’t exist in the size they needed and they weren’t at a stage in their company where they could design their own. They took a risk to achieve the product vision that they were determined to bring into the world given the technical constraints of the age. I’d say overall it paid off. IMO it wasn’t arrogance.
- jsight 5y ago
  Yeah, and their typical failure mode, yellowing around the edges, still looked better than a lot of other vehicles from its era. I don't like their handling of it from a warranty perspective, but I'm also not sure that their initial decision was actually the wrong one.
- da_chicken 5y ago
  I agree, it wasn't arrogance. Deliberately going forward with something in spite of knowing for certain that it's not fit for purpose is called engineering malfeasance, not arrogance.
  - eganist 5y ago
    > Deliberately going forward with something in spite of knowing for certain that it's not fit for purpose is called engineering malfeasance, not arrogance. An argument can be made for both. Malfeasance in that it was a significant and potentially catastrophic engineering tradeoff due to unknown failure modes, and arrogant in that the decision valued company vision over potential consumer safety risks as well as over the expertise of the component manufacturers.
    - MobiusHorizons 5y ago
      +1. This is something software engineers (of which I am one) often miss-understand about other engineering disciplines. You can totally ship software that you know will run into problems down the line, because you can reasonably expect to solve those challenges and ship updates before they come into play. But when you sell someone a durable good that is expected to last 10-20 years (eg a car) there are a whole host of things that go into designing that thing that are not ok to fix later. The result is that these engineering disciplines have practices that are much more rigorous. If you think about it, a recall is essentially a hardware patch, and they are typically highly embarrassing and costly. There is also the whole separate aspect of professional licensure which certain engineers get. Licensed engineers are expected to hold their work to certain standards regardless of the business needs of the company they work for. Now i'm not saying what Tesla did falls in this category, but it certainly looks like designing a product below the standard expected for that product. (what most people would call shoddy engineering). In software we often get away with this, because we can put in the work later on, and also because people often have insanely low expectations of how well software will work.
tzs 5y ago
I wonder if they are doing that for other components too, or just the LCD?
- rtkwe 5y ago
  IIRC there were issues with the flash memory in the media unit (afaik that means the whole giant center screen) where it would fail to boot after a while because the flash chips were failing. They actually issued a recall for that one at least.
0xTJ 5y ago
I don't know, so this is a genuine question, but is that a safety-critical part in that case? If it's not it's still not ideal, you want to make sure that your parts will happily survive the car's environment, but it's not as bad.
- shakna 5y ago
  > Safety concerns revolve around cars losing their display for rear-facing cameras (now required by law) and consumers complaining it’s impossible to defog windshields with the center console inoperable. [0] > Another complaint said the failure disabled safety monitors associated with Tesla’s driver-assistance system Autopilot. [1] It looks like losing access to the touchscreen does actually effect safety components, and driver's abilities to use them. (Though I'm not clear on to what extent - driver's might just need to crack out the manual to see how else to defog, for example.) [0] https://www.thetruthaboutcars.com/2020/06/tesla-investigated-over-touchscreen-failure-complaints/ https://www.thetruthaboutcars.com/2020/06/tesla-investigated... [1] https://www.reuters.com/article/us-tesla-safety/u-s-probes-touchscreen-failures-in-tesla-model-s-cars-idUSKBN23V1VG https://www.reuters.com/article/us-tesla-safety/u-s-probes-t...
- brewdad 5y ago
  Safety critical? Who knows, but probably depending on how you want to define "safety critical." Tesla runs practically every user control through the touch screen. Climate control, lighting, wipers. Did they design with reasonable fail-safes? I wouldn't want to bet my life on it.

jrockway 5y ago

I doubt that everyone on the team missed the fact that electronics get hot outdoors in the summer. Rather, they probably chose to launch a product that breaks when it gets hot, because it's an easy problem to fix later. Now that something is out in the field, they can debug their network, satellites, RF hardware, installation instructions, management applications, and figure out what kind of customers want to buy this thing -- all in parallel while some tiny EE team redoes the BOM to use better-spec'd components. Everyone at the company now has a ton of work to do to produce the next version, taking into account real-world feedback as they do it. Would it be worth blocking all those projects another 3 months just so that some guy in Arizona gets more Internet uptime on the hottest day of the year? Probably not. It sucks to invest in an early product and have it break on you, but it's also probably not possible to have a company that releases a totally flawless product on day 1. If you demand perfection, what you are really demanding is for the product to never exist. (It's also completely possible that the parts simply aren't available. You may have noticed that there's been a pandemic going on for the last year or so, and that's hurt the availability of many electronic components, among other things.)

kortex 5y ago
This is exactly the whole "agile" vs traditional/waterfall mentality, scooters before motorcars, etc., put into practice. Hardware is hard. > It's also completely possible that the parts simply aren't available Fun fact: they don't make (commercially available) GPUs, NICs, pretty much any PC components rated below 0°F. If you need to design a system which needs to sit in an aircraft in Deadhorse, Alaska, you either pay out the nose for some serious engineering, or you just roll the dice. We rolled, and it turned out fine. That won't always be the case. What Spacex is doing is smart, not sloppy.
- MichaelApproved 5y ago
  > We rolled, and it turned out fine. Who is “we”?
  - kortex 5y ago

alisonkisk 5y ago

There is an interesting article about this. https://arstechnica.com/information-technology/2021/06/starlink-dish-overheats-in-arizona-sun-knocking-user-offline-for-7-hours/ https://arstechnica.com/information-technology/2021/06/starl... > Officially, SpaceX has said that "Dishy McFlatface" is certified to operate from 22° below zero up to 104° Fahrenheit. Temperatures reached about 120° yesterday in Martin's town of Topock, > the Starlink app provided an error message saying, "Offline: Thermal shutdown." The dish "overheated" and "Starlink will reconnect after cooling down," the error message said.

shaklee3 5y ago
This is a problem when Arizona was above 100 degrees for 144 days out of the year last year.

washadjeffmad 5y ago

I mean, the units didn't fail randomly, they shutdown. They're beta, so it could be that the thermistor was placed poorly in a redesign, that they intended 122°C instead of °F, that they shipped with a dummy value used in an older fw during testing, etc. All in all, a little embarrassing but not that significant, assuming this is certifiably not a scam.

froh 5y ago
I dont get the °C vs F one? with 122C youre covered for 122F. 122C = 251.6F and 122F = 50C
- nine_k 5y ago
  The device detects overheating and shuts down orderly. If they planned to shut it down at 122C (which is indeed close to melting the solder) but by mistake made it shut down at 122F (not that much for most electronics), it would be a completely understandable blooper. It doesn't mean that the hardware isn't capable of operating at 122F.
  - nitrogen 5y ago
    122F is exactly 50C, so it's probably just a nice round number that was chosen for the shutdown temperature.
  - deleted 5y ago
    [deleted]
rtkwe 5y ago
The stated temperature range tops out at 104F so I doubt it's a units issue, it's just not designed to stand up to temperatures a good portion of the US reaches over the summer.

Meph504 5y ago

Your missing optimization engineers who come behind the original engineers and optimize design with a focus on production cost over performance. This often leads to some of the most spectacular failures, and unforseen issues. Or someone fucked up

only_as_i_fall 5y ago

According to the article the dish literally reported to the user that it had overheated, and then it worked once it had cooled down. Maybe not the greatest behavior, but clearly they already accounted for this.

matheweis 5y ago

Do aircraft engineers get a pass? > the Bombardier CRJ aircraft used on some shorter routes have a maximum operating temperature of 118 degrees. https://www.npr.org/sections/thetwo-way/2017/06/20/533662790/its-too-hot-for-some-planes-to-fly-in-phoenix https://www.npr.org/sections/thetwo-way/2017/06/20/533662790...

FartyMcFarter 5y ago
Is that plane temperature, or air temperature in the shade? Probably the latter? In the case of Starlink's dish, it's not clear which temperature they mean.
t0mas88 5y ago
You may think this is a smart example, but you're mostly illustrating that you know nothing about aviation engineering. The temperature limit for takeoff performance has nothing to do with the maximum temperature for electronics or other components. It's about air density decreasing which reduces the engine thrust as well as the maximum lift from the wings. So the warmer it gets the lower the maximum weight you can take off with and the higher the speed required before lift-off. At some point that's not safe anymore so the manufacturers only publish verified performance data up to a maximum temperature. You can't legally fly above that.
- matheweis 5y ago
  I think this particular issue might not be related to the safe flight envelope, although that was the hot take at the time... I should have added more context. About a year or so after the article I linked above was published, Bombardier obtained ISA+40C certification for the CRJ700 and CRJ900 from both the FAA and TC. For the flights in and out of Phoenix, AZ, this meant that those aircraft have since been able to legally fly in temps of up to 123.8F. I don’t know the details of the certification upgrade, but my understanding is that there were no changes to either the aircraft or weight restrictions. I don’t personally know anyone at Bombardier to know exactly why they didn’t certify the aircraft for ISA+40C to begin with, but given that there were apparently no changes to either the aircraft or the weight restrictions, this seemed very much a case of “your engineering team had to wait ... to realize an item that's going to be outdoors in the sun is going to get hot”.

robertlagrant 5y ago

They didn't wait. That's why it shuts itself down.