5 ms·

ECUs have been notoriously badly programmed for years, they're usually contracted out rather than built in-house. Across wide swaths of Fortune 500 companies,

by ihsw 11y ago

ECUs have been notoriously badly programmed for years, they're usually contracted out rather than built in-house. Across wide swaths of Fortune 500 companies, they abhor the idea of having tech people on staff (for reasons I will not go into here) and just now it's starting to bite them in the ass hard. Proper discipline and established standards are, well, not established, so we get companies serving up product experiences to hundreds of millions of individuals where they are held together by duct tape and super glue (with varying degrees of quality but generally leaning towards overengineered+badly implemented). That said, there are some standards in place for these devices, but regulation is lax and punitive action rarely taken (it's an honor system). Compensatory and reactive costs are going to continue to rise as the steady flow of bad code borne from incompetence continues to accrue at exponential rates as more and more companies need tech built but aren't willing to expend the resources for it to be built properly. Thankfully, as mentioned in the article, some auto companies are starting to embrace the idea of maintaining a healthy staff of developers in-house. Tesla, notably, has over-the-air updates pushed to vehicles directly rather than requiring owners to bring their cars to the dealer.

pjc50 11y ago
abhor the idea of having tech people on staff (for reasons I will not go into here) Ah, go on. I'm guessing it's partly a Two Cultures problem (not wanting to give tech people status) and partly a cost problem?
- nmrm2 11y ago
  That doesn't make sense to me because car companies hire huge numbers of automotive engineers (I think?) I always thought this was an engineering culture war thing -- software engineers aren't really engineers and software is sort of this silly thing that you use to glue together "real" engineering products. Hopefully the author of your parent post will elaborate some :-) (edit: I assumed you meant business-engineering culture war, not engineering-software. Might've been mistaken upon rereading)
  - structural 11y ago
    The culture war is alive and well, many companies (mine included, unfortunately) treat software engineers as barely qualified typists of the "algorithms" designed by "real engineers". They'd get rid all of them in a second if they could, every so often a new manager tries.
    - at-fates-hands 11y ago
      The culture war has been going on forever. When I was growing up in the 80's, my father was a software engineer at Honeywell and said they discriminated against the software guys, even when they knew software was the future of computing. He said it had been going on since the 60's and 70's as related to him by previous employees. He was told if he was a software engineer to just keep his head down and not talk to the "real engineers" working on "real problems". Likewise when I was in college for computer science, he reminded me of the same struggles he had when I used to get fed shit from the other engineering students about how "If all else failed, they could become a CS major." and just be nice because one day they'll be coming to me for help. And here we are today. Same thing, same arguments, same outdated viewpoints.
  - rwmj 11y ago
    Well, unfortunately many software engineers don't behave like real engineers, so is it any surprise? I mean to say, most people here will be highly familiar with version control, but back in the days when I used to audit companies, version control was barely[1] used, and editing files on the live server was the norm. And even here we're still having discussions about how "hard" it is to manage dependencies, use strong typing, use safe languages, garbage collection, bounds checking, formal proofs, etc. [1] I think I may be correct in saying never used. I can't recall a case where we saw a company using any kind of VCS, but perhaps I just forgot.
    - ScottBurson 11y ago
      Your criticism is valid from our perspective, but the non-software engineers on the other side of the culture divide are even less aware of proper software practice, so I don't see how the failure of software people to follow those practices could be the source of the prejudice. I think it's a different problem. Software engineering is fundamentally about the management of complexity. People who have only worked on small programs -- like most non-software engineers and scientists -- don't understand the need to manage complexity because they've never worked on any program large enough to require it. Programming in the small is easy, particularly when you're mostly doing numerics. Other engineers see that and think that all programming must be easy.
aleh 11y ago
over-the-air updates for safety-critical components of a car is a risky practice at best.
- titomc 11y ago
  Having worked in exactly the same domain , I can very well say that OTA is risky, moreover the data providers for the car is not really happy about OTA ( Sprint , AT& T provides connectivity for the projects I worked on.). They lease out a certain bandwidth to the connected cars which sends small bursts of data at a low contracted price. If the car manufacturers need OTA, they will have to pay a premium which will be slapped on the customer. There is also a technicality of downloading a huge image ( possibly 1 GB ) over a spotty network. ( a running car will switch cell networks , network types 3G , LTE , 1x , lost connections etc ).
  - kaftoy 11y ago
    Why only Tesla has OTA? Some people really think nobody in the industry ever thought of that since the internet was born? Musk is doing a nice publicity stunt. Tesla sold, what?.... 30000 cars all time? All models above $50000? So addressing people living in places where OTA is not an issue. How are all the other car makers going to handle a production of a few milion units PER YEAR, being updated OTA? How are they going to update cars sold in countries where Internet is a dream? If you change your release strategy to fit an OTA update model, you better be able to support it. This is not a problem for the volume Tesla is selling, and neither for it's audience.
titomc 11y ago
Yes you are absolutely right. The connected car project which I worked for had ECUs outsourced. The car manufacturer said "hey we are a car company". You deal with the technicalities of ECUs giving you proper data. The ECUs were giving me frames with bits turned on at the wrong places. We used to call the telematics hardware/software provider which gives us an abstracted API to talk to ECUs quoting the problem. Their reply was "thats the ECUs messing up, we will "fix it" by giving you the bit turned on at the correct position or you change your code, because changing the API needs an OTA and OTA is not supported for that car manufacturer.
- ethbro 11y ago
  Which I think is symptomatic of the bigger issue. Car companies look at anything that goes into a car as a physical part, and they've gotten extraordinarily efficient (through ruthless competition) at process for dealing with physical parts. Except... anything that contains code has orders of magnitude more test dimensions than a typical physical part. I would probably stop driving altogether if I knew exact figures on how many auto manufacturers even have in-house test coverage of their outsourced ECU / microcontrollers.
kaftoy 11y ago
What makes you believe in-house SW is more reliable? Anecdotal or not, I know people who worked for both a X's (German car maker, not VW) supplier and X themselves in SW engineering and he told me the supplier was on a totally another level above the in-house development process. Suppliers also have tradition in car sub-systems. They also provide complete sub-systems (hardware and software). Much easier to implement the SW when you also build the full HW (injectors, pump, rail, sensors, ECU, etc.). You should see some of the development labs. Tens/hundreds of millions of EURO invested in equipment over time. The only guys I know, who are still keeping a lot in-house, are the Japanese car makers (though they work with some locals in Japan, they can't be that independent). That market is opening a lot more to EU suppliers in next few years for sure. In general car makers just lack the technological know-how to do a lot of stuff in-house. If they give up suppliers tomorrow, gonna take many years just to catch-up.

by ihsw 11y ago

pjc50 11y ago

abhor the idea of having tech people on staff (for reasons I will not go into here) Ah, go on. I'm guessing it's partly a Two Cultures problem (not wanting to give tech people status) and partly a cost problem?

nmrm2 11y ago
That doesn't make sense to me because car companies hire huge numbers of automotive engineers (I think?) I always thought this was an engineering culture war thing -- software engineers aren't really engineers and software is sort of this silly thing that you use to glue together "real" engineering products. Hopefully the author of your parent post will elaborate some :-) (edit: I assumed you meant business-engineering culture war, not engineering-software. Might've been mistaken upon rereading)
- structural 11y ago
  The culture war is alive and well, many companies (mine included, unfortunately) treat software engineers as barely qualified typists of the "algorithms" designed by "real engineers". They'd get rid all of them in a second if they could, every so often a new manager tries.
  - at-fates-hands 11y ago
    The culture war has been going on forever. When I was growing up in the 80's, my father was a software engineer at Honeywell and said they discriminated against the software guys, even when they knew software was the future of computing. He said it had been going on since the 60's and 70's as related to him by previous employees. He was told if he was a software engineer to just keep his head down and not talk to the "real engineers" working on "real problems". Likewise when I was in college for computer science, he reminded me of the same struggles he had when I used to get fed shit from the other engineering students about how "If all else failed, they could become a CS major." and just be nice because one day they'll be coming to me for help. And here we are today. Same thing, same arguments, same outdated viewpoints.
- rwmj 11y ago
  Well, unfortunately many software engineers don't behave like real engineers, so is it any surprise? I mean to say, most people here will be highly familiar with version control, but back in the days when I used to audit companies, version control was barely[1] used, and editing files on the live server was the norm. And even here we're still having discussions about how "hard" it is to manage dependencies, use strong typing, use safe languages, garbage collection, bounds checking, formal proofs, etc. [1] I think I may be correct in saying never used. I can't recall a case where we saw a company using any kind of VCS, but perhaps I just forgot.
  - ScottBurson 11y ago
    Your criticism is valid from our perspective, but the non-software engineers on the other side of the culture divide are even less aware of proper software practice, so I don't see how the failure of software people to follow those practices could be the source of the prejudice. I think it's a different problem. Software engineering is fundamentally about the management of complexity. People who have only worked on small programs -- like most non-software engineers and scientists -- don't understand the need to manage complexity because they've never worked on any program large enough to require it. Programming in the small is easy, particularly when you're mostly doing numerics. Other engineers see that and think that all programming must be easy.

aleh 11y ago

over-the-air updates for safety-critical components of a car is a risky practice at best.

titomc 11y ago
Having worked in exactly the same domain , I can very well say that OTA is risky, moreover the data providers for the car is not really happy about OTA ( Sprint , AT& T provides connectivity for the projects I worked on.). They lease out a certain bandwidth to the connected cars which sends small bursts of data at a low contracted price. If the car manufacturers need OTA, they will have to pay a premium which will be slapped on the customer. There is also a technicality of downloading a huge image ( possibly 1 GB ) over a spotty network. ( a running car will switch cell networks , network types 3G , LTE , 1x , lost connections etc ).
- kaftoy 11y ago
  Why only Tesla has OTA? Some people really think nobody in the industry ever thought of that since the internet was born? Musk is doing a nice publicity stunt. Tesla sold, what?.... 30000 cars all time? All models above $50000? So addressing people living in places where OTA is not an issue. How are all the other car makers going to handle a production of a few milion units PER YEAR, being updated OTA? How are they going to update cars sold in countries where Internet is a dream? If you change your release strategy to fit an OTA update model, you better be able to support it. This is not a problem for the volume Tesla is selling, and neither for it's audience.

titomc 11y ago

Yes you are absolutely right. The connected car project which I worked for had ECUs outsourced. The car manufacturer said "hey we are a car company". You deal with the technicalities of ECUs giving you proper data. The ECUs were giving me frames with bits turned on at the wrong places. We used to call the telematics hardware/software provider which gives us an abstracted API to talk to ECUs quoting the problem. Their reply was "thats the ECUs messing up, we will "fix it" by giving you the bit turned on at the correct position or you change your code, because changing the API needs an OTA and OTA is not supported for that car manufacturer.

ethbro 11y ago
Which I think is symptomatic of the bigger issue. Car companies look at anything that goes into a car as a physical part, and they've gotten extraordinarily efficient (through ruthless competition) at process for dealing with physical parts. Except... anything that contains code has orders of magnitude more test dimensions than a typical physical part. I would probably stop driving altogether if I knew exact figures on how many auto manufacturers even have in-house test coverage of their outsourced ECU / microcontrollers.

kaftoy 11y ago

What makes you believe in-house SW is more reliable? Anecdotal or not, I know people who worked for both a X's (German car maker, not VW) supplier and X themselves in SW engineering and he told me the supplier was on a totally another level above the in-house development process. Suppliers also have tradition in car sub-systems. They also provide complete sub-systems (hardware and software). Much easier to implement the SW when you also build the full HW (injectors, pump, rail, sensors, ECU, etc.). You should see some of the development labs. Tens/hundreds of millions of EURO invested in equipment over time. The only guys I know, who are still keeping a lot in-house, are the Japanese car makers (though they work with some locals in Japan, they can't be that independent). That market is opening a lot more to EU suppliers in next few years for sure. In general car makers just lack the technological know-how to do a lot of stuff in-house. If they give up suppliers tomorrow, gonna take many years just to catch-up.