10 ms·
> because it was written by under-experienced software "engineers." I used to be an infotainment software engineer for an Auto maker. There’s this idea in the
by sircastor 4y ago
> because it was written by under-experienced software "engineers."
I used to be an infotainment software engineer for an Auto maker. There’s this idea in the software industry that because the software behaves poorly on your car’s system, it must’ve been written poorly by someone who didn’t know what they were doing. In my experience it was almost always a matter of the hardware being stretched way past its limits. Your brand new 2024 car’s system has been in development for 4 years. Product people have been picking at the design and asking for changes way too late, and someone in accounting has asked the hardware contractor to cut more cost off the BOM.
My colleagues were tremendously talented. Some of the brightest engineers I’ve had the pleasure of knowing. The developers aren’t lazy or inexperienced, they’re just being asked to turn lead into gold and often get silver.
- andrewmcwatters 4y agoThat still just sounds like people not knowing what they're doing. In no world would automakers say, "Hey you know, we just did all of this tire engineering with this rubber, but let's change the composition at the last minute and not retest all of its physical properties." So, why should it be any different with software? Because some sucker PM didn't have the brains to say, "We can't actually do that."?
- teeray 4y agoIt feels like there’s some derision towards the infotainment software engineering since it’s perceived as not safety critical. It’s in the name, “infotainment”! Fun and optional! It’s not like you’re working on the PFD/MFD/EICAS on an Airbus. “So what, the seat heater button lags sometimes. Not gonna kill someone like the tires. Ship it” says some MBA higher-up. But then the seats kick on automatically when the car starts and the driver tucks in with her morning coffee for a commute down the highway. The coffee kicks in, the cabin heats up, and suddenly that warm seat is like sitting on a griddle. She taps on the menu to bring up the seat controls, then lag, then has to hit the seat heat button twice to go from “high” to “low” to “off”, lagging each time. Oh, she hit it three times, now it’s on high again. Tap, tap, tap, tap… Meanwhile she has crossed three lanes of traffic and is staring down a jersey barrier at 70+ mph.
- switchbak 4y agoI know you're getting a lot of pushback here, but you're totally right. Qt (among others) has been a thing in the embedded space for decades, and we've had snappy UIs on sub gigahertz CPUs for decades. I just don't buy that's it's a matter of under specced hardware. I'd actually buy that it's more a matter of the corporate sausage making machine that turns ok hardware and talented developers with good intentions into an output of barely usable poorly performing crap. Ie: exactly how we end up with something like Spotify.
- hikawaii 4y agoThe corporate sausage making machine is merely only doing what it's customers want, for the price they'll pay, nothing more, nothing less. A lot of these are good impulses but also feel like an unfunded mandate, like rear back up cameras being mandatory - obligating everyone pay a thousand dollars more every year. The feel good comes from the DOT, the cost gets paid by the customer.
- pySSK 4y ago> I'd actually buy that it's more a matter of the corporate sausage making machine This. The devs don't have a say in the tech stack. The decsion is usually outsourced to the lowest bidder. Other than Tesla and Rivian, I can't think of any OEMs that I've dealt with who have full control over control over their display. Almost everyone outsources it to the Koreans who have a lock on that market – i.e. OEMs send in their designs but it's actually implemented by LG, Samsung etc. From what I've seen, any changes or new additions to any on-screen components has a 2 year lead-time unless (not a typo) it's safety critical or catastrophic in some other way.
- pySSK 4y agoIt's not that that the sucker program manager doesn't have the brains, or at least less so than PMs in other fields. It's that the PMs in automotive are not empowered to say that. There are 7-8 layers of management above them, all of whom are being shat on by the layer above them, and there's a greater disincentive to causing delays to a vastly interconnected program vs. staying in their lane and polishing the turd they've been assigned the best they can. Another big problem is a very high employee turnover and there is no redundancy in roles and skillsets because of severe cost cutting through the ages. Tech is chosen by senior engineers but vision, knowledge and experience are lost and implementation suffers every time anyone moves companies. My company has been working on an automotive project for the last 3.5 years. I deal with around 10 people on at least a weekly basis. Out of those 10, only one was around at the start of the project. We're also working with another Tier 1, where only 2 out the 12 have been around for 3 years.
- berkle4455 4y agoDude Winamp was a clean, simple and wickedly responsive UI that allowed a multitude of real-time audio tweaks and supported playlists with thousands of files. This was in the late 90’s running on 166Mhz processors with maybe 16MB of system ram. Are you saying modern car infotainment systems have fewer resources allocated to the UI?
- randomdata 4y agoAre you saying that you want to control a car’s infotainment system with a keyboard and mouse while staring at a CRT monitor?
- uncletaco 4y agoYes, unironically. Give me tactile buttons and a near zero latency screen.
- philsnow 4y agoYou’ve just reminded me that I lived in a time when my desktop machine was not powerful enough to decode and play MP3s in real time. I had to decode to wav files and play those. This was in a time when far fewer instructions were accelerated on the cpu, and my cpu at the time was something like a 66MHz 486-dx2, and even the (Frauenhofer) mp3 codec wasn’t as optimized as it was later.
- berkle4455 4y agoGotta use that turbo button bro
- Dalewyn 4y ago>Are you saying modern car infotainment systems have fewer resources allocated to the UI? Relatively speaking: Yes. Response lag is always because the hardware is chugging to process all the bits in time (they're not in time). Between underprovisioned hardware and shittily written software, hell yeah fewer resources are allocated to the UI. We need a small supercomputer's worth of resources allocated to bring the software of today up to snap like it was the 90s.
- ambicapter 4y agoThere's a 0% chance 4 yr old hardware can't handle what GP is asking it to do. > Product people have been picking at the design and asking for changes way too late Sounds like classic everything-bolted-on-never-refactored-always-add-features you see in the vast majority of companies. Which is fine if you're making a social app or CRM but not okay when talking about heavy machinery. No company ever prioritizes making things snappy and it's ultimately always on the engineers to push back. Sorry, this one is still on the software developers to me.
- kaba0 4y ago4 yr old what hardware? There are many many categories ranging from microchips barely using any electricity.
- randerson 4y agoEven if the infotainment has been in development for 15 years, all new cars I've played with seem less responsive than a 1st-generation iPhone despite having a UI that is no more capable. Either the engineers aren't making the most of the hardware or the hardware is underpowered by a few decades.
- arlcode 4y agoI can imagine that customers are annoyed by laggy UIs but don't weigh that very highly when deciding to buy a new car. Especially if the competition is roughly equally slow, the costs to engineer a responsive solution might be to high for its impact on sales. Consumers, being used to feature rich phones, may indeed value feature count over quality (for the entertainment System). Despite what OEMs claim a car is first and foremost a means of transportation and a "software Plattform" only after that.
- donalhunt 4y agoI think it's a growing factor and the linked article would indicates that manufacturers are receiving feedback about it. It's not clear whether that feedback is from sales channels (i.e. potential customers are shunning particular products) or from service channels (customers are reporting issues that require additional development spend resulting in profits being eroded). Personally, I've been looking for a new car for 3+ years and the quality of controls (which are nearly all touchscreen based these days) is a major factor. This is also a reflection that trust in car manufacturers to address software related issues is low - they are used to designing and building cars that are churned out in 3+ year cycles. Moving to a scenario where the product is changing monthly is a huge change. My 13 year old current car is extremely solid and the controls don't change month to month - something that does happen without any customer choice in more recent times. :/
- wruza 4y agocustomers are annoyed by laggy UIs but don't weigh that very highly when deciding to buy a new car UI is not something you can evaluate on a 30 min test drive under sales pressure. Sometimes there’s no way to test it (no rain, no snow, no free straight road, no immediate glitches). And there’s no reviews that cover it. It’s not a weighted decision, it’s a decision out of only parameters that are most obvious. The same way you play a lottery with e.g. a washing machine, whether it beeps after unlocking the door or you have to go and sit near it like an idiot for a whole minute.
- amluto 4y agoI'm willing to credit this type of feature creep for some of the problem, but only some. Decades ago, we had hardware that was so much less capable than anything anyone has put in a car in ten years that the comparison is absurd. Nonetheless, those old systems were often very capable of doing useful things, and the UIs were often very, very fast by modern standards. And a lot of this was that the software stack was tuned for performance and latency and did not do things that hurt latency too much, and this came at a substantial modularity cost. Back when a TI calculator with a whole OS running on a Z80 was an order of magnitude or so faster than your average new car, it didn't have a compositor (which still eats a good frame or so of latency, maybe more, and, as an industry we still don't know how to solve this). It ran an OS that was surely far harder to tweak than a modern car stack, but it had fewer layers and could be kept performant. An Apple ][ and its clones were, of course, much faster than a TI calculator, and we still can't match that level of performance. A group of engineers that were willing to keep the managers away and make a simple, limited-purpose system, could surely get excellent performance out of any currently available car computer hardware, but it would come at a cost. Buzzwords would not happen :) Also, the software developers had access to full data sheets for the hardware, and when you told the hardware to put something in its framebuffer, it was there, and no blobs or firmware were in the middle. I still remember when Windows 2000 launched with "layered windows": an application could opt in to being composited, and it would gain the ability to be translucent, but it came at a very obvious performance cost. Sadly, 23 years later, that performance cost is still there, but it's not opt-in any more.
- klodolph 4y agoThe kind of development you’re describing happens in aerospace. I know some people who work in aerospace. They’re not any smarter than the people who work in automotive, but the people in aerospace have a much higher tolerance for dealing with bureaucracy, and management is much happier to get something less impressive but reliable.
- DanielHB 4y agoautomative main problem is the constant push to cut costs which leads to worse SoC being shipped. Aerospace profit margins are much higher (or don't matter for gov programs) so they don't need to cut costs that much on computer hardware > management is much happier to get something less impressive but reliable. ^ this is accurate though. Management wants bells and whistles (animations) which are hard to build without abstraction layers
- prng2021 4y agoIt's great to get your perspective on this. Aside from the GPS/navigation features, aren't car infotainment systems extremely basic? We've been controlling heating, cooling, audio source, and audio volume since the dawn of time. How are technical requirements related to these basic functions changing such that their UIs haven't been absolutely perfected by now? Is the entire infotainment code being rewritten all the time so there's no chance to iterate and improve on the previous versions work?
- the_overseer 4y agoYes. The infotainment code is rewritten every time. By different devs. Because that is how automotive industry works. This time BMW wants infotainment and a third party wins the contract (because it's the cheapest usually). For the next BMW car another company wins the contract. Everything is rebuilt from scratch. The components that make up the infotainment a lot of times are actually developed by different companies. So navigation is one company, media player another etc. Also, the deadlines usually make no sense and there is zero time for any optimization work. It's not the devs, it's the way the automotive industry works outsourcing every little button on the dash to another (cheapest) company. They just don't understand that that mentality doesn't work with software. Most of them are mechanical engineers not software people.
- AlotOfReading 4y agoI'm currently going through a quote cycle and one of my requirements for the component is a sub 8min boot, which the tier 1/2s are pushing back on as unreasonable. The insanity is real.
- carlmr 4y ago8 minutes? What component takes 8 minutes to start?
- AlotOfReading 4y agoThat can include booting after an update and self tests, but yes, nothing about the component should take that anywhere near that long. We just needed a number to design the manufacturing workflow and assumed it was a reasonable ceiling. They simply don't want to do the (potential) optimization work as part of their initial contract if they somehow manage to exceed it.
- jes5199 4y agothe thing is, even the brightest of us are still terrible. There’s no such thing as a good software developer
- bluSCALE4 4y agoI buy this. Did you all forget the GTA5 loading delay caused by a JSON added after the game was developed in game purchase items.
- ISO-morphism 4y agoI believe that the engineers are competent at developing software, and think that a lot of the quality issues are due to engineers abdicating authority over the software to managers and never saying "No, adding in that feature at this point in the timeline is going to make this a steaming pile of shit and I refuse to do it and still call myself a software engineer." I get that we have families to feed, but I've seen far too much of a mindset shift in fellow engineers into thinking that we're warcraft peons rather than professionals. "The business" has engineering feedback as a necessary input, and speaking individually with steakholders they expect this - they'll push until we push back.
- Transfinity 4y agoThere's no incentive for an engineer to do that. Saying yes and delivering crap gets you a bonus, hard truths get you shuffled around or made redundant. There's no real consequence for delivering crap, so that's what happens. Contrast this with other engineering fields, where the engineer is truly responsible for the decisions they make. My civil engineer friends face losing their licenses, fines or jail time if they are found professionally negligent. The same is true of other high stakes professions - think doctors, lawyers, even accountants. It's probably not appropriate for most software engineering roles, but for safety critical systems it doesn't seem far-fetched to me.
- alkonaut 4y agoThe incentive is having a rewarding job where you develop products you are proud of. Once I have food to eat, this is by far the most important incentive for me and it greatly outweighs e.g my desire for promotions, raises and bonuses. If I can have both, great. If I need to choose one, it’s the fulfilling job and product pride every day.
- cto_of_antifa 4y ago[dead]
- geraldwhen 4y ago
- deleted 4y ago[deleted]
- IIAOPSW 4y ago> they’re just being asked to turn lead into gold and often get silver. TBF thats literally leaded gasoline
- chinathrow 4y ago> In my experience it was almost always a matter of the hardware being stretched way past its limits. Then someone else didn't know what they were doing.
- 4gotunameagain 4y agoA computer running at 2.048 MHz and with 2048 words RAM took men to the moon. I don't buy it that a 4 year old SoC cannot run GPS navigation, play music, do whatever a car infotainment system does, without half a second delay
- alkonaut 4y agoUnless the hardware can and will be upped from a $20 to a $200 SoC because it’s required to keep response times low, then someone isn’t doing their job. There has obviously been mountains of specs during these projects, but the most important one - the response time - was obviously overlooked. I’m not saying like the GP that this indicates inexperienced developers, but it does indicate poor technical leadership and priorities. Further, good/senior developers would have been more likely to call this out.
- londons_explore 4y agoA higher powered SoC might help make it lag-free... But the real solution is simply to put in the requirements that 90% of taps must be fully responded to within 100ms. Animations must complete within 350ms and be 60 fps. Dragging must have a glass to glass latency of under 50 ms. Booting from door open to usable must be done in 5000 ms. While we're there, lets put in the requirement that a random joe from the street, with no training, when given the UI and asked to complete a task (eg. Play me some coldplay, turn on the window defroster, pair a bluetooth phone, get directions to the nearest gas station), must be able to figure it out within 20 seconds and 6 taps on average.
- hsbauauvhabzb 4y agoA talented software developer should know how to use the phrase ‘no’ Effectively.
- the_overseer 4y agoAnd a talented manager should know when to say "Ok. I will find somebody else to do it then. For cheaper". You are easily replaceable.
- wiseowise 4y ago> And a talented manager should know when to say "Ok. I will find somebody else to do it then. For cheaper". You are easily replaceable. Our hiring rate disagrees. That might be true for run-of-the-mill sweatshops, but not for any tier 2+ companies.
- hsbauauvhabzb 4y agoThe kind of manager I’d happily walk from. Either way, I won’t be doing pointless work that inevitably leads to failure - said manager can enjoy their low quality development team and failed outcomes. When the managers bosses see the manager failed a project after firing all the people who said it would fail, it won’t look good for them.
- the_overseer 4y agoIt's not pointless work. It's actually very profitable work. Low quality or not.
- hsbauauvhabzb 4y agoI’ll think of you every time I hate on my car computer system, I guess.
- madmax108 4y agoThe solution is quite evident is it not? Regulatory bodies to set response latency and limits with over-life degradation as an additional parameter. Only reason "they’re just being asked to turn lead into gold and often get silver." happens is because automakers know they can get past all regulatory approval with lead/silver or whatever metal you prefer.
- szundi 4y agoWindows 95 runs pretty well on 10 year old hw and does much more than these car screens do. So no, software is fundamentally crap there.
- anthk 4y agoWindows 95 runs on 25 years old hardware now. I can run a full XFCE based GNU/Linux DE with ease on a 10 yo laptop.
- jonathankoren 4y agoI wouldn’t necessarily the engineers, but I’ll certainly blame the designers and product managers. I absolutely hate my Acura’s infotainment system. My “favorite” feature isGPS autocomplete. You type in part of an address (which of course has a very slow response time), then you pick the completion off the list. Sounds fine right? But when you click it, you’re met with a SLOW ANIMATION OF THE COMPLETION BEING TYPED INTO THE SEARCH BOX. Motherfuck dude! What the hell is that? Why does anyone want to watch the keyboard flash for five seconds?
- Gordonjcp 4y ago"He fired up the car’s onboard GPS device and began coping with its user interface." REAMDE, Neal Stephenson
- ryan-allen 4y agoMy top of the line Android Auto compatable head unit is pretty ordinary. A RAM mount on the windscreen holding my phone is significantly more responsive!
- tasuki 4y ago> In my experience it was almost always a matter of the hardware being stretched way past its limits. What aspects of the hardware would lead to 500ms response times?
- zamnos 4y agoUnderpowered CPU being asked to do way more work than it properly has capacity for.
- fps_doug 4y agoPlus 400 layers of abstraction because the lowest bidder doesn't know any actual software engineering and every time they face a problem they add yet another library to the project that someone on SO mentions because the accompanying sample code in the answer has the least amount of lines.
- vultour 4y agoThis absolutely boggles my mind. I had to do some javascript recently, and every time I had what I thought was a very basic question the most upvoted answer was "just npm install this library, and use this one-liner". I honestly never experienced this before with any other language, it's horrifying.
- tasuki 4y agoI bet the "underpowered CPU" is way faster than the supercomputers of not that long ago. Way more work, as in, turn up the volume or display another thing on the screen? How do you explain that not that long ago, when CPUs were slower, interfaces were faster?
- DanielHB 4y agomy experience working with industrial manufacturing is that there is ALWAYS a push to cut costs and the SoC that runs your software is usually the first thing to get to the chopping block. Given it is extremely hard to gauge how well software will perform before it is actually built and the SoC is chosen way ahead of time it leads to bad performance And from the other side, there is a big push to add more and more fluff to make the UI design sleek, you could make it performant if you used DOS-style interfaces but no product manager is going to allow that and making animations from scratch (vs using a platform like Android or browsers that has a ton of abstractions to create fancy UIs) is extremely costly
- joshgachnang 4y agoMeanwhile some car company is putting a Ryzen APU capable of playing Steam games in the infotainment system.
- closeparen 4y agoCar infotainment as responsive as a 2004 iPod would be earth-shattering. I refuse to believe that in 2019 the hardware to do this was a material expense on a $25,000 car.
- Vvector 4y ago> In my experience it was almost always a matter of the hardware being stretched way past its limits. I find this hard to believe. An Intel 486 from 30+ years ago could run Doom full screen with nearly instant key press response. A 6502/6510 C64 could repaint the entire text screen each frame. Why would today's auto hardware require 500ms to respond to touches on a static menu?
- P_I_Staker 4y agoYeah this is nonsense. If not for hardware considerations, probably some bureaucratic nonesense. Disputes between teams, or suppliers*, something fell through the cracks because it was nobody's problem; stuff like that. Probably not because they lack 10x l33t coding skills, and have fat fingers. * I'm not sure if people realize that many automotive dev teams are buying enormous stacks of software from outside parties that often have motivation to throw you under the bus... if it's not an external supplier, it's probably another team on a different continent... but maybe it's different in infotainment?
- ricardobeat 4y agoWe had responsive hardware/software 20 years ago, and car UIs are not doing anything out of the ordinary. To what do you attribute the low quality then?
- throwaway292939 4y agoWhat is the compensation range for software engineers in the auto industry? Is there a place to look at job descriptions?