5 ms·
It sounds like these issues are relatively fixable. It's a classic victim of the Unity engine's tech debt though. I use Unity myself and they desperately need t
by wilg 3y ago
It sounds like these issues are relatively fixable. It's a classic victim of the Unity engine's tech debt though. I use Unity myself and they desperately need to decide on how they want people to make video games in their engine. They can't have three rendering pipelines and two ways of adding game logic that have a complicated matrix of interactions and missing features. And not great documentation and a bad bug reporting process.
- Someone1234 3y agoIt makes one wonder what their internal employee incentives are and if they're problematic. Microsoft has a similar problem where nobody gets promoted from fixing bugs or maintaining stuff, everyone gets rewarded for new innovative [thing] so every two-three years there's a completely new UI framework or similar. Although I feel like wanting to start-a-new is a common tech problem, where there are problems and everyone wants to just reboot to "fix" it rather than fixing it head-on inc. backwards compatibility headaches.
- cipheredStones 3y ago> Microsoft has a similar problem where nobody gets promoted from fixing bugs or maintaining stuff, everyone gets rewarded for new innovative [thing] so every two-three years there's a completely new UI framework or similar. Is there any big (or even medium-sized) company where this isn't true? I feel like it's just a rule of corporate culture that flashy overpromising projects get you promoted and regularly doing important but mundane and hard-to-measure things gets you PIP'd.
- ajmurmann 3y agoIs it only big companies? The fact that many companies in our industry need to do "bug squash" events because we are unable to prioritize bugs properly speaks books to meet.
- Jochim 3y agoTop down decision making, typically by non-technical people who often have no idea what software development even involves. Eventually things get so bad that there's no choice but to abandon feature work to fix them. The business loses out multiple times. Feature work slows down as developers are forced to waste time finding workarounds for debt and bugs. The improvements/fixes take more time than they would have due to layers of crap being piled on top, and the event that forces a clean up generally has financial or reputational consequence. Collaborative decision making is the only way around this. Most engineers understand that improvements must be balanced with feature work. I find it very strange that the industry operates in the way it does. Where the people with the most knowledge of the requirements and repercussions are so often stripped of any decision making power.
- ghaff 3y agoThis is pretty much a universal thing--whether it's software development or home maintenance. It's really tempting to kick the can down the road to the point where 1.) You HAVE to do something; 2.) It's not your problem any longer; or 3.) Something happens that the can doesn't matter any more. I won't say procrastination is a virtue. But sometimes the deferred task really does cease to matter.
- 3seashells 3y agoAt least we wouldn't do that on a planetary scale, right?
- throw3823423 3y agoIt's a matter of letting things degrade so that the maintenance becomes outright firefighting. I am currently working on a project where a processing pipeline has a maximum practical throughput of 1x, and a median day's for said pipeline is... 0.95x. So any outage becomes unrecoverable. Getting that project approved 6 month from now would have been basically impossible. Right now, it's valued at a promotion-level difficulty instead. At another job, at a financial firm I got a big bonus after I went live on November 28th with an upgrade that let a system 10x their max throughput, and scaled linearly instead of being completely stuck. at their 1x. Median number of requests per second received in dec 1st? 1.8x... the system would have failed under load, causing significant losses to the company. Prevention is underrated, but firefighting heroics are so well regarded that sometimes it might even be worthwhile to be the arsonist
- piaste 3y agoIntuitively, "fixing life-or-death disasterss is more visible and gets better rewards than preventing them" doesn't seem like it should be a unique problem of software engineering. Any engineering or technical discipline, executed as part of a large company, ought to have the potential for this particular dysfunction. So I wonder: do the same dynamics appear in any non-software companies? If not, why not? If yes, have they already found a way to solve them?
- dmoy 3y ago> If yes, have they already found a way to solve them? A long history of blood, lawsuits, and regulations. Preventing a building from collapsing is done ahead of time, because buildings have previously collapsed, and cost a lot of lives / money etc.
- harimau777 3y agoI remember my very first day of studying engineering, the professor said: "Do you know the difference between an engineer and a doctor? When a doctor messes up, people die. When an engineer messes up LOTS of people die."
- dymk 3y agoFacebook was pretty good about this on the infra teams. No, not perfect, but a lot better than the other big companies I was exposed to. If anything, big companies are better about tech-debt squashing, and it's the little tiny companies and startups that are, on average, spending less time on it.
- hutzlibu 3y agoI think it is a bit tricky to get the incentives right ( since the bookkeeping people like to quantize everything). If you reward finding and fixing bugs too much - you might push developers to write more sloppy code in the first place. Because then those who loudly fix their own written mess gets promoted - and those who quietly write solid code gets overlooked.
- xctr94 3y agoGoodhart’s law at work, or “why you shouldn’t force information workers to chase after arbitrary metrics”. Basecamp has been famously just letting people do good work, on their terms, without KPIs. I will preemptively agree that this isn’t possible everywhere; but if you create a good work environment where people don’t feel like puppets executing the PM’s vision, they might actually care and want to do a solid day’s work (which we’re wired for).
- bluedino 3y agoI spent a few weeks migrating and then fixing a bunch of bugs in 20-year old Perl codebase (cyber security had their sights set on it). Basically used by a huge amount of people to record data for all kinds of processes at work. Original developer is long gone. Me and another guy are two of the only people (we aren't a tech company) who can re-learn Perl, upgrade multiple versions of Linux/Apache/MySQL, make everything else work like Kerberos etc... Or maybe I'm one of the only people dumb enough to take it on. Either way, nobody will get so much as an attaboy at the next department meeting. But, they'll know who to go to the next time some other project is resurrected from the depths of hell and needs to be brought up to date.
- brucethemoose2 3y ago> Is there any big (or even medium-sized) company where this isn't true? Valve?
- uolmir 3y agoFrom everything I've read Valve has exactly the same problem. Stack rating isn't immune. New features still get rewarded the most.
- deleted 3y ago[deleted]
- flukus 3y agoIt seems endemic, especially everywhere that's not a product company. I think it was mythical man month (maybe earlier) that pointed out the 90% of the cost of software is in maintenance, yet 50 years on this cost isn't accounted for in project planning. Consultancies are by far the worst, a project is done and everyone moves on, yet the clients still expect quick fixes and the occasional added feature but there's no one familiar with the code base. Developers don't help either, a lot move from green field to green field like locusts and never learn the lessons of maintaining something, so they make the same mistakes over and over again.
- Mistletoe 3y agohttps://www.thepeoplespace.com/practice/articles/leadership-action-patagonia-case-study https://www.thepeoplespace.com/practice/articles/leadership-... It’s very rare, this is one of the only places I can imagine something like that happening.
- bandrami 3y agoAviation. Software will often spend ten times as long in QA and testing as it will in principal development.
- capableweb 3y agoThe developers of Cities Skylines has less than 50 employees in total, it's a small developer based in Finland (Colossal Order), I doubt they have those sort of issues at that scale, that's usually something that happens with medium/large companies. Edit: seems I misunderstood, ignore me
- thrillgore 3y agoWe're weeks past a very public pricing change that cost Unity market reach amidst competitors and open source projects; and that led to a CEO change. There are problems beyond what the employees can realistically fix.
- hnthrowaway0315 3y agoIt's a combination of team not given enough time amd headcount to maintain and develop a product and another team's manager wants to grab a fief. So old products are thrown away while new products with similar functionalities are being created. Both teams are happy. The users suffer.
- johnnyanmac 3y ago> It makes one wonder what their internal employee incentives are and if they're problematic. not a very strong one until recently. They implemented the FAANG esque "levels" in late 2021. Promotion lines weren't too atypical from FAANG, but the new system was not around long enough to cause the problems people complain about today. The biggest issue IME was that teams were isolated. Maybe DOTS should have talked more with strongly integrating HDRP/URP into its workflow, but DOTS was busy getting off the ground itself. HDRP and URP are two very different teams and they were trying to solve very different problems. Perhaps that was a bad move for the people it was serving, who'd want to have the flexibility to migrate to/from HDRP and URP. There weren't really much product management that was trying to make the engine cohesive, so you end up with a rendering core and a bunch of different plugins with different philosophies and whatnot. >I feel like wanting to start-a-new is a common tech problem, where there are problems and everyone wants to just reboot to "fix" it rather than fixing it head-on inc. backwards compatibility headaches. to some extent, yes. I think the one huge downside of Unity compared to modern companies is its antiquated CI/CD. Getting changes into the core c++ engine took 10x longer than it really needed to. iterations on repo builds were slow because Unity simply didn't cough out the money for proper server farms, and the interface to interact with the status of PRs felt like it was from 2005. Much of DOTS was iterated upon separately on with modern Jira/Github/etc. pipelines and the DOTS repo was very lean (and it was public too... until it wasn't. I think they moved it in 2022?), moving to make a change to the core engine was like traveling back in time 15 years ago. Legacy code is a pain as is. And Unity definitely needed to revamp some non-feature workflows before it could really dig into the core Unity engine issues.
- tus666 3y agoIt's a classic victim of shitty, shitty software developers who blames tools rather than taking ownership. Or shitty software dev companies that push out crap to meet marketing deadlines. Either way, take your money elsewhere.
- gamblor956 3y agoGiven that a number of other Unity-based games have had the same or similar performance issues, including KSP1, the Endless games, and others, it seems the problem is very much that Cities Skylines 2 is hitting up against the performance limits that the Unity engine is capable of without custom modifications to the engine-layer codebase.
- raincole 3y agoI'll be really surprised if City Skylines's team didn't have access to Unity's source code.
- kimixa 3y agoAnd do they have the number of engineers with the required skills to rewrite half the engine? Especially if the reason why they developed using those tools and engine is they expected not to have to do it themselves in the first place? It's not like there's just some "go_slow=true" constant that just needs changing.
- bananaboy 3y agoIt's pretty unlikely. A source code license is negotiated with the sales team directly and costs at least USD100k last I heard (the price is not publicly disclosed). They're also reluctant to give source code licenses at all.
- MrLeap 3y agoI have personally been responsible for optimizing unity games you haven't heard issues like this about ;) This write-up really points the finger at not solving occlusion culling or having good LOD discipline. Give a person a dedicated optimization mandate and you can avoid most of this. One of the first things I do when I'm profiling is to sort assets by tris count and scan for excess. I wonder if they had somebody go through and strategically disable shadowcasting on things like those teeth? I am guessing that they made optimization "everybody's responsibility" but nobody had it as their only responsibility.
- dexwiz 3y agoSounds like every other enterprise software platform. Unity has reach the IBM level of “no one gets fired for choosing X,” even though X only makes the business people happy.
- moffkalast 3y agoI think it's a good thing in the long run, one more reason to switch away from Unity to add to the ever growing pile.
- Auracle 3y agoIt's honestly a bit insane. Just the other night I wanted to know what it'd take to do some AR development for the Quest 3 using Unity. 10 minutes in I was straight up confused. There's AR Foundation, AR Core, AR Kit, and I think at least one other thing. I have no idea the difference between those, if they're even wholly separate. That's on top of using either the OpenXR or Unity plugin for the actual headset.
- andybak 3y agoAR Kit is Apple's thing. AR Core is Google's thing. Neither of those are Unity's fault. AR Foundation is a Unity layer to present a common interface. Which of my books is a good thing. Open XR is also an an attempt to make a cross platform layer for vendor specific APIs. Again not Unity's fault. The Unity plugin system is a common interface for all XR devices. I'd generally support your sentiment but in this case you're picking on things where Unity had mostly got it right.
- deleted 3y ago[deleted]
- Auracle 3y agoThis comment made things more clear to me than the documentation that I saw, perhaps because I was looking at it from a Quest 3 perspective and their main page about it doesn't mention that at all. I know what Open XR is all about but again, it's not clear which you should actually use for development if you're only targeting Quest devices, for instance. A little extra documentation would go a long way. The same goes for all of the other things so frequently mentioned like their renderers.
- squeaky-clean 3y agoHonestly, like a lot of Unity functionality, what you do there is pay $50-100 for an asset from the store that handles things in a sane way.
- unihd2kno 3y ago[flagged]
- frozenfoxx 3y agoI worked at Unity on Build Automation/Cloud Build for nearly a decade. Let me assure you, that tech debt is NOT being fixed any year soon. It’s due to a fundamental disconnect between executive leadership wanting to run the company like Adobe (explicitly) and every engineer wanting to work like a large scale open source project (Kubernetes, Linux, and Apache are pretty close in style). The only way anything gets built is as a Skunkworks project and you can only do so much without funding and executive support.
- LeanderK 3y ago> run the company like Adobe (explicitly) what does this mean?
- robterrell 3y agoCan you elaborate on this?
- johnnyanmac 3y ago> I worked at Unity on Build Automation/Cloud Build my condolances. the Build iterations and ancient CI workflow was by far the biggest complaint for the teams I was on and talked to. it takes so long getting stuff properly landed into trunk that I can't really blame the teams wanting to break off if possible (it fortunately was for my division). But I'm guessing it was hard to convince product or execs that dev velocity matters, so we were all just wading in muck. I heard things were improving... but I heard that every month I was there. >executive leadership wanting to run the company like Adobe I think I know what you mean by this, but can you clarify?
- KronisLV 3y agoHonestly, automatic LOD generation would solve at least some of the performance issues: add the functionality, make it opt-out for those that don't need LODs and enjoy performance improvements in most projects, in addition to some folks getting a simpler workflow (e.g. using auto-generated models instead of having to create your own, which could at the very least have passable quality). Godot has this: https://docs.godotengine.org/en/stable/tutorials/3d/mesh_lod.html#generating-mesh-lod https://docs.godotengine.org/en/stable/tutorials/3d/mesh_lod... Unreal has this (for static meshes): https://docs.unrealengine.com/5.3/en-US/static-mesh-automatic-lod-generation-in-unreal-engine/ https://docs.unrealengine.com/5.3/en-US/static-mesh-automati... Aside from that, agreed: the multiple render pipelines, the multiple UI solutions, the multiple types of programming (ECS vs GameObject) all feel very confusing, especially since the differences between them are pretty major.
- mardifoufs 3y agoI'm pretty sure unity already has that.
- KronisLV 3y agoOut of the box, it only has manual LOD support for meshes: https://docs.unity3d.com/Manual/importing-lod-meshes.html https://docs.unity3d.com/Manual/importing-lod-meshes.html (where you create the models yourself) They played around with the idea of automatic LOD, but the repo they had hasn't gotten updated in a while: https://github.com/Unity-Technologies/AutoLOD https://github.com/Unity-Technologies/AutoLOD The closest to that would be looking at assets on the Asset Store, for example: https://assetstore.unity.com/packages/tools/utilities/poly-few-mesh-simplifier-and-auto-lod-generator-160139 https://assetstore.unity.com/packages/tools/utilities/poly-f... An exception to that is something like the terrain, which generates the model on the fly and decreases detail for further away chunks as necessary, but that's pretty much the same with the other engines (except for Godot, which doesn't have a terrain solution built in, but the terrain plugins do have that functionality). I guess in Unity's case you can still get that functionality with bought assets, which won't be an issue for most studios (provided that the assets get updated and aren't a liability in that way), but might be for someone who just wants that functionality for free.
- araes 3y agoIt sounds like they need to implement easy to use Level of Detail (LOD) and progressive meshes. 100,000 vertices on far away objects will break most rendering pipelines that do not somehow reduce them. 100,000 complicated matrix interactions instead of the like, 8, it probably takes really far away. [1] https://en.wikipedia.org/wiki/Level_of_detail_(computer_graphics) https://en.wikipedia.org/wiki/Level_of_detail_(computer_grap... [2] https://en.wikipedia.org/wiki/Progressive_meshes https://en.wikipedia.org/wiki/Progressive_meshes
- deleted 3y ago[deleted]