8 ms·

I'd be careful about painting the developers at Rockstar as "not curious," that feels like an incredibly unfair conclusion to draw from this. Those developers h

I'd be careful about painting the developers at Rockstar as "not curious," that feels like an incredibly unfair conclusion to draw from this. Those developers have other pressures, and deadlines to hit. There are a million reasons why this bug wasn't solved by the team there. When a user complains about a bug in software you wrote, is it because you just "weren't curious" enough? Or is the situation more complicated? Lots of incredibly talented people worked on GTA Online, it's worth trying to think through why a team of smart developers might not have prioritized this, rather than assuming something negative about them.

delusional 4y ago
You might say that assuming this was due to inherent incuriousity amongst the developers is quite incurious.
azemetre 4y ago
I wasn’t trying to blame engineers that wasn’t my intention, but someone needs to own establishing the environment that actively encourages or doesn’t encourage curiosity. Talented people are found everywhere, but hopefully someone with authority over them doesn’t slowly poison the drive out of them.
- blowski 4y ago
  Why should developers necessarily be trying to cut loading times? Perhaps GTA is phenomenally successful because they have good BAs and POs who are able to identify and convince engineers to work on what matters to the bottom line.
  - dingleberry420 4y ago
    You've clearly never played it nor read any of the community discussions about the game. The #1 complaint is (or was) loading times. It's why I've personally stopped playing years ago.
    - blowski 4y ago
      I play GTA, am annoyed by those loading screens, and have found those forums while looking for solutions. Companies are successful not because they delight every customer consistently, but because they do just enough to make the customer spend money with them. After all, I will still buy the next version.
  - azemetre 4y ago
    I mean that's fair, but I'm talking about wanting to be curious and encouraging environments where curious individuals can be lauded instead of dismissed. Not everyone on a team HAS to be curious but for those that aren't they shouldn't be smacked upside the head with corporate bureaucracy.
  - s0l1dsnak3123 4y ago
    Screw this. Developers should advocate for their craft and for quality, and business metrics should hold an adversarial position to balance things out. We definitely should not be accepting the narrative that developers just do what they are told and don't push back on what we know is good for product - we are in a unique position of power, we should recognise this and use it for the betterment of the industry as a whole. The alternative is we let corporate greed slide our standards of quality into the gutter thereby making the job at hand miserable.
    - sreekotay 4y ago
      Very much agree with this. That said given the quality of what they have put out, I'd surmise the issue here is morelikely they don't feel the same pain on load times? Bandwidth too good, rigs too powerful, dev envs overpowered, etc. to enable developer velocity can often also mask real world problems. Hell, even hot reload can mean you don't see load as often and are able to easily rationalize as "not that big a deal" - I wouldn't necessarily take it as a sign they are not deeply invested in their craft.
      - s0l1dsnak3123 4y ago
        I very much agree with you as well actually. One of the things that I've instituted at my work is our methodology for acquiring test phones for our product (WhatsApp for hospitals, basically). We go to a supermarket and buy the most eye-catching box of the budget phones available in the phone aisle. We've squashed so many bugs and performance problems this way. It's cheaper, it instils empathy in product decisions, and our customers love the user experience as a result. Similarly, one of my first big decisions after joining was to wean ourselves from our AWS addiction using Kubernetes. This also allowed us to design a dev env that fits on a modern machine comfortably, but can scale to entire Hospital Trusts in production. Once again: focusing on portability now means we can sell our product on-prem and hybrid, because we designed our developer UX this way.
      - foobarian 4y ago
        I don't know, this seems like it was a pure CPU problem on good hardware. But anyway a developer who has 10 hour days and backlog stretching beyond the horizon will not have idle time to go try things out. It needs to be someone's explicit job to go hunt down things like this. And given it was data-size dependent, I wouldn't be surprised this was not an issue at the start while someone who's job it was to look for this kind of issue was looking. And the issue appeared after they got retasked to something else. In fact maybe this is the only kind of issue that you end up with after all is said and done. Just like in WW2 they reinforced the parts of the fuselage on returning fighter craft without bullet holes, because if a bullet hit there, the craft would not return.
  - deleted 4y ago
    [deleted]
  - nwallin 4y ago
    Loading times were the #1 complaint from users about the game. I personally wouldn't play a game where 5 minute loading times were the norm, 10+ minute times are common, and 15+ minute times are not unheard of. It's the #1 complaint. The thing that makes this egregious isn't so much that they didn't fix the #1 complain, the thing that makes it egregious is that it's a simple enough flaw that a mediocre developer could have just run WPR/WPA against it and see that 90% of those 5+ minutes was spent in strlen. Maybe it's not their area, maybe they don't know how to fix it, but they will recognize that it's a problem with a likely easy fix. So even if we completely ignore the fact that it's the #1 complaint, it's still egregious that it's been allowed to go on for this long.
- Griffinsauce 4y ago
  You are assuming curiosity was the problem instead of a lot of other possible factors like bandwidth.
  - azemetre 4y ago
    I wasn't blaming this problem solely existing due to being incurious but more of lamenting how corporate environments don't encourage people to be curious.
    - nkellenicki 4y ago
      That may have been your intention, but your statement of "environments where people aren't curious" comes across as putting the blame on the people themselves. As has been stated elsewhere, often times engineering is lacking in bandwidth to do anything but meet the deadlines. I don't disagree that the "environment" itself could do more to further making improvements outside of the assigned work - initiatives such as 20% time, cafe days, etc. exist in other companies. But I don't put the blame on the curiosity of the people themselves.
      - azemetre 4y ago
        I mean at the end of the day we’re products of our environments. If SWEs can only work on features and do nothing else that is still a sign of poor project management. Just accepting that fate is worse for your growth I’d wager. But idk, I never worked in an environment that encouraged curious people either. I suppose the closest for me was working at Comcast but that was only one manager with a small team of 4 people and no real deadlines. I feel like two separate thoughts came from replies to my comment. One advocating for engineers to push back and care about their crafts, the other absolving the choices that were clearly made by people as some mechanized process that could never be changed. The only thing I’m wondering is what the replies would be like 10 years ago on HN.
867-5309 4y ago
it was a feature - they served ads during loading screens
asveikau 4y ago
I agree with the sentiment. However, it is a tad surprising when you consider the productivity hit, even just internally speaking, that ~5min of cpu time in strlen() on every start of the app incurs. How much time did perfectly qualified engineers, who could have attached a profiler at any moment, spend sitting at a loading screen spending most of its time in unnecessary strlen() calls? I understand we see this through 20/20 hindsight, serendipity can work against you, etc., but that's a crazy irony.
- r00fus 4y ago
  A toxic culture resistant to change can obtain these results no matter how brilliant/skilled the people.
  - asveikau 4y ago
    It's true, but also, you can get this result with a competent team and good culture, curious people, etc. You could just have bad luck and nobody happened to check it.
terafo 4y ago
Game was out for almost EIGHT YEARS by the time this bug was fixed. There can be no reasonable explanation for them not fixing that. 5 minute load every time you try to go into multiplayer. The amount of revenue lost to that one single bug is staggering. This bug was a reason why I never played GTA online, for example.
- badlucklottery 4y ago
  >The amount of revenue lost to that one single bug is staggering. The players who are still there are used to it so the value is likely near zero. It's the sort of thing that's 100% worth fixing before launch or soon after so you don't lose players in the first place. But the fix loses value every day it goes unfixed because you're already lost those load time-sensitive players. Keep in mind that years after release it still makes Rockstar/Take-Two millions per day (https://www.thegamer.com/gta-5-rockstar-2-5-million-per-day/ https://www.thegamer.com/gta-5-rockstar-2-5-million-per-day/) so I guess a lot of players just don't care. For the record: I'm surprised anyone is willing to tolerate that shit.
  - Bjartr 4y ago
    Lots of people cite long load times as why they stopped playing GTA5. They are certainly making bank, but they also certainly left money on the table because those players who left could have still been there and spending money.
  - Drew_ 4y ago
    Them making a lot of money doesn't prove that they couldn't have made more money. Everyone knows better performance means better conversion which means more $$$.
  - terafo 4y ago
    It is definitely non-zero, I think since the launch of game the number of revenue lost to this single bug is in tens, possibly hundreds of millions of dollars. Considering that game made somewhere around 10 billions USD, 1-2% of total revenue lost to loading time seems plausible, given how horrible the problem was.

delusional 4y ago

You might say that assuming this was due to inherent incuriousity amongst the developers is quite incurious.

azemetre 4y ago

I wasn’t trying to blame engineers that wasn’t my intention, but someone needs to own establishing the environment that actively encourages or doesn’t encourage curiosity. Talented people are found everywhere, but hopefully someone with authority over them doesn’t slowly poison the drive out of them.

blowski 4y ago
Why should developers necessarily be trying to cut loading times? Perhaps GTA is phenomenally successful because they have good BAs and POs who are able to identify and convince engineers to work on what matters to the bottom line.
- dingleberry420 4y ago
  You've clearly never played it nor read any of the community discussions about the game. The #1 complaint is (or was) loading times. It's why I've personally stopped playing years ago.
  - blowski 4y ago
    I play GTA, am annoyed by those loading screens, and have found those forums while looking for solutions. Companies are successful not because they delight every customer consistently, but because they do just enough to make the customer spend money with them. After all, I will still buy the next version.
- azemetre 4y ago
  I mean that's fair, but I'm talking about wanting to be curious and encouraging environments where curious individuals can be lauded instead of dismissed. Not everyone on a team HAS to be curious but for those that aren't they shouldn't be smacked upside the head with corporate bureaucracy.
- s0l1dsnak3123 4y ago
  Screw this. Developers should advocate for their craft and for quality, and business metrics should hold an adversarial position to balance things out. We definitely should not be accepting the narrative that developers just do what they are told and don't push back on what we know is good for product - we are in a unique position of power, we should recognise this and use it for the betterment of the industry as a whole. The alternative is we let corporate greed slide our standards of quality into the gutter thereby making the job at hand miserable.
  - sreekotay 4y ago
    Very much agree with this. That said given the quality of what they have put out, I'd surmise the issue here is morelikely they don't feel the same pain on load times? Bandwidth too good, rigs too powerful, dev envs overpowered, etc. to enable developer velocity can often also mask real world problems. Hell, even hot reload can mean you don't see load as often and are able to easily rationalize as "not that big a deal" - I wouldn't necessarily take it as a sign they are not deeply invested in their craft.
    - s0l1dsnak3123 4y ago
      I very much agree with you as well actually. One of the things that I've instituted at my work is our methodology for acquiring test phones for our product (WhatsApp for hospitals, basically). We go to a supermarket and buy the most eye-catching box of the budget phones available in the phone aisle. We've squashed so many bugs and performance problems this way. It's cheaper, it instils empathy in product decisions, and our customers love the user experience as a result. Similarly, one of my first big decisions after joining was to wean ourselves from our AWS addiction using Kubernetes. This also allowed us to design a dev env that fits on a modern machine comfortably, but can scale to entire Hospital Trusts in production. Once again: focusing on portability now means we can sell our product on-prem and hybrid, because we designed our developer UX this way.
    - foobarian 4y ago
      I don't know, this seems like it was a pure CPU problem on good hardware. But anyway a developer who has 10 hour days and backlog stretching beyond the horizon will not have idle time to go try things out. It needs to be someone's explicit job to go hunt down things like this. And given it was data-size dependent, I wouldn't be surprised this was not an issue at the start while someone who's job it was to look for this kind of issue was looking. And the issue appeared after they got retasked to something else. In fact maybe this is the only kind of issue that you end up with after all is said and done. Just like in WW2 they reinforced the parts of the fuselage on returning fighter craft without bullet holes, because if a bullet hit there, the craft would not return.
- deleted 4y ago
  [deleted]
- nwallin 4y ago
  Loading times were the #1 complaint from users about the game. I personally wouldn't play a game where 5 minute loading times were the norm, 10+ minute times are common, and 15+ minute times are not unheard of. It's the #1 complaint. The thing that makes this egregious isn't so much that they didn't fix the #1 complain, the thing that makes it egregious is that it's a simple enough flaw that a mediocre developer could have just run WPR/WPA against it and see that 90% of those 5+ minutes was spent in strlen. Maybe it's not their area, maybe they don't know how to fix it, but they will recognize that it's a problem with a likely easy fix. So even if we completely ignore the fact that it's the #1 complaint, it's still egregious that it's been allowed to go on for this long.
Griffinsauce 4y ago
You are assuming curiosity was the problem instead of a lot of other possible factors like bandwidth.
- azemetre 4y ago
  I wasn't blaming this problem solely existing due to being incurious but more of lamenting how corporate environments don't encourage people to be curious.
  - nkellenicki 4y ago
    That may have been your intention, but your statement of "environments where people aren't curious" comes across as putting the blame on the people themselves. As has been stated elsewhere, often times engineering is lacking in bandwidth to do anything but meet the deadlines. I don't disagree that the "environment" itself could do more to further making improvements outside of the assigned work - initiatives such as 20% time, cafe days, etc. exist in other companies. But I don't put the blame on the curiosity of the people themselves.
    - azemetre 4y ago
      I mean at the end of the day we’re products of our environments. If SWEs can only work on features and do nothing else that is still a sign of poor project management. Just accepting that fate is worse for your growth I’d wager. But idk, I never worked in an environment that encouraged curious people either. I suppose the closest for me was working at Comcast but that was only one manager with a small team of 4 people and no real deadlines. I feel like two separate thoughts came from replies to my comment. One advocating for engineers to push back and care about their crafts, the other absolving the choices that were clearly made by people as some mechanized process that could never be changed. The only thing I’m wondering is what the replies would be like 10 years ago on HN.

867-5309 4y ago

it was a feature - they served ads during loading screens

asveikau 4y ago

I agree with the sentiment. However, it is a tad surprising when you consider the productivity hit, even just internally speaking, that ~5min of cpu time in strlen() on every start of the app incurs. How much time did perfectly qualified engineers, who could have attached a profiler at any moment, spend sitting at a loading screen spending most of its time in unnecessary strlen() calls? I understand we see this through 20/20 hindsight, serendipity can work against you, etc., but that's a crazy irony.

r00fus 4y ago
A toxic culture resistant to change can obtain these results no matter how brilliant/skilled the people.
- asveikau 4y ago
  It's true, but also, you can get this result with a competent team and good culture, curious people, etc. You could just have bad luck and nobody happened to check it.

terafo 4y ago

Game was out for almost EIGHT YEARS by the time this bug was fixed. There can be no reasonable explanation for them not fixing that. 5 minute load every time you try to go into multiplayer. The amount of revenue lost to that one single bug is staggering. This bug was a reason why I never played GTA online, for example.

badlucklottery 4y ago
>The amount of revenue lost to that one single bug is staggering. The players who are still there are used to it so the value is likely near zero. It's the sort of thing that's 100% worth fixing before launch or soon after so you don't lose players in the first place. But the fix loses value every day it goes unfixed because you're already lost those load time-sensitive players. Keep in mind that years after release it still makes Rockstar/Take-Two millions per day (https://www.thegamer.com/gta-5-rockstar-2-5-million-per-day/ https://www.thegamer.com/gta-5-rockstar-2-5-million-per-day/) so I guess a lot of players just don't care. For the record: I'm surprised anyone is willing to tolerate that shit.
- Bjartr 4y ago
  Lots of people cite long load times as why they stopped playing GTA5. They are certainly making bank, but they also certainly left money on the table because those players who left could have still been there and spending money.
- Drew_ 4y ago
  Them making a lot of money doesn't prove that they couldn't have made more money. Everyone knows better performance means better conversion which means more $$$.
- terafo 4y ago
  It is definitely non-zero, I think since the launch of game the number of revenue lost to this single bug is in tens, possibly hundreds of millions of dollars. Considering that game made somewhere around 10 billions USD, 1-2% of total revenue lost to loading time seems plausible, given how horrible the problem was.