8 ms·

I'm not sure if "embarrassing" is the right word here. This bug has existed for years and has lead to hundreds, if not thousands of people quitting the game ci

I'm not sure if "embarrassing" is the right word here. This bug has existed for years and has lead to hundreds, if not thousands of people quitting the game citing this exact reason yet Rockstar did nothing. I think "shameful" is really more appropriate. It's not like they simply didn't know this existed. I think they have known all along it existed and didn't give 2 shits to fix it.

monkeypizza 6y ago
Watch out not to condemn them no matter what they do. Sure, they messed up in the past, but given the situation they found themselves in when they saw this report, this response is still good. You risk making discourse difficult when you continue punishing people who are attempting to do the right after doing wrong for a while.
- thitcanh 6y ago
  I disagree. They ignored the issue for years and that part won’t be erased from memory. They should be given credit for finally fixing it, but nothing more. What happened before this week was and remains forever shameful.
  - epivosism 6y ago
    Very likely that internally there were people who had an inkling this was going on, but could never convince product to devote time to put it in the backlog. This change is a win for that engineer. This example will be ammunition in internal debates about how much to weigh product pressure vs engineering suggestions that improvement is technically possible; the next time an eng says "let's look at this slowness one more time, it may be fixable more easily than you think", they might be listened to.
  - ramblerman 6y ago
    > What happened before this week was and remains forever shameful. lol without context you'd think something tragic actually happened here.
    - thitcanh 6y ago
      Losing customers over such a simple, known problem, over years… is tragic for a company like Rockstar. This is like Microsoft ignoring a bug that made Windows take 10 minutes to start and just letting that happen for 5 years. Now tell me that’s not tragic.
      - fomine3 6y ago
        Anyway GTA5 is just a game.
        thitcanh 6y ago
        To you it's just a game, to Rockstar GTA is literally a multi-billion dollar product with a household name that politicians talk about. I can't believe you're missing this huge point.
- sellyme 6y ago
  > You risk making discourse difficult when you continue punishing people who are attempting to do the right after doing wrong for a while. On the other hand, a few days of doing the absolute bare minimum of quality control doesn't undo a decade of actively hostile practices. They're still very very deeply in the red when it comes to their relations with the playerbase and for good reason. Paying $10k to the person who documented this bug is a good start, but you need to do a whole lot of work to make up for the impressions caused by that kind of prolonged systemic negligence.
  - monkeypizza 6y ago
    Right. I'm not saying eval(their net behavior)>0; I'm saying that if your response function is always negative, you lose the ability to influence their response. Have you heard the one about the country with the death penalty for all crime? It just leads criminals to maximally try not to get caught, without regard to damage done [i.e. removing all witnesses to even the smallest crimes]. This isn't what you want; even within "negative" relationships both sides lose when you don't have a gradient of responses.
    - sellyme 6y ago
      The crux of this is that most readers presumably found out about both the bug and the fix in relatively short proximity, and their response is negative to the combined events, not to the knowledge of just the patch. A criminal prosecuted for murder and petty theft simultaneously can't really go around complaining that they got 25 years in gaol for stealing an apple, but if everyone had already known about the murder and dealt with that accordingly it'd be pretty harsh. (not that fixing a bug is exactly a criminal act, but the general principle remains)
NoPicklez 6y ago
I think "embarrassing" could still be the correct word. If you were to ask the development teams responsible they might feel a little embarrassed over the solution being so small (yet technical). And if they didn't care at all given what you say then why would they care to implement a fix for it all these years later?
awill 6y ago
I understand your frustration, but anyone who's run (or been on) a dev team knows you have a million things you could/should be doing, and many, many priorities. Often supporting new launches/features is the priority. Sure, ideally there would be someone internal advocating for a fix like this on behalf of customers. But Shameful isn't the right word. It's a miss like any other.
- shock-value 6y ago
  The egg that's on Rockstar's face has more to do with how much money they likely left on the table by not diagnosing and fixing this issue early on. (And assuming they had a team overseeing this aspect of performance with adequate tools to observe and debug it should have been trivial.) I personally feel like the magnitude of this -- especially compared to how simple the solution is -- makes it a miss that's really much more extreme than most others. EDIT: Though I agree that "shameful" isn't the right word to use. Rockstar wronged themselves much more so than any other party.
- SV_BubbleTime 6y ago
  Obviously you are right... but... If the fix had been some super application specific super obscure thing, we would all understand. Instead... the bug is easily describable to anyone who knows what JSON is and the fix is “don’t parse JSON the slow way”. So I think some of the leniency afforded to the devs may be misplaced. Then again, hindsight and all that.
  - thitcanh 6y ago
    The cause doesn’t actually matter much, usually one doesn’t know the issue until they investigate. So often issues remain open for lack of interest, then someone finally does and it’s a missing comma. This doesn’t affect the fact that the issue has been ignored. I’m sure GTA has unlimited bugs to look into, but a bug that turns people away before they even start should have probably been given higher priority.
    - deleted 6y ago
      [deleted]
lambda_obrien 6y ago
It's a video game, so maybe give them some slack. We're not talking a SolarWinds breach here.
aantix 6y ago
The game supports 8 platforms, has virtually miles of terrain, and now Rockstar is suppose to feel ashamed because they didn't fix a big performance issue that affected startup times? The game probably has tens of millions of lines of code. I'm glad they shipped. If they can enhance the startup time, great, bonus for them. Additional credit for thanking the third party that uncovered the perf issue.
- mdoms 6y ago
  > now Rockstar is suppose to feel ashamed because they didn't fix a big performance issue that affected startup times? Yes, they should be ashamed of this.
- swagmoney69 6y ago
  It's true that the game is probably absurdly complex, but how did they never profile loading times?
- malwarebytess 6y ago
  I don't think you've been following this story. If you looked into the technical details [0], and surrounding context, you would understand why your comment isn't particularly insightful. For example, terrain has nothing to do with this bug and it isn't a complex fix that interacts with "millions of lines of code." A single run of a profiler would have caught this. I also think it's relevant to note that GTA V has sold 140 million copies, 20 million last year alone, and Rockstar makes hundreds of millions of dollars a year on the title. For a multi billion dollar product to have a known, relatively simple, problem is shameful. I don't know what went wrong, but I don't think it was the competency of engineers or the complexity of the problem that allowed this to fester. [0] https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times-by-70/ https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times...
- mlindner 6y ago
  I think you missed previous news about this. This was an incredibly trivial issue that would show up in any use of a profiler on the slow startup screens. The fact they didn't is that they're either unable to run profilers on their code at Rockstar (which would be shameful indeed) or they never bothered to do so. Either case shows egg on face at Rockstar.
867-5309 6y ago
you forgot to mention that ads are displayed during the loading screen
rectang 6y ago
An alternative possibility: there were 100 bugs of similar consequence, all of which they cared about, and they got 99 of them. This one escaped, because stuff happens. > I think "shameful" is really more appropriate. When you make an inadvertent mistake in the future, karma is coming for you: people will speculate that you "didn't give 2 shits".
- jacobr1 6y ago
  More likely, there were 99 other bugs that they had clearer metrics related to monetization that they prioritized (not that this bug didn't impact monetization - I bet it did, just not in a way they were measuring). Or that their QA engineers identified and flagged as high priority or major new features with their own bugs and regressions or whatever. These things get lost in the shuffle. More likely there were 1000s of bugs in the tracker, probably 10s of thousands.
  - rectang 6y ago
    Yes, I agree with your elaboration. Also, of the 100 bugs, they didn't get 99 of them, because there are surely some left. Maybe they got 90, and there are 9 still hiding out.
- rubicon33 6y ago
  I’m sorry but no. This bug existed for a very long time and was the #1 outcry from gamers. It is well documented all over the internet. Every company I’ve ever worked at and every code base I’ve ever worked on would have made this a #1 priority. If it were a technical limitation of the devs I would agree it is “embarrassing” but this was not outside of their ability. I simply don’t blame the devs, I blame management. Management should have allocated the resources to fix this truly game breaking bug but they didn’t because money was still flowing. Is it any coincidence that ads played during the 6+ minutes you would wait to get into a game?
  - ryandrake 6y ago
    Yea, if it was a 6 month old bug or it was a hard to repro edge case, then sure it’s embarrassing. Oops! But years long, and highly visible/reproducible, without correcting? That’s veering into shameful.
agumonkey 6y ago
If so then they still acted in a gentle way that is rare afaik. If they were so careless they'd just lift the fix and ignore the author.
pluc 6y ago
This is akin to that guy buying a whole-page ad to complain about his ISP recently, resulting in enough shame to trigger action. It's a bad look for Rockstar considering the income that title generates and at most it was a few weeks worth of dev time. Very interesting to watch the public reaction to this in comparison though, seems to be a lot more sympathy for Rockstar than for ISPs.
seppel 6y ago
There probably is a story behind it. Like some developer looked into the bug in the beginning (when when the load time was not that bad because the json file was smaller), saw that the time was spend parsing json, and then, without thinking too much whether this makes sense, wrote in the bug ticket that, to make it faster, they would have to switch from json to something better/faster/different. Since switching the file format is probably some work, maybe involving many differnt teams and network protocols, the bug was closed because it too much efford. And then, whenever somebody complained, the old bug was referred which unfortunately never got a proper analysis. Sad, but I saw this happening many times in the past in some projects. Since the old and wrong analysis will be in many powerpoint slides and heads, it is incredible hard to argue against or get even time to work on such a problem. Usually, you need someone who just takes some time form other projects without telling anyone and ninja fixes the problem to solve it.
lumberingjack 6y ago
They still made like a billion dollars in shark cards so apparently all the people buying shark cards didn't give a shit about the load time issue and that's the real problem.
deleted 6y ago
[deleted]
specialist 6y ago
Based on my own experiences, I'm absolutely certain many people over many years tried to get the load time fixed, and then were beaten down for their earnestness. Just as I'm certain that this unsolicited 3rd party fix was used to settle some old scores. Corporate politics is a blood sport.

monkeypizza 6y ago

Watch out not to condemn them no matter what they do. Sure, they messed up in the past, but given the situation they found themselves in when they saw this report, this response is still good. You risk making discourse difficult when you continue punishing people who are attempting to do the right after doing wrong for a while.

thitcanh 6y ago
I disagree. They ignored the issue for years and that part won’t be erased from memory. They should be given credit for finally fixing it, but nothing more. What happened before this week was and remains forever shameful.
- epivosism 6y ago
  Very likely that internally there were people who had an inkling this was going on, but could never convince product to devote time to put it in the backlog. This change is a win for that engineer. This example will be ammunition in internal debates about how much to weigh product pressure vs engineering suggestions that improvement is technically possible; the next time an eng says "let's look at this slowness one more time, it may be fixable more easily than you think", they might be listened to.
- ramblerman 6y ago
  > What happened before this week was and remains forever shameful. lol without context you'd think something tragic actually happened here.
  - thitcanh 6y ago
    Losing customers over such a simple, known problem, over years… is tragic for a company like Rockstar. This is like Microsoft ignoring a bug that made Windows take 10 minutes to start and just letting that happen for 5 years. Now tell me that’s not tragic.
    - fomine3 6y ago
      Anyway GTA5 is just a game.
      - thitcanh 6y ago
        To you it's just a game, to Rockstar GTA is literally a multi-billion dollar product with a household name that politicians talk about. I can't believe you're missing this huge point.
sellyme 6y ago
> You risk making discourse difficult when you continue punishing people who are attempting to do the right after doing wrong for a while. On the other hand, a few days of doing the absolute bare minimum of quality control doesn't undo a decade of actively hostile practices. They're still very very deeply in the red when it comes to their relations with the playerbase and for good reason. Paying $10k to the person who documented this bug is a good start, but you need to do a whole lot of work to make up for the impressions caused by that kind of prolonged systemic negligence.
- monkeypizza 6y ago
  Right. I'm not saying eval(their net behavior)>0; I'm saying that if your response function is always negative, you lose the ability to influence their response. Have you heard the one about the country with the death penalty for all crime? It just leads criminals to maximally try not to get caught, without regard to damage done [i.e. removing all witnesses to even the smallest crimes]. This isn't what you want; even within "negative" relationships both sides lose when you don't have a gradient of responses.
  - sellyme 6y ago
    The crux of this is that most readers presumably found out about both the bug and the fix in relatively short proximity, and their response is negative to the combined events, not to the knowledge of just the patch. A criminal prosecuted for murder and petty theft simultaneously can't really go around complaining that they got 25 years in gaol for stealing an apple, but if everyone had already known about the murder and dealt with that accordingly it'd be pretty harsh. (not that fixing a bug is exactly a criminal act, but the general principle remains)

NoPicklez 6y ago

I think "embarrassing" could still be the correct word. If you were to ask the development teams responsible they might feel a little embarrassed over the solution being so small (yet technical). And if they didn't care at all given what you say then why would they care to implement a fix for it all these years later?

awill 6y ago

I understand your frustration, but anyone who's run (or been on) a dev team knows you have a million things you could/should be doing, and many, many priorities. Often supporting new launches/features is the priority. Sure, ideally there would be someone internal advocating for a fix like this on behalf of customers. But Shameful isn't the right word. It's a miss like any other.

shock-value 6y ago
The egg that's on Rockstar's face has more to do with how much money they likely left on the table by not diagnosing and fixing this issue early on. (And assuming they had a team overseeing this aspect of performance with adequate tools to observe and debug it should have been trivial.) I personally feel like the magnitude of this -- especially compared to how simple the solution is -- makes it a miss that's really much more extreme than most others. EDIT: Though I agree that "shameful" isn't the right word to use. Rockstar wronged themselves much more so than any other party.
SV_BubbleTime 6y ago
Obviously you are right... but... If the fix had been some super application specific super obscure thing, we would all understand. Instead... the bug is easily describable to anyone who knows what JSON is and the fix is “don’t parse JSON the slow way”. So I think some of the leniency afforded to the devs may be misplaced. Then again, hindsight and all that.
- thitcanh 6y ago
  The cause doesn’t actually matter much, usually one doesn’t know the issue until they investigate. So often issues remain open for lack of interest, then someone finally does and it’s a missing comma. This doesn’t affect the fact that the issue has been ignored. I’m sure GTA has unlimited bugs to look into, but a bug that turns people away before they even start should have probably been given higher priority.
  - deleted 6y ago
    [deleted]

lambda_obrien 6y ago

It's a video game, so maybe give them some slack. We're not talking a SolarWinds breach here.

aantix 6y ago

The game supports 8 platforms, has virtually miles of terrain, and now Rockstar is suppose to feel ashamed because they didn't fix a big performance issue that affected startup times? The game probably has tens of millions of lines of code. I'm glad they shipped. If they can enhance the startup time, great, bonus for them. Additional credit for thanking the third party that uncovered the perf issue.

mdoms 6y ago
> now Rockstar is suppose to feel ashamed because they didn't fix a big performance issue that affected startup times? Yes, they should be ashamed of this.
swagmoney69 6y ago
It's true that the game is probably absurdly complex, but how did they never profile loading times?
malwarebytess 6y ago
I don't think you've been following this story. If you looked into the technical details [0], and surrounding context, you would understand why your comment isn't particularly insightful. For example, terrain has nothing to do with this bug and it isn't a complex fix that interacts with "millions of lines of code." A single run of a profiler would have caught this. I also think it's relevant to note that GTA V has sold 140 million copies, 20 million last year alone, and Rockstar makes hundreds of millions of dollars a year on the title. For a multi billion dollar product to have a known, relatively simple, problem is shameful. I don't know what went wrong, but I don't think it was the competency of engineers or the complexity of the problem that allowed this to fester. [0] https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times-by-70/ https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times...
mlindner 6y ago
I think you missed previous news about this. This was an incredibly trivial issue that would show up in any use of a profiler on the slow startup screens. The fact they didn't is that they're either unable to run profilers on their code at Rockstar (which would be shameful indeed) or they never bothered to do so. Either case shows egg on face at Rockstar.

867-5309 6y ago

you forgot to mention that ads are displayed during the loading screen

rectang 6y ago

An alternative possibility: there were 100 bugs of similar consequence, all of which they cared about, and they got 99 of them. This one escaped, because stuff happens. > I think "shameful" is really more appropriate. When you make an inadvertent mistake in the future, karma is coming for you: people will speculate that you "didn't give 2 shits".

jacobr1 6y ago
More likely, there were 99 other bugs that they had clearer metrics related to monetization that they prioritized (not that this bug didn't impact monetization - I bet it did, just not in a way they were measuring). Or that their QA engineers identified and flagged as high priority or major new features with their own bugs and regressions or whatever. These things get lost in the shuffle. More likely there were 1000s of bugs in the tracker, probably 10s of thousands.
- rectang 6y ago
  Yes, I agree with your elaboration. Also, of the 100 bugs, they didn't get 99 of them, because there are surely some left. Maybe they got 90, and there are 9 still hiding out.
rubicon33 6y ago
I’m sorry but no. This bug existed for a very long time and was the #1 outcry from gamers. It is well documented all over the internet. Every company I’ve ever worked at and every code base I’ve ever worked on would have made this a #1 priority. If it were a technical limitation of the devs I would agree it is “embarrassing” but this was not outside of their ability. I simply don’t blame the devs, I blame management. Management should have allocated the resources to fix this truly game breaking bug but they didn’t because money was still flowing. Is it any coincidence that ads played during the 6+ minutes you would wait to get into a game?
- ryandrake 6y ago
  Yea, if it was a 6 month old bug or it was a hard to repro edge case, then sure it’s embarrassing. Oops! But years long, and highly visible/reproducible, without correcting? That’s veering into shameful.

agumonkey 6y ago

If so then they still acted in a gentle way that is rare afaik. If they were so careless they'd just lift the fix and ignore the author.

pluc 6y ago

This is akin to that guy buying a whole-page ad to complain about his ISP recently, resulting in enough shame to trigger action. It's a bad look for Rockstar considering the income that title generates and at most it was a few weeks worth of dev time. Very interesting to watch the public reaction to this in comparison though, seems to be a lot more sympathy for Rockstar than for ISPs.

seppel 6y ago

There probably is a story behind it. Like some developer looked into the bug in the beginning (when when the load time was not that bad because the json file was smaller), saw that the time was spend parsing json, and then, without thinking too much whether this makes sense, wrote in the bug ticket that, to make it faster, they would have to switch from json to something better/faster/different. Since switching the file format is probably some work, maybe involving many differnt teams and network protocols, the bug was closed because it too much efford. And then, whenever somebody complained, the old bug was referred which unfortunately never got a proper analysis. Sad, but I saw this happening many times in the past in some projects. Since the old and wrong analysis will be in many powerpoint slides and heads, it is incredible hard to argue against or get even time to work on such a problem. Usually, you need someone who just takes some time form other projects without telling anyone and ninja fixes the problem to solve it.

lumberingjack 6y ago

They still made like a billion dollars in shark cards so apparently all the people buying shark cards didn't give a shit about the load time issue and that's the real problem.

deleted 6y ago

[deleted]

specialist 6y ago

Based on my own experiences, I'm absolutely certain many people over many years tried to get the load time fixed, and then were beaten down for their earnestness. Just as I'm certain that this unsolicited 3rd party fix was used to settle some old scores. Corporate politics is a blood sport.