21 ms·
Rockstar thanks GTA Online player who fixed load times, official update coming
- marecile 6y agoGood job on the person who found they could reduce GTA Online's abysmal loading times; it shows how greatly they could improve the game. Keep in mind Rockstar has almost 0 interactivity with its fanbase and doesn't listen to any feedback. So the possibility of negative attention forced them to do something about it.
- torgian 6y agoThanks, but we won’t give you anything.
- pugworthy 6y agoActually they did
- TheRealSteel 6y agoThe article has been updated to note that Rockstar gave him ten thousand dollars :)
- meowster 6y agoPlease don't pretend to be someone you're not.
- torgian 6y agoNice, good that they gave him something. I didn’t think they would
- pixelpoet 6y agoLove the comment someone made on his buy me a coffee page: "Please fix Cyberpunk!"
- kjjjjjjjjjjjjjj 6y agoThis is really cool to witness. A random guy fixed a really obscure bug that millions of people have been complaining of since the game launched.
- deleted 6y ago[deleted]
- matsemann 6y agoDiscussion about the findings two weeks ago: https://news.ycombinator.com/item?id=26296339 https://news.ycombinator.com/item?id=26296339
- brink 6y agoFor all the amazing things Rockstar has made, for doing what no other studio could do on the Xbox360, it's crazy that they haven't fixed this before now.
- dmos62 6y ago> doing what no other studio could do on the Xbox360 What's that?
- brink 6y agoGTA5, two console generations later and most games still don't come close.
- Thaxll 6y agoBecause it's a game that has been in developpment for 8years, the game as of now is very different from 8 years ago from when it released. You can't compare a game just released with a game that had that much iteration with thousand of people.
- mclightning 6y agoFor the most part, it is not the tech, it is the content & artistic value they put into the game. So I don't think it relates in this tech situation. Tech-wise GTA5 is not that interesting for few years now.
- zamadatix 6y agoGTA 5 on the 360 runs at 720p@30 FPS and even then it drops to the low 20s any time something happens even considering the graphics are cranked extremely low by today's standards. How well they had it running on the 360 is absolutely amazing given the limitations but nostalgia is taking over if you think most games on 4th gen Xbox don't even come close.
- dkersten 6y agoMost games don't come close in what sense? World size? Lots of open world games are as big or bigger. Graphics? I thought this too, but then I had another look (been a few years since I played it) and its quite dated looking now. Not bad, but definitely not on the same level as more recent games. Understandable though, it came out an entire console generation ago! Even gameplay-wise, yeah, it has much better driving mechanics than most non-racing games, but shooting and such is fine, but nothing special. Its a great game and was absolutely fantastic for when it came out. I'm not at all surprised that its as popular as it is! But "most games still don't come close" just isn't true, at least for large AAA titles.
- adwi 6y agoGood on them for acknowledging something a little embarrassing and responding with gratitude instead ignoring to try to save some face.
- dna_polymerase 6y agoIf shorter load times there translate to higher conversions in the same way this connection is observed by Amazon [0] this guy just made Rockstar a little fortune. Hiring the blogger outright and giving them a VP of Something Something Role would be more adequate. [0]: https://www.gigaspaces.com/blog/amazon-found-every-100ms-of-latency-cost-them-1-in-sales https://www.gigaspaces.com/blog/amazon-found-every-100ms-of-...
- swinglock 6y agoI'm genuinely surprised. Given how long they've left this untouched I thought they didn't care at all.
- de6u99er 6y agoTbh: Looks more like they haven't been aware of it. This can result in severe consequences for the developers who worked on this particular part of the codebase.
- aaomidi 6y agoNo it won't. It's ridiculous to blame a few devs for this. C/C++ has unknown quirks like this.
- bogwog 6y agoThey probably just didn't have anyone working on it that had the knowledge (or incentive and/or permission) to fix it. AFAIK they've just been adding content to the online portion for the past 7~8 years, and you don't need engine programmers to do that.
- rubicon33 6y ago
- IshKebab 6y ago"Turns out, they were right." err yeah of course. Nobody doubted them.
- saagarjha 6y agoExcept some commenters in the original thread ¯\_(ツ)_/¯
- atum47 6y agoEpic!!! Imagine doing something for fun and it ends up in production.
- ZephyrBlu 6y agoThis is going to be a great point on his resume. How many people get to say something like, "Reduced GTA 5 load times by 70%"?!
- megablast 6y agoOne person. Only one person gets to say that.
- DevX101 6y agoThat's pretty amazing by itself. But it was doing it without access to the source code that blew me away.
- ZephyrBlu 6y agoThat really is the cherry on top. Ya know on second thought, it's so unbelievable that he did this whole thing that I wouldn't be surprised if someone reading his resume thought he was bullshitting.
- xdrosenheim 6y agoAnd that's why you have source in your resume.
- redrobein 6y agoAlthought it is impressive, it isn't that uncommon in the reverse engineering world and quite believable imo.
- lumberingjack 6y agoGamers playing this game have all kinds of tricks to try to bypass the load screen itself because it takes a long time to go through it but the game by design forces you to look at the load screen multiple times just to do one task. One way that I remember was to play in a Solo session that way you don't have to load every time you come out of multiplayer. They even make a GTA online software tool that blocks GTA on the firewall at very specific intervals to force an online session in order to speed up load times.
- yNeolh 6y agoI really was thinking that it was intended... I remember watching GTA V on Xbox 360 and being impressed by how good it looked, how it performed in such a "limited" device, so something like this would resonate in a team so good. But it was almost 6 years ago, so probably a good part of that team has moved on to other companies, and the ones left, maybe are in other projects, hopefully, GTA VI. Glad to see Rockstar addressing this; I will try the update once it is released, and maybe I can play some online games without getting frustrated.
- rightbyte 6y agoMaybe scanf didn't redo strlen on the Playstation stdlib. I mean the PC version was released one year later. Maybe by another team.
- SXX 6y agoGame itself had amazing performance on PC when releases, but multiplayer part had lots of issues for years even though they primary profit from it.
- imwillofficial 6y agoI want to see more of this from giant companies. This gives me hope that there are real people amongst the mindless drones in these hulking corporations.
- deleted 6y ago[deleted]
- hutzlibu 6y agoYou mean real marketing people, who know that talk is cheap and the small bounty might mitigate some of the negative PR they received because of this?
- imwillofficial 6y agoDon't put words in my mouth. I meant exactly what I said.
- hutzlibu 6y agoI know what you meant, but with those words I hinted, that the decision from Rockstar was likely more a move out of PR motivation, not moral standards, that you assumed.
- A12-B 6y agoI'd like to see this person actually get paid for what is essentially a million dollar fix
- Frost1x 6y agoYup. I'm not deeply familiar with the situation and circumstances but it sounds like this person should be very well compensated for what they did. The amount of free skilled labor businesses benefit from and abuse while not donating back to society is laughable. It's great they helped the GTA 5 community and a fun read but keep your pride and fame, I'd take cash anyday over a pat on the back.
- deleted 6y ago[deleted]
- DavidVoid 6y ago"Just got awarded $10k through their H1 in-game bounty as an exception :)) (usually only for security issues)" From their blog post about it: https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times-by-70/ https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times...
- A4ET8a8uTh0 6y agoTell me that is not cool.. being able to say you had that much of an impact on million of players across the globe. Not to mention, bragging rights and cool achievement you can put on your resume ( that anyone should be able to understand ).
- utf_8x 6y agoNot to mention the $10,000 bug bounty...
- sim_card_map 6y agoGiven how simple this bug is, I was 100% certain they did it on purpose to show more ads.
- indeedmug 6y agoI don't think it was simple. It came down to the odd specifics of scanf making the algorithm O(n^2) which wasn't obvious at all from the programmer standpoint. A lot of people pan Rockstar for being incompetent but in reality, it's a C footgun that any programmer would make. Edit: I reread the article and I overlooked how there was a second issue of a poorly implemented "hash table" that was a linked list that checked the entire list for dups in every update. Since we don't have access to the source code, it's hard to say if this is poorly implemented piece of code on Rockstar's part or someone else. It could be some deeply buried piece of code that no one ever looked at for years until now.
- 295310e0 6y agoOn the gripping hand, if someone had run a profiler on the code in all of these years, it would have popped out immediately that some investigation was warranted.
- indeedmug 6y agoIt seems like the original GTAV is a efficient and well optimized game. I played it and it looks amazing for how fast it runs. They certainly ran a lot of profilers to nail the performance down. It's amazing what video games can do, look at the Doom series. https://www.youtube.com/watch?v=UsmqWSZpgJY https://www.youtube.com/watch?v=UsmqWSZpgJY But this is online GTA. I can imagine that they handed it off to another department that didn't have the same amount of discipline and expertise. it's probably an off shoot team that creates content for the engine but don't necessarily need to dig down into how the engine works.
- bredren 6y agoThen it is not a bug.
- pityJuke 6y agot0st's original post has an update that says he got $10k from R*'s bug bounty program. [1]: https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times-by-70/ https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times...
- nickspag 6y agoThe original fix’s author updated the post about this: https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times-by-70/ https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times... Rockstar awarded them 10k for it.
- sdfhbdf 6y agoIt's a really nice and wholesome use of HackerOne. Nice that they acknowledged and awarded as opposed to some companies that try their best to push a security disclosure out of scope and not pay out a bounty [0] [1] [0]: https://news.ycombinator.com/item?id=9278832 https://news.ycombinator.com/item?id=9278832 [1]: https://news.ycombinator.com/item?id=16000550 https://news.ycombinator.com/item?id=16000550
- rvz 6y ago> Rockstar awarded them 10k for it. The original post, analysis and the actual fix of this bug is beyond impressive. Strangely, Rockstar awarding $10k for that is like someone searching for lost treasure, identifies historical references which leads to finding a ton of it and somehow leaves with 1/8th of it, even though they did 98% of all the work. For that detailed effort, analysis and with a fix, I'd say that is worth around $80k. At least Rockstar was generous enough to award them in the first place; so I'll give them that.
- PragmaticPulp 6y ago> For that detailed effort, analysis and with a fix, I'd say that is worth around $80k. No good deed goes unpunished. Rockstar went out of their way to thank this person and send them a check equivalent to several weeks of median developer salary. Can we not shame them for this goodwill gesture? $80K would be on the order of half of the annual salary of most developers outside of SFBA.
- scrapcode 6y agoHalf? I live in the Midwestern US. Have been dabbling in software for most of my life, have some decent working examples and am finishing my BS:CS this year. I can't even get close to my current salary in my current unrelated field ($85k) to even consider a career switch. Are the jumps from junior to mid-level/pm that high? I am a PM in construction so that may end up being my path.
- rocky1138 6y agoImagine a world where companies open-sourced their most boring bits of their software so we, the users, could fix the problems we find with them or extend them in new ways without having to rely on the parent company to get around to it.
- lupire 6y agoMicrosoft essentially does this. They call it Shared Source. Most companies won't do more due to competition or embarrassment.
- varispeed 6y agoIt used to be a practice in the 90s that companies would insert Sleep instructions or other means to delay program execution e.g. to make appearance of heavy computations being done or to make people buy a "pro" version that was without such elements. Given how frankly stupid is the error, could there be some merit to that? For example to give themselves some ammo for the next versions - if they won't find anything to improve they could pull the rabbit out of the hat and said hey we optimised our code and it loads much faster, buy the new version!
- lupire 6y agoThey still do that today. It's why TurboTax is so darn slow and hard to use, wasting user's time.
- kylewatson 6y agoI wouldn't be surprised if they found a faster turbotax made people trust it less. You want to see it chugging away to "find your deductions" and "catch your mistakes". Ironically the hardest thing I needed TT to do this year was handle my wash sales and it didn't even blink as it imported the 1099s. Wait, maybe it didn't handle them correctly... See what I mean? Now I better double check.
- jtsiskin 6y agoYep, that’s exactly what they are saying
- viraptor 6y ago[citation... wanted] I mean I know it happened, but beyond one or two famous examples which get repeated on tdwtf, was it really a practice? (As in something done regularly, and not in shareware which had explicit visible delays)
- fao_ 6y agoDid you actually read the code. I think if they were going to do that it would be a literal sleep instruction rather than a naievely used way of looping over data edit: Akchually, the DRY method of doing this would be to make a function like: /* Remove this when it's time for a performance improvement! */ function Optimize(int N) /* do stuff N times (so you can limit the amount of performance improvement) */ end and then put that in the places you want to be able to optimize in the future
- LinuxBender 6y agoOff topic: Has Rockstar considered also making improvements to their identity management? Currently you can get the account details of any player from their social club. [1] [1] - https://www.youtube.com/watch?v=C_fdhCXlPI0 https://www.youtube.com/watch?v=C_fdhCXlPI0
- aligray 6y agoCan’t imagine how many hundreds of thousands of hours of people’s lives have been wasted just sitting and waiting for it to load. Oof.
- bluedino 6y ago"Well, let's say you can shave 10 seconds off of the boot time. Multiply that by five million users and thats 50 million seconds, every single day. Over a year, that's probably dozens of lifetimes. So if you make it boot ten seconds faster, you've saved a dozen lives. That's really worth it, don't you think?" https://www.folklore.org/StoryView.py?story=Saving_Lives.txt https://www.folklore.org/StoryView.py?story=Saving_Lives.txt
- LukeShu 6y agoI was curious: (10 seconds)*(5 million)*(365) = 564.1 years If I add on "/(human lifetime)", Wolfram Alpha uses 72.6 years, which it attributes to a 2019 UN estimate. ((10 seconds)*(5 million)*(365))/(human lifetime) = 7.775 lifetimes
- someperson 6y agoLife expectancy for the average Macintosh user in 1983 was shorter than it is in 2021. But 72 years about sounds right for US 1983 Mac users, who were already closer to middle aged.
- m463 6y agoCorrelation does not imply causation. But all bets are off with time traveling macintosh users.
- Xevi 6y agoDidn't they shave off a couple of minutes?
- 6y ago
- dcfccortex 6y agoAs much as author deserves a much bigger reward IMO, I feel like the value of this on their resume / CV is much more than 10k
- t-writescode 6y agoI would hope Rockstar is considering interviewing them.
- kirubakaran 6y agoI'd say he has already passed the interview. What else do they need to know?
- NwtnsMthd 6y agoThe horrendous load time was one of the reasons I stopped playing GTA Online. But at this point I don't think I'll ever pick it up again. Cool fix, nonetheless!
- mdoms 6y agoIt's the one and only reason my friends and I stopped playing. I hope they take this patching opportunity to resolve a few more load time problems, because even a 70% reduction isn't really enough in my opinion. The load times in GTA are at least 5-10x longer than most games I play.
- joshu 6y agonot the constant barrage of modders?
- effingwewt 6y agoWe used to call it Loading Screen Online. Really was a shame because I made some long-lasting friendships from that one game, many of whom I still play other games with. Had they bothered to fix this early on as many of us begged them to do from day one of online's release, many of us would have stuck around far longer. Oh well actually the shark card garbage coupled with ever-increasing item costs, as well as releasing the casino ~5 years late probably would have made me stop anyway.
- kasper93 6y agoDon't get overexcited. On "normal" CPU the improvement is more like 30%, still huge, but don't expect GTA Online to load instantly after the patch.
- lefstathiou 6y agoThe scale of this fix is mind boggling. Millions of people play this game every day. This lone man, driven by his love of solving puzzles, is single handedly unlocking potentially hundreds of millions of hours of “lost time” across the globe.
- triggercut 6y agoNot to mention the huge amount of cumulative energy consumption on those wasted cycles.
- winrid 6y agoMore like, opened up more time for them to play! :)
- machiaweliczny 6y agoI like that Steve Jobs was calculating people "killed"(lifetimes wasted) by slow start of OS to motivate engineers. We need more people like him.
- wuwuno 6y agoWhat I find extraordinary is that none of the developers who play the game never bothered to fix this problem. If I was a developer at Rockstar and played this game I would have have spent my own time to fix this problem just so that I didn't need to sit around and wait every time I played the game.
- gbrayut 6y agoReminds me of a Windows installer I working on many years ago, that seemed to take a long time using CPU but otherwise not doing any real work. Turned out the SDK we were using tried to update the progress bar every kilobyte or something like that, and our near-gigabyte sized packages filled the message loop with a bunch of pointless updates (30.001% -> 30.002% but both displayed as 30%). Once the vendor fixed the issue it shaved a few minutes off the install time that had previously just been a UI glitch (processing the large backlog of progress bar update messages).
- easton 6y agoThis must be why when you turn off the progress bar for Invoke-WebResource in PowerShell your download speeds become reasonable, as otherwise it updates the terminal every byte and takes a million years to download stuff that takes a second in Firefox or via wget.
- strogonoff 6y ago> Turned out the SDK we were using tried to update the progress bar every kilobyte or something like that, and our near-gigabyte sized packages filled the message loop with a bunch of pointless updates (30.001% -> 30.002% but both displayed as 30%). Had a similar situation with an Electron app I worked on. App’s window was getting very stuttery at some point during a long-running operation done by a third-party library, even though it happened in a background worker thread and wasn’t supposed to block the UI (which simply showed the progress of the operation). Turned out, that third-party library called my progress notifier function (in which I used Electron’s IPC to notify the GUI) on increments that were fairly spaced out most of the time, but really frequent during some operations. Mere overhead from IPC calls and GUI updates added up to a point where everything lagged, even though the operation itself wasn’t that intensive. I wrapped my progress notifier in a throttler, which I frankly should’ve done much earlier, and that solved it.
- fomine3 6y agoNPM had similar story https://news.ycombinator.com/item?id=10974929 https://news.ycombinator.com/item?id=10974929
- neilv 6y agoI wonder how much money the load time slowdown has cost Rockstar -- given their revenue from "Shark Cards", which are bought by players who want the fun of playing with all the expensive in-game items without spending time grinding for them. What's the correlation between being averse to spending time on less-fun aspects of the game (to the point that you'd pay Rockstar extra money to get only the more-fun time)... and being averse to waiting for long load times, when you just want to play a quick session of more-fun? Given Rockstar's investment in a steady stream of new big-ticket in-game items, and their huge revenue from that, I wouldn't be surprised if the slow load times cost them many millions.
- lumberingjack 6y agoRed Dead redemption has the same load time issue I bought that game for five bucks. Which was rockstar's cry for help in a dying online game.I tried it out for about 15 minutes realized it suffered the same issue turned it off and never turned it back on
- lumberingjack 6y agoYou have to be joking me. after years of launching solo sessions to bypass this ridiculous loading time they fix it now lol WTF
- est 6y agoHopefully Rockstar use a real JSON parser instead of handwritten strlen and offsets inside nested loop for a megabyte sized json.
- zelon88 6y agoI've found that when writing something like this which iterates over and over many millions of times a helpful test can be to place a simple counter in your loop. If you have an algorithm which you expect to loop only one million times and it loops ten million times, that is something you want to investigate.
- xyst 6y agoI am actually impressed R* followed up with this. I thought it would be sent to the backlog blackhole. I guess the social media impact really pushed it up the chain. glad to see I am wrong. many GTAO players will definitely be very happy once the new update is live.
- reverendhomer 6y ago"You forget a thousand of bugs every day. Make sure this is one of them"
- lars_francke 6y agoThe author mentions he might need help with a W8 form: I've filled out plenty of those. I can't find any contact information for him to offer help. If you're interested feel free to reach out and I might be able to help you. Contact details in my profile.
- diveanon 6y agoWasn't there an ad shown on that loading screen?
- JoeyBananas 6y agoNone of the programmers at rockstar who had familliarity with the codebase and the platform ever thought "There's no reason these load times should be so long, I'm going to look into this"
- xroche 6y ago> Yeah, that’s gonna take a while… To be fair I had no idea most sscanf implementations called strlen so I can’t blame the developer who wrote this. I would assume it just scanned byte by byte and could stop on a NULL. Darn, I got the exact same issue ten years ago, with sscanf on Linux/glibc. The sscanf was forcing the huge XML mmaped file region to be entirely read. The reason behind is that the glibc library was using some "smart" streams (to have generic functions for file/string streams, I guess) that basically all do strlen as first step to buildup this stream structure.
- cosmotic 6y agoImagine being a developer at rockstar whom could have fixed that bug but didn't because no one ever prioritized that ticket.