518 ms·
1500 Archers on a 28.8: Network Programming in Age of Empires and Beyond (2001)
- Waterluvian 4y agoI think this is a “must-read” for anyone, even if you aren’t in game dev or doing networking stuff. There’s just so many mini “eureka!” nuggets to enjoy. I think that internet multiplayer is possibly one of the hardest things in game dev (use an engine that gives it to you!!!) despite feeling like some little sideshow. I imagine Blizzard and Westwood have similar stories from the 90s.
- skocznymroczny 4y agohttps://www.codeofhonor.com/blog/the-making-of-warcraft-part-3 https://www.codeofhonor.com/blog/the-making-of-warcraft-part... there's this for Warcraft 1 development
- CSMastermind 4y agoLots and lots of great things in here, thank you for sharing. > At first take it might seem that getting two pieces of identical code to run the same should be fairly easy and straightforward I found that a little funny. I would certainly not assume that. Synchronization problems are one of the scariest areas of programming in my opinion.
- hot_gril 4y agoIRL it was not simple. The game usually worked, but occasionally things would go out of sync. The biggest downside is mentioned in the article: If one person lags, everyone lags. In an 8-player game, the chance that at least one person lags was pretty high. I used to play on the Mac version of GameRanger, a small community. That was 2008, when reliable internet wasn't as common. Players had personal reputations, and those who lagged too much were often excluded from games.
- ComputerGuru 4y agoMan, this brings back memories. People fighting over char in the lobbies, some derivative of “laggy” used a slur to try and get someone you don’t like to leave the game before enough people joined, etc. fun times!
- hot_gril 4y agoWhen the game lags, the witch hunt begins, and the red ping or turtle shows on your name despite it not being your fault.
- IntelMiner 4y agoEternally immortalized in the sequel as a taunt "Sure, blame it on your ISP!" https://www.youtube.com/watch?v=HGBOeLdm-1s https://www.youtube.com/watch?v=HGBOeLdm-1s
- hot_gril 4y agoI was always confused about that taunt. It sounds like, blame your poor play on your ISP, but this is one of the few games where laggy internet doesn't affect your moves.
- IntelMiner 4y agoRed Alert 2: Yuri's Revenge was pretty similarly infamous for this If your ping in the lobby was 1000ms (effectively "timeout") or you took too long to download a map, most players would just kick you preemptively
- hot_gril 4y agoIn AoE2 custom scenario lobbies like CBA, people would often see the file transfer as an indicator that you're a n00b since you didn't already have the scenario, and you'd get kicked. "Bro I'm just playing on my other iMac G3!" kick
- 4y ago
- xupybd 4y agoMaybe a 2001 in the title would help?
- hotpotamus 4y agoThe 28.8 was more than enough for me to place the article in time.
- djmips 4y agoIf you're old enough to know what 28.8 means
- scaramanga 4y agoKids these days... :)))
- iso1631 4y ago28.8mbit a second? Wow speeds were slow back then /s
- hotpotamus 4y agoI actually remembered a tidbit that will show my age as well. The first 28.8Kbps modem in the house was bought by my dad and he told me not to tell my mom that he got it because it cost around $500.
- Dalewyn 4y agoAs someone who grew up playing Age of Empires II and still consider it among the best games ever made, reading this is nothing short of amazing.
- eloff 4y agoI still play it (the definitive edition) with my wife. Best game ever in my opinion.
- kyleee 4y agoMonk! I need a monk!
- chrisco255 4y agoWololooo!
- Dalewyn 4y agoWood, please!
- the__alchemist 4y agoSame. Played 6 games with my brother and friends today.
- paulirish 4y agoSame
- 12907835202 4y agoI still regularly have dreams about it 20 years later. I take it as a sign that I'm in a good healthy place mentally when the most stressful and anxiety inducing thing my brain can conjure up is me being rushed by 3 units in the dark age.
- IntelMiner 4y ago
- poglet 4y agoIt's interesting how the online gameplay has changed between the different versions of the game as network latency has improved. For example, being able to micro manage archers to dodge other archers, and time their shots is something that wasn't possible previously.
- spywaregorilla 4y agoThat sounds terrible. I'm pretty sure the next game in the series, Age of Mythology, didn't permit missed shots.
- thesandlord 4y agoArcher micro is a huge part of the competitive AoE2 scene. AoE4 had auto tracking arrows and I think folks didn't like it as much as it lowered the skill ceiling
- spywaregorilla 4y agoI understand what was said. I'm saying that sucks. I don't really care about the tiny minority of people who care about the skill ceiling within a super low level micro. RTS entices people into a strategy game. But the games are so similar that everyone but the top 1% of players will do better by simply following a predefined strategy and executing it as tightly as they possible can. And that sucks. Nudging guys a few inches to the side is the opposite of what makes the game fun for most people. I'd like to see RTS games that randomize game parameters each time, requiring you to actually strategize.
- the__alchemist 4y agoGet ballistics!
- dayjaby 4y agoYou can still dodge ballistic shots, if you move your units after the projectiles got shot.
- ironSkillet 4y agoI love seeing the inside of the game development cycle from a technical perspective, thanks for sharing.
- tigerlily 4y agoThis site used to be Gamasutra didn't it?
- acdha 4y agoYou’re right: https://web.archive.org/web/20210823163259/https://www.gamasutra.com/view/news/387227/Gamasutra_is_becoming_Game_Developer.php https://web.archive.org/web/20210823163259/https://www.gamas...
- teraflop 4y agoYup, this post is how I found out that they changed their name. > Game Developer was first founded in 1997 as Gamasutra and has strived since its inception to be a leading resource and reference for game development and industry knowledge. Following the shift from Gamasutra to Game Developer in August 2021, the site has maintained that mission while embracing the in-depth content its namesake Game Developer magazine is known for. https://www.gamedeveloper.com/about-game-developer https://www.gamedeveloper.com/about-game-developer
- astrange 4y agoThey renamed it because they decided being a pun on "Kama Sutra" was just weird and unprofessional. (Same for the NIPS conference. But oddly not for the science journal PNAS.)
- Haga 4y ago[dead]
- recuter 4y agoShame that. Sutra is a word all on its own. Gamasutra was a nifty name. It works. One day sex positive triple-breasted feminist whores of Eroticon Six are all the rage the next you're sunsetting harmless Sanskrit puns. Y'all strange.
- IntelMiner 4y agoMisogyny isn't cool nor wanted here
- fragmede 4y ago(2001)
- idorosen 4y agoFor those who don’t want to suffer from eye-bleeds due to ads, interstitial scrolling ads, more ads, and awful formatting, Googling the title yields this PDF: https://zoo.cs.yale.edu/classes/cs538/readings/papers/terrano_1500arch.pdf https://zoo.cs.yale.edu/classes/cs538/readings/papers/terran... Maybe the link can be replaced? Also can (2001) be added to the title?
- xivzgrev 4y agoI enjoyed reading how they solved synch problem: schedule this moments commands for 2 turns from now, and the turn length is dynamic to smooth out experience.
- Centigonal 4y agoSeeing stuff like this on here fills me with nostalgia. I basically learned to code thanks to a few helpful and extremely patient IRC members in the Spring RTS engine[1] community (some of whom I spot in these comment sections on occasion - hi!), and deterministic lockstep simulation for online play was one of the first ideas that totally blew my mind when I learned about it ("b-but that means all the "random" numbers on every computer need to be the same! and so do all the floating point calculations across different hardware!"). Of course, to me at the time, that was deep magic that I thought I'd never be able to fully understand. [1] see https://springrts.com/ https://springrts.com/ - or check out http://zero-k.info/ http://zero-k.info/ and https://www.beyondallreason.info/ https://www.beyondallreason.info/ for games on the engine that are still pretty active
- paulryanrogers 4y agoIf the randomness is deterministic doesn't that mean players could peek into the future?
- Centigonal 4y agoGenerally (I hope I get this explanation right), the PRNGs on all players' computers are the same algorithm, with the same seed, executing the same player commands in the same order. So as soon as the game starts, you can predict what all future calls to random() will return. However, you can't predict the other players' actions, and those actions can insert an arbitrary (and unpredictable) number of random() calls between now and when your next action executes, so practically there is no gameplay advantage you can get from that avenue (AFAIK).
- taberiand 4y agoI'm gonna take an uneducated guess that though the simulation is in lockstep, the order of inputs from all players is not known ahead of time so you can't peek too far. Though maybe a cheat engine could simulate a few branches of possible inputs a few steps ahead and adjust inputs when it discovers split-second advantages?
- 4y ago
- lamacase 4y ago> Cheating to reveal information locally was still possible, but these few leaks were relatively easy to secure in subsequent patches and revisions. I don't understand this. Running the whole simulation locally on both ends means that a modified client would have access to the whole game state, and I don't really see how you could patch that out. Anyone have any idea what they actually did? Try to detect modified clients? Obfuscate the game state to make it harder to interpret?
- sonjaqql 4y agoIn age II, it was CRC checks of game state that prevented cheating! Games would get out of sync if there was a mismatch, which could happen for various reasons. I worked at MacSoft during the Age2 and Age3 days. The ports were faithful to the windows versions, but cross platform hadn’t been solved at the time because of this problem. It also made long game play sessions longer because the calculated state kept getting more complicated. There was one particular Mac OS X update that broke math interoperability for multiplayer because they changed how math worked on the OS. This meant we had to bundle a common math library to ensure the game the game states would line up, preventing unnecessary CRC checks.
- hot_gril 4y agoIn the Definitive Edition, there has been observed cheating in online ranked mode where players give themselves infinite resources. Evidently there's a way to do that without going OoS. This was probably the same in the original. This is besides the less complex cheat, removing fog of war. Seems the only thing stopping most people from cheating is, they don't want to. Also, I used to play the Mac version a lot. It was great! Shame they made the remakes Windows-only.
- alternatetwo 4y agoYou can call the command creation functions yourself, making the game think it's part of the simulation [1]. These types of cheat have been around ever since the original game, and the HD edition [2]. [1]: https://redrocket.club/posts/age_of_empires/ https://redrocket.club/posts/age_of_empires/ [2]: https://www.youtube.com/watch?v=3tl8AjDfnBk https://www.youtube.com/watch?v=3tl8AjDfnBk
- mirchiseth 4y agoWho remembers tourneys for Age of Empires at Sunnyvale Fry's.
- deleted 4y ago[deleted]
- Emjayen 4y agoAs someone who fixes up old games in my spare time, and Age of Empires II being one of them, I'll provide a bit of trivia about the game's internals: - The AI system is not part of the deterministic simulation. This was surprising to me, and after contacting one of the original programmers it was explained that it was due to a desynchronization bug that the "AI and network programmers weren't able to fix it in time". A consequence of this design regression is that, due to the AI now being authoritatively run by the designated host player, network congestion issues arose which lead to a clear series of progressively more aggressive optimizations to reduce egress traffic. This primarily consisted of a very simple filter (mentioned in the article) which dropped duplicate commands in the common submission path, meaning it applied to both local user and AI commands, along with batching of AI-submitted commands which would be flushed at rather arbitrary times. I'll note that I've restored AI being deterministic in my project. - A rather obscure determinism bug resulted from their compiler's implementation of a few CRT routines, namely fsin/fcos and a few others, which leveraged the specialized ISA instructions of the same name. The problem being that these transcendental functions are beyond the scope of the IEEE 754 spec. and ergo are hardware implementation-dependent. In practice, contemporary Intel/AMD chip families produce bitwise the same result, however those around the time of AoE are known to diverge on results to some small margin (as confirmed by an Intel engineer on a thread I came across while researching). - The game employs a dirty-update system for rendering, not only that it's at a scanline granularity. This was something I was very pleased to see, as it's a exceedingly rare to see such an important optimization in games of this era (although common in earlier eras) - There's some "interesting" naming conventions, one being prefixing member variables names with "value" -- there's even a "valueValue". Very little consistency in general in this regard, a reflection of independence between teams working on different components. - While there are attempts at validation of input into the simulation (albeit woefully inadequate), it relies on ad hoc inclusion of PID (player ID) fields within commands. This is entirely useless, as this information is not authoritative and controlled by the players, permitting them to "spoof" the contextual information required for validation. This is one of the more perplexing aspects of the engine, especially given the necessary information about the origin of a command is ofcourse available. (As this information was later publicly published by other individuals I don't see a problem with elaborating on it here as I have) - An example of missing the wood for the trees: session information goes through an ad hoc compression for its wire form (just bitpacks fields) to conserve bandwidth, however the architectural choice is to synchronize this session state by just having the host broadcast the state --dirty or not-- every 200ms, flooding the network pointlessly (atleast in the 18.8k days) - While Age of Empires is somewhat notorious for its poor multiplayer performance (notably when contrasted with say, the recent "Definitive Edition"), and while its implementation of this synchronization model is certainly rather juvenile, it should be understood that the final MP gameplay issues are primarily due to the choice of a peer-to-peer topology over the public internet, which at the time was the most reasonable. While superficially P2P may seem like it should achieve the lowest latencies for instance, the reality is that the primary determinant is the characteristics --jitter, delays and packet-loss, reordering, ect-- of the path between two hosts and a P2P architecture means there's n*n paths (network egress/ingress paths are typically asymmetric). In contrast, a server/client model you not only have far fewer routes, but datacenters are located at critical points in the network, roughly analogous to comparing travelling A->B along a freeway versus via the maze of residential streets. - The state checksum "algorithms" involved are bordering on useless, atleast for state-tracing (such as when debugging desync. bugs.). They appear to have been devised by way of believing doing "a bunch of random bitwise ops" constitutes sufficient mixing -- a quick test demonstrated that csum collisions were not just possible, but occured sometimes for over 80% of inputs. -- There's a lot more that could be said but I feel that's enough for now.
- khalidx 4y agoThat's crazy I see this post. Haven't read it yet, but I just played a 3v3 game out of nostalgia. People still play this game years later, and it works great.
- AshleysBrain 4y agoWhat I find interesting about the lockstep approach described here is it seems a lot of modern RTS games still use the same approach, even though the bandwidth calculations have changed radically. When developing my own RTS game I was able to fairly straightforwardly get 1000 units in combat to live stream using about 50 KiB/s bandwidth, which is nothing these days: https://www.construct.net/en/blogs/ashleys-blog-2/rts-devlog-units-bandwidth-1603 https://www.construct.net/en/blogs/ashleys-blog-2/rts-devlog... So as much as this is a fascinating piece of history and an impressive technical solution to the constraints of the time, I think modern games ought to move past it.
- qikInNdOutReply 4y agoDid those units shoot? What happens when there is a few thousand projectiles in the air, fired by those 1000 units? What about map deformations? What about map wide physic simulations, like springs tsunami water sim or the lava flows of a volcano, changing directions?
- nothis 4y agoNone of this should matter as long as the algorithm determining randomness is deterministic. The bottleneck is player input which is the most overestimated bandwidth stat in gaming. It's mouse movements and a couple of keys strokes per second. Top Starcraft players are in the 300 actions per minute range, that's still just 5 per second.
- qikInNdOutReply 4y agoBut parent wanted to ditch the effort for determinism.
- AshleysBrain 4y agoYeah, the units all shoot. One-off events turn out to take negligible bandwidth if they are cosmetic and the client can predict what happens.
- dale_glass 4y ago
- TacticalCoder 4y agoIIRC, and I've got nearly all my emails since the nineties until now so I could find out, I sent an email to the author of that article back when I read it, I think, still IIRC, on the Gamasutra website. And he answered me: we chit-chatted about deterministic game engines, for I wrote one in... 1991! (not for a networked-game though). The subject already came up here on HN and some posted about games using deterministic engines before AoE. The reason I did it is I had a bug which happened ultra-rarely and couldn't figure it out so I thought a long time about this and realized a could make the game deterministic and that would maybe allow me to record the bug happening and then be able to replay it. I found that by myself: back in 1991 I had never heard of anyone writing a deterministic engine back then. So I took a few days and rewrote the engine to be fully deterministic. Sure enough it eventually caught the bug: some case where the hero would clear a level after having fired two shots at once, which was an extra (by default it only had one shot at any time). The shot still on the previous level would continue to "live", invisible, in the following level, and would corrupt the memory. Classic. Oh the memories to see that article again!
- lithander 4y agoAn RTS we developed just recently (Iron Harvest) is still based on the same basic principle. But instead of peer to peer we use a "smart" proxy server that receives the commands of all clients and sends the entire sequence back to the client. That way if one client has a high ping the other players latency isn't affected.
- deleted 4y ago[deleted]