9 ms·
OpenTTD's Infrastructure in 2023
- AceJohnny2 3y agoHow much does this infrastructure cost? How does the OpenTTD project pay for it?
- Macha 3y ago< 3500 USD/year (that was the cost of the previous iteration on AWS, and they mentioned one reason they moved to the current setup was because it was cheaper) It's paid for by donations They go into this in part 2: https://www.openttd.org/news/2023/07/09/infra-migration https://www.openttd.org/news/2023/07/09/infra-migration
- bshipp 3y agoHaving played (and lost countless hours) Transport Tycoon Deluxe in the 90s and then OpenTTD 10 or 15 years ago, I'm just tickled that this game adaptation is still going strong. Chris Sawyer's ability to create addictive building games that remain fun to play long after their contemporaries have ended up in the dustbin of history is superhuman, in my humble opinion. Add to the fact that he did it all in Assembly, and it's hard not to place his achievements on a bit of a pedestal.
- rightbyte 3y agoI would say Roller Coaster Tycoon is his best design. It was just before 3D became forced mandatory for games for some reason and not a one man job any more. I mean at the same time Sim City was actually good too.
- massysett 3y agoThere are still new 2D games out there: Factorio, Dwarf Fortress. For some games 3D just doesn’t add anything of value.
- arp242 3y agoThese are much more recent games; things have bounced back and there's loads of 2D games today, but starting around the year ~2000 there was a decade or so everything had to be 3D, which often meant "3D FPS shooter", whether it made sense or not. This often meant a regression in graphics quality (e.g. Baldur's Gate 2 vs. Neverwinter Nights) and frequently a regression in gameplay too (usually due to horrible controls and/or camera, something like Monkey Island 4 is a good example).
- Vvector 3y agoI'd say there is still plenty of room for topdown/isometric games. Factorio for example.
- paxys 3y ago> The question: how does OpenTTD’s infrastructure look, or even: why is it so complex, is a rather complicated question to answer in a few words. [...] Was waiting for them to get to the "why is it so complex" part, but after all the details of Cloudflare Pages, Cloudflare R2, Cloudflare Workers, Cloudflare Access, EC2 instances, multiple CDNs, hosted Redis, Nomad, Pulumi, web of proxies and APIs and front doors, a dozen microservices and an IaaS repo to make sense of all of this, it came down to: > In total, we store over 150GiB of data, transfer over 6TiB of data monthly, have more than 10M requests a month, and serve thousands of unique visitors every week. Basically my MacBook Pro from 2019 could host all their infra and data and serve the entire load (~3 RPS) with room to spare for my day-to-day work. For anyone else who is reading the post looking to get inspired – ignore everything they did and start small. A single web service to handle all business logic, hosted on two rented VPS instances which split traffic. Data stored in mysql or postgres with some regular backup. Start scaling only when the load from this setup overwhelms you (and I can guarantee it won't for 99.9% of cases, including the one in this post).
- mynameisvlad 3y agoAnd if your MBP were to die today? Or if a burst of traffic came through? Or if you closed the lid and forgot to turn Caffeinate on? Surely you can understand why people don't routinely host all their project's infrastructure on their laptops, even if the technical specs are enough.
- LordShredda 3y agoIt doesn't have to be _your_ macbook, it could be someone else's, like a vps somewhere else with a 99.9 uptime. That would be more than enough to serve 150GB of static data + a forum. You don't need to involve amazon and cloudflare for a 90's game.
- Macha 3y agoNote that the forum is a separate entity to the openttd project (though there is some overlap in people), and while openttd is the most commonly played variety these days, was started to discuss TTD and TTDPatch
- cdchn 3y agoThis sounds like practically a text book example of how to run a small-ish hosted service, and I like they got to take more interesting choices with Pulumi and Nomad vs. the "nobody ever got fired for buying IBM" of Terraform and Kubernetes. If I need to build the infrastructure of an early (or even mid depending on the complexity) startup I'd practically use this as a playbook.
- The_Colonel 3y ago> This sounds like practically a text book example of how to run a small-ish hosted service The stuff of nightmares. Our $200M/year revenue web app serving >2M daily users runs on a simpler setup than this.
- cdchn 3y agoHow much is your hosting/staff cost?
- metalforever 3y agoThis.
- mattlondon 3y ago> This means that if people want to play nasty and find issues in our services, they first need to bypass Cloudflare’s WAF. And this is not an easy thing to do. Is it not the case that you just need to use their IP address and you bypass essentially 100% of what cloudflare offers? I guess a targeted attack is hard (how do you find the "real" IP?), but there.may be speculative attacks just scanning through IP ranges
- bamfly 3y agoYou default-deny all source IPs, then allowlist your CDN's IPs on your "origin server" or its network's firewall box (if you have such a thing). Is the usual way to solve this problem, anyway, IDK if that's what they're doing. Then it doesn't matter if someone finds the IP of the actual server. Worst they can do is flood you with instantly-dropped connection attempts, but not probe services or run up your server hosting bill with large data transfers or anything like that. Scans won't find listening ports.
- jcrawfordor 3y agoYou can also set up TLS client authentication as a more complicated but a bit more assured method of refusing connections from anyone other than Cloudflare.
- viraptor 3y agoThere are two ways you can handle this: - accept traffic only from the published CloudFlare IP ranges - connect to CloudFlare rather than accept traffic (Argo tunnel) The first one unfortunately doesn't protect you from someone scanning from the CloudFlare ranges themselves. You can add a custom header in that case so that any traffic without the shared secret is not accepted.
- Natuerich 3y agoI'm pretty sure this is not as complex as it reads (private network, ipv6, cloudflare etc). But I would assume that two hetzner server for 100$/month an nginx properly configured would do the same thing. Nice writeup cool to get insight:)
- jaimex2 3y agoHoly hell that's over-engineered.
- lionkor 3y agoThey could probably host this on a $60 Hetzner dedi, with maybe a $5 VPS from some other provider in case the big one goes down, at least to host the website and possibly a status page from two locations. That can easily handle TiB of traffic (Say 20+ TiB/month), have a few thousand requests a second (depends on what kind of request), and you can easily parallelize a lot of this by running your backend in a few docker containers on this dedi. That's, say, $70 including the cost for a domain and such, and all it requires in maintenance is regular system updates. Auction boxes on Hetzner often come with >1TiB of SSD storage, if you're so inclined. Then again, I'm the kind of guy to play OpenTTD by making a few simple networks and watching the thing generate stable revenue, and then messing around with building something else, so maybe the mentality is just different.
- berkle4455 3y agoRunning anything you care about on Hetzner is a needless risk. There's numerous other discount providers who don't abruptly turn off traffic to your box or close your account the moment they see something they don't understand.