7 ms·
Folia – Multithreading Coming to your Minecraft server
- brucethemoose2 4y ago> provide a no-compromise solution Except for fabric/forge compatibility. :( Tick threading with server mods is just a mad dream anyway. Even dimensions threading is apparently impractical.
- gjsman-1000 4y agoPaper and Spigot, the parent projects, have never been compatible with Forge and Fabric, and are more popular for servers.
- brucethemoose2 4y agoOf course, though there have been some mad projects to try an merge the server implementations with fabric/forge. Paper + Fabric is currently dead though, so not much hope there.
- c7DJTLrn 4y agoThe modding/custom server situation with Minecraft is so exhausting. I can't believe after all these years there is still no official modding or plugin API. So now we have Folia, which is a fork of Paper, which is a fork of Spigot, which I believe was originally a fork of Bukkit. Nice. I guess people don't want to upstream patches for some reason.
- brucethemoose2 4y agoWe kinda do... in a limited fashion... for Bedrock. Java MC's development had always been extremely conservative, and it always will be under Mojang. Its both saved it from going downhill and weighed it down.
- c7DJTLrn 4y agoIt's a shame Bedrock modding (behaviour packs?) is so limited. The performance is so much better than the Java version. It's also a shame it has Xbox shit and microtransactions shoved in.
- brucethemoose2 4y agoWell the Java version is so moddable because its Java, thats almost impossible for Bedrock unless it goes open source or moves to unobfuscated C# (which is impossible because of the MTX of course).
- nottorp 4y agoBedrock has IAPs now? I mean it was useless anyway for running a private server, and only runs on Windows, and automation is limited because they crippled the world update distance to have the omg raytracing! or whatever. And now it has IAPs too?
- zamadatix 4y agoBedrock has had IAPs for many years. It's fine for a private server and they offer Windows and Linux versions of it (client side it supports pretty much every platform except Mac/Linux, including oddities like ChromeOS). Just don't go running it expecting to set up what you would on a Java server - it's two different environments. World update distance default is lower in Realms but can be configured on a private server or client hosted session. It's not really related to raytracing, simulation range has little to do with that. Both clients/servers have their ups and downs. I do wish they'd release the Bedrock client for Mac/Linux though. Then all platforms would have an official way to interop.
- nottorp 4y ago> It's fine for a private server Land claim / griefing protection? Multiplayer sleep? Custom recipes? Dynmap? Multiverse? WorldGuard rules? :) Edit because I can't reply any more: > to use a 3rd party bridging solution with it instead like GeyserMC+Floodgate All that for the privilege of running the Bedrock client on a Mac with Crossover, I guess. Nah, they can keep it. I didn't even claim the free Bedrock when they were giving it free to java owners. On the server I play we're looking forward to testing Folia so redstone addicts will only lag themselves and not everyone btw.
- robotnikman 4y agoI remember, back before Mojang was acquired by Microsoft the devs mentioned working on a modding API, and how it would allow for all kinds of things. I have a feeling such plans were killed off during the acquisition, as it has never been mentioned since then. Even now it would be simple for the devs to allow players to add new blocks and items via server sent resource packs without needed a modded client, and yet for some reason they do not. I still have to thanks Bukkit/Spigot for getting me in to Java in the first place tough, and thus my current position as a Java dev. I don't follow it as often but I still maintain a small server my friends play on.
- gjsman-1000 4y agoThere's a lot of drama there. Basically, there was Bukkit. In 2011, Mojang hired a bunch of Bukkit's developers, but didn't tell anyone that they also legally took control of the Bukkit project. This was because Mojang didn't really want Bukkit to be their official API as much as being a temporary API for the time being (and also didn't want to have anyone assume they would be responsible for it), but from that point, Bukkit was legally their project and the community didn't know. Then, Mojang changed their EULA to ban Pay-to-win servers. This caused a lot of controversy, and one of the Bukkit developers left. Mojang then revealed that, yeah, they owned Bukkit for years, so they didn't need him. Another developer, feeling betrayed, who had contributed to Bukkit, submitted a DMCA takedown against his own code bringing Bukkit down with it, angry at having done "free labor." (You might say, "but the GPL!," but being tricked into a contract under false pretenses can legally invalidate contracts, making the GPL-license for his contributions potentially invalid.) Mojang panicked at the takedown and a ton of developers abandoned ship, working on Bukkit's community fork Spigot. Mojang then 2.5 months later brought Bukkit back and that time really just threw it into the community's hands. The result of the aftermath was everyone had restructured around Spigot, based on Bukkit; and Mojang wasn't going to deal with APIs and open-source stuff for a long while afterwards. (They did later try purchasing parts of OptiFine, but the developer wanted compatibility to be all-or-nothing which Mojang wouldn't accept.) https://www.reddit.com/r/Minecraft/comments/6g1tqn/psa_no_bukkit_isnt_dead_and_it_basically_never_was/ https://www.reddit.com/r/Minecraft/comments/6g1tqn/psa_no_bu... https://blog.jwf.io/2020/04/open-source-minecraft-bukkit-gpl/ https://blog.jwf.io/2020/04/open-source-minecraft-bukkit-gpl... https://www.reddit.com/r/Minecraft/comments/2fk5nn/my_response_to_vubui_mojang_and_the_hundreds_yes/ https://www.reddit.com/r/Minecraft/comments/2fk5nn/my_respon...
- whalesalad 4y agoIn other words, grass roots open source development is hard :)
- perihelions 4y ago- "Folia, which is a fork of Paper, which is a fork of Spigot, which I believe was originally a fork of Bukkit" We're not old, the forks just fly by so very fast
- gjsman-1000 4y agoYeah, but Bukkit is ancient - so ancient, upstreaming is basically impossible and even discouraged because it would break compatibility with the few people who still need it. It's really only maintained as a reference architecture by Spigot because it helps with initial Minecraft release reverse-engineering. Spigot, meanwhile, is run by one developer who only posts his code within a few weeks of a Minecraft launch after working on it completely privately. Paper has examined taking over that role, but has decided against it for the time being until they are ready to fully take on that job because... it's not an easy job and he's basically mastered it (Paper would be much slower if they were doing it themselves for the first few releases). Also he runs the plugin repository and Paper isn't ready to launch their own yet. Also, Spigot and Bukkit have this ancient API that not many plugins use, but a few do... and it'd be nice to not break them. So Paper is the first remotely reasonable downstream with a big open-source community for it, which then makes it more like... one fork. And even though Paper is technically reliant on Spigot (and Spigot on Bukkit), Paper can really cut them off whenever convenient. And there's a good reason to make Folia a fork - the way it works under the hood is very different than a traditional Minecraft server, and it basically breaks all the Spigot and Paper and Bukkit plugin compatibility.
- jeroenhd 4y agoMinecraft server forks almost seem to develop as fast as Javascript toolkits. It's impressive, in a way. I think this has to do with the way that ultimate, true upstream involves the closed source Microsoft components. None of these servers seem to be built from the ground up, each extending an existing package (closed or open source) in one way or another.
- bakugo 4y agoWhat exactly would the point of an official modding API be, even? Anything Mojang could provide would inevitably be inferior to what can be done by directly modifying the game's code via existing third party mod APIs.
- nightpool 4y agoFolia is a complete re-write of Minecraft's core game loop to support "true multithreading and regionalized ticking" ("ticking" being the term for a single turn of Minecraft's core game event loop). Even the most ambitious official modding API in the world would never ever contemplate supporting something as thorough and involved as this. Minecraft does in fact have an official plugin API, known as data packs—they're obviously more limited then the ability to completely rewrite the game's code, but they can create new blocks, items, entities, customize their behavior, change worldgen, add achievements, and more. Here's an example of a "bendy pistons" mod that was built entirely using the datapacks API, with no mods required: https://www.youtube.com/watch?v=_Wk7ZUYkrpI https://www.youtube.com/watch?v=_Wk7ZUYkrpI (and 7 more features from the original video: https://www.youtube.com/watch?v=2LRvq6H11bs https://www.youtube.com/watch?v=2LRvq6H11bs)
- Aeolun 4y agoOh my god, bendy pistons are ridiculous. Thanks for that.
- sobkas 4y ago> The modding/custom server situation with Minecraft is so exhausting. I can't believe after all these years there is still no official modding or plugin API. > So now we have Folia, which is a fork of Paper, which is a fork of Spigot, which I believe was originally a fork of Bukkit. Nice. I guess people don't want to upstream patches for some reason. Time and effort you put into upstreaming patches is almost always wasted, either you get it included and then have to maintain it forever or it will get removed due to bitrot or developers will just say no and all your effort was wasted. Don't forget that developers might also want to share their opinions about you based on your commit. There is no better way to build community than saying that no one cares about you inside of it.
- bscphil 4y ago> I guess people don't want to upstream patches for some reason. My insanely biased impression based on my experience is that this is just the norm for Windows-centric development. Generally you have no expectation of being given access to source code at all; if software is free then it's "freeware", but for whatever reason the developers don't generally want you modifying it. So many power user tools just died out when their developers quit working on them because the source code was not available, at least not under a free license. I remember back in the early days of Minecraft (beta+ era) I'd just go on the forum and download random jar files uploaded by whomever, and mod Minecraft with them. Nobody asked for source code. Or thinking about doom9, for years and years the norm for tool development was that you'd download a random exe from someone on a forum, with a single thread for "support". If they disappeared one day, that was it. That's also the way a bunch of Android stuff is/was developed on xda-dev. The more Linux-centric set of norms has expanded outward into a pretty robust open source ecosystem, but it's by no means the universal norm for free (as in beer) software development. Even in cases like Paper / Bukkit where the source is free, building the software yourself is very much not the norm, often is not even supported.
- Cyphase 4y agoI've gotten the same impression. So many Windows tools are given away for free, no hint of ever monetizing or anything, just a hobby / service by the developer, and yet no code is provided. I remember hearing an explanation from one such solo developer with a relatively recognizable brand that he doesn't want people to be able to easily replicate his tools with added malware / adware. But I feel like download sites have been doing that for years anyway. He has released at least one tiny but useful piece of code as public domain. Also he does have one product that charges for. I suppose one potential reason to keep code private is to maintain more control of the free tools for purposes of branding and bringing attention to the pay tool? That's just a thought.
- imtringued 4y agoFolia most likely breaks compatibility with plugins so forking is to be expected. If you ignore server plugins, the only effort in actually trying to build a modding API is Quilt and it's Quilt Standard Libraries, which you probably guesses further splits the modding community into Forge, Fabric and now Quilt but that is on the Forge and Fabric guys for not even trying.
- irascibeth 4y agoBukkit was in fact preceded by hey0's mod, the project of a 15 year old kid who was largely responsible for bootstrapping the entire line of Minecraft server mod APIs. After all these years though it is extremely exciting to see this advancement, I'd expect this to allow extremely large servers. Makes me want to start playing again ^_^
- LelouBil 4y agoMojang releases the deobfuscation mappings for each version nowadays.
- gjsman-1000 4y agoThis is, for those who know a little bit about Minecraft hosting, extremely impressive stuff. Stuff I never thought I would see for years. Minecraft has been notoriously single-threaded and breaking it apart into regions, even though discussed for years, has always been a crapshoot whenever attempted - so getting it even close to finally possible in a reasonable and self-adjusting way is astounding.
- cooljacob204 4y agoI wonder how it handles the seams between the regions. How does a player interact with another player on the other side of this border.
- gjsman-1000 4y agoIf I read the documentation, it appears to be that it will expand the region to accommodate both players or merge the regions, so that you will never have a situation where two players from different regions are visible to each other.
- mkeedlinger 4y agoMakes sense, also very cool.
- TOMDM 4y agoSo worst case is you're essentially back to single threaded. With a slight overhead of tracking connected regions. I guess that makes sense. I wonder what this looks like on practice, what kinds of play styles will see the biggest boost from this. I also wonder how this plays with things like hyper long range TNT/Ender pearl cannons. Entities are their own pile of worms, but distant regions can suddenly influence each other in some cases.
- Gigachad 4y agoMy hope is this allows expanding the player count on servers like 2b2t. The normal play style here is small groups of players spread out massively over the map. It would be incredible if you could effectively have each user on their own thread and mini server with it all just seamlessly working. Would be outstanding to see a server with something like 4000 players online all on the same map.
- colingoodman 4y agoMy mind went to 2b2t as well. Unfortunately players still tend to congregate somewhat around spawn. Perhaps Folia lets them raise the server cap to a degree, but it appears here that you still can’t have more than a couple hundred players in the same area. Allowing more than that limit on the server invites a mess in the event enough of those players end up in the same place for whatever reason.
- anticrymactic 4y agoI have thought about this a lot already and basically came to exact same conclusion with the "bubble" system. But don't the bubbles have to tick at the same time to be 100% vanilla? They still compute their ticks independently but only "roll over" together. Which would still increase performance extremely, don't get me wrong. But will not stop lag machines, since global mspt would be the min of all bubble mspt's. Although it would make tracking lag generation a lot easier.
- DefineOutside 4y agoStuff is already used to ticking at a different pace because only parts are loading in the world at a time. Lag machines don't work as it will only lag your region, not all regions have to tick at the same pace.
- nightpool 4y agoRegions would need some level of synchronization to allow the merging / splitting algorithm to work, I think is GP's point.
- the_gipsy 4y agoI wonder how "bubble merging" would look like. Some hiccup? I used to play on civcraft and at some point they decided the lag was unbearable and restarted the map. The new map was a number of map shards connected with portals (bungee?). It fixed the lag but getting around got more cumbersome.
- TOMDM 4y agoYeah would you roll back the smaller bubble and propagate the larger one? Try and merge them? What happens with 3 bubbles? Is the worst case here much worse than single threaded? At the moment there are some weird nuances around update order based on world location and direction that the more technical players have learnt to work with/around for the most part. Predictable update orders are a big deal for those super technical players.
- binkHN 4y ago> It is recommended to have a minimum 16 physical cores in order to fully benefit from Folia. I didn't expect this. I wonder how many Minecraft servers are using more than 16 cores in production today.
- Gigachad 4y agoI'd assume minimal numbers since the existing software is mostly single threaded. The comment seems to suggest that to make use of the new multi threading, you have to first have some large number of threads to actually use.
- infogulch 4y agoSo does time advance differently in different regions? If two regions start redstone clocks and one of them has a lag machine, will they desync? Will it get to a point where one is day and the other is night at the same time? What happens when the regions are merged or a player crosses between them?
- colingoodman 4y agoIf I understand correctly, there’s still a global thread for things like day/night cycle, and then region threads handle their own redstone ticks and can indeed fall out of sync.
- infogulch 4y agoI see. I wonder how they sync up again and how dynamic the splitting and joining is. I guess long distance instant wire would be the worst case scenario for this system.
- skerit 4y agoThat's nice... when you like to use the Spigot API (which, of course, is used by the majority of servers) Something like this for Fabric would be great though. Server-side modding with Fabric offers a lot of opportunities you can't do with the Spigot API. In fact there have been multiple mods that tried to implement this for Fabric or Forge. It's keeping everything in sync with Mojang's codebase that's the hard part. Right now I just use DimensionalThreading. Putting different dimensions on different threads is quite easy to do.
- FastEatSlow 4y agoThis has been attempted by many people in the past, including myself, but I'm sure that if anyone is to have a shot, it's SpottedLeaf. He's done a lot for Paper performance and is very familiar with the workings of Minecraft.