13 ms·
SpacetimeDB
- matthewfcarlson 2y agoA relational DB + serverless compute + suspiciously fast updates
- thefourthchime 2y agoHaha, yes very suspiciously fast!
- _huayra_ 2y agoThis is indeed interesting, but FYI this is a BSL-covered project with somewhat onerous times: only one instance per service (i.e. don't scale out, if I understand correctly) [0]. [0] https://github.com/ClockworkLabs/SpacetimeDB?tab=License-1-ov-file#readme https://github.com/ClockworkLabs/SpacetimeDB?tab=License-1-o...
- weego 2y agoThis looks really interesting, and bitcraft looks cute. I did hit a personal bias: when I saw "maincloud" as a phrase to do with pricing my heart sank because my brain went "they've wedged in some crypto shit!". Turns out no, it's just a naming thing for hosting credits.
- pmx 2y agoI feel like they've been inspired by crypto for the way they handle billing. They sell you "Energy" which you use by running on their infra. Seems like a needless abstraction to me.
- anentropic 2y agoDoesn't Snowflake do something similar though?
- cloutiertyler 2y agoThey do indeed.
- Tomuus 2y agoAnd Vercel's "compute units"
- cmrdporcupine 2y agoThere was a lot of crypto-adjacent talk in earlier versions of their marketing and product design. Without it actually being anything actually crypto-adjacent. Analogies drawn to ethereum etc concepts, I guess because they figured somehow there was an audience that would get those references and it would allow them to understand the product better? I have no idea why. I was hired to work on very early versions of it, and left very early on in part because I wanted nothing to do with that angle. IMO it's a liability.
- cloutiertyler 2y agoIt had nothing to do with marketing, it was because conceptually stored procedures are similar to smart-contracts and share technical aspects. We thought engineers would find the similarities interesting. Instead they immediately assumed we were shilling tokens, so we pulled it down. Kind of sad that people can't see past that, but I also understand. It is what it is.
- cmrdporcupine 2y agoHey Tyler; looking back I think maybe a better language than "reducers" etc would have been to just something analogous to the ECS term "systems", which I understand to be "processes that operate on entities / components" and always seemed overly general and non-specific to me but at least is familiar to game programmers operating in data driven design. You may recall I was partial to the term "verb" :-)
- cloutiertyler 2y agoI think that's very fair, we've got quite a lot of momentum behind reducers at this point though. That doesn't make it a good, name but it is a sticky one. "mutation" may also have been a good choice?
- cloutiertyler 2y agoOh it's you! Sorry cmrdporcupine! I didn't see.
- ksynwa 2y agoIs there a place where I can read about what technologies online games use? As a terminal gamer I am ashamed to say I have no idea what goes on behind the scenes.
- spmurrayzzz 2y agoAssuming you're referring to the backend side of things for online/multiplayer games, working through the Hathora tutorials in the docs can be a good first introduction to general idioms related to backend systems architecture: https://docs.hathora.dev/#/builder/ https://docs.hathora.dev/#/builder/ From my past consulting experience though, the distributed systems side of games is all over the place. Designs tend to need to be customized for each games needs, which usually directly influences which technologies are chosen.
- LarsDu88 2y agoA server DB monolith that explicitly doesn't scale? You put some of your game logic on the server and some on the client with no clear dividing line? When PoE2 uses PostGres and Eve Online uses SQLServer? Also doesn't tackle the main challenges of online multi-player games (physics sim rollback)? I'm sorry but there might not be a good market for this.
- cloutiertyler 2y agoIt scales much the same way that Erlang/BEAM scales. Each database is an actor in a distributed actor system. That's how we operate BitCraft, however we haven't released inter-module communication (IMC) to the wider world yet.
- deleted 2y ago[deleted]
- ksec 2y agoPrevious Discussions https://news.ycombinator.com/item?id=43593913 https://news.ycombinator.com/item?id=43593913
- paulluuk 2y agoYouTube keeps recommending SpacetimeDB to me even though I have never made a videogame. Now I'm seeing it on HN too. Are there any game devs here who can comment on how useful this actually is? The pitch seems pretty cool but I can't tell how much of it is just good marketing.
- matthewfcarlson 2y agoNot a developer affiliated with the project. But here's my take: The problem is that the proof is in the pudding. Some of their claims feel outlandish. Imagine writing your game with SQL queries. Rather than crafting update packets that only contain relevant game info you just do `select * from players where abs(player.x - current_player.x) + abs(player.y - current_player.y) + abs(player.z - current_player.z) < 100^2` and it just updates automagically 60 times a second. Running physics sim in SQL stored functions sounds insane but that's exactly what they're advertising. They say it's due to in-mem DB optimization but I'm not sure how they horizontally scale that (they allow you to run standalone but any scaling requires their cloud product)
- cloutiertyler 2y agoI mean you can download it and give it a try! For BitCraft we scale it with IMC (inter-module communication). It works very similarly to Elixir.
- Rohansi 2y agoIt's in closed alpha so there's a limited number of players but I am curious to hear more about how BitCraft is scaling out. Is it a single, seamless world for all players? How much scaling is currently required? How is data replicated when horizontal scaling comes into the picture?
- deleted 2y ago[deleted]
- jackb4040 2y ago
- jackb4040 2y agoBest comment from last time this got posted: I kept on reading expecting to see motion prediction, multiversion, or similar given the name and focus on games, but no. This is a totally normal database, designed for low latency and with support for WASM stored procedures. You can host your own server or they will rent you one. Don't get me wrong, this looks very nice. It looks like a solid building block for persistent worlds in multiplayer games. You'll just have to do your lag masking netcode yourself. https://news.ycombinator.com/item?id=43593913 https://news.ycombinator.com/item?id=43593913
- neuroelectron 2y agoWasm? What a great idea.
- gafferongames 2y ago[flagged]
- cloutiertyler 2y agoWe can and plan to handle the complex lag masking netcode as well. We've just focused so far on games that don't need very sophisticated lag compensation
- gafferongames 2y agoPlanning to support complex lag masking netcode as a TODO is not a good sign.
- pphysch 2y agoSeems too biased and harsh. The best lag masking and clientside prediction systems I've experienced as a player are those that are closely integrated with the idiosyncratic gameplay mechanics. I would never expect SpacetimeDB to have a one-size-fits-all solution to this, nor for it to be the central focus of their product.
- ChrisArchitect 2y agoDiscussion last week: https://news.ycombinator.com/item?id=43593913 https://news.ycombinator.com/item?id=43593913
- hanska 2y agoi've tried it, it works, it's nice :) got a mini mmo running with several people at once running around
- thefourthchime 2y agoYouTube suggested the video on this. It seems pretty interesting, but I'm curious how well it works in the real world. https://www.youtube.com/watch?v=kzDnA_EVhTU&t=753s https://www.youtube.com/watch?v=kzDnA_EVhTU&t=753s
- aeturnum 2y agoSpacetimeDB fits into a genre of tech that I want to call "universe brain reorgs" - structural re-arrangements that might unlock wild performance increases. The challenge with these is that they require devs to re-align their thinking and it's hard to disentangle if "the problem" is that the technology isn't being fully embraced or if the promises of the tech don't work for your use case. Other techs in this category have seen a lot of success. I'd classify Elixir and Node as being similar in that you adopt a stack to get access to some rare guarantees and also accept new or exotic limits. I think these things tend to rely on having clear-eyed visionaries out in front, able to show off their strengths in open source. Without a successful project it's hard to believe the claims. I hope clockwork labs is able to deliver their example game (it seems high risk because, even if the game is a technical wonder - what if it isn't fun?!).
- cloutiertyler 2y agoWe are definitely inspired by Elixir/BEAM. IMC (inter-module communication) is directly inspired by it. It is high risk, but I think a risk worth taking.
- aeturnum 2y agoAbsolutely! I work in Elixir and I feel very good about its tradeoffs - and a lot of being good with the tool is knowing when to not use it. This tech is very much out of my wheelhouse but it seems very cool and I hope y'all find a nice fit!
- MasterYoda 2y agoWhich tradeoffs does elixir have?
- aeturnum 2y agoWell, it totally gives up memory mutation. Each value is final (under the hood). So you lose an entire category of optimizations and an entire category of errors. You also gain some new optimizations you can do because values never change. In general that makes it worse at tight loops where you really want optimal machine code and better at handling heterogeneous calls like HTTP requests.
- beders 2y ago#[table(name = person, public)] pub struct Person { name: String, } No, just say no. Do not conflate your relational database schema with however you treat that data in your app. You want explicit control, not annotation magic.
- robmccoll 2y agoThis sounds like a fundamental disagreement with using ORM. Are you advocating for always writing queries (not excluding the possibility of a query builder) over treating database rows as application-side entities that can have CRUD operations performed on them / persisted?
- bob1029 2y ago> SpacetimeDB wasn't built in a vacuum. It's the system powers our own large-scale MMORPG BitCraft. We designed it specifically for the performance requirements of games. That means extremely low latency (~100 us/Tx) and extremely high throughput (~1,000,000 Tx/s). If you are trying to build a new MMO that scales to the moon, you might want to take a look at how the existing market goes about things. Take World of Warcraft for example. You could probably put a $20 MySQL VPS behind a realm and have it reliably persist state with how authoritative and infrequently updated the clients are. Being clever with what you can trust the client with and how the server reconciles events is where I think you will find most of the scaling hacks.
- drivingmenuts 2y agoIs this similar to the way HeroEngine works? IIR, that was basically a big database server (that wasn't compatible with anything else).
- hesdeadjim 2y ago$0.28/gb for net traffic on the Maincloud is pretty brutal.
- gafferongames 2y agoYeah this alone makes this a total non-starter. Do the math at 100k CCU
- cloutiertyler 2y agoWe want to dial these numbers in, but we have to err on the side of too high, because it's easier to lower prices than to raise them. We want to bring it down as low as we possibly can.
- kriper 2y agoI don't really understand how you would write a game server using this tech. Lets say you need to simulate physics, pathfinding, animation etc on server. How one would integrate all of this? In regular world you just use unreal/unity headless mode that includes all of this but using spacetimedb requires ditching the game engine and reimplementing everything from scratch?
- nerflad 2y agoJust put it all in stored procedures. What could go wrong?
- janderland 2y agoForgive my ignorance, but why would you implement physics and animations on the server? I feel like you can get away with basic positional data and do the rest of that on the clients.
- kriper 2y agoYou need physics in order to make movement server authoritative. Otherwise player will be able to fly around and ignore collusions. You might need animation state in server if you are making a fps shooter and want to calculate raycasts properly.
- calebm 2y agoMinor critique - the logo is almost exactly the same as Grok's
- cloutiertyler 2y agoOurs came first :/
- xixixao 2y agoThe “server in DB” is also the Convex model, but Convex is more tailored towards JS full stack apps: https://www.convex.dev/ https://www.convex.dev/
- vivzkestrel 2y agoHow do you integrate this with unreal engine 5.5
- cloutiertyler 2y agoUnreal (and C++) support is coming!
- deleted 2y ago[deleted]
- arittr 2y agoam i the only one who hoped (and thought from the logo) that this was by the Spaceteam devs?
- R4tY9jQ2 2y ago[dead]
- MPSimmons 2y agoThis feels like very old-school java client/server architectures. >This means that you can write your entire application in a single language and deploy it as a single binary. No more microservices, no more containers, no more Kubernetes, no more Docker, no more VMs, no more DevOps, no more infrastructure, no more ops, no more servers. This is an abstraction layer that will end up being applied on top of most or all of those technologies. You can't hide from the fact that the database instances need to run on something.
- wy35 2y ago>SpacetimeDB wasn't built in a vacuum. It's the system powers our own large-scale MMORPG BitCraft. I checked out BitCraft and it's not released yet. Don't know if I'm being unfair here, but isn't this a bit misleading?
- bsbsnjzjkkdjk 2y ago[flagged]
- pianoben 2y agoThe website and naming (and product space, sorta) remind me a lot of Parallel Universe. Any direct inspiration there, or shared personnel?
- cloutiertyler 2y agoI haven't heard of Parallel Universe. What is it?
- pianoben 2y agoThey were a YC company from a while back: https://techcrunch.com/2012/07/09/parallel-universe-spacebase-will-make-for-new-matrix-style-mmo-games/ https://techcrunch.com/2012/07/09/parallel-universe-spacebas... They had an MMO-targeted database called Spacebase, and a bunch of awesome JVM technology. Their founder actually works for Oracle now, if I'm not mistaken.
- alexpadula 2y agoVery cool, keep it up. Always good to see innovations, creativity and curiosity in the database space.
- alexpadula 2y agoOne question I have is what is the plan on implementing full SQL? I see currently you're only implementing maybe 15% of SQL92. Are you implementing your own SQL-like language?
- alexpadula 2y agoAnother few pieces here listening to your video. https://www.youtube.com/watch?v=kzDnA_EVhTU https://www.youtube.com/watch?v=kzDnA_EVhTU You mention it runs in process? How does this occur? You mention running cached in memory tables. So you're caching on at the application level, not the storage level? There's a few things with the video I find misleading. You mention 400ms at best to parse, plan and execute a query; That's not true, what about the type of query, etc. Overall I can see it's still early, which is ok. I will review the code and see what else I can find.
- alexpadula 2y agoOne more question I have is how are you handling the cluster architecture. I don't see that on the website. You mention on the pricing a cluster and replication but scaling horizontally is rather important and we definitely want to know how you're achieving scalability.
- 3371 2y agoI wish you success! And hopefully you don't end up disappearing like SpatialOS. They tried very hard promoting its capacity for large scale MMO, even published a game (Scavenger IIRC?) made with it.
- drwiggly 2y agoThis looks to be an in memory db, with a wasm runtime to host domain logic. The hand wavy part was how do they handle scale and clustering? Are we sharding the data our selves (atm it seems so.). This is nice and all but the hard part is replication and consistency in a distributed database. In memory has its uses, also disk backed tables can have their uses. Pretty much normal databases already do this, just writing domain logic in stored procs is kind of annoying. I'd imagine embedding sqlite in your binary using memory tables is equivalent at the moment. Well you'd have to write code to publish table updates to clients. I suppose it has that going for it. I've seen some hand wavy docs about clustering but nothing concrete.
- darkbatman 2y agoIs it similar to rethinkdb, I remember using RethinkDB for similar use case for live queries running directly on database layer back in 2016.
- suspended_state 2y agoI found the idea of polling DB queries interesting. They basically took what is done with HTTP and pushed it down the stack directly in the DB server, eliminating the extra SQL round trips one needs to make server-side in order to provide updates. Neat!
- Geee 2y agoDoes this make any sense in single-player games, where the db would run locally on the user's machine? Does it have any benefit to run all game state through a local db? I'm not a game dev, but I'm imagining that if I'd write a custom game engine, I'd have to write just the "frontend" aka. graphics engine if I'd have a robust state manager (+tooling) as a separate piece of software.
- hansvm 2y agoSome algorithms are much more easily represented as SQL or that plus a little wrapper code. E.g., somebody re-wrote E-Graphs Good using SQL as their primitives, and that viewpoint made a bunch of other features and optimizations obvious. Stated somewhat differently, do you want to write the algorithm to do a 3-connected recursive join in linear time, or do you want to slap the join into a DB and move on with your life? For a concrete example, a DB could make the Entity-Component architecture even simpler to use than it already is (give me all the npcs in this radius who haven't attacked in the last second, have a distance attack, and are facing the right direction). Would it have performance pitfalls if you weren't careful? Almost certainly. It could be a fun way to structure a game though.
- sitkack 2y agoIf you know all of your queries ahead of time, then compiling them efficiently has already been shown to be effective. Noria https://github.com/mit-pdos/noria https://github.com/mit-pdos/noria https://www.youtube.com/watch?v=kVv9Pik6QGY https://www.youtube.com/watch?v=kVv9Pik6QGY https://thesquareplanet.com/research/ https://thesquareplanet.com/research/ Materialize/TimelyDataflow https://scholar.google.com/citations?hl=en&user=YYJ3aycAAAAJ&view_op=list_works&sortby=pubdate https://scholar.google.com/citations?hl=en&user=YYJ3aycAAAAJ...
- hansvm 2y agoOh, sure, the complaint wasn't about SQL per se, but that making complicated things easy to write isn't necessarily a win. Even if a big query perfectly represents the problem, and even if you can use compiled SQL to match hand-written efficiency, it still might not be the optimal way to approach the problem in the first place. If you have experience and plan for that it needn't be bad at all, but I think it's easier to write performance footguns in SQL (the "right" data model is more prescriptive, even with some number of years of experience, and is less likely to be tuned to your actual problems) than with traditional code, so I'd be quite surprised if the majority of game developers (even those with some degree of success currently average <3 YOE) wouldn't run into major performance issues with a SQL-based engine, even a good one.
- 999900000999 2y agoSeems like a neat idea, but for the love of Jesus Christ offer me a flat rate billing solution. Or at least an option of doing that. This seems like the type of thing that will accidentally burn through all of your credits if you code something wrong, and then you're going to have to beg their tech support to refund you. I had an idea last night for an RTS like game where the units can only be controlled via radio commands which are only sent every minute or so, but this feels like a very strange way to do it. Not to mention hey it's not open source, but after 4 years will open source everything ? Why, why wouldn't I just want to pay render or something $7 a month for a normal server versus this ? And use an actual open source stack.
- Cieric 2y agoSpacetimeDB is open source[1], the open source version doesn't currently have all the features of maincloud, but most of those features are solely for scalability. I've done all of my work on my own local servers. I would even suggest paying for your own server up front until you're ready to scale out. I believe the 4 years is when the license changes to a more open license. And the scalability features on local are coming, they're just not here yet. [1] https://github.com/ClockworkLabs/SpacetimeDB https://github.com/ClockworkLabs/SpacetimeDB
- 999900000999 2y ago>I believe the 4 years is when the license changes to a more open license. Why not use something with a vastly friendlier license. Most companies don't want to play the GPL dance when alternatives exist.
- truth_seeker 2y agoIts areally great effort, but i would stick with PostgreSQL with PL/SQL procedure and tons of extensions. Lowering down the TRSACTION LEVEL, Async transctions on disk backed tables and UNLOGGED Tables can help you go a long way. Also, in next major version (v18) of PG, they are trying to implment io_uring support which will further drastically improve the random read/write performance.
- whizzter 2y agoInteresting, been tinkering on a similar project of my own. Main similarity is also combining database/server-simulation even if it seems like some parts of the design might differ. Gonna take a deeper look when I get home later.