14 ms·
Reflect – Multiplayer web app framework with game-style synchronization
- btbuildem 3y agoIf you want true concurrency, in an elegant format, with a lot of the common problems abstracted away -- use Erlang. They solved this 30 years ago.
- szundi 3y agoSomeone should bring those logics to js and java then. Ehm, please.
- OkayPhysicist 3y agoWhat makes Erlang (and Elixir) great isn't just the actor model, it's the language-level commitment to it. Things that would mess with concurrency (say, race conditions on the reading and modification of a global variable) are simply unexpressible in BEAM languages.
- bnchrch 3y agoSomeone has! https://github.com/untu/comedy https://github.com/untu/comedy However to the below point, Bolt ons are never as good as baked ins. This is especially true for concurrency.
- jitl 3y agoI don’t think Erlang solves this problem: multiplayer editing of data structures while preserving user intent. Specifically does BEAM build in a data structure for preserving user intent while multiple users type/delete/apply styling in the same text field?
- simonw 3y agoErlang is a programming language. Reflect is an implementation of a hosted service that helps user's build realtime collaborative interfaces. Reflect could be built using Erlang. Erlang could not be used to replace what it does.
- equilibrium 3y agoIs this much different from croquet.io?
- holoduke 3y agoWhen will it finally be possible to use UDP protocol within browsers? TCP got way too big overhead. A multiplayer engine in which you send 30 messages per second for 128 users is what i need. Not possible with websockets withput having super computers.
- ermir 3y agoWebRTC makes this possible, but server implementations are quite difficult to set up. I recommend you check out Pion (Golang) as a good implementation that's also easy to deploy and integrate with Node (presumably your game server is implemented in JS).
- hnav 3y agoWhy does TCP have so much overhead? 30 messages that you don't batch is 30 packets either with TCP or UDP.
- holoduke 3y agoTCP got a much higher ping delay. For real fps like multiplayer experience you definitely need UDP. Packet loss should be ignored, not retried.
- meheleventyone 3y agoThis isn’t as true anymore but it’s definitely best practice to build on UDP. Games built on UDP normally have several different policies including the ability to send messages that guarantee delivery even if packets get lost.
- hahn-kev 3y agoHTTP3 is your friend here
- anonzzzies 3y agoBesides trystero which is great but depends on things I cannot get into our company, anything open source? I cannot get this into our company either as it's vital infra and as such needs to be replaceable with open source at will. We have that as a rule. The only things that are exceptions are necessary evil; payments & cloudflare (bot-fight specifically).
- wim 3y agoVery elegant solution, and neat demo page! I'm curious what trade-offs you looked at regarding sending state vs mutations from the server back to clients. As the server is authoritative, I imagine it could also send mutations to the clients in the order in which the server has determined parallel operations need to be resolved (and adjust mutations as needed to resolve conflicts or permissions issues). Clients would then need to rebase any pending local mutations, so that would involve more logic vs the data sent for mutations most likely being smaller than the "full" state of certain objects being updated?
- woggy 3y agoSo comparing this with Replicache, basically same client but with a real-time managed server?
- tlhunter 3y agoUnfortunate that's it's named after a JavaScript built-in.
- deleted 3y ago[deleted]
- aboodman 3y agoHey hacker news! I'm one of the people behind this, happy to answer any questions.
- woggy 3y agoCurious what your thoughts are on ElectricSQL?
- aboodman 3y agoIt seems really interesting. The key thing to understand is that there are deep architectural choices made in each of these systems that really influence what you can do. There is "mechanical sympathy" with certain kinds of applications. ElectricSQL is a distributed database. It's not going to run anywhere near 60fps with tons of users, because it's not running in memory on the server like Reflect/PartyKit/Liveblocks do. OTOH it will allow you to filter/query interact with much more data at the same time than Reflect/PartyKit/Liveblocks do – Reflect has a limit currently of 50MB per room and it's not practical to have dozens of rooms open at one time to get around this. Systems in the Reflect room/document based model are well suited for applications where you primarily interact with one "document" at a time and you want that to be as realtime as possible. Figma is the canonical example. You want the entire document to move together at 60 FPS, completely fluidly. It's not going to be possible to do this well in the ElectricSQL model IMO. Spreadsheets, presentations and documents are other examples. Systems in the ElectricSQL model are more suited for applications where you are interacting with lots of documents at the same time. Think CRMs, bug trackers, etc. Of course real applications are messy. Even document editors always have a dashboard where you can at least see all your documents at once. And CRMs of course have a detail view that looks like a document editor. Both types of systems are going to track toward the other to support the needs of real applications. HTH!
- coreyh14444 3y agoAaron is humble, so I'll do it for him! He created Greasemonkey and has had a hand in several groundbreaking browser innovations over the years.
- jckahn 3y agoI strongly endorse Trystero (https://github.com/dmotz/trystero https://github.com/dmotz/trystero) for enabling P2P communication in web apps. It’s open source and leverages public infrastructure for matchmaking.
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- vitalurk 3y ago[flagged]
- alexanderking 3y agoYou killed ants with a magnifying glass when you were a kid, didn't you.
- vitalurk 3y agoJust looking for ways to break things, so that i can help rebuild them stronger. And no, i ate them.
- chrisweekly 3y agoInteresting alternative to CRDTs, "Transactional Conflict Resolution". Bookmarked.
- deleted 3y ago[deleted]
- jakelazaroff 3y agoIt looks very slick, and I'd probably be interested in this if I were building a centralized local-first app. But I think this serves a fundamentally different use case vs. CRDTs. Reflect says "Your mutation code runs server-side and is authoritative" (emphasis theirs). - If you can have an authoritative server, CRDTs come with unnecessary restrictions and overhead. Reflect presumably gains a lot of efficiency by loosening those constraints. - Conversely, if you want clients to collaborate without a central server, you can't use a service like Reflect, since there's no server to run your conflict resolution logic.
- aboodman 3y agoAbsolutely – CRDTs have some unique benefits, this among them. I recommended a potential user to CRDTs a few weeks ago. But most apps people build today do in fact have a central server. And by leveraging that you can get some really nice benefits.
- jakelazaroff 3y agoYeah, it's interesting to me how from a user's point of view both technologies do something similar, but from an architecture point of view the use cases are mostly disjoint. I totally agree that something like Reflect makes more sense for most apps people build today. Anyway, congrats on the launch! Reflect looks great. I'm really excited to see the building blocks emerge for local-first software.
- matlin 3y agoWhile I do love the simplicity of this approach on the surface. Some of the complexity gets brushed under the rug. Firstly is rebasing these operations in this style of system takes a bigger hit on performance than I expected in my experience. Sure, managing counters and doing integer math can consistently run at 120FPS but more complex operations can really bog down the system when a lot of users are interacting with it once. The second challenge is that when operations fail (e.g. for authorization) undoing that operation isn't always straightforward. You either pop the operation and replay ops from a earlier snapshot (which necessitates storing all ops in order), writing an inverse operation manually, or having the server send the entire state back to rebase your unconfirmed ops on which can also end being expensive. In general, the challenges of this approach are true for all operation-based CRDTs too. State-based approaches are a good deal simpler just tend to over-send data over the network.
- aboodman 3y agoHey Matlin, it's true that this approach is harder to implement, but that's a one-time cost, and it's one we take on for our users and they don't have to worry about.
- matlin 3y agoCan you share how y'all made it fast? Obviously it can be made fast because multiplayer online games are living proof but I'm curious what it takes to get there or what's missing in my mental model.
- aboodman 3y agoNot really any magic, just a lot of careful engineering.
- ramses0 3y agoWhat's your opinion on using this library as a base for (eg) a simplified "world-of-warcraft" clone? I've done a lot of thinking (but not coding) around "simply replicating state" being the right answer to interconnected apps. The degenerate case is to model everything as a git transaction log, but be able to run it "faster" than really using git. It's very intriguing to hear your discussion of "branching, sequentialization, etc..." because that seems like the right way to go. ( edit: and implicit _physics_ vs. state transfer in some cases... it's cheaper to send nuke@[x,y,z] instead of the literal calculations and changes to state that single action may imply ... physics in games maybe being gravity and ballistics, but 'business-physics' might be update/select/modify, etc. ) There's a few interesting optimizations/tradeoffs in some of the "real" multiplayer networking libraries: https://torque-3d.readthedocs.io/en/latest/script/network.html https://torque-3d.readthedocs.io/en/latest/script/network.ht... ...eg: Unguaranteed, Guaranteed, Guaranteed Order, Most Recent State, Guaranteed Quickest (and each client has it's own "camera" / perspective against the "scene", where +180 might be "need it now b/c I'm looking at it", but -180 might be "meh, need close-to-the-latest-state before I turn around and look at it"). I'm rooting for you b/c "multiplayer + physics" seems like the right answer (eg: the ghost of couchdb), but no one seems to have (yet) cracked the semantic code as to how to describe "as fast as local, but synchronized with $THESE tradeoffs and $EXCEPTIONS".
- pensono 3y agoThis technique is pretty similar to Operational Transformations, except that it requires an authority for serialization and must re-apply any state which is out of sync.
- aboodman 3y agoIt's not the same thing. OT works by having a discrete set of operations that are known to the system so that they can be reordered properly server-side. Reflect is based on Server Reconciliation: https://www.gabrielgambetta.com/client-side-prediction-server-reconciliation.html https://www.gabrielgambetta.com/client-side-prediction-serve.... The critical difference is that Reflect doesn't need to know the operations up front. That is what allows developers to provide their own operations, or as we call them "mutators".
- tablatom 3y agoI'm using Replicache in a big project - this simple fact is a big deal. It's basically the same coding model that all the front-end frameworks have gravitated to (central store, unidirectional dataflow, mutators separate from data access) but it all stays in sync across the distributed system.
- ggambetta 3y agoWell hello there!
- joshelgar 3y agoHey Aaron, cool product! Well done so far. Curious why you think no-one has done TCR before? Also, any insights on why you're bullish on the "multiplayer web"? Will be cool to see example docs / repos as they come out!
- aboodman 3y agoThe examples and docs are live here: https://hello.reflect.net/ https://hello.reflect.net/ TCR is just a small generalization of what the game industry has been doing for awhile. So in that sense, we are not at all the first. But I'm honestly not sure why it's not been done on the web yet. To me, it's a really elegant approach. It is harder to implement in a general way (like as a library) because you need to run code on the server. And as others have pointed out, it's harder to make fast. Perhaps multiplayer is just new enough to the web that there hasn't been time for these approaches to evolve. I mean, React was also inspired by game techniques that at that time were decades old.
- flupke 3y agoWhat does TCR mean please?
- tablatom 3y agoPeople might remember this showing up once or twice before as Replicache. Reflect adds a fully managed, incredibly fast sync server. The local-first/realtime space is getting busy these days. Replicache/Reflect is well worth checking out for the beautifully simple data model / coding model. I'm very happy with it. One plus point compared to CRDTs is that you get to decide how you want to deal with conflicts yourself, with simple, sequential code. I'm not up to date with the latest in CRDT-land but I believe it can be complicated to add application specific conflict resolution if the built-in rules don't fit your needs.
- dang 3y agoAh thanks - related past threads: A JavaScript framework to build offline-first but collaborative webapp - https://news.ycombinator.com/item?id=33269440 https://news.ycombinator.com/item?id=33269440 - Oct 2022 (2 comments) Linear clone, with realtime sync and instant UI - built with Replicache - https://news.ycombinator.com/item?id=31331660 https://news.ycombinator.com/item?id=31331660 - May 2022 (3 comments) Replicache: Easy Offline-First for Existing Applications - https://news.ycombinator.com/item?id=22173500 https://news.ycombinator.com/item?id=22173500 - Jan 2020 (22 comments)
- ochiba 3y ago> One plus point compared to CRDTs is that you get to decide how you want to deal with conflicts yourself, with simple, sequential code. I'm not up to date with the latest in CRDT-land but I believe it can be complicated to add application specific conflict resolution if the built-in rules don't fit your needs. I agree wholeheartedly — we took the same approach for PowerSync with a server reconciliation architecture [1] over CRDTs, which is also mentioned elsewhere in the comments here. For applications that have a central server, I think the simplicity of this kind of architecture is very appealing. Props to Aaron for bringing awareness to and evangelizing server reconciliation. [1] https://www.gabrielgambetta.com/client-side-prediction-server-reconciliation.html https://www.gabrielgambetta.com/client-side-prediction-serve...
- joloooo 3y agoLooks great, congrats on launching. Is this in a similar vein to https://partykit.io/ https://partykit.io/?
- aboodman 3y agoYes they are in the same space. The key difference is in how opinionated each is. PartyKit is extremely unopinionated. It's essentially lightweight javascript server, that launches fast and autoscales (I don't say this as as a bad thing, it's a useful primitive). Most people seem to run yjs in PartyKit, but you can also run automerge or even Replicache – my company's other project. Reflect is entirely focused on providing the best possible multiplayer experience. We make a lot of choices up and down the stack to tightly integrate everything so that multiplayer just works and you can focus on building your app. That's the plan anyway :).
- joloooo 3y agoThanks for taking the time to answer! I've been trying to keep up with the rise of more collaborative tooling, so I appreciate the comparison. Will have to try out reflect sometime on a future project :)
- joshelgar 3y agoAny idea how this compares to Supabase's Realtime & Presence libraries? https://supabase.com/docs/guides/realtime https://supabase.com/docs/guides/realtime
- aboodman 3y agoWe'll post something soon comparing all of the options (in our opinion). But as I understand it: - Supabase Broadcast: one client sends every other client a one-time message - Supabase Presence: one client tells all other clients that its own state has changed. This is similar to above, except that server keeps track of all the last-state for each client so that new members of the channel can get up to date. - Supabase Realtime: server tells all clients when server state changes (so you will not get optimistic client-side changes this way)
- jitl 3y agoYou forgot to compare them :) Supabase’s real-time stuff lets various parts of your system send messages about changes to each other. But when it comes to realtime editing, that’s the easy part! Reflect is a much higher level abstraction that handles the hard parts: - maintaining a cache on the device (this alone is annoying hard to get right) - queuing updates to send to the server reliably - handling conflicts between different users while preserving the intent of each user, by rebasing histories locally, and choosing the winner on the server
- jauntywundrkind 3y agoVery allagmatic. Instead of storing & transferring structures transfer operations. Iw as wondering what kind of checkingpoint or snapshotting there might be. Such that one doesn't have to "replay-the-world" if something goes awry (localstack persistence did/l (does?) this, storing state by just recording/replaying API calls). It sort of seems like having a globally accepted state is enough & makes sense, but it still seems like work to rollback & do again, still seems to require snapshotting. Something like this feels like a 100% fit for having immutable data structures. Where you can snapshot the world very very quickly/at low cost.
- aboodman 3y agoReflect is indeed built on immutable data structures internally, and it’s part of what makes it performant.
- revskill 3y agoI need a simple explanation like i am 6 years old because i have a mind of a 6 years old.
- simonw 3y agoThis is a hosted service that makes it much easier for you add multiplayer real-time collaboration (as seen in Google Docs and Figma) to your own web application.
- windowshopping 3y agoI would add a link to the documentation on the homepage.
- alexanderking 3y agoThere is. Under “Get Started”
- Jeff_Brown 3y agoFrameworks for hard problems like this seem like such a good public service. Is there much low-hanging fruit in this area -- common concurrency problems where people are likely to roll their own solutions because it's hard to find a preexisting framework?
- gfodor 3y agoThe main value prop of CRDTs is they can be used without a central server (for example, in p2p networking, or when offline.) If you're choosing between a CRDT or something else like this for your centralized server, you're probably not thinking about things properly. Edit: I have no idea why this was downvoted, it's true
- phatskat 3y ago> If you're choosing between a CRDT or something else like this for your centralized server, you're probably not thinking about things properly. I think that’s the point of comparing this to CRDTs - to show that there may be a better tool for the job if you’ve been considering CRDT because it’s seen some recent popular exposure
- linux2647 3y agoThe demo at the top of the homepage (https://reflect.net/ https://reflect.net/) is a lot of fun. While I was watching, every time the puzzle got completed, people would shake their cursor in excitement, like saying, "yay we did it!!"
- FrostKiwi 3y agoSeems to run worse on Firefox than on Chromium / Edge / Chrome. Rather choppy, mouse cursor lagging behind and laptop fans spinning up. Smooth on Chromium and friends though.
- qup 3y agoGranollers!!!
- bbx 3y agoVery fun experience indeed. Here's a video of what you're describing: https://streamable.com/asu261 https://streamable.com/asu261
- automatoney 3y agoThere's a visual bug where you can hide pieces of the 'e' behind the completed portions of the 'e' - it doesn't work for other letters because they still show the outline.
- aboodman 3y agoHm, good point. Thanks!
- automatoney 3y agoIt's very fun to troll other players with, especially that middle bar piece is easy to hide. I've also seen someone play keep-away with the last piece. Just overall very funny how much you can goof off with a simple puzzle!
- 3y ago
- hgs3 3y agoThis is a pedantic comment, but the terminology is confusing. Games have "players" so their sync systems are called "multiplayer," but regular software has "users" so their sync systems should be called "multiuser." The page reads strangely mixing the terms "user" with "multiplayer."
- itjustisthough 3y agoThat’s just the terminology these days. Multiplayer == “live” multi-user including visualizing what other users are doing at every moment.
- nwenzel 3y agoMaybe. But user is like a consumer. So HN is multiuser, but seems weird to call HN multi-player. Live concurrent interactions feels like something more than multiuser. Just my 2-cents, but multiplayer as in multiple users acting and interacting is a good use of the term multiplayer.
- Slartie 3y agoLet me guess: you are not regularly playing multiplayer computer games? I totally feel hgs3's confusion when reading the discussion here or that website. I always confuse the use cases - "are they now talking about computer games, or are they talking about the broader scope of any kind of software in which multiple users can act concurrently within a shared environment?". I think this confusion would be much less severe if I hadn't been an avid gamer for decades and thus had a very specific idea of the meaning of the term "multiplayer".
- mrkeen 3y agoIts logic is based on the logic that's been in "multiplayer games" for decades. On the other hand all web software is "multiuser", which doesn't tell you anything.
- lgessler 3y agoAs someone who's casually been eyeing up CRDTs for two years I've wondered continually what the story for authorization is, and this article seems to suggest (in line with my own understanding) that a CRDT library on its own like Y.js can't really handle applications where mutations need to be checked for authorization because it lacks a central authority (i.e., the server) while Reflect assumes that a server will be mediating client interactions. Is this all correct?
- hem777 3y agoOne way to do authorization is to sign each operation/message and then verify the signature to match a public key in an access control list. This also enables CRDTs to work in a peer to peer context. But as aboodman says in a sibling comment, if there’s a server as an authority, it can simply reject messages from unauthorized clients.
- foota 3y agoCRDTs can still involve a server, just not necessarily in the middle of everything. Consider you could have connections authenticated by the server, for one. E.g., someone connects peer to peer to you and claims some privilege etc., you could use the server to verify their claim. Also, I don't think CRDT necessarily implies peer to peer, as you can use a central server for message passing, but keep the CRDT model for resolving the current state on both server and client.
- aboodman 3y ago"Can't" is a strong word. You can get auth with a CRDT with effort, for example you can put a server in the middle of everything and have the server reverse any changes it sees which are unauthorized. This ends up being a lot of work to maintain and easy to break as the application gets bigger, and it also defeats some of the benefits of the CRDT in the first place (now the server has to mediate everything and you can't have peer-to-peer sync). Also if there are side-effects of any actions which require auth which aren't undoable, then that gets more complicated. If you already have a server in the middle, it's a lot simpler to just use a protocol that allows the server to reject messages in the first place.
- antidnan 3y agoReflect is awesome. We're currently using an alpha version in production, and I've been really happy with the system as well as Aaron and team! Happy to answer any questions as a customer :)
- moralestapia 3y agoI worked extensively on this (also implemented it on JS) about 15 years ago, to escape boredom while on my grandma's place. I wish I had open sourced something, but back then I was a young boy with no ulterior motivations :^). Anyway, I want to comment on this: >For example, any kind of arithmetic just works: Of course, it is extremely trivial to set those up as idempotent operations. >List operations just work: Nope ... or at least, I'd have to take a closer look at your implementation, consider: Some sort of array/list that looks like: [1, 2, 3, 4, 5] * User A deletes the range from 2-4. * User B deletes the range from 3-5. * User C inserts (uh-oh) something between 3 and 4. All updates reach the server at the same time. What is the solution to that? Timestamps? Fallback to LLW? This breaks the transactional model. Pick any of those updates as "the winner", what do the other users see? How do you communicate this to them? If your answer is like "we just send them the whole array again", this breaks the transactional model (x2).
- pmontra 3y agoIn a collaborative application the server could resolve the three operations in any order. The three users observe the result, realize that they are operating on the same data at the same moment and messed with the state. Then they fix it. Think about editing a document. If they can't see each other working on the data together they could be surprised but they could eventually fix the data to the desired state. If they don't check the result or can't check the result, the state won't be OK. However it's not what probably happens in interactive apps like collaborative editing or games.
- gizmo 3y agoMost of these systems do deletes with tombstones. Then you can safely insert after/inbetween deleted items.
- moralestapia 3y agoIs the solution to that, a) [1, 5] or b) [1, <new element>, 5] ?
- 3y ago
- stu_k 3y agoHow does one handle an upgrade to mutators? If a client is running old code then the operation will differ to the server. Obvious answer would be to version them independently: `increment_v1`, `increment_v2`, but wondering if there is a better answer?
- erikarvidsson 3y agoAt the moment there is no better answer than do not remove your old mutators until you know there are no more clients out there that might have pending changes. Right now that window is pretty small because we haven't turned on persistence yet.
- wheybags 3y agoFor reference, this strategy is called "deterministic lockstep" in the gamedev world, and is used especially often in games which have many entities which need their state synced. The canonical example is rtses, which pretty much all use it.
- Oberdiah 3y agoSpecifically "Deterministic Rollback", as the clients don't wait for the server's response before displaying the change locally. For an in-depth explanation, see this excellent GDC talk on its implementation in Mortal Kombat and Injustice 2: https://youtu.be/7jb0FOcImdg https://youtu.be/7jb0FOcImdg
- DecoPerson 3y agoHey wheybags, this is not deterministic lockstep. That is an algorithm for peer-to-peer games, where each peer waits for the input from all other players before advancing the game simulation. The deterministic part is because players share inputs (not simulation results) AND the simulation is deterministic for the same inputs. The lockstep part is because all clients advance at a coordinated pace. The Age of Empires series use this approach, and that’s why units don’t move immediately when you click. Starcraft uses this too, but it has some tricks to smooth the gameplay experience. Both are peer-to-peer with no single “server”. In the case of Reflect, we have a server-authorative simulation (not peer-to-peer). Clients send their inputs to the server, but they do not wait for the result, they instead predict the result locally without confirmation from the server. The server also rolls back time and then replays inputs to compensate for individual client latency. And the client corrects/reconciles their local simulation once the server sends a simulation result that included one of their inputs. The keywords for this algorithm are: - Server-authoritative - Predicted - Lag compensated (simulation rollback / server rewind / input replay) - Prediction reconciliation (local simulation rollback / local input replay / misprediction smoothing) I’m not sure if Reflect has it, but client-side interpolation is also a common feature in FPS games, where upon receiving a world update, the client will tween entities to their new positions/rotations over a fixed interval (such as 0.1s). This allows you to send updates to clients at only 10Hz but have entities move smoothly (without needing the client to locally simulate the physics of every entity). There is only one authority, the server, so determinism isn’t super important. However, it is nice to have for client predictions to reduce the occurrences of mispredictions. Mispredictions occur when the server state did not advance the way the client predicted. These can happen when: (1) Another client’s input changed the world state in an important way, (2) The simulation is not deterministic (e.g. the random number generation is not synced). In FPS games, (1) is impossible to eliminate. These mispredictions are often smoothed using interpolation. (2) should be minimised, but may actually be desirable. For example, Counter Strike does not sync the RNG for bullet spread randomness, to prevent “nospread” cheat programs from predicting where each bullet will go and instantly adjusting the player’s aim so the bullet lines up perfectly. https://www.gabrielgambetta.com/client-side-prediction-live-demo.html https://www.gabrielgambetta.com/client-side-prediction-live-... https://developer.valvesoftware.com/wiki/Latency_Compensating_Methods_in_Client/Server_In-game_Protocol_Design_and_Optimization https://developer.valvesoftware.com/wiki/Latency_Compensatin... https://developer.valvesoftware.com/wiki/Source_Multiplayer_Networking https://developer.valvesoftware.com/wiki/Source_Multiplayer_...
- Aeolun 3y agoWhat’s the transport for this? Websockets? I have a hard time imagining 120 messages for 100 clients over websocket in a CF Durable Object, but maybe I’m just completely misimagining.
- aboodman 3y agoIt's websockets, and yes we do make use of Durable Objects. But we do extensive batching and buffering on top of the raw platform. You can't literally send 120 messages / second to and from 100 concurrent clients. Actually the math tops out at about 8 :).
- asadm 3y agoCan you explain what are the limitations in Durable Objects?
- aboodman 3y agoThere are a variety of relevant limitations. Some of them are listed here: developers.cloudflare.com/durable-objects/platform/limits/ For this particular problem, it just comes down to CPU time. At the bottom of the stack ws.send() is a syscall and it takes significant time. You can only do so many of these calls per second. We measured it at about 8k/sec a year ago. With a conservative limit of 2k calls to ws.send() (so that we can stay about about 50% utilization and only use half of the time for calling send), this implies 6 clients at 60 FPS using a naive approach (6 * 6 * 60 = ~2k). It's not really a DO problem, it's just that doing n^2 messages can't really work in any platform.
- deleted 3y ago[deleted]
- plopz 3y agoCan this be used to let multiple people edit a very large image concurrently?
- aboodman 3y agoDepends what you mean by very large. Reflect can store up to about 50MB/room. For a PNG, that's "fairly large" I guess. We plan to increase this limit in the future. You'd have to split up the image into chunks of about 1KB each, this would be fairly easy to do by pixels. It would be a fun project to try. See https://hello.reflect.net/rooms#data-model https://hello.reflect.net/rooms#data-model for more information.
- heron88 3y agoThis looks great! I'm curious what the recommended use cases are for something like this. Is it worth adding the additional complexity for complex crud-y saas? I know Linear is built on this local-first model but from seeing their presentations on the challenges they've had scaling it, I'm left wondering if it makes sense for something like a full-featured CRM with hundreds of tables. Is Reflect/Replicache suited for that kind of use case?
- meheleventyone 3y agoMy 2c as a game dev is that you really want to know the cases where you need real-time collaboration. Otherwise the still collaborative but slower updating techniques people normally use are perfectly fine. Even in something like an MMO parts of the game will use real-time game servers and others might just use a REST API. The other side of it is to make sure that real-time collab is something users actually want and use. There’s a lot of froth at the moment about real-time but IMO and IME it has a much smaller set of useful cases than most people actually think. But where it is useful, it is a step up for collaboration. For example with Google Docs I find myself and our broader team mostly solo write but collaboratively review and edit. Where we collaboratively write its carefully structured so people aren’t stomping on each other and often done with someone acting as a mediator.
- erikarvidsson 3y agoWe have an old demo of our Linear clone using Replicache at https://repliear.herokuapp.com/ https://repliear.herokuapp.com/
- cornfutes 3y agoLooks great for a hobby project. Has anyone done the cost + risk assessment of building a for-profit product on top of this? Would love to know, as I am working on a web IDE with collaboration. There’s also the matter of obscuring client data from 3rd and even 1st party.
- holoduke 3y agoA risk assessment? Just go and try it out in a prototype and see if you can make money with it. You can always replace it once successful.
- jamil7 3y agoIf what you're building on top of leaks into other parts of your codebase, can't be isolated and can't easily be swapped out, I'd say avoid building a product on top of it.
- reactordev 3y agoDon’t call it multiplayer, call it multiuser or concurrent user. Player assumes it’s a game. It’s not a game. It’s very very real. We are the agents. Cool project. Collaboration is hard. Synchronicity even harder. This makes steel become butter.
- aboodman 3y agoI like multiplayer because it connotes games and games have worked so hard at this and done so well. Our goal is to be as good as games. But I get your argument too. Naming is hard.
- zharknado 3y agoI think multiplayer is a good name for a different reason. I think it’s a clear and accessible metaphor for how people will interact, i.e. by concurrently working toward a shared goal with a coherent shared state. I don’t think it trivializes the goal or the work being done, it just distinguishes it from parallel non-coordinated use. Also, players are not just for games; the Bard had some thoughts on that. ;)
- reactordev 3y agoTwo hardest things, Naming and Time synchronization. Thanks for building this.
- bostonvaulter2 3y agoVery neat project! What is the business model for this project?
- yewenjie 3y agoNot at all any expert in this space, so maybe a dumb question, but how do you model plain text data for something like a collaborative markdown editor with things like this or CRDTs?
- cco 3y agoWow this looks incredible. I had the most fun I've had on a computer in a long time this last weekend using Partykit.io to build https://is-sf-back.cconeill.partykit.dev/ https://is-sf-back.cconeill.partykit.dev/ and this looks so much smoother. Going to have to check this out.
- appleaday1 3y agoThis kills the game engine devs dreams.
- knubie 3y agoOne consequence of this approach is that your mutators will need to be versioned, and you can pretty much never delete or change them once they're live. (Please correct me if I'm wrong).
- bradhe 3y agoI mean…how many things we gonna call Reflect? Signed, the former founder of reflect.io.
- alanwreath 3y agoA lot. - Signed not the creator of the word reflect. :)
- cadbox1 3y agoFigma uses a similar solution for their realtime feature. They have a famous article that explains why CRDTs aren’t necessary when you have a central server authority. https://www.figma.com/blog/how-figmas-multiplayer-technology-works/ https://www.figma.com/blog/how-figmas-multiplayer-technology... I implemented the same solution with Redux actions and Cloudflare Durable Workers. https://emojis.cadell.dev https://emojis.cadell.dev