3 ms·

This is cool, but how is state managed here? Does the server need to maintain any per-client state between renders? Does that have scalability or concurrency im

This is cool, but how is state managed here? Does the server need to maintain any per-client state between renders? Does that have scalability or concurrency implications? I'm having trouble discerning the ramifications.

com2kid 4y ago
Modern stateless rest microservices are popular for many reasons, including ease of scaling and ease of reasoning about performance. This entire system seems very stateful. Also, in the real world, requests to one web service end up cascading out to multiple web services, with differing permissions models. For example user comes in with a token, makes a request to a web service, which then has its own secret store that it fetches an API key from to make a request to another web service. Very rarely in life has it been as simple as "fetch data from this API". It is "fetch data from this API given this auth token and that API takes that auth token and uses it to get another auth token, attaches some tracing info for diagnostics in case things go wrong, writes some metrics to a DB so the team can track API usage, then goes out to a couple more services, gets data from them, collates it, and eventually gives some data back to the user."
- Twisol 4y ago
  I'm hopeful about this approach, honestly. We do build stateful flows in web applications, today, but we have to manually manage the serialization of that state to a shared store so that whichever worker the next request hits can pick up where things were left off. It seems to me that a system like Hyperfiddle / Photon could automatically manage the collection and persistence of this kind of state, so that the "server"-located code can be scaled horizontally without change to the written program. I can even imagine an Erlang-like solution to updating application code, so that existing flows can gracefully upgrade or time out. "Could", of course, is the operative word! But given that we have to manage state now anyway, this seems like an approach with a lot of potential.
  - dustingetz 4y ago
    Sure, i think you're basically describing distributed actors with durable state, and yes the Photon model can be viewed as distributed actors (with amazing syntax). Other great teams are working on the infinity scale streaming infrastructure, we'd like to be the last-mile UI for next-gen end-user applications.
- dustingetz 4y ago
  (speaker here) regarding the limits of stateless apps see https://driftingin.space/posts/session-lived-application-backends https://driftingin.space/posts/session-lived-application-bac..., the trend is that the more complex your app (think an IDE), the more the "stateless" approach is too slow and breaks down. (Note that what people call "stateless" is not stateless at all; there's database query effects, network effects, session state – and then on the client we have dom effects, reactive rendering and component local state. Modern web apps are actually entirely imperative except in isolated little components where we pretend) The system we describe in the talk is indeed stateful, but it's worth pointing out that the Photon runtime is referentially transparent in the pure functional programming sense. People think functional programming is about avoiding state and effects; it's not. It's about gaining control over our effects so that we can orchestrate scaled up fabrics of millions of fine grained effects without loss of composition, reasoning or control. This is the promise of modern functional effect systems, and is the underpinning abstraction that powers Photon.
  - capn_duck 4y ago
    +1 for understanding that "stateless" doesn't necessarily mean better OR more performant. There's too much reductionist thinking based on not enough evidence. I'm not building a twitter clone, i'm building something that needs to frequently re-evaluate business rules at each step of a lengthy workflow. Reactivity is a natural fit. And there's state that exists there. Where do you want it to stay? (please don't say 'in a jwt') There's room for so many different solutions to so many different problems. Shooting something down because its "stateful" well, okay if you know your problem domain well and that's significant to you then so be it, but that smells to me like "I have nothing valuable to contribute so I'm going to throw out a meaningless non-sequitur". I'll happily prioritize developer productivity first when user experience is not meaningfully impacted. I suspect that ~95% of web developers are in that same position.
  - dataflow 4y ago
    Does this approach end up causing you to keep around more state (or other resources, like connections or whatnot) than would be typical in the traditional approaches? That's kind of where my mind was wandering with my comment earlier—not expecting statelessness for every application, but wondering if it becomes any more difficult to manage state than it would be normally. (Part of why I thought this might be the case is that people naturally try to minimize resources when they have to keep it around explicitly, but when it's done implicitly, I'm not sure if that becomes more of a concern or not. I could see it going either way.)

com2kid 4y ago

Modern stateless rest microservices are popular for many reasons, including ease of scaling and ease of reasoning about performance. This entire system seems very stateful. Also, in the real world, requests to one web service end up cascading out to multiple web services, with differing permissions models. For example user comes in with a token, makes a request to a web service, which then has its own secret store that it fetches an API key from to make a request to another web service. Very rarely in life has it been as simple as "fetch data from this API". It is "fetch data from this API given this auth token and that API takes that auth token and uses it to get another auth token, attaches some tracing info for diagnostics in case things go wrong, writes some metrics to a DB so the team can track API usage, then goes out to a couple more services, gets data from them, collates it, and eventually gives some data back to the user."

Twisol 4y ago
I'm hopeful about this approach, honestly. We do build stateful flows in web applications, today, but we have to manually manage the serialization of that state to a shared store so that whichever worker the next request hits can pick up where things were left off. It seems to me that a system like Hyperfiddle / Photon could automatically manage the collection and persistence of this kind of state, so that the "server"-located code can be scaled horizontally without change to the written program. I can even imagine an Erlang-like solution to updating application code, so that existing flows can gracefully upgrade or time out. "Could", of course, is the operative word! But given that we have to manage state now anyway, this seems like an approach with a lot of potential.
- dustingetz 4y ago
  Sure, i think you're basically describing distributed actors with durable state, and yes the Photon model can be viewed as distributed actors (with amazing syntax). Other great teams are working on the infinity scale streaming infrastructure, we'd like to be the last-mile UI for next-gen end-user applications.
dustingetz 4y ago
(speaker here) regarding the limits of stateless apps see https://driftingin.space/posts/session-lived-application-backends https://driftingin.space/posts/session-lived-application-bac..., the trend is that the more complex your app (think an IDE), the more the "stateless" approach is too slow and breaks down. (Note that what people call "stateless" is not stateless at all; there's database query effects, network effects, session state – and then on the client we have dom effects, reactive rendering and component local state. Modern web apps are actually entirely imperative except in isolated little components where we pretend) The system we describe in the talk is indeed stateful, but it's worth pointing out that the Photon runtime is referentially transparent in the pure functional programming sense. People think functional programming is about avoiding state and effects; it's not. It's about gaining control over our effects so that we can orchestrate scaled up fabrics of millions of fine grained effects without loss of composition, reasoning or control. This is the promise of modern functional effect systems, and is the underpinning abstraction that powers Photon.
- capn_duck 4y ago
  +1 for understanding that "stateless" doesn't necessarily mean better OR more performant. There's too much reductionist thinking based on not enough evidence. I'm not building a twitter clone, i'm building something that needs to frequently re-evaluate business rules at each step of a lengthy workflow. Reactivity is a natural fit. And there's state that exists there. Where do you want it to stay? (please don't say 'in a jwt') There's room for so many different solutions to so many different problems. Shooting something down because its "stateful" well, okay if you know your problem domain well and that's significant to you then so be it, but that smells to me like "I have nothing valuable to contribute so I'm going to throw out a meaningless non-sequitur". I'll happily prioritize developer productivity first when user experience is not meaningfully impacted. I suspect that ~95% of web developers are in that same position.
- dataflow 4y ago
  Does this approach end up causing you to keep around more state (or other resources, like connections or whatnot) than would be typical in the traditional approaches? That's kind of where my mind was wandering with my comment earlier—not expecting statelessness for every application, but wondering if it becomes any more difficult to manage state than it would be normally. (Part of why I thought this might be the case is that people naturally try to minimize resources when they have to keep it around explicitly, but when it's done implicitly, I'm not sure if that becomes more of a concern or not. I could see it going either way.)