4 ms·

I don't think "serverless is bad" is necessarily the full lesson here. The bigger lesson is when a service has dependencies, moving that service closer to the c

I don't think "serverless is bad" is necessarily the full lesson here. The bigger lesson is when a service has dependencies, moving that service closer to the client (without also moving those dependencies) will counterintuitively make the e2e experience slower, not faster. Prefer building physically near your dependencies. If that's not fast enough, then you have to figure out how to move or sync all your dependencies closer to the client, which except in very simple cases, is almost always a huge can of worms.

brap 1y ago
I don’t know if this is a good rule of thumb, I think it really depends on what you use the dependencies for, how often you need them, etc. Consider for example a single DB dependency. Should the server be close to the DB or the client? It depends. How often does the client need the server? How often does the server need the DB? Which usecases are expected to be fast and which can be sacrificed as slow? What can be cached in the server? What can be cached in the client? etc etc. And then of course you can split and do some things on the server and some in the edge…
- marcosdumay 1y ago
  Oh, it's a very good rule of thumb. It's probably not universal, but it's really close to it. the problem is that nobody designs the dependencies flexible enough to let them run without fine-control. And the main application always wants to change the way it uses the dependencies, so it always needs further flexibility. You can build an exception to the rule if you explicitly try. But I'm not sure one appears naturally. The natural way to migrate your server into the edge is by migrating entire workloads, dependencies included. You can split the work like you said, you just can't split single endpoints.
  - daxfohl 1y ago
    Well, I guess one can take one more step back and say this is all merely an example of "premature optimization is the root of all evil". Unless you know a-priori that you have some very hard latency requirements, start with something simple and low-maintenance. If low-latency requirements come in later, then design that holistically, not just looking at your component. Make sure you're measuring the right things; OOTB metrics often miss the e2e experience. And IME most latency issues come from unexpected places; I know I've spent weeks optimizing services to get an extra percent or two out of them, only to realize there's a config setting that reduced latency by half. So generally, simplicity is your friend when it comes to latencies (among other things). Fewer things to cause long-tail spikes, more simple things you can try out that don't break the whole system, whereas if you start with a highly-optimized thing up-front, fixing some unexpected long-tail issue may require a complete rewrite. Also, check with your PM or end users as to whether latency is even important. If the call to your service is generally followed up to a call to some ten-second process, users aren't going to notice the 20ms improvement to your own thing.
swiftcoder 1y ago
> The bigger lesson is when a service has dependencies, moving that service closer to the client (without also moving those dependencies) will counterintuitively make the e2e experience slower, not faster The problem here is that pretty much all services have dependencies - if they didn't, you could have already moved that logic client-side. This bites edge-compute architectures really hard. You move compute closer to the client, but far from the DB, and in doing so your overall latency is almost always worse. The most latency-friendly architecture is almost always going to be an in-memory DB on a monolithic server (albeit this comes with its own challenges around scaling and redundancy)

brap 1y ago

I don’t know if this is a good rule of thumb, I think it really depends on what you use the dependencies for, how often you need them, etc. Consider for example a single DB dependency. Should the server be close to the DB or the client? It depends. How often does the client need the server? How often does the server need the DB? Which usecases are expected to be fast and which can be sacrificed as slow? What can be cached in the server? What can be cached in the client? etc etc. And then of course you can split and do some things on the server and some in the edge…

marcosdumay 1y ago
Oh, it's a very good rule of thumb. It's probably not universal, but it's really close to it. the problem is that nobody designs the dependencies flexible enough to let them run without fine-control. And the main application always wants to change the way it uses the dependencies, so it always needs further flexibility. You can build an exception to the rule if you explicitly try. But I'm not sure one appears naturally. The natural way to migrate your server into the edge is by migrating entire workloads, dependencies included. You can split the work like you said, you just can't split single endpoints.
- daxfohl 1y ago
  Well, I guess one can take one more step back and say this is all merely an example of "premature optimization is the root of all evil". Unless you know a-priori that you have some very hard latency requirements, start with something simple and low-maintenance. If low-latency requirements come in later, then design that holistically, not just looking at your component. Make sure you're measuring the right things; OOTB metrics often miss the e2e experience. And IME most latency issues come from unexpected places; I know I've spent weeks optimizing services to get an extra percent or two out of them, only to realize there's a config setting that reduced latency by half. So generally, simplicity is your friend when it comes to latencies (among other things). Fewer things to cause long-tail spikes, more simple things you can try out that don't break the whole system, whereas if you start with a highly-optimized thing up-front, fixing some unexpected long-tail issue may require a complete rewrite. Also, check with your PM or end users as to whether latency is even important. If the call to your service is generally followed up to a call to some ten-second process, users aren't going to notice the 20ms improvement to your own thing.

swiftcoder 1y ago

> The bigger lesson is when a service has dependencies, moving that service closer to the client (without also moving those dependencies) will counterintuitively make the e2e experience slower, not faster The problem here is that pretty much all services have dependencies - if they didn't, you could have already moved that logic client-side. This bites edge-compute architectures really hard. You move compute closer to the client, but far from the DB, and in doing so your overall latency is almost always worse. The most latency-friendly architecture is almost always going to be an in-memory DB on a monolithic server (albeit this comes with its own challenges around scaling and redundancy)