4 ms·

The confusing part for me is why I’m dicking around with the client when the server’s broken.

MontyCarloHall 1y ago
Exactly. If I were asked this question during an interview, the first thing I'd say is "why should the client bother with anything more complex than jittered exponential backoff?"
- efitz 1y ago
  From a security perspective, it’s axiomatic that you can’t make a client behave in any particular way; we just assume the client is hostile.
- diesal11 1y ago
  I've dealt with backends that refresh a CSRF token on each valid request and return it in the response as a cookie. In those cases a solution like this may be needed. Not optimal but, we don't always have control over the backends we use, especially then they're provided by a third party.
  - deleted 1y ago
    [deleted]
mikeocool 1y ago
FWIW I’ve basically been given basically this exact requirement by a partner with a crappy API. We’d get on calls with them and they’d be like “you can’t do multithreading!” we eventually parsed out that what they literally meant was that we could only make a single request to their API at a time. We’d had to integrate with them, and they weren’t going to fix it on their side. (Our solve ended being a lot more complicated than this, as we had multiple processes across multiple machines that were potentially making concurrent requests.)
- balder1991 1y ago
  So in the end it had to be funneled to a single server keeping a list of requests to make serially to their API?
  - lmm 1y ago
    That's the easy way to do it, the fun one is implementing a distributed lock/queue.
    - reillyse 1y ago
      Not actually that hard with a redis lock or any database (Postgres has a specific lock for this but you could also just use a record in a table) Far easier than the original single threaded solution - and has fault tolerance baked in cause you can run it on multiple clients
      - lmm 1y ago
        > Not actually that hard with a redis lock or any database (Postgres has a specific lock for this but you could also just use a record in a table) Redis is just another SPOF, and so is Postgres without fiddly third party extensions (that are pretty unreliable in practice, IME). I'm talking about something truly distributed.
        virgilp 1y ago
        What, you need something truly "internet-scale" to make sure your thousands of clients can hit, sequentially, that one faulty api? Would you really be concerned more about Redis failure rates, than said API's failure rates?
        lmm 1y ago
        If you get into that situation then it's probably because that API is critical and irreplaceable (otherwise you wouldn't be tolerating its problems), so you really don't want to get stuck and be unable to query it. And if you can tolerate a SPOF then there's no reason to bring Redis/Postgres into the picture, you might as well just have a single server doing it. Plus it's just good practice that I'd want to be following anyway. Once you get in the habit of doing it it doesn't really cost much to design the dataflow right up-front, and it can save you from getting trapped down the line when it's much harder to fix things. Especially for an interview-type situation, why not design it right?
rustyminnow 1y ago
Because it was written in ALGOL 60, none of the mainframe devs are willing to touch that code, and the dozen other clients probably depend on the broken functionality.
Jarwain 1y ago
I've had to implement this exact logic at work, because we have to talk to devices using modbus tcp, where a lot of devices only supports having one active request per connection at a time. One device we talk to only supports having 5 active connections to it.
- xnorswap 1y ago
  I've dealt with low-limit APIs, and I've always considered the better approach to the problem is to create a proxy rather than trust the clients to manage, limit and coordinate themselves. Then we send all traffic to proxy.service instead of canon.service, and implement queuing, rate limiting, caching, etc on the proxy. This shields the weak part of the system from the clients.
  - Jarwain 1y ago
    Makes sense as a broad pattern! I guess I hadn't considered the existence of generic TCP proxies And I'd probably have concerns around how much latency it would introduce in an environment where we have some requirements on the rate of data collection. That said our service Does act as a kind of proxy for a few protocols.
Xss3 1y ago
I implemented an incredibly similar async queue logic for a CLI tool with the option to build sequences of commands
layer8 1y ago
Because you only control the client, but you need to integrate with that broken server of a third party. It’s a pretty common situation to find oneself in.
- mechanicalpulse 1y ago
  Yup. I've even found myself in situations where the owner of the third-party service is another team or department within the organization I'm working for or partnering with. Oftentimes, the product/project people on our team tries to make it a business issue with the partner only to find that they don't have the leverage to effect a fix, they get told that the service doesn't offer the SLA you require, or you hear back from the team some hilarious quote like six weeks of development that can't begin until the next quarter. Meanwhile, your feature or product has to launch by the end of the current sprint or quarter. What happens when the unstoppable force meets the immovable object? The unstoppable force works over the weekend to implement a store-and-forward solution.
lazyant 1y ago
A server that has a spike of load and can't cope with it is pretty normal, hard to characterize as "broken". When the client(s) can send more work than the server can handle there are three options: 1 - Do nothing; server drops requests. 2 - Server notifies the clients (429 in HTTP) and client backs-off (exponential, jitter). 3 - Put the client requests in a queue. Interview question/solution does 2 in a poor way (just adding a pause), it's part of the client and does 3 in the client, when usually this is done in an intermediate component (RMQ/Kafka/Redis/Db/whatever).

Exactly. If I were asked this question during an interview, the first thing I'd say is "why should the client bother with anything more complex than jittered exponential backoff?"

efitz 1y ago
From a security perspective, it’s axiomatic that you can’t make a client behave in any particular way; we just assume the client is hostile.
diesal11 1y ago
I've dealt with backends that refresh a CSRF token on each valid request and return it in the response as a cookie. In those cases a solution like this may be needed. Not optimal but, we don't always have control over the backends we use, especially then they're provided by a third party.
- deleted 1y ago
  [deleted]

mikeocool 1y ago

FWIW I’ve basically been given basically this exact requirement by a partner with a crappy API. We’d get on calls with them and they’d be like “you can’t do multithreading!” we eventually parsed out that what they literally meant was that we could only make a single request to their API at a time. We’d had to integrate with them, and they weren’t going to fix it on their side. (Our solve ended being a lot more complicated than this, as we had multiple processes across multiple machines that were potentially making concurrent requests.)

balder1991 1y ago
So in the end it had to be funneled to a single server keeping a list of requests to make serially to their API?
- lmm 1y ago
  That's the easy way to do it, the fun one is implementing a distributed lock/queue.
  - reillyse 1y ago
    Not actually that hard with a redis lock or any database (Postgres has a specific lock for this but you could also just use a record in a table) Far easier than the original single threaded solution - and has fault tolerance baked in cause you can run it on multiple clients
    - lmm 1y ago
      > Not actually that hard with a redis lock or any database (Postgres has a specific lock for this but you could also just use a record in a table) Redis is just another SPOF, and so is Postgres without fiddly third party extensions (that are pretty unreliable in practice, IME). I'm talking about something truly distributed.
      - virgilp 1y ago
        What, you need something truly "internet-scale" to make sure your thousands of clients can hit, sequentially, that one faulty api? Would you really be concerned more about Redis failure rates, than said API's failure rates?
        lmm 1y ago
        If you get into that situation then it's probably because that API is critical and irreplaceable (otherwise you wouldn't be tolerating its problems), so you really don't want to get stuck and be unable to query it. And if you can tolerate a SPOF then there's no reason to bring Redis/Postgres into the picture, you might as well just have a single server doing it. Plus it's just good practice that I'd want to be following anyway. Once you get in the habit of doing it it doesn't really cost much to design the dataflow right up-front, and it can save you from getting trapped down the line when it's much harder to fix things. Especially for an interview-type situation, why not design it right?

rustyminnow 1y ago

Because it was written in ALGOL 60, none of the mainframe devs are willing to touch that code, and the dozen other clients probably depend on the broken functionality.

Jarwain 1y ago

I've had to implement this exact logic at work, because we have to talk to devices using modbus tcp, where a lot of devices only supports having one active request per connection at a time. One device we talk to only supports having 5 active connections to it.

xnorswap 1y ago
I've dealt with low-limit APIs, and I've always considered the better approach to the problem is to create a proxy rather than trust the clients to manage, limit and coordinate themselves. Then we send all traffic to proxy.service instead of canon.service, and implement queuing, rate limiting, caching, etc on the proxy. This shields the weak part of the system from the clients.
- Jarwain 1y ago
  Makes sense as a broad pattern! I guess I hadn't considered the existence of generic TCP proxies And I'd probably have concerns around how much latency it would introduce in an environment where we have some requirements on the rate of data collection. That said our service Does act as a kind of proxy for a few protocols.

Xss3 1y ago

I implemented an incredibly similar async queue logic for a CLI tool with the option to build sequences of commands

layer8 1y ago

Because you only control the client, but you need to integrate with that broken server of a third party. It’s a pretty common situation to find oneself in.

mechanicalpulse 1y ago
Yup. I've even found myself in situations where the owner of the third-party service is another team or department within the organization I'm working for or partnering with. Oftentimes, the product/project people on our team tries to make it a business issue with the partner only to find that they don't have the leverage to effect a fix, they get told that the service doesn't offer the SLA you require, or you hear back from the team some hilarious quote like six weeks of development that can't begin until the next quarter. Meanwhile, your feature or product has to launch by the end of the current sprint or quarter. What happens when the unstoppable force meets the immovable object? The unstoppable force works over the weekend to implement a store-and-forward solution.

lazyant 1y ago

A server that has a spike of load and can't cope with it is pretty normal, hard to characterize as "broken". When the client(s) can send more work than the server can handle there are three options: 1 - Do nothing; server drops requests. 2 - Server notifies the clients (429 in HTTP) and client backs-off (exponential, jitter). 3 - Put the client requests in a queue. Interview question/solution does 2 in a poor way (just adding a pause), it's part of the client and does 3 in the client, when usually this is done in an intermediate component (RMQ/Kafka/Redis/Db/whatever).