3 ms·

Replying to mattmanser in the sibling comment since I cannot replying to it directly: It comes in faster than you think. One of the company I work for was a sm

Replying to mattmanser in the sibling comment since I cannot replying to it directly: It comes in faster than you think. One of the company I work for was a small company, at the time I think we were 4 or 5 including the 2 founders, they sell some specific information as a service through APIs. They charge per call, so you have to update the customer funds in their account on each API call. Yes, people can want and make lots of APIs call very fast, and in a business which charges per API call, this is a good thing, but you have to find strategies and make business decisions on how to handle accepting and replying to new API calls while maintaining their account balance.

nostrademons 7y ago
Load the remaining customer funds into RAM, and decrement them on each API. Periodically flush the remaining balance to DB at set intervals (say every hour). If the server goes down, well, your customers got free API calls for an hour to compensate them for their trouble. With any reasonable uptime you'll save more on DB traffic than you lose in lost revenues. (BTW, the "can't reply" issue happens on HN when there's a long comment thread between just two people that's being rapidly updated. If you wait 10 minutes or so you'll be able to reply.)
- jimktrains2 7y ago
  > Load the remaining customer funds into RAM, and decrement them on each API. Periodically flush the remaining balance to DB at set intervals (say every hour). So you only ever have one app server? Sure, this specific issues could be handled other ways, but in general global state is hard.
  - nostrademons 7y ago
    If you have more than one app server load the customer funds into Redis and use atomic operations to decrement it. Serving out of RAM is going to be orders of magnitude faster than updating disk regardless of how much you spend on network overhead. If you have so many customers that they blow the RAM limit on a Redis box....well, you won't, because that's billions of customers. You can get surprisingly far on one app server (or sometimes one app server with 2 mirrors for failover). See eg. the latest TechEmpower benchmarks [1], where common Java and C++ web frameworks can serve 300k+ reqs/sec off of bare-metal hardware. The OP indicated that these API requests are basically straight queries anyway, and only need to write to update the billing information. Reads can run incredibly fast (both because they can usually be served out of cache and because they can be mirrored), so if your only bottleneck is the write, take the write out of the critical path. In general global state is hard. Don't solve general problems, solve specific ones. Atomic counters have a well-understood and highly performant solution. [1] https://www.techempower.com/benchmarks/#section=data-r17&hw=ph&test=fortune https://www.techempower.com/benchmarks/#section=data-r17&hw=...
    - jimktrains2 7y ago
      Using a database like redis isn't quite the same as just "loading it into ram". Also, I never said that you couldn't use a single app server because of performance; as you said, you can handle quite a bit of traffic on a single box, even with slower languages.
      - deleted 7y ago
        [deleted]
      - deleted 7y ago
        [deleted]
deleted 7y ago
[deleted]
deleted 7y ago
[deleted]
deleted 7y ago
[deleted]
mattmanser 7y ago
I find it hard to believe this actually caused a DB problem, updating an int column on an indexed table should have taken nanoseconds. Far faster than any HTTP calls should have been happening. Sounds like something was very wrong. Someone fucked up with the network causing some severe DB latency or they'd written some really shitty code. Even if it did, as the other comment says, you could easily work round it. If that was the behaviour there was no good reason to have it utterly perfect.
- nostrademons 7y ago
  Depends if autocommit is enabled. If it is (and on many DBs it's the default) then you're telling the DB that by the time the increment completes, it should be written to durable storage. That's a minimum of one seek, which on a hard disk is about 4ms, which = max 250 TPS. On a SSD you might get 0.1ms seek times = 10k TPS. It's possible to kill your performance very quickly with databases if you request features that you don't actually need.
- dorfsmay 7y ago
  That's where decisions around consistency kick in. You need to decide if you absolutely do not want to serve any API call unless you are sure they've been paid for, in which case you have to create a commit transaction on that account for each call. Or you decide how much you can let a would be rotten customer get away with, and uses queues which leads to eventual consistencies.

nostrademons 7y ago

Load the remaining customer funds into RAM, and decrement them on each API. Periodically flush the remaining balance to DB at set intervals (say every hour). If the server goes down, well, your customers got free API calls for an hour to compensate them for their trouble. With any reasonable uptime you'll save more on DB traffic than you lose in lost revenues. (BTW, the "can't reply" issue happens on HN when there's a long comment thread between just two people that's being rapidly updated. If you wait 10 minutes or so you'll be able to reply.)

jimktrains2 7y ago
> Load the remaining customer funds into RAM, and decrement them on each API. Periodically flush the remaining balance to DB at set intervals (say every hour). So you only ever have one app server? Sure, this specific issues could be handled other ways, but in general global state is hard.
- nostrademons 7y ago
  If you have more than one app server load the customer funds into Redis and use atomic operations to decrement it. Serving out of RAM is going to be orders of magnitude faster than updating disk regardless of how much you spend on network overhead. If you have so many customers that they blow the RAM limit on a Redis box....well, you won't, because that's billions of customers. You can get surprisingly far on one app server (or sometimes one app server with 2 mirrors for failover). See eg. the latest TechEmpower benchmarks [1], where common Java and C++ web frameworks can serve 300k+ reqs/sec off of bare-metal hardware. The OP indicated that these API requests are basically straight queries anyway, and only need to write to update the billing information. Reads can run incredibly fast (both because they can usually be served out of cache and because they can be mirrored), so if your only bottleneck is the write, take the write out of the critical path. In general global state is hard. Don't solve general problems, solve specific ones. Atomic counters have a well-understood and highly performant solution. [1] https://www.techempower.com/benchmarks/#section=data-r17&hw=ph&test=fortune https://www.techempower.com/benchmarks/#section=data-r17&hw=...
  - jimktrains2 7y ago
    Using a database like redis isn't quite the same as just "loading it into ram". Also, I never said that you couldn't use a single app server because of performance; as you said, you can handle quite a bit of traffic on a single box, even with slower languages.
    - deleted 7y ago
      [deleted]
    - deleted 7y ago
      [deleted]

deleted 7y ago

[deleted]

deleted 7y ago

[deleted]

deleted 7y ago

[deleted]

mattmanser 7y ago

I find it hard to believe this actually caused a DB problem, updating an int column on an indexed table should have taken nanoseconds. Far faster than any HTTP calls should have been happening. Sounds like something was very wrong. Someone fucked up with the network causing some severe DB latency or they'd written some really shitty code. Even if it did, as the other comment says, you could easily work round it. If that was the behaviour there was no good reason to have it utterly perfect.

nostrademons 7y ago
Depends if autocommit is enabled. If it is (and on many DBs it's the default) then you're telling the DB that by the time the increment completes, it should be written to durable storage. That's a minimum of one seek, which on a hard disk is about 4ms, which = max 250 TPS. On a SSD you might get 0.1ms seek times = 10k TPS. It's possible to kill your performance very quickly with databases if you request features that you don't actually need.
dorfsmay 7y ago
That's where decisions around consistency kick in. You need to decide if you absolutely do not want to serve any API call unless you are sure they've been paid for, in which case you have to create a commit transaction on that account for each call. Or you decide how much you can let a would be rotten customer get away with, and uses queues which leads to eventual consistencies.