5 ms·

Wasn't this done recently?

by dtgriscom 4y ago

Waterluvian 4y ago
I think Reddit did a button thing. But I’m trying to figure out the simplest, cheapest way to implement it.
- hedora 4y ago
  Postgres or MySQL on a free tier VM will be fine for this use case. With one paid VM, you could easily get to something ridiculous, like 1M TPS. If you get past that, you could add memcache to the read path. If you need more than 1M button pushes per second, each frontend server could see if the timer was reset less than a second ago. At this point, you're limited by data center bandwidth. You could move the frontend servers into edge infrastructure. That should scale linearly to billions of requests per second. At some point, you'll probably need to hire a team to negotiate with cloud providers, or invest in hardware procurement infrastructure. Anyway, that free tier VM would be able to handle the load, even at such a scale. You might want to add a secondary in a second DC if fault tolerance is important. What was this for again? Writing this makes me want to go play Universal Paperclips: https://www.decisionproblem.com/paperclips/ https://www.decisionproblem.com/paperclips/
  - cmeacham98 4y ago
    To get 1M TPS, you will need to recieve 1M PPS (assuming each button reset is a single packet, which it's probably not if you're doing HTTP) and that is really hard (tm): https://blog.cloudflare.com/how-to-receive-a-million-packets/ https://blog.cloudflare.com/how-to-receive-a-million-packets...
    - hedora 4y ago
      I skipped a step where you break the frontend servers out into separate machines when you add the memcache. iouring or dpdk on a many core machine might be needed to get to 1M TPS when scaling up. (HTTP would probably also be non-trivial / impossible) Anyway, with connection pooling / sharing, a database server should be able to hit it without those tricks.
  - Waterluvian 4y ago
    Thanks. I think your napkin estimates work out and tell me this is trivially possible for free. I’m not sure I need a database though. Just a single timestamp in memory with a file to back it up and a “just assume the button was pushed” if that ever gets corrupted somehow. Of course I didn’t specify how rigorous the solution needed to be: not very. I want a website where if the timer ever hits 0 I donate $100 to a charity. And it’s called “This is why we can’t have nice things.”
    - hedora 4y ago
      The file thing with cache you describe is just a little database. An off the shelf one is probably easier. (If it's just a hobby project, then writing a little database server could be fun. Go for it.)
  - withinboredom 4y ago
    Why do we need to handle that many POST requests? We only need to update the client and do a POST if and only if it would make a meaningful change to the remaining time. If we're going to keep it up and running forever anyway and this is just a marketing gimmick, we can randomly reset the server side time every few minutes and do zero POST requests.
    - withinboredom 4y ago
      Alternatively you can serve a static response over nginx with async logging. Then ingest the logs through some streaming pipeline…
- janci 4y ago
  if($_SERVER['REQUEST_METHOD'] == "POST") touch("button.txt"); echo "<form method=POST><button>".(60+filemtime("button.txt")-time());
  - Waterluvian 4y ago
    I love this so much. I’m going to try it.

Wasn't this done recently?

Waterluvian 4y ago

I think Reddit did a button thing. But I’m trying to figure out the simplest, cheapest way to implement it.

hedora 4y ago
Postgres or MySQL on a free tier VM will be fine for this use case. With one paid VM, you could easily get to something ridiculous, like 1M TPS. If you get past that, you could add memcache to the read path. If you need more than 1M button pushes per second, each frontend server could see if the timer was reset less than a second ago. At this point, you're limited by data center bandwidth. You could move the frontend servers into edge infrastructure. That should scale linearly to billions of requests per second. At some point, you'll probably need to hire a team to negotiate with cloud providers, or invest in hardware procurement infrastructure. Anyway, that free tier VM would be able to handle the load, even at such a scale. You might want to add a secondary in a second DC if fault tolerance is important. What was this for again? Writing this makes me want to go play Universal Paperclips: https://www.decisionproblem.com/paperclips/ https://www.decisionproblem.com/paperclips/
- cmeacham98 4y ago
  To get 1M TPS, you will need to recieve 1M PPS (assuming each button reset is a single packet, which it's probably not if you're doing HTTP) and that is really hard (tm): https://blog.cloudflare.com/how-to-receive-a-million-packets/ https://blog.cloudflare.com/how-to-receive-a-million-packets...
  - hedora 4y ago
    I skipped a step where you break the frontend servers out into separate machines when you add the memcache. iouring or dpdk on a many core machine might be needed to get to 1M TPS when scaling up. (HTTP would probably also be non-trivial / impossible) Anyway, with connection pooling / sharing, a database server should be able to hit it without those tricks.
- Waterluvian 4y ago
  Thanks. I think your napkin estimates work out and tell me this is trivially possible for free. I’m not sure I need a database though. Just a single timestamp in memory with a file to back it up and a “just assume the button was pushed” if that ever gets corrupted somehow. Of course I didn’t specify how rigorous the solution needed to be: not very. I want a website where if the timer ever hits 0 I donate $100 to a charity. And it’s called “This is why we can’t have nice things.”
  - hedora 4y ago
    The file thing with cache you describe is just a little database. An off the shelf one is probably easier. (If it's just a hobby project, then writing a little database server could be fun. Go for it.)
- withinboredom 4y ago
  Why do we need to handle that many POST requests? We only need to update the client and do a POST if and only if it would make a meaningful change to the remaining time. If we're going to keep it up and running forever anyway and this is just a marketing gimmick, we can randomly reset the server side time every few minutes and do zero POST requests.
  - withinboredom 4y ago
    Alternatively you can serve a static response over nginx with async logging. Then ingest the logs through some streaming pipeline…
janci 4y ago
if($_SERVER['REQUEST_METHOD'] == "POST") touch("button.txt"); echo "<form method=POST><button>".(60+filemtime("button.txt")-time());
- Waterluvian 4y ago
  I love this so much. I’m going to try it.