You should definitely try to enable HTML caching - that's what CDNs are far, and it will the workload much cheaper to serve and an origin server less likely to

You should definitely try to enable HTML caching - that's what CDNs are far, and it will the workload much cheaper to serve and an origin server less likely to fall over. Besides that, looking at the Rust code for the origin you link on twitter, you should make some changes to make it reliable at scale. First of all, I recommend to add timeouts. Otherwise the amount of open sockets will just creep up if RSTs got lost/dropped and there is no data to send - which ultimately makes things prone to resource exhaustion. Also be aware that the tower ConcurrencyLimitLayer alone is not a great solution for this problem - it will build a queue of requests and if clients don't give up the queue gets longer and longer until also no current requests are served anymore and clients will again time out (=> website becomes unreachable). It's better to reject requests fast once a limit is reached than to build infinite queues. Regarding observability, one can log the amount of processed requests, connections, active requests and connections (to determine which things are stuck), maybe status codes. All these things should require emitting one datapoint per minute or so, which is cheap.