4 ms·

Can anyone recommend monitoring solutions to help identify these issues?

by yyx 8y ago

matt2000 8y ago
Strongly recommend Cloudflare in combination with some uptime monitoring service. Cloudflare gives you so many options on ways to mitigate attacks, and you get CDN and other services for free. Pretty great.
jeffshek 8y ago
1. CloudFlare. That's like the first thing I normally do for any new project. (which he then switched to). 2. Your clients should not be the first one to alert your site is down. Pingdom does a great job to alert you before your customers/users do. 3. The author brought up some of the queries weren't paginated and running an expensive SQL query, so there's a few options. Since he's using Django there's some Django specific options in this list. A) Implement a backend cache that will return back the JSON query (throw it on a redis). Cache and return that from the backend. B) Add a Django throttle to the view (can be done via IP / username). C) Enable logged in users only to access endpoint (harder to do on the fly though, since you need to make changes to your frontend). If a logged in user is causing you hell, turn off signups and kick that user off. D) Have CloudFlare cache a public response for you on endpoints and return it (you need to make sure the JSON should always be the same for every API call though, which is very very risky). E) Author brought up DRF JSON serialization is slow. Another alternative is to use Serpy which sees a 50-100x speedup. I'd only recommend that for complex JSON payloads. Not because it's hard, but because it's additional complexity. The author is also using Dokku which is fine for most projects, but you'd imagine at some point it'll probably be switched onto a load balancer + web machines. Alerting can also be set on the load balancer level if it goes above % threshold. Since he's using Dokku (so by that definition docker), they could probably use a log aggregation service that would allow him to access his logs much faster to see what's going on. Papertrail, etc. Monitoring CPU usage would also be helpful here, but I'm not sure if Dokku allows that.

Can anyone recommend monitoring solutions to help identify these issues?

by yyx 8y ago

Can anyone recommend monitoring solutions to help identify these issues?

matt2000 8y ago

Strongly recommend Cloudflare in combination with some uptime monitoring service. Cloudflare gives you so many options on ways to mitigate attacks, and you get CDN and other services for free. Pretty great.

jeffshek 8y ago

1. CloudFlare. That's like the first thing I normally do for any new project. (which he then switched to). 2. Your clients should not be the first one to alert your site is down. Pingdom does a great job to alert you before your customers/users do. 3. The author brought up some of the queries weren't paginated and running an expensive SQL query, so there's a few options. Since he's using Django there's some Django specific options in this list. A) Implement a backend cache that will return back the JSON query (throw it on a redis). Cache and return that from the backend. B) Add a Django throttle to the view (can be done via IP / username). C) Enable logged in users only to access endpoint (harder to do on the fly though, since you need to make changes to your frontend). If a logged in user is causing you hell, turn off signups and kick that user off. D) Have CloudFlare cache a public response for you on endpoints and return it (you need to make sure the JSON should always be the same for every API call though, which is very very risky). E) Author brought up DRF JSON serialization is slow. Another alternative is to use Serpy which sees a 50-100x speedup. I'd only recommend that for complex JSON payloads. Not because it's hard, but because it's additional complexity. The author is also using Dokku which is fine for most projects, but you'd imagine at some point it'll probably be switched onto a load balancer + web machines. Alerting can also be set on the load balancer level if it goes above % threshold. Since he's using Dokku (so by that definition docker), they could probably use a log aggregation service that would allow him to access his logs much faster to see what's going on. Papertrail, etc. Monitoring CPU usage would also be helpful here, but I'm not sure if Dokku allows that.