6 ms·

I'm not sure resource scaling is what's needed here. Just good ol' caching. Shameless plug: I built Cachoid[0] for caching as a service (leveraging Varnish main

by jjoe 9y ago

I'm not sure resource scaling is what's needed here. Just good ol' caching. Shameless plug: I built Cachoid[0] for caching as a service (leveraging Varnish mainly) to help with situations like this. [0] https://www.cachoid.com/ https://www.cachoid.com/

slifin 9y ago
I understand that this is a shameless plug but I've met teams where the first response to any performance problem is caching If you're reading this and have performance issues look at caches as a last resort, first figure out why things are slow I'm assuming a traditional stack here but use mysql jet profiler, or SQL equivalent first, then your languages profiler xhprof? If your queries are slow or impossible make sure your tables are normalised and indexed, make sure opcache is on and xdebug off Mysql out of the box is very conservative with its resources, the limits can often be raised and the web server can often be configured to be faster, make sure you're sending the correct http cache headers There's so many things to consider with performance Please don't denormalise your data slap it in redis/ use varnish and call it a day No cache is free, eventually you have to pay the pied piper when you do make sure you can afford to have never used the cache at all Some of my hardest bugs have come from out of sync caches & trying to build performant data architecture around systems that had data stuffed into a redis black hole aged for years..
- slifin 9y ago
  Also if you're doing key scans in redis, you're doing it wrong, it's possible to work around with indexes you have to manually maintain, but it's not worth it
- jjoe 9y ago
  Lots of good common sense advice and practical experience there. I've had my fair share as a devops/sysadmin working on crazy systems (more worse than better haha). So it's unsurprising how common sense isn't always put into practice. I do believe caching to be the proverbial Swiss Army Knife of performance. Some of its tools can be used properly and others as band-aid / stop gap. Even band-aids serve a useful purpose. But I can definitely see how a system can accrue technical debt when over leveraging caching. But to believe in prevalent, perfectly built systems is to deny the status quo of the landscape. When you democratize development like WordPress has done with its plugins and Node has done with npm, as popular examples, you definitely have it coming.
- jedberg 9y ago
  I dunno, I'd say solve your customer pain first, then mess around with your profiler. If throwing up a cache solves your immediate problem, by all means throw up a cache. Just make sure it's your first step, not your last step.
arghwhat 9y ago
Maybe state that it is a Varnish configurator. I initially thought it was a new caching proxy, thinking "Why would anyone need that when they have Varnish?". Also, adding caches is still resource scaling.
- jjoe 9y ago
  Edited to mention Varnish. Also, adding caches is still resource scaling. This is a semantically correct. Caching can indeed cut down on resource usage. It isn't always a scale-up operation.
  - arghwhat 9y ago
    My point is that a caching server is a resource like any other. Adding additional backend capacity, and adding additional caching capacity are both "adding resources"/"scaling up". Doing smart things to do more with the same resources, do the same with fewer resources, or by replacing expensive resources with cheaper resources, is optimization, not scaling. We're being really pedantic right now, though. :)

by jjoe 9y ago

slifin 9y ago

I understand that this is a shameless plug but I've met teams where the first response to any performance problem is caching If you're reading this and have performance issues look at caches as a last resort, first figure out why things are slow I'm assuming a traditional stack here but use mysql jet profiler, or SQL equivalent first, then your languages profiler xhprof? If your queries are slow or impossible make sure your tables are normalised and indexed, make sure opcache is on and xdebug off Mysql out of the box is very conservative with its resources, the limits can often be raised and the web server can often be configured to be faster, make sure you're sending the correct http cache headers There's so many things to consider with performance Please don't denormalise your data slap it in redis/ use varnish and call it a day No cache is free, eventually you have to pay the pied piper when you do make sure you can afford to have never used the cache at all Some of my hardest bugs have come from out of sync caches & trying to build performant data architecture around systems that had data stuffed into a redis black hole aged for years..

slifin 9y ago
Also if you're doing key scans in redis, you're doing it wrong, it's possible to work around with indexes you have to manually maintain, but it's not worth it
jjoe 9y ago
Lots of good common sense advice and practical experience there. I've had my fair share as a devops/sysadmin working on crazy systems (more worse than better haha). So it's unsurprising how common sense isn't always put into practice. I do believe caching to be the proverbial Swiss Army Knife of performance. Some of its tools can be used properly and others as band-aid / stop gap. Even band-aids serve a useful purpose. But I can definitely see how a system can accrue technical debt when over leveraging caching. But to believe in prevalent, perfectly built systems is to deny the status quo of the landscape. When you democratize development like WordPress has done with its plugins and Node has done with npm, as popular examples, you definitely have it coming.
jedberg 9y ago
I dunno, I'd say solve your customer pain first, then mess around with your profiler. If throwing up a cache solves your immediate problem, by all means throw up a cache. Just make sure it's your first step, not your last step.

arghwhat 9y ago

Maybe state that it is a Varnish configurator. I initially thought it was a new caching proxy, thinking "Why would anyone need that when they have Varnish?". Also, adding caches is still resource scaling.

jjoe 9y ago
Edited to mention Varnish. Also, adding caches is still resource scaling. This is a semantically correct. Caching can indeed cut down on resource usage. It isn't always a scale-up operation.
- arghwhat 9y ago
  My point is that a caching server is a resource like any other. Adding additional backend capacity, and adding additional caching capacity are both "adding resources"/"scaling up". Doing smart things to do more with the same resources, do the same with fewer resources, or by replacing expensive resources with cheaper resources, is optimization, not scaling. We're being really pedantic right now, though. :)