3 ms·

Do you see any issues with scaling in this project? I'd be very interested to know what you chose as backend architecture and how you handle so many requests pa

by akuji1993 8y ago

Do you see any issues with scaling in this project? I'd be very interested to know what you chose as backend architecture and how you handle so many requests parallel to each other, as I've struggled to build a good multi-threaded express backend in the past, would be awesome to hear some other stories. Do you think your service can handle a massive surge, let's say a really big player would pick you up? Edit: Also really surprised about your pricing strategy. What if somebody wants to surpass the 50k pageviews? Wouldn't it be better to at least give those another tier?

adtac 8y ago
I've done some basic benchmarks and it has scaled pretty well. Mostly thanks to the excellent performance and scalability that comes from choosing Go and Postgres. IMO backend servers are where Go excels. I'm planning on adding Redis support soon enough so that sudden spikes would have little impact on the database server (which is nearly always the bottleneck, unless you're compute-intensive, which isn't the case with Commento). So that if you suddenly get a few hundred thousand visitors, most API responses are cached and only a fraction will actually require disk access. Like reddit does, where it'll temporarily say there N comments, but when you open the page, there wouldn't be that many. A few seconds of inconsistency is a fine compromise in the comments world. Of course, Redis isn't going to magically make everything faster, so I'll have to be careful with where I introduce it. Optimise for the common case and all that. I'm still in uni, and I've never actually worked on large-throughput, high-scalability stuff before, so all this has been a great learning experience for me. If you have any suggestions or war stories that might be relevant, I'm all ears!
- akuji1993 8y ago
  I'm by no means an expert in web backend architecture, but what you have done seems pretty solid to me with my limited knowledge. We've been mostly working with Express, that's why I asked to hear some feedback about how others have done. And also, very well done! Your software seems to work very fine, a lot of thought already went into scaling and general stability of the backend. And for me, as fullstack working for a design company, your website layout and design is pretty good. If you didn't have help with that, kudos to you, good work, especially for someone still in college :) Edit: Website performs pretty good in a Lighthouse audit, apart from Accessibility features.
- raviatluri 8y ago
  I am happy to help/pitch in for the Redis support. Let me know, if you would like to collaborate.
  - adtac 8y ago
    I'd love to get help with Redis. I don't know anything beyond the absolute basics, so it'd really be useful if someone with more experience give lend a hand! If you'd like to discuss, please add your input here: https://gitlab.com/commento/commento/issues/75 https://gitlab.com/commento/commento/issues/75

adtac 8y ago

I've done some basic benchmarks and it has scaled pretty well. Mostly thanks to the excellent performance and scalability that comes from choosing Go and Postgres. IMO backend servers are where Go excels. I'm planning on adding Redis support soon enough so that sudden spikes would have little impact on the database server (which is nearly always the bottleneck, unless you're compute-intensive, which isn't the case with Commento). So that if you suddenly get a few hundred thousand visitors, most API responses are cached and only a fraction will actually require disk access. Like reddit does, where it'll temporarily say there N comments, but when you open the page, there wouldn't be that many. A few seconds of inconsistency is a fine compromise in the comments world. Of course, Redis isn't going to magically make everything faster, so I'll have to be careful with where I introduce it. Optimise for the common case and all that. I'm still in uni, and I've never actually worked on large-throughput, high-scalability stuff before, so all this has been a great learning experience for me. If you have any suggestions or war stories that might be relevant, I'm all ears!

akuji1993 8y ago
I'm by no means an expert in web backend architecture, but what you have done seems pretty solid to me with my limited knowledge. We've been mostly working with Express, that's why I asked to hear some feedback about how others have done. And also, very well done! Your software seems to work very fine, a lot of thought already went into scaling and general stability of the backend. And for me, as fullstack working for a design company, your website layout and design is pretty good. If you didn't have help with that, kudos to you, good work, especially for someone still in college :) Edit: Website performs pretty good in a Lighthouse audit, apart from Accessibility features.
raviatluri 8y ago
I am happy to help/pitch in for the Redis support. Let me know, if you would like to collaborate.
- adtac 8y ago
  I'd love to get help with Redis. I don't know anything beyond the absolute basics, so it'd really be useful if someone with more experience give lend a hand! If you'd like to discuss, please add your input here: https://gitlab.com/commento/commento/issues/75 https://gitlab.com/commento/commento/issues/75