5 ms·

Lol it's so surreal seeing your project when you're randomly browsing the frontpage. Anyway, creator here, AMA! >adtac has succeeded in reducing download to ju

by adtac 8y ago

Lol it's so surreal seeing your project when you're randomly browsing the frontpage. Anyway, creator here, AMA! >adtac has succeeded in reducing download to just 11 kB and isn’t done yet Someone raised an issue to support Brotli in Commento (which I wasn't aware of) [1] and I did a quick test; it'd decrease the JS+CSS payload even further to under 8-9 KB. Nearly all modern browsers support Brotli [2], so I'm really looking forward to this. I now realise this could be a modern take on the famous C10K problem, just with a different meaning: developing web services with under 10 kilobytes of payload :) [1] https://gitlab.com/commento/commento/issues/125 https://gitlab.com/commento/commento/issues/125 [2] https://en.wikipedia.org/wiki/Brotli#Browser_support https://en.wikipedia.org/wiki/Brotli#Browser_support

akuji1993 8y ago
Do you see any issues with scaling in this project? I'd be very interested to know what you chose as backend architecture and how you handle so many requests parallel to each other, as I've struggled to build a good multi-threaded express backend in the past, would be awesome to hear some other stories. Do you think your service can handle a massive surge, let's say a really big player would pick you up? Edit: Also really surprised about your pricing strategy. What if somebody wants to surpass the 50k pageviews? Wouldn't it be better to at least give those another tier?
- adtac 8y ago
  I've done some basic benchmarks and it has scaled pretty well. Mostly thanks to the excellent performance and scalability that comes from choosing Go and Postgres. IMO backend servers are where Go excels. I'm planning on adding Redis support soon enough so that sudden spikes would have little impact on the database server (which is nearly always the bottleneck, unless you're compute-intensive, which isn't the case with Commento). So that if you suddenly get a few hundred thousand visitors, most API responses are cached and only a fraction will actually require disk access. Like reddit does, where it'll temporarily say there N comments, but when you open the page, there wouldn't be that many. A few seconds of inconsistency is a fine compromise in the comments world. Of course, Redis isn't going to magically make everything faster, so I'll have to be careful with where I introduce it. Optimise for the common case and all that. I'm still in uni, and I've never actually worked on large-throughput, high-scalability stuff before, so all this has been a great learning experience for me. If you have any suggestions or war stories that might be relevant, I'm all ears!
  - akuji1993 8y ago
    I'm by no means an expert in web backend architecture, but what you have done seems pretty solid to me with my limited knowledge. We've been mostly working with Express, that's why I asked to hear some feedback about how others have done. And also, very well done! Your software seems to work very fine, a lot of thought already went into scaling and general stability of the backend. And for me, as fullstack working for a design company, your website layout and design is pretty good. If you didn't have help with that, kudos to you, good work, especially for someone still in college :) Edit: Website performs pretty good in a Lighthouse audit, apart from Accessibility features.
  - raviatluri 8y ago
    I am happy to help/pitch in for the Redis support. Let me know, if you would like to collaborate.
    - adtac 8y ago
      I'd love to get help with Redis. I don't know anything beyond the absolute basics, so it'd really be useful if someone with more experience give lend a hand! If you'd like to discuss, please add your input here: https://gitlab.com/commento/commento/issues/75 https://gitlab.com/commento/commento/issues/75
technion 8y ago
The brotli experience has been frustrating. All modern browsers support it, so you'd think it would be an easy win. The top nginx search is an official looking Google project which is long abandoned and doesn't support current versions of the library. There's an official nginx module which is commercial, and then finally there's a supported fork that that sits on about page three of Google. Shared hosts don't generally support it. AWS Cloudfront doesn't support it, along with several other CDNs. Big + to Cloudflare for proper support. It reduced my bundle sizes measurably. We all hate bloat and want to reduce bundles, but there seems to be a community inertia around mass deployment.
- adtac 8y ago
  Ah, that sucks. The hosted version uses Cloudfront as CDN for static assets, too bad they don't support it. >official looking Google project which is long abandoned Is [1] what you're talking about? [1] https://godoc.org/github.com/google/brotli/go/cbrotli https://godoc.org/github.com/google/brotli/go/cbrotli
  - technion 8y ago
    The [1] I should have linked is here: https://github.com/google/ngx_brotli/issues/72 https://github.com/google/ngx_brotli/issues/72
    - wyday 8y ago
      Maintained fork: https://github.com/eustas/ngx_brotli https://github.com/eustas/ngx_brotli
- fouc 8y ago
  Anyone know why brotli needs to be generated at the web server level? Just curious why/if it can't be done at the web app level.
  - samschooler 8y ago
    Its generally used as a data transfer compression system. So the web server needs to compress the data before its sent to the browser. If you did it at the web app level (client/browser side), the data would already be transferred so there are no size savings.
orkon 8y ago
When did you start working on Commento?
- adtac 8y ago
  Around Apr 2017. Released v1.0 (almost a full rewrite, but still in Go) in Jan 2018. Launched the hosted version in Feb 2019 after almost 6 months of private beta.
  - orkon 8y ago
    Cool! Did you consider the pay-per-use model instead of a fixed monthly subscription?
tekkk 8y ago
That's pretty cool. Last I checked on my own site Disqus downloaded 1.3 MBs of bunch of stuff, which I think is just ridiculous. After caching it's only 2 KBs but it still bothers me - what in the world it's downloading? main.js seems to be 1.2 MBs and the main culprit. EDIT: well that was the ungzipped size, 263 KBs gzipped the CSS was larger with 353 KBs. But however, it works and has a free-tier so I guess it'll do. Maybe one day I'll try this out, but for now I can't be bothered with self-hosting.
kawsper 8y ago
Is it easy to move from hosted Commento into self-hosted Commento, and vice-versa?
- adtac 8y ago
  Not yet. You can export comments from Commento as a JSON, but you can't import from the same format. Sucks, I know, but I'll try to implement this soon.
simplehuman 8y ago
How is the hosted version doing?
- adtac 8y ago
  It's doing alright. Pays for its server bill, so I'm happy about that :)
  - 9999px 8y ago
    Just emailed you about an enterprise account. My biggest concern would be longevity. Our org is an institution decades old and can't have a startup sunsetting on us a year after implementation.
BeetleB 8y ago
How does it take care of spam? I hope it's not via Akismet, as I've not found it to be good enough.

by adtac 8y ago

akuji1993 8y ago

Do you see any issues with scaling in this project? I'd be very interested to know what you chose as backend architecture and how you handle so many requests parallel to each other, as I've struggled to build a good multi-threaded express backend in the past, would be awesome to hear some other stories. Do you think your service can handle a massive surge, let's say a really big player would pick you up? Edit: Also really surprised about your pricing strategy. What if somebody wants to surpass the 50k pageviews? Wouldn't it be better to at least give those another tier?

adtac 8y ago
I've done some basic benchmarks and it has scaled pretty well. Mostly thanks to the excellent performance and scalability that comes from choosing Go and Postgres. IMO backend servers are where Go excels. I'm planning on adding Redis support soon enough so that sudden spikes would have little impact on the database server (which is nearly always the bottleneck, unless you're compute-intensive, which isn't the case with Commento). So that if you suddenly get a few hundred thousand visitors, most API responses are cached and only a fraction will actually require disk access. Like reddit does, where it'll temporarily say there N comments, but when you open the page, there wouldn't be that many. A few seconds of inconsistency is a fine compromise in the comments world. Of course, Redis isn't going to magically make everything faster, so I'll have to be careful with where I introduce it. Optimise for the common case and all that. I'm still in uni, and I've never actually worked on large-throughput, high-scalability stuff before, so all this has been a great learning experience for me. If you have any suggestions or war stories that might be relevant, I'm all ears!
- akuji1993 8y ago
  I'm by no means an expert in web backend architecture, but what you have done seems pretty solid to me with my limited knowledge. We've been mostly working with Express, that's why I asked to hear some feedback about how others have done. And also, very well done! Your software seems to work very fine, a lot of thought already went into scaling and general stability of the backend. And for me, as fullstack working for a design company, your website layout and design is pretty good. If you didn't have help with that, kudos to you, good work, especially for someone still in college :) Edit: Website performs pretty good in a Lighthouse audit, apart from Accessibility features.
- raviatluri 8y ago
  I am happy to help/pitch in for the Redis support. Let me know, if you would like to collaborate.
  - adtac 8y ago
    I'd love to get help with Redis. I don't know anything beyond the absolute basics, so it'd really be useful if someone with more experience give lend a hand! If you'd like to discuss, please add your input here: https://gitlab.com/commento/commento/issues/75 https://gitlab.com/commento/commento/issues/75

technion 8y ago

The brotli experience has been frustrating. All modern browsers support it, so you'd think it would be an easy win. The top nginx search is an official looking Google project which is long abandoned and doesn't support current versions of the library. There's an official nginx module which is commercial, and then finally there's a supported fork that that sits on about page three of Google. Shared hosts don't generally support it. AWS Cloudfront doesn't support it, along with several other CDNs. Big + to Cloudflare for proper support. It reduced my bundle sizes measurably. We all hate bloat and want to reduce bundles, but there seems to be a community inertia around mass deployment.

adtac 8y ago
Ah, that sucks. The hosted version uses Cloudfront as CDN for static assets, too bad they don't support it. >official looking Google project which is long abandoned Is [1] what you're talking about? [1] https://godoc.org/github.com/google/brotli/go/cbrotli https://godoc.org/github.com/google/brotli/go/cbrotli
- technion 8y ago
  The [1] I should have linked is here: https://github.com/google/ngx_brotli/issues/72 https://github.com/google/ngx_brotli/issues/72
  - wyday 8y ago
    Maintained fork: https://github.com/eustas/ngx_brotli https://github.com/eustas/ngx_brotli
fouc 8y ago
Anyone know why brotli needs to be generated at the web server level? Just curious why/if it can't be done at the web app level.
- samschooler 8y ago
  Its generally used as a data transfer compression system. So the web server needs to compress the data before its sent to the browser. If you did it at the web app level (client/browser side), the data would already be transferred so there are no size savings.

orkon 8y ago

When did you start working on Commento?

adtac 8y ago
Around Apr 2017. Released v1.0 (almost a full rewrite, but still in Go) in Jan 2018. Launched the hosted version in Feb 2019 after almost 6 months of private beta.
- orkon 8y ago
  Cool! Did you consider the pay-per-use model instead of a fixed monthly subscription?

tekkk 8y ago

That's pretty cool. Last I checked on my own site Disqus downloaded 1.3 MBs of bunch of stuff, which I think is just ridiculous. After caching it's only 2 KBs but it still bothers me - what in the world it's downloading? main.js seems to be 1.2 MBs and the main culprit. EDIT: well that was the ungzipped size, 263 KBs gzipped the CSS was larger with 353 KBs. But however, it works and has a free-tier so I guess it'll do. Maybe one day I'll try this out, but for now I can't be bothered with self-hosting.

kawsper 8y ago

Is it easy to move from hosted Commento into self-hosted Commento, and vice-versa?

adtac 8y ago
Not yet. You can export comments from Commento as a JSON, but you can't import from the same format. Sucks, I know, but I'll try to implement this soon.

simplehuman 8y ago

How is the hosted version doing?

adtac 8y ago
It's doing alright. Pays for its server bill, so I'm happy about that :)
- 9999px 8y ago
  Just emailed you about an enterprise account. My biggest concern would be longevity. Our org is an institution decades old and can't have a startup sunsetting on us a year after implementation.

BeetleB 8y ago

How does it take care of spam? I hope it's not via Akismet, as I've not found it to be good enough.