54 ms·
Static HTML Comments
- cirrus3 4y agoHm. Is it really a "static" website if you have some JS constantly listening for updates?
- reustle 4y agoJS isn’t listening for updates, but simply pulling in a static html file filled with comments. document.getElementById('commentlist').innerHTML = xhr.responseText
- cirrus3 4y agoah, ok.. so it just dynamically loads comments on load?
- pontilanda 4y agoStatic as in "the page is pre-generated, not generated on every HTTP request" It's the right setup for write rarely, read often scenarios.
- cirrus3 4y agoWhere are you are sanitizing the comments? You have literally provided a perfect SQL injection mechanism here and allowed arbitrary HTML to be inserted onto your page.
- zamadatix 4y agoI can tell it's Friday because it took until the section ID until I realized this was going to be about user comments on a website not <!-- HTML comments --> LOL. If you've got Ruby listening for updates with a postgres database all tied into a content generator it's probably time to admit you don't have a static website anymore and just look for best practice ways to go about this (with a bit more security!).
- cirrus3 4y agoSame here... and I wouldn't call this a static page anymore if it runs JS on load, calls a HTTP endpoint which makes DB query which returns HTML that modifies the DOM.
- dylan604 4y agobut if you can host your site from an S3 bucket, would it not be called static? to me, static means no server side languages necessary to render the page. if your statically served JS can call a remote host to pull down some data that updates things client side, it's still considered static, from a hosting perspective.
- codeflo 4y agoWouldn’t that definition would apply to standard SPAs as well?
- dylan604 4y agoI guess it depends on perspective. What a host service that will charge you more money for a non-static site, a client-side JS app can still be considered static. We might also be blurring lines of static sites vs interactive sites. You can have a static site that does nothing but allows the user to scroll to view it. You can also have static sites that change and do things based on the user's input. Entire games have been made using client-side JS but is still served statically.
- zamadatix 4y ago
- krapp 4y agoThis used to be relatively common. Messageboards and forums (going all the way back to Matt's cgi board, which didn't even use a database) would cache static HTML pages and only regenerate them when necessary. I think Wordpress' cache plugin works in much the same way. I guess it's a completely foreign concept in the age of SPAs when everything is dynamically generated in the browser.
- layer8 4y agoThe main point of the article is using the Postgres notify/listen mechanism to trigger the update.
- krapp 4y agoThat's the least interesting thing about it, though.
- acidburnNSA 4y agoYeah my guestbooks from the 90s and early 00s were all just cgis that directly edited the html with the new comments. Great times.
- superkuh 4y agoI implemented static HTML comments on my website by tail'ing the /var/logs/nginx/access.log with a perl script. No CGI, no database, just the actual text of the comments stored as a single comment per line in a .html file. To comment a visitor takes any url on the domain and appends "/@say/". Like http://notmyurl.com/somepage.html/@say/Some http://notmyurl.com/somepage.html/@say/Some response to somepage. Or "...lakephoto.jpg/@say/Cool fish! How long was it?" The perl script sees the /@say/ in the logs and adds the parsed out and sanitized comment to an .html file. There's some nginx location hijinks for matching /@say/ URLs that goes to a confirmation page and redirects to the comment listing page.
- dylan604 4y agowhat executes the perl script? the POST doing a CGI type thingy i have so forgotten how to utilize from not using it since the 90s? on first read, i missed that you're modifying the contents of the html and was concerned about log rotation before realizing that was just dumb to be concerned about.
- superkuh 4y agoI have it start when my OS starts. It runs continuously thereafter (~5MB ram). The webserver does not know or care about it. The script leans heavily on perl's POE::Wheel::FollowTail module to get updates to nginx's access.log automagically then I abuse regexes to note log entries with /@say/, extract what URL-file-path commented on, what the comment is, and prepend the line to the algorithmically appropriate .html file with some html mark-up. The webserver serves the static html as it would any html.
- pbhjpbhj 4y agoYou've told everyone on HN how it works, have you been root-ed yet?
- superkuh 4y agoI've used this comment system on my tor onion services sites for the last decade. I get plenty of people trying to exploit it. It's kind of fun. If the Tor folk haven't pwned it I doubt the HN folk will. Not for lack of skill but mostly a lack of motivation relative to the tor folk.
- jetpackjoe 4y agoThis seems similar to Incremental Static Regeneration (ISR) provided by NextJS/Vercel https://nextjs.org/docs/basic-features/data-fetching/incremental-static-regeneration https://nextjs.org/docs/basic-features/data-fetching/increme...
- victor106 4y ago> When comments change, PostgreSQL trigger sends NOTIFY If your application is down when the NOTIFY is triggered that comment is lost, isn’t it?
- nurettin 4y agoThere is a queue that will prevent further writes when full. https://www.postgresql.org/docs/current/sql-notify.html https://www.postgresql.org/docs/current/sql-notify.html
- thom 4y agoThe normal way around this in Postgres is to use NOTIFY to merely signal that there is something new, and use a separate (perhaps transactional) work queue to claim a row to process. You’re correct that relying on parameters in the actual notification will lose work.
- nurettin 4y agoWeb users can insert, the author can moderate, and database changes are reflected on the cache instead of being generated on demand. It looks useful.
- ckdot 4y ago[flagged]
- geerlingguy 4y agoThis doesn't deal with the deluge of spam you'll get once any page on your site gets even modestly popular. (But it seems great for simpler use cases). It would be so nice to have a comment system like Disqus but simple like described here, that also had an easy tie-in to a spam prevention service (or would work with a trained spam prevention model).
- smitty1e 4y agoThe Famous Article is just about the content management. A separate authentication layer is entirely possible, at the loss of anonymity.
- aae42 4y agoOk, but instead of inserting into db, it'd be cool to have the script commit the comment into source control and have your ci rebuild/deploy your static site
- jaza 4y agoThat's what I did: https://greenash.net.au/thoughts/2022/03/email-based-comment-moderation-with-netlify-functions/ https://greenash.net.au/thoughts/2022/03/email-based-comment...
- classified 4y agoWhy not use a blockchain if you want an append-only store?
- aae42 4y agoDidn't say anything about an append-only. Blockchain is the answer to exactly 0 technical problems.
- classified 4y agoSource control repos are effectively append-only. How else could you recover earlier revisions?
- yashap 4y agoSo, you’ve got a static-ish page, a Ruby server, Postgres and NGINX. The page both POSTs and GETs comments from the backend via an Ajax calls. All very conventional, but then you’re worried about traffic spikes, and to deal with that, the comments on the server are stored in pre-generated HTML files, updated by some wild PG triggers setup, and served up by NGINX. IMO you could go way more traditional, with the same result. Worried that Postgres will tip over? Just read from the DB, but cache the results in memory with a short TTL on your Ruby server. Or, if you’re worried about not just Postgres, but also your Ruby server getting overloaded, just go with a fully server rendered page, and cache the whole page. Could use NGNIX cache, as you’re already using NGINX, or a CDN, or Varnish, whatever. No JS loaded comments at all. Either way, caching should be able to get you tonnes of scalability, easily avoid hug of death, without resorting to PG triggers calling a Ruby server that writes files to disk. With that being said, this approach works too, and if it’s just a personal page with you as the only dev, you understand it, so it’s all good. But I’d certainly avoid this approach at a company, largely because it’s unusual and will take new devs longer to wrap their heads around.
- thom 4y agoThe approach in the article is functionally different though, in that the cache is always immediately invalidated when a new comment comes in. So it’s not the same result as using a cache with a TTL.
- 411111111111111 4y agoYou can always set a super short TTL like 15s or even 1s. It's still going to make scaling a non-issue like that and at some point the caching will be fresher then the html generation, as that needs to be done too (and read after by the http server). It's fun to make architecture like in the article though, I'm guilty of it as well with previous hobby projects
- yashap 4y agoOP’s approach has fast cache updates, but it’s still eventually consistent. Like, the whole flow of PG trigger, handled by Ruby server, then writing to a file, that happens outside the flow of POST comments, and could sometimes finish after POST comments returns to the caller. Also, with my suggestions, you can cache invalidate too, if a short TTL isn’t good enough. Based on writing files to local disk, it seems like a “single server” scenario, so invalidating the in-memory cache on POST comments is dead simple, and can be truly immediately consistent. Likewise, for full page caching, if using Varnish as a cache, Varnish lets you invalidate easily too. NGINX and CDNs don’t (generally), but if a short TTL isn’t enough, and you need instant invalidation on new comments, then just go in-memory cache or Varnish.
- pier25 4y agoIf you already have a DB and backend app just use a CDN.
- pyrolistical 4y agoHave the static comment creation trigger an upload of static html to cdn and you will have a hug safe solution
- yawnxyz 4y ago> The big idea is to write the comments as static HTML, only when comments change, instead of doing a database query to display them every time I'm using Cloudflare to do this, and any kind of db updates will update the cache. I'm new to all this, but wouldn't this be spiritually the same, e.g. if the cache would serve the latest db content instead of hammering the db? (I'm doing this because I'm using Airtable as db, and it's incredibly slow, rather than a "hug of death" kind of scenario)
- userbinator 4y agoThe big idea is to write the comments as static HTML, only when comments change, instead of doing a database query to display them every time. This prevents the “hug of death” if you get a burst of traffic. I believe this was the technique used (and still being used?) by 2chan, the ancestor of the infamous 4chan. It didn't even need a database, as it was entirely file-based and simply appended each post to the appropriate file. Visitors were served static HTML with no JS required.
- masklinn 4y agoYep, I'm surprised the OP doesn't use such a scheme, possibly with something SSE-ish so you don't mix the content and comments on-disk but they're both retrieved as a single request. This way all you need is a POST endpoint (which might just be a cgi script) to receive the comment and update the comment file. Gets more complicated if you support threaded comments or pagination, but for a linear comment system it should be more than enough.
- russelg 4y agoThe main 4chan open source clone project (tinyboard -> vichan -> infinitychan) works this way.
- janci 4y agoThat's similar to what was my firtst message board done. It was a static HTML file with a form. The form submitted to a separate PHP script that appended the comment to the original HTML.
- markdown 4y ago> Another interesting approach would be to write the comments into each HTML file directly, instead of in a separate file, so you wouldn’t need JavaScript at all. A better approach IMO. Why require js for something as trivial as adding a small block static html to a page every time it loads? Makes no sense.
- habibur 4y agoThis is how the first discussion board on the web was written to work. Search for Matt's script WWWBoard. Perl script added the posted comment with date into the HTML file's insertion point. At that time in the 90s this was heavily deployed everywhere. This was before PHP-BB and other database based boards got popular.
- connordoner 4y agoThis was my first memory on reading the post too. Ah, Matt Wright, and FormMail, WWWBoard and even… OMG… guestbooks (remember those?!).
- ksec 4y agoYes, and we have 100x the DRAM capacity, 1000x CPU Performance if we include MultiCore, and 10,000x faster IOPS on SSD. We could have gone back to the old ways of doing things and it would ( or should ) actually be more scalable.
- hawski 4y agoI've got an idea to allow comments to be appended to the HTML and counting on the fs implementation synchronization, so only allow comments around 4k (or whatever is prooved to be safe) for a complete single append. It would require the page to not have a footer and closing tags for </body></html>. Here is an incomplete try: https://github.com/hadrianw/compend https://github.com/hadrianw/compend Lock and unlock is there for moments when someone wants to modify the content or moderate comments while still allowing commenting.
- bestest 4y agoLooks quite overengineered. Most modern web frameworks already do it aswell, and using a much nicer static generation approach.
- ckdot 4y agoThis has to be a joke. If the Ruby implementation that saves the comment would simply contain the logic to save the static HTML file - the obvious implementation - no Postgres events are required.
- tracnar 4y agoHaving a database separates the view from the model. If you want to modify the generated HTML, it's much easier with the comments being separate from the HTML itself.
- ckdot 4y agoI don’t say to get rid of the database. Of course it makes sense to store the comments there. I say get rid of 1) the database events and 2) the Ruby logic to listen to these events. This can be done by the Ruby saving logic. It’s just over engineering to use events.
- darekkay 4y agoI have written and maintained a blog post about different methods and tools to include comments on a static website: https://darekkay.com/blog/static-site-comments/ https://darekkay.com/blog/static-site-comments/
- rolandog 4y agoHave you tried Hypothesis[0]? It also seems like a nice solution that offers other features in addition to comments. [0] https://web.hypothes.is/ https://web.hypothes.is/
- darekkay 4y agoThanks for the suggestion! As I understand Hypothesis, it is a social tool for users to annotate any web page; as a website owner, there is nothing I need to do, right? In that case, it doesn't match the original intent of my blog post, but I think I should still mention it.
- rolandog 4y agoYou're totally right that this would be something that the users would be using... perhaps the only requirement as a website owner would be to properly configure the Content Security Policy (CSP) HTTP headers [0]. [0] https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP
- quickthrower2 4y agoYou could also do it by chucking something like a Varnish cache in front of it, and invalidate a page on a new comment.
- Kwpolska 4y agoIs there a point in using NOTIFY in PostgreSQL, instead of just generating the HTML file when saving a new comment? The performance cost of doing a simple SELECT and writing a file in the request handler is neglible (especially since you are talking to the database anyway), but a NOTIFY-less solution is far simpler and will work with any database, even sqlite.
- nesarkvechnep 4y agoHTTP caching + conditional requests are a much easier approach.
- ljm 4y agoIt looks like you could trivially exploit this with XSS? The example code has glaring vulnerabilities in terms of taking user input and then inserting it into the DOM verbatim. Anyone who copy/pastes that example is going to have a rough time.
- amelius 4y agoThis is a very specific implementation of a very general idea.
- aloukissas 4y agoOne thing to know if you’re using this Postgres feature is that the max payload is 8000 bytes [1] That was the initial implementation for supabase’s real-time updates (afaik) and for this reason they had to resort to read the WAL. [1] https://www.postgresql.org/docs/9.4/sql-notify.html https://www.postgresql.org/docs/9.4/sql-notify.html
- aloukissas 4y agoThis is another cool way of doing it /w Elixir + Lit https://launchscout.com/blog/embedded-web-apps-with-livestate https://launchscout.com/blog/embedded-web-apps-with-livestat...
- samsquire 4y agoI call this hybrid static websites. It allows you to statially generate pages for even people who are logged in. It means the generation step is powered by interaction with the static site. I use a special api.subdomain.tld proxy_pass to a server to trigger the site generation. I wrote a forum that regenerated the static files when a post is added. Including the user's own account pages, those were statically generated too. In theory a change to a user's account should only change that user's account pages. I wrote a medium article about it. I was trying to get as many requests per second while logged in. It's an extension of the cache invalidation problem. I got to 3288 requests per second on a virtual machine. A static site should get at least 12,000 requests per second. For reference, Visa and Mastercard process 40,000 requests in total per second for financial transactions. I use an nginx authenticator and Rust authentication program that talks to Redis to check the IP address of a user's session token in the URL. You could probbaly improve the performance further by using an embedded Lua script in Nginx and using that to authenticate IP address rather than doing a HTTP request for each request that the nginx authenticator does. You can see the static site as a cache that needs to be updated based on new inputs. https://medium.com/@samuelmichaelsquire/1633-requests-a-second-while-logged-in-3bca95cf3cf1 https://medium.com/@samuelmichaelsquire/1633-requests-a-seco...
- funstuff007 4y agoIf you go this route isn't your server spending a lot of CPU cycles generating pages that will never be accessed?
- krapp 4y agoYou're only regenerating pages once when necessary, then they just sit on the server as flat files until someone reads them. That takes up fewer CPU cycles than dynamically generating pages for each request, unless you're rebuilding multiple pages at a time, as with a messageboard or forum. But even then I would expect file I/O to be a bigger bottleneck than the CPU, or race conditions if posts are still being sent during a rewrite.