5 ms·

Unnamed Forum | San Francisco | Remote | Contract You may know me here as the owner and founder of rsync.net. This is not a job posting for rsync.net. Rather

by rsync 4y ago

Unnamed Forum | San Francisco | Remote | Contract You may know me here as the owner and founder of rsync.net. This is not a job posting for rsync.net. Rather, I am starting a new venture that involves creating a threaded discussion software very much like what we are using right here on HN. The full job description, which is likely to be a contract position but could evolve into a full time development position, is here: https://kozubik.com/forumjob.html https://kozubik.com/forumjob.html

sroussey 4y ago
Having written forum stuff a few times over, I’m not sure you realize what you should even ask for. We never released the last rewrite, so could sell that, but it’s old… Once a community is started, you will spend all your time on content moderation features (less if more like HN than Reddit). Be sure the dev separates out the messages from ordering. I just saved you a rewrite, lol.
- mod 4y ago
  > Be sure the dev separates out the messages from ordering. Can you explain? I'm not sure what you mean by this.
  - sroussey 4y ago
    The temptation is to have the message date and/or ranking number in the message table. This is fine and good. The next temptation is to have the index on that date or (date, rank) and use that for displaying the list of messages. This is bad. For the list of threads (messages without a parent) you want to know which to show… but there are a lot of calculations that would make this inefficient. Is it a root message? Is it deleted? Is it marked by spam or other content filters? Is it approved? You can start to see how this gets messy. How do you show messages that are approved for one group of readers and not for others? And what is their order? So separate the concerns… in the db. SQL is your friend if the db has the right data packaged and indexed the right way. It’s a different problem and has a different solution. Now mind you, I cut my teeth on this in the late 1990s and early 2000s when we grew quickly into the Alexa 1000 and farther up. China blocked us as one of the original dozen sites on the great firewall (via dns resolved IPs, not the domain itself… hahaha, oh the chaos they empowered!). Databases were more expensive than engineering time back then, so efficiency mattered. People do dumb things now and use giant clusters when a single machine would do, but I digress…
    - evandwight 4y ago
      Why not an index on created for speeding up the "new" sort? What indexes should you create? One for each sort you have? (I'm just playing around cloning Reddit and running into all sorts of performance problems. Mostly due to celery taking all my memory)
      - sroussey 4y ago
        Yeah, these are the performance issue solutions. Based on the way open source databases work (clustered indexes), move your indexes to a table that has the sole purpose of indexing, and index everything you sort by index{search constants, then sorting columns, then id }. You can test with your workload, but I found that full covering indexes sat in ram better (so included the id even though not needed). 20yr ago MySQL didn’t have reverse index columns so we had to make (1-timestamp) cols and index those! Also used int for time as it was easier to index back then and you don’t need sub second or anything. Smaller indexes are better.
314 4y ago
Your certificate is broken, clicked on the click but decided not to bother after the five warnings from Safari and the need to unlock the system security settings to set an exception.
- rsync 4y ago
  Is your date set correctly on your computer ? I just double checked kozubik.com SSL cert and it expires in 7 days (it is an acme.sh generated certificate, etc.). Or is Safari complaining about this cert chain ? Genuinely curious ...
  - iammiles 4y ago
    Not the person you're responding to, but it appears broken on Chrome and Safari on macOS (Firefox is fine) with a correctly set date. On further inspection it looks like there's a problem with the R3 certificate being expired. After a little Googling and I found this: https://community.letsencrypt.org/t/help-thread-for-dst-root-ca-x3-expiration-september-2021/149190 https://community.letsencrypt.org/t/help-thread-for-dst-root... Hopefully that helps!
Dana_abramenko 4y ago

by rsync 4y ago

sroussey 4y ago

Having written forum stuff a few times over, I’m not sure you realize what you should even ask for. We never released the last rewrite, so could sell that, but it’s old… Once a community is started, you will spend all your time on content moderation features (less if more like HN than Reddit). Be sure the dev separates out the messages from ordering. I just saved you a rewrite, lol.

mod 4y ago
> Be sure the dev separates out the messages from ordering. Can you explain? I'm not sure what you mean by this.
- sroussey 4y ago
  The temptation is to have the message date and/or ranking number in the message table. This is fine and good. The next temptation is to have the index on that date or (date, rank) and use that for displaying the list of messages. This is bad. For the list of threads (messages without a parent) you want to know which to show… but there are a lot of calculations that would make this inefficient. Is it a root message? Is it deleted? Is it marked by spam or other content filters? Is it approved? You can start to see how this gets messy. How do you show messages that are approved for one group of readers and not for others? And what is their order? So separate the concerns… in the db. SQL is your friend if the db has the right data packaged and indexed the right way. It’s a different problem and has a different solution. Now mind you, I cut my teeth on this in the late 1990s and early 2000s when we grew quickly into the Alexa 1000 and farther up. China blocked us as one of the original dozen sites on the great firewall (via dns resolved IPs, not the domain itself… hahaha, oh the chaos they empowered!). Databases were more expensive than engineering time back then, so efficiency mattered. People do dumb things now and use giant clusters when a single machine would do, but I digress…
  - evandwight 4y ago
    Why not an index on created for speeding up the "new" sort? What indexes should you create? One for each sort you have? (I'm just playing around cloning Reddit and running into all sorts of performance problems. Mostly due to celery taking all my memory)
    - sroussey 4y ago
      Yeah, these are the performance issue solutions. Based on the way open source databases work (clustered indexes), move your indexes to a table that has the sole purpose of indexing, and index everything you sort by index{search constants, then sorting columns, then id }. You can test with your workload, but I found that full covering indexes sat in ram better (so included the id even though not needed). 20yr ago MySQL didn’t have reverse index columns so we had to make (1-timestamp) cols and index those! Also used int for time as it was easier to index back then and you don’t need sub second or anything. Smaller indexes are better.

314 4y ago

Your certificate is broken, clicked on the click but decided not to bother after the five warnings from Safari and the need to unlock the system security settings to set an exception.

rsync 4y ago
Is your date set correctly on your computer ? I just double checked kozubik.com SSL cert and it expires in 7 days (it is an acme.sh generated certificate, etc.). Or is Safari complaining about this cert chain ? Genuinely curious ...
- iammiles 4y ago
  Not the person you're responding to, but it appears broken on Chrome and Safari on macOS (Firefox is fine) with a correctly set date. On further inspection it looks like there's a problem with the R3 certificate being expired. After a little Googling and I found this: https://community.letsencrypt.org/t/help-thread-for-dst-root-ca-x3-expiration-september-2021/149190 https://community.letsencrypt.org/t/help-thread-for-dst-root... Hopefully that helps!

Dana_abramenko 4y ago