4 ms·

First off, I am not saying that youporn guys are not decent but IMHO, people shouldn't get too excited about this. Youporn is somewhat quite a simple site I'd

by clu3 15y ago

First off, I am not saying that youporn guys are not decent but IMHO, people shouldn't get too excited about this. Youporn is somewhat quite a simple site I'd imagine. It's just categorized videos with a bunch of stats (views, related video...), and users. There's almost no social interaction between users (i've just registered an account just to confirm that). That's a very clear (and good) reason why 100% redis suits for this website, apart from the goodness of redis like sorted sets are perfect for the "top rated" or "being watched" sections Imagine the site now wants to add a little bit of social interaction, like allowing users to post video, follow each others... I'd say 100% redis wouldn't work any more

raphman 15y ago
Imagine the site now wants to add a little bit of social interaction, like allowing users to post video As far as I know, Youporn is all about users posting videos (whereby amateur porn is probably only a small part of all uploads, I guess)
asto 15y ago
Imagine the site now wants to add a little bit of social interaction, like allowing users to post video, follow each others Follow each other? I'm not sure you know what the point of a porn website is.
- clu3 15y ago
  I'm pretty sure i know what I'm looking for in a porn site. There's many things youporn can do to help me watch better stuff. Say I'm a fan of a porn star XXX and I'd like to see a simple notification that there's recently a new video tagging her. Same thing if I like some particular user's taste for porn and would like to get noticed if he/she has posted a video. Basically if it gets more social, my wall or dashboard or whatever, would be more sorta relevant.
CWIZO 15y ago
You would imagine wrong. I'm running a video sharing page (think FB&youtube mashed together with a dash of blogger) with about 800k unique/month users and it's no picnic.
- lmm 15y ago
  Sure, but the sharing is what makes it hard - as soon as your pages are per-user it gets interesting. For youporn, each video page is essentially static (maybe there are comments? Even if there are, I doubt they display a different set of comments to each /viewer/). So video pages are easy to cache, the database layout is simple, and sure there's some effort involved in scaling up but it's nowhere near as hard as e.g. tumblr's architecture that was featured recently (where every viewer sees a page with all the posts from everyone they're following, and so everyone's page is different) /briefly worked at last.fm
- hello_moto 15y ago
  Do you store the video using some sort of distributed file systems? specialized machine? SAN/NAS? Or the good old' NFS?
  - CWIZO 15y ago
    Nothing fancy actually. We have 3 dedicated storage servers (two in the same DC as our web server, one in germany), with lots and lots of disk space. We then SCP converted videos to them and serve them trough nginx (supports seeking).
    - hello_moto 15y ago
      Hey, thank you so much for the information. I hope you don't mind to share a little bit more (it's okay if you don't reply due to IP or whatnot, I'll understand). How do you handle where to put the files on which disks? do you write custom software or there's some sort of off-the-shelf software to do this? The reason I asked about this is because it looked like a few people I talked to seem to prefer to use Distributed File System such as HBase, GlusterFS or something else so that they don't have to write extra code. And I'm guessing you back them all up on daily base as well?
      - CWIZO 15y ago
        We split files into two levels of folders. For instance if a video has a filename "somethingsomethingXY" it will go into "videos/y/x/" folder. Then it's up to our sys admins to mount those folders to whichever disk they want. So from the app's point of view it doesn't matter where the file actually is. And we backup everything every couple of hours. Like I said, nothing fancy but it works without any issues (apart from a small delay when you have to copy the file to 2 or 3 different servers (redundancy) before it becomes available, but it's not a big problem but I have few ideas how to solve that if it becomes one :)).

by clu3 15y ago

raphman 15y ago

Imagine the site now wants to add a little bit of social interaction, like allowing users to post video As far as I know, Youporn is all about users posting videos (whereby amateur porn is probably only a small part of all uploads, I guess)

asto 15y ago

Imagine the site now wants to add a little bit of social interaction, like allowing users to post video, follow each others Follow each other? I'm not sure you know what the point of a porn website is.

clu3 15y ago
I'm pretty sure i know what I'm looking for in a porn site. There's many things youporn can do to help me watch better stuff. Say I'm a fan of a porn star XXX and I'd like to see a simple notification that there's recently a new video tagging her. Same thing if I like some particular user's taste for porn and would like to get noticed if he/she has posted a video. Basically if it gets more social, my wall or dashboard or whatever, would be more sorta relevant.

CWIZO 15y ago

You would imagine wrong. I'm running a video sharing page (think FB&youtube mashed together with a dash of blogger) with about 800k unique/month users and it's no picnic.

lmm 15y ago
Sure, but the sharing is what makes it hard - as soon as your pages are per-user it gets interesting. For youporn, each video page is essentially static (maybe there are comments? Even if there are, I doubt they display a different set of comments to each /viewer/). So video pages are easy to cache, the database layout is simple, and sure there's some effort involved in scaling up but it's nowhere near as hard as e.g. tumblr's architecture that was featured recently (where every viewer sees a page with all the posts from everyone they're following, and so everyone's page is different) /briefly worked at last.fm
hello_moto 15y ago
Do you store the video using some sort of distributed file systems? specialized machine? SAN/NAS? Or the good old' NFS?
- CWIZO 15y ago
  Nothing fancy actually. We have 3 dedicated storage servers (two in the same DC as our web server, one in germany), with lots and lots of disk space. We then SCP converted videos to them and serve them trough nginx (supports seeking).
  - hello_moto 15y ago
    Hey, thank you so much for the information. I hope you don't mind to share a little bit more (it's okay if you don't reply due to IP or whatnot, I'll understand). How do you handle where to put the files on which disks? do you write custom software or there's some sort of off-the-shelf software to do this? The reason I asked about this is because it looked like a few people I talked to seem to prefer to use Distributed File System such as HBase, GlusterFS or something else so that they don't have to write extra code. And I'm guessing you back them all up on daily base as well?
    - CWIZO 15y ago
      We split files into two levels of folders. For instance if a video has a filename "somethingsomethingXY" it will go into "videos/y/x/" folder. Then it's up to our sys admins to mount those folders to whichever disk they want. So from the app's point of view it doesn't matter where the file actually is. And we backup everything every couple of hours. Like I said, nothing fancy but it works without any issues (apart from a small delay when you have to copy the file to 2 or 3 different servers (redundancy) before it becomes available, but it's not a big problem but I have few ideas how to solve that if it becomes one :)).