13 ms·
Just how big are porn sites?
- furyofantares 15y ago> While it obviously varies from site to site, most adult sites will probably store in the region of 50 to 200 terabytes of porn. This is quite a lot for a website (only something like Google, Facebook, Blogger, or YouTube would store more data), Netflix, Hulu, Apple, Flickr, Dropbox, Steam... I find it disappointing that this list (and the one about bandwidth saying only YouTube or Hulu comes close to Xvideos) are incomplete but they aren't really presented as such.
- TazeTSchnitzel 15y agoSteam? Are you sure their catalogue and save files take up that much space?
- jetti 15y agoThis and the article about YouPorn's stack make me really want to go work for these places. I'm sure that the day to day challenges would be fascinating and it would be a thrilling technical experience.
- dude_abides 15y agoYep! I wish it was more socially acceptable :)
- jetti 15y agoSeriously! I think it wouldn't be hard getting a job after wards, but the real challenge would be explaining it to the missus! :)
- SystemOut 15y agoPron doesn't have that much of a negative stigma attached to it when it comes to the tech industry. At least none that I've experienced. I do admit that I use the holding company's name on my resume/linked in profile but I always disclose pretty early on what it was just to make sure. If they have a problem with it I don't want work for them anyways. Also, you try to stop saying the word hard after working in the adult business....everyone snickers when they know where you've worked. ;-)
- rmc 15y agoHaters gonna hate. Ignore the people who tell you you're stupid. Us geeks have been picked on for years.
- igrekel 15y agoI know some companies build a separate IT entity to have a more socially acceptable front for their employees.
- Killswitch 15y agoI worked in porn running sites like these (Not in that scale, but hundreds of thousands of daily visitors) for about 10 years... Sure traffic is easy to come by, but making money is a lot harder. You have to sell memberships to porn sites that have premium memberships and you get either revenue sharing or a flat fee.
- Killswitch 15y agoAlso you're fighting to sell a membership at sitea.com when some torrent or pirating site offers full rips of sitea.com within the top 10 results on Google for that search. So basically you're trying to sell a $5 cup of coffee to a guy while someone next to you is giving away that same coffee for free + a bagel.
- jetti 15y agoWhich is why I would want to work for one and not run one. I just focus on technical, they just give me my money and everybody is happy!
- Killswitch 15y agoUnless you're targetting working for a major company (Brazzers who owns PornHub, Tube8, KeezMovies, Extremetube, etc) you've gotta have some serious skills, otherwise you have to create your own site... Not many in the realm of porn actually have full companies behind them, most of them are just your average joe running a site by himself. Not to mention most porn website owners are cheap... Doing any type of work in that realm is a pain.
- jetti 15y agoInteresting. I guess the other factors come in and make the technical aspect less...appealing.
- Killswitch 15y ago
- stickfigure 15y agoKink.com is almost always looking for talented developers. Message me (my username @ infohazard.org) and I'll put you in contact with the right people. It runs on a JBoss/Hibernate stack.
- va_coder 15y agoMasochism and Hibernate go together like peanut butter and jelly
- stickfigure 15y agoThanks to Hibernate's second level caching, Kink.com's entire user-facing website traffic could be served by a single appserver and single MySQL instance.* Try that with Ruby. * Not including NATS, which is an abomination.
- jaggederest 15y agoHave done, thanks, varnish is the answer.
- stickfigure 15y agoUnless you serve personalized pages. Which Kink does.
- moe 15y agoA small bit of javascript tends to help with that.
- stickfigure 15y agoThat doesn't solve the problem at all, it just moves it from the generate-HTML request to a followon get-the-content-appropriate-to-this-user ajax request. Generating the information for each page is complicated, especially at Kink where your ability to interact with each "shoot" is determined by subscription rights, admin rights, microcurrency purchases, and who knows what else these days (I haven't seen the codebase in over four years). Hibernate's clustered 2nd-level cache is still pretty magical. It means that the vast majority of web page requests are serviced out of RAM with zero database hits - without writing any special caching code. And it's transactional. For a certain set of scaling problems, this feature is golden.
- pgrote 15y agoDoes anyone have real world experience monetizing porn sites on a such a large scale? I am not familiar with the business aspects of it. Is it driven through affiliate? Direct advertising? Something else?
- mcphilip 15y agoThe guy who made videobox did an AMA on reddit not too long ago. You may find some of it relevant to your questions. http://www.reddit.com/r/IAmA/comments/gt0l1/i_have_a_cs_degree_from_harvard_and_i_run_a_porn/ http://www.reddit.com/r/IAmA/comments/gt0l1/i_have_a_cs_degr...
- bdreadz 15y agoGuy that runs 4tube as well as a bunch of other high traffic sites did an ama as well. http://www.reddit.com/r/IAmA/comments/htn7x/i_run_porntubecom_4tubecom_among_others_ama/ http://www.reddit.com/r/IAmA/comments/htn7x/i_run_porntubeco...
- Killswitch 15y agoYou have two options, outright selling the ad spaces (like what the PornHub network does) or attempt to sell memberships at other sites for commission. Not easy.
- SystemOut 15y agoIt depends on the kind of business you are. If you are actually a producer of porn (bang bros, naughty america, kink, etc.) you sell subscriptions or "points" packages for pay-per-view/live streaming. To get those subscriptions you use just about every method out there and continue to try since the way traffic is driven to porn sites changes every year. It's a mix of affiliate traffic with a different mix of programs like revenue sharing, pay-per-signup, etc. and then you throw in a mix of other direct ad programs like ad words. One of the difficult parts of being in the porn business, aside from the difficulty in getting a bank that will actually process your transactions, is that the way traffic is driven to your business changes annually. The affiliates bear the brunt of that, though, but you pay them about 50% of the revenue they bring in for that so it is costly. But you're always trying to find ways to bring in traffic yourself since you don't like giving up 50% of your revenue to an affiliate.
- NameNickHN 15y agoFrom the article: [...] when you factor in what those porn surfers are actually doing [...] Ahem, I'd rather not. ;-)
- benologist 15y agoI wish HN would cap how many times ExtremeTech, ITWorld, MacObserver etc can be submitted.... by employees of those sites.
- wyclif 15y agoAdd TechCrunch, Mashable, TheNextWeb, and PandoDaily to that list. I'm burned out and fatigued by the middlemen. Seriously, just link to the fucking content.
- Groxx 15y agoSounds like a job for an extension, honestly. Something to remove entries which link to URLs which match a regex. Interested? I could probably make one when I get some time, though my knowledge is currently limited to Chrome.
- wmf 15y agoSpammers (er, social media experts) don't use anti-spam browser extensions. Let's fix this on the server side.
- corin_ 15y agoClearly the plugin he's talking about is for people like you to hide these submissions from you - not in the hope that spammers will install it themselves and find themselves unable to submit.
- wmf 15y agoAh, I didn't think about it that way. Then you're back to Usenet killfiles; just hide the bad stuff (driving away newcomers who haven't learned to killfile yet) instead of getting rid of it.
- simcop2387 15y agoi'd go for greasemonkey myself since that will work in chrome and firefox if done right. I think it can also work in safari, but i don't know about any others.
- commiebob 15y ago>To put that 800Gbps figure into perspective, the internet only handles around half an exabyte of traffic every day, which equates to around 50Tbps — in other words, a single porn site accounts for almost 25% of the internet’s total traffic. That should be more like 1.6% if those numbers are correct... Still an absurd amount of traffic.
- mrsebastian 15y agoSorry, it should've read 2%. Shift-5 = % -- damn my sticky shift key (I did a lot of hands-on research in writing this story...)
- fourmii 15y agoI love it how the article ends with >The Internet really is for porn< >It’s probably not unrealistic to say that porn makes up 30% of the total data transferred across the internet.< If this is the case, is the online porn industry held up as models of high tech and innovation? I thought I heard somewhere investors and VC's in particular, shy away from porn...
- feral 15y agoIts interesting that the article starts with a riff on the opening line of Pride and Prejudice. "It is a truth universally acknowledged, that a single man in possession of a good fortune must be in want of a wife." It seems like there could be several intended meanings behind that.
- yaix 15y agoMost new technologies were. Photography, film cameras, home video, etc. Porn was always a great help in getting it to the main stream (years ago a read a good article about the history of porn in technology, but can't find it now). Now I'm really looking forward to the creative things to be done to Google's new "Glass" product.
- medinism 15y agoIs anyone at all surprised their tech-stack is php? is it because of legacy or is it because any sensible person moving petabytes of data would use? or does it even matter
- jetti 15y agoI'm assuming you are talking about YouPorn. I'm surprised they went from Perl to PHP. But they obviously know what they are doing. I wonder if they use HipHop at all.
- cinquemb 15y agoI think your spot on. I wonder if PHP might get more usage at larger sites because of HipHop.
- Killswitch 15y agoPHP is the most used language in the adult realm.
- jetti 15y agoDo you know if there is a reason for that?
- Killswitch 15y agoLike stated before, most adult website owners are average joes, and PHP being easy to learn with a low entry barrier it was the logical reason to get a site up and running fast. Also because many of the tools written for the adult industry were done in PHP. Just like ICQ it won't ever be replaced as the standard for the industry.
- smcl 15y ago"Just like ICQ" What's it used for in the porn industry, do you know? I didn't know there'd much need for IMing
- deleted 15y ago[deleted]
- SystemOut 15y agosigh The article makes it sound like these sites are doing everything themselves including pushing the bits. Maybe some are but I can say from personal experience that most of your traffic, if you're smart, comes out of a CDN. The sites themselves are definitely not that interactive which makes them simpler to publish. The pages are almost all cached and that doesn't take much horsepower to serve up. The big video sites have ratings and comments but they are not that big of a deal. People go to watch porn sites to watch porn, not interact. Customer analytics have shown that over and over. I know of virtually no porn company that handles their own transactions, either. They all go through billing companies that handle things like PCI compliance for them. Most sites also use a system like NATs to do their affiliate management. You need one that the affiliates trust isn't shaving sign-ups from their account. They tend to trust NATs. For the data on the backend you just have a SAN to manage the data or you just manage it on a few servers with lots of disks but if you are really at the 100TB mark then you get a SAN I would think. That's what we did. Sure, it's a lot of space but they're big files so managing them isn't that hard. I'd say the largest issue that a company like YouPorn will have is the amount of data in their working set for a CDN. CDNs generally charge you for the size of your working set that they keep at each POP in their network so you want to keep it as small as possible. At the end of the day running a large porn network is more about integrating the myriad of partners you need to run the network. The infrastructure is interesting for a while but once you have it working the business of doing deals and handling promotions and figuring out why integration point A isn't working like it should is what keeps you busy.
- stickfigure 15y agoThis comment needs a huge +1. It's really quite easy to serve large volumes of porn. The dataset doesn't change often and 90% of your working set is the first couple pages of content. Back in 2007 when I was in the business, there was only one CDN that would touch porn (LimeLight) and they were absurdly expensive. Today there are hundreds of porn-friendly CDNs and they charge 1/10th the price (no exaggeration). Storing a couple hundred terabytes of porn is not expensive or complicated. Sites like YouPorn don't authenticate their content. Most of the high-volume web page content is static. Even then, you're looking at just a few page views before the user spends 5 minutes watching a video that streams from the CDN. Payment transactions are handled by third parties, and usually abstracted through third-party affiliate software like NATS. Which, BTW, is a piece of junk and the one part of our system which we had trouble scaling. Big bandwidth numbers sound impressive but the truth is running even a mildly successful social network with heavily personalized pages is ten times harder than running even the largest porn sites.
- zerostar07 15y agoWhat would the traffic be if there these sites were not blocked in a large part of the world?
- jsherry 15y ago"While it’s difficult domain to penetrate" I'm deeply sorry, but couldn't resist...
- omgsean 15y agoI think that was intentional.
- caycep 15y agoi heard it offhand from an acquaintance who did a google internship that google has to downrank porn sites by several orders of magnitude, otherwise all that would ever come up in google searches would be porn...
- luser001 15y agoI call bs. Everything I've seen about the link structure of the Internet indicates that this is false.
- patrickaljord 15y agoYou are correct, Matt Cutts explains it here: http://www.youtube.com/watch?feature=player_embedded&v=Dn87YqI73mY http://www.youtube.com/watch?feature=player_embedded&v=D...
- naryad 15y agoI noticed this a lot with the regional languages. For example searchig for a simple word 'bhabhi' (this means brother's wife) returns a whole lot of erotic stories instead of the meaning of it or its usage etc. Looks like Google does not downrank regional porn/erotica sites.
- kragen 15y agoMaybe porn is more popular in India? I mean, the US state with highest porn subscriptions is Utah, which is also the most religiously conservative, and India's even more so.
- digisth 15y agoWhat's really interesting to me personally is how porn continues to stay ahead of, or at least at/near the front of, the pack technology/performance-wise. Back in the late 1990s and early 2000s I co-ran the technology department at a very large network of high-traffic adult web sites (I'm not sure exactly where we would have been in the rankings, but I'd take a wild guess to say it was top 20, if not top 10.) We were doing streaming video (in Real, QT, and WM) at a time when it was still images as the default. Reading comments from SystemOut and stickfigure reminded me of just how (obviously) primitive everything seems compared to today, but we still made it work. Some broad notes from the period: - Started with single processor Sun SPARCs, which were later replaced by a dual and quad core ones (went from 32 to 64 bit early due to file size limitations), along with a collection of Linux boxes from Penguin Computing (remember them?) Most were in the mid-hundreds MHz range, topping out at a blazing 1GHz by the end. - Apache, mod_perl, MySQL (postgres for one system), later replaced some of the front end code with PHP. - No CDNs! Akamai was more or less the only game in town and was still unproven/considered too expensive at the time so we did traditional multiple-host setups (things like image1, image2, along with RRDNS for some other bits) - No really good, well-integrated turnkey billing systems. The ones at the time often took too large a chunk of the revenue or were designed for low volume/were very inflexible. Custom billing code to directly talk to charge processors (we spoke a custom protocol right over UDP to ours. We had a dedicated line to the processor, too IIRC. Every time a transaction was processed, you got to hear a classic modem-like noise. The hardware on our side was connected to a text-terminal (Monochrome, orange text.) - In-browser video started out using NPH tricks(!), later used a custom Java applet. Most, however, was served directly to separate client applications. In the days before the YouTubes and Vimeos came along, you had to yes, have your customers download 3rd party software and then provide support for it. - RAID 1 under Linux at the time had some ugly bugs which would partially corrupt one of the mirrors, requiring weekly manual rebuilds. I had a script monitoring for corruption which would send an email to this crazy old device called a "pager." The corruption always seemed to occur 15 minutes after I fell asleep, too. Anyhow, interesting to see just how far things have come. Impressive numbers.
- silentscope 15y ago"While it’s difficult domain to penetrate..." Very punny.
- yaix 15y agoBut how do they make money? They give all the naked girls and guys away for free (and the users probably can achieve the purpose of their visit by means of that free content alone).
- kanchax 15y agoOn a non-tech note, I read on the internet (yes, aldaily, do not have the link right now) that 1/3 of Casanova's autobiography was about his affairs with women. It came to me when the article said that a third of the internet was dedicated to porn. Conclusion: A third of our lives are dedicated to sex.