12 ms·

Reddit costs $33k/month to operate?

125 points by there 16y ago

d0m 16y ago
So.. how much cost HN? :) And what/who pay for that?
- jackowayed 16y ago
  I'm pretty sure YC pays for it. It's one big server, so it's probably like $1k/month or something. But HN isn't anywhere near 8M active users, and it's pretty slow much of the time and has brief downtime often (multiple times per day, I think) so they can reboot the server to speed it back up.
- tomjen3 16y ago
  Very little compared to reddit. PG said recently that HN has 60K unique/day. Reddit has around 8Million. In addition to that, the software behind HN doesn't do nearly as much as the reddit software: there is no automatic checking of new messages, there are no sub reddits you need to work on, there are only so many votes to be processed, etc. I have no clue about who pays for what, but I guess PG pays.
  - robryan 16y ago
    I would assume though it would be even easier here than on reddit to get people to donate towards server cost should the need ever arise.
    - rick888 16y ago
      The users of reddit are downright hostile toward anyone trying to make money. There was a thread last week about paid accounts. Out of 2000 comments, I think I only found one or two people supporting the idea. Most of the comments were somewhere along the lines of "paid accounts creates two classes of users and everyone should be equal" and "it's not my job to figure out how reddit makes money" (advertisements are actively blocked by 30% of users). When your userbase is filled with college kids and anti-capitalist, you are going to have a tough time making money,
      - zavulon 16y ago
        Yep. I think it's a valuable business lesson: unless you know what your plan to make money is, don't announce that your attitude is "ads are bad". Although I think they'll figure out how to make money, this situation could've been avoided if they ran ads from early on and weren't loudly against them.
      - Locke1689 16y ago
        Many of us are also gainfully employed/work in the "real world" ;)
      - saulhoward 16y ago
        There had been a donation drive the week prior to that, which was very well supported. I think people were saying that they thought the idea of 'paid accounts' just wasn't going to work as well as the donation drive had. The idea being that once people are paying for a actual service they expect more for their money. After all, they're aiming for 2% of the userbase subscribing. That's a large number when they're offering basically nothing of value in return. I know other 'freemium' business get 1-2% subscribers, but surely that's when they're offering a real improvement in the service, such as Dropbox's 2GB -> 20GB.
      - Super_Jambo 16y ago
        Being anti-advertising is no the same as being anti-capitalist. There is a good argument that advertising actively moves you away from the EMH which is one of the strongest arguments in favor of capitalism.
        rick888 16y ago
        "There is a good argument that advertising actively moves you away from the EMH which is one of the strongest arguments in favor of capitalism." I said anti-capitalist because many people were against advertisements and the gold accounts. While there may be many people lurking on reddit that are not anti-capitalist, it's a sentiment I see in almost every topic related to making money on that site.
      - jasonlotito 16y ago
    - masklinn 16y ago
      You shouldn't. It probably isn't the case.
jbail 16y ago
A comment or two down, jedberg explains they use reserved instances from Amazon, so their hosting fees are more like $22,390.37 a month.
- bkrausz 16y ago
  Actually they're even lower, since the 3 year instances are so much cheaper than the 1 year over time. (1820 * 57) + (910 * 23) = $124,670/yr in reservation fees Changes to (2800 * 57) + (1400 * 23) = $191,800/3yr in reservation fees Which, when spread over the 3 years, brings the cost down by $5,061.39/mo, making the final cost there actually $17,328.98. Of course this assumes they're doing 3 year reservations, but I don't see any reason not to on their XL instances (I could be wrong though, not sure how open Conde Nast is to logic like this).
  - masklinn 16y ago
    Conde Nast isn't involved in Reddit's operations. Also, > Actually they're even lower Yeah I like that you declare knowing the costs of Reddit's servers.
    - bsiemon 16y ago
      I think all the math is based on the known costs of EC2 services at various rate plans. All the people posting dollar figures here and on reddit are drawing from these know costs.
      - masklinn 16y ago
        > All the people posting dollar figures here and on reddit are drawing from these know costs. The others are quite clearly marked as estimates, this one not really, and the second estimate from the top in the reddit thread (yielding an estimate of $22,390.37/mo) got the following comment by one jedberg: > Yes, once again, you are totally accurate. That is almost exactly what it costs to run reddit, as of today. However, with our projected growth, we're looking to be closer to 350K by the end of the year. note that the comment got that estimate based off 1-year reserved servers, which jedberg's comment suggest is broadly correct rather than 3-year reservations.
deleted 16y ago
[deleted]
acangiano 16y ago
Amusingly, I can't read it because I get an error: "the service you request is temporarily unavailable. please try again later.".
- ronnier 16y ago
  I have a screenshot of it here: http://toadjaw.com/screenshot?id=http://www.reddit.com/r/blog/comments/ctz7c/your_gold_dollars_at_work/c0v8ug8?context=1 http://toadjaw.com/screenshot?id=http://www.reddit.com/r/blo...
  - memoryfault 16y ago
    What is toadjaw? The homepage didn't seem to have any info.
    - ronnier 16y ago
      It's my after hours/weekend project. It'll eventually be a bookmarking site. I'm about 75% of the way there before I open it up to the public. Some of the features are nearly completed, like the screenshots and article text viewer. Along the way, I realized it would be trivial to make a mobile HN reader, so I spent some time doing that http://toadjaw.com/hn http://toadjaw.com/hn
      - ritonlajoie 16y ago
        Hey I use your website everyday on my Iphone. Love it, and thanks for the new 'theming' and new buttons, that's better than last week ! Keep up your website. Maybe you should put an ad on it, I would click it once a day to help :)
  - acangiano 16y ago
    Thank you, ronnier. Very kind of you.
rufugee 16y ago
Wow...this is hard to believe. Numbers like that make me wonder if the cost of going with ruby or python (in reddit's case) or insert-your-slower-dynamic-language-here outweigh the benefits in the long run. How can anyone bootstrap a company and scale to the size of Reddit without quickly running into a "must fund or sale" situation? If it's truly a python issue, it should not be taken lightly by entrepreneurs. I hope it's not...I'm an avid user of both ruby and python, and want to believe they can both be used to create successful and maintainable (in terms of effort and cost) sites.
- joshu 16y ago
  Actually, I bet you it's a "we hit the database on every pageload" problem.
  - giantfuzzypanda 16y ago
    Yep. That's why you shouldn't have pylons noobs running a site like reddit.
  - emmett 16y ago
    I know the reddit guys, and I can tell you it's definitely not that. Hitting the DB is extremely rare - almost everything comes straight out of cache (either memcache, or memcachedb for caches which are more expensive to rebuild). Execution cost is dominated by the python app servers.
  - blhack 16y ago
    The overwhelming majority of reddit's pageviews are users that are not logged in, which is very very cacheable.
    - emmett 16y ago
      I am pretty sure this is false. While the overwhelming number of unique visitors to reddit are not logged in, those users are far less engaged than average. Logged in users make up a huge percentage of the pageviews because of their higher engagement level.
cheald 16y ago
That's actually not terribly surprising. Reddit's always been extremely conservative with ad space, and it seems like they could offset a lot of that if they started running more "traditional" ad slots. By their own numbers, they do about 14.3m pageviews/day. Let's assume half of that is AdBlocked. A single 300x250 at an exceptionally low $0.05 CPM would yield $3,575/day in ad revenue, or $107,250/month. I'll guarantee that reddit could pull far, far higher CPMs. Like, an order magnitude or two higher. Edit: I'm off by an order of magnitude, see below. Point stands, though, that there is a lot of ad money they're leaving on the table.
- _pi 16y ago
  Reddit's actual CPM is 8$. One of the admins said that Conde wants to keep it a premium brand and they don't actually set the CPM. I'll try to dig up a link. It's also why there's tons of cross reddit advertising, and adverts that are paid are only seen for a tiny bit of time.
  - endlessvoid94 16y ago
    I'm really interested in hearing more about this. Were you able to find a link?
    - _pi 16y ago
      http://www.reddit.com/r/ads/comments/cpm2d/i_cant_believe_somebody_bought_this_ad/c0ubpto http://www.reddit.com/r/ads/comments/cpm2d/i_cant_believe_so... Here you go. Took a bit. Also it's risen about 1$ in the last year. http://mediamemo.allthingsd.com/20090327/reddits-ad-experiment-is-good-news-for-conde-nast-maybe-for-digg-too/ http://mediamemo.allthingsd.com/20090327/reddits-ad-experime...
      - corin_ 16y ago
        Pretty depressing seeing comments like "Also there is no way a website has a $8CPM" and wondering why people who don't know what they're talking about keep on talking. I sell at double that rate for smaller sites on a daily basis ;/
deleted 16y ago
[deleted]
apower 16y ago
That's just the server instance cost. What about bandwidth and S3 cost? Bandwidth would cost a bundle.
- keefe 16y ago
  I haven't studied their arch, but I could imagine scenarios where spot instances would reduce the bill some, but yeah. Bandwidth. Ouch.
- jedberg 16y ago
  The bandwidth is less than 5% of the total cost. It's about $2500 a month.
pbhjpbhj 16y ago
I wonder what the best money-saving optimisation would be? Whilst it sounds like a lot of money remember that the whole sites annual EC2 bill could easily be paid for several years by one of the Newhouse brothers (who own Advance Publications, who own Conde Nast) selling one of their Reubens or probably just giving up their bank interest for a month (est. about ¼-million USD per day for the poor one : 1day of 2% annual interest on $4.5 billion).
- ryanwaggoner 16y ago
  Of course the Newhouse brothers could do that, but they could do it precisely for the reason they'd be unlikely to: if it's a bad investment, they shouldn't keep throwing money at it. Not saying that it is a bad investment, just that ignoring the problem because it's owned by a rich person who could easily bail it out ignores the way rich people think.
  - pbhjpbhj 16y ago
    >Not saying that it is a bad investment, just that ignoring the problem because it's owned by a rich person who could easily bail it out ignores the way rich people think. I don't want to argue that at all. I'd argue that bailing them out as if they're a charity is wrong given that supporting it would be pocket change for the owners - less than they'd pay for a painting that they probably don't even bother to look at (I know that these paintings are not to look at, humour me).
known 16y ago
Would moving reddit to a cloud will make it cheaper?
- dgudkov 16y ago
  It's already there.
theprodigy 16y ago
People are usually the highest cost for most companies. Reddit for sure needs admins, developers, sales, marketing, etc.
Revisor 16y ago
There is a great talk by an (ex-) developer from Reddit about the technical background, scalability problems, caching, storing redundant data, queued offline computing etc. This might shed light on some questions in the comments here. http://thinkvitamin.com/dev/steve-huffman-on-lessons-learned-at-reddit/ http://thinkvitamin.com/dev/steve-huffman-on-lessons-learned...
eleitl 16y ago
Presumably, they could halve the costs buy racking a bunch of SuperMicro boxes. Renting an empty rack is about 200 EUR/month, and you can fill it with back to back mounted Atoms with SSDs for less than 40 kUSD. 160 cores and 320 GByte RAM with some 16 TByte/s peak throughput for some ~2 kW is not that bad for the price.

125 points by there 16y ago

d0m 16y ago

So.. how much cost HN? :) And what/who pay for that?

jackowayed 16y ago
I'm pretty sure YC pays for it. It's one big server, so it's probably like $1k/month or something. But HN isn't anywhere near 8M active users, and it's pretty slow much of the time and has brief downtime often (multiple times per day, I think) so they can reboot the server to speed it back up.
tomjen3 16y ago
Very little compared to reddit. PG said recently that HN has 60K unique/day. Reddit has around 8Million. In addition to that, the software behind HN doesn't do nearly as much as the reddit software: there is no automatic checking of new messages, there are no sub reddits you need to work on, there are only so many votes to be processed, etc. I have no clue about who pays for what, but I guess PG pays.
- robryan 16y ago
  I would assume though it would be even easier here than on reddit to get people to donate towards server cost should the need ever arise.
  - rick888 16y ago
    The users of reddit are downright hostile toward anyone trying to make money. There was a thread last week about paid accounts. Out of 2000 comments, I think I only found one or two people supporting the idea. Most of the comments were somewhere along the lines of "paid accounts creates two classes of users and everyone should be equal" and "it's not my job to figure out how reddit makes money" (advertisements are actively blocked by 30% of users). When your userbase is filled with college kids and anti-capitalist, you are going to have a tough time making money,
    - zavulon 16y ago
      Yep. I think it's a valuable business lesson: unless you know what your plan to make money is, don't announce that your attitude is "ads are bad". Although I think they'll figure out how to make money, this situation could've been avoided if they ran ads from early on and weren't loudly against them.
    - Locke1689 16y ago
      Many of us are also gainfully employed/work in the "real world" ;)
    - saulhoward 16y ago
      There had been a donation drive the week prior to that, which was very well supported. I think people were saying that they thought the idea of 'paid accounts' just wasn't going to work as well as the donation drive had. The idea being that once people are paying for a actual service they expect more for their money. After all, they're aiming for 2% of the userbase subscribing. That's a large number when they're offering basically nothing of value in return. I know other 'freemium' business get 1-2% subscribers, but surely that's when they're offering a real improvement in the service, such as Dropbox's 2GB -> 20GB.
    - Super_Jambo 16y ago
      Being anti-advertising is no the same as being anti-capitalist. There is a good argument that advertising actively moves you away from the EMH which is one of the strongest arguments in favor of capitalism.
      - rick888 16y ago
        "There is a good argument that advertising actively moves you away from the EMH which is one of the strongest arguments in favor of capitalism." I said anti-capitalist because many people were against advertisements and the gold accounts. While there may be many people lurking on reddit that are not anti-capitalist, it's a sentiment I see in almost every topic related to making money on that site.
    - jasonlotito 16y ago
  - masklinn 16y ago
    You shouldn't. It probably isn't the case.

jbail 16y ago

A comment or two down, jedberg explains they use reserved instances from Amazon, so their hosting fees are more like $22,390.37 a month.

bkrausz 16y ago
Actually they're even lower, since the 3 year instances are so much cheaper than the 1 year over time. (1820 * 57) + (910 * 23) = $124,670/yr in reservation fees Changes to (2800 * 57) + (1400 * 23) = $191,800/3yr in reservation fees Which, when spread over the 3 years, brings the cost down by $5,061.39/mo, making the final cost there actually $17,328.98. Of course this assumes they're doing 3 year reservations, but I don't see any reason not to on their XL instances (I could be wrong though, not sure how open Conde Nast is to logic like this).
- masklinn 16y ago
  Conde Nast isn't involved in Reddit's operations. Also, > Actually they're even lower Yeah I like that you declare knowing the costs of Reddit's servers.
  - bsiemon 16y ago
    I think all the math is based on the known costs of EC2 services at various rate plans. All the people posting dollar figures here and on reddit are drawing from these know costs.
    - masklinn 16y ago
      > All the people posting dollar figures here and on reddit are drawing from these know costs. The others are quite clearly marked as estimates, this one not really, and the second estimate from the top in the reddit thread (yielding an estimate of $22,390.37/mo) got the following comment by one jedberg: > Yes, once again, you are totally accurate. That is almost exactly what it costs to run reddit, as of today. However, with our projected growth, we're looking to be closer to 350K by the end of the year. note that the comment got that estimate based off 1-year reserved servers, which jedberg's comment suggest is broadly correct rather than 3-year reservations.

deleted 16y ago

[deleted]

acangiano 16y ago

Amusingly, I can't read it because I get an error: "the service you request is temporarily unavailable. please try again later.".

ronnier 16y ago
I have a screenshot of it here: http://toadjaw.com/screenshot?id=http://www.reddit.com/r/blog/comments/ctz7c/your_gold_dollars_at_work/c0v8ug8?context=1 http://toadjaw.com/screenshot?id=http://www.reddit.com/r/blo...
- memoryfault 16y ago
  What is toadjaw? The homepage didn't seem to have any info.
  - ronnier 16y ago
    It's my after hours/weekend project. It'll eventually be a bookmarking site. I'm about 75% of the way there before I open it up to the public. Some of the features are nearly completed, like the screenshots and article text viewer. Along the way, I realized it would be trivial to make a mobile HN reader, so I spent some time doing that http://toadjaw.com/hn http://toadjaw.com/hn
    - ritonlajoie 16y ago
      Hey I use your website everyday on my Iphone. Love it, and thanks for the new 'theming' and new buttons, that's better than last week ! Keep up your website. Maybe you should put an ad on it, I would click it once a day to help :)
- acangiano 16y ago
  Thank you, ronnier. Very kind of you.

rufugee 16y ago

Wow...this is hard to believe. Numbers like that make me wonder if the cost of going with ruby or python (in reddit's case) or insert-your-slower-dynamic-language-here outweigh the benefits in the long run. How can anyone bootstrap a company and scale to the size of Reddit without quickly running into a "must fund or sale" situation? If it's truly a python issue, it should not be taken lightly by entrepreneurs. I hope it's not...I'm an avid user of both ruby and python, and want to believe they can both be used to create successful and maintainable (in terms of effort and cost) sites.

joshu 16y ago
Actually, I bet you it's a "we hit the database on every pageload" problem.
- giantfuzzypanda 16y ago
  Yep. That's why you shouldn't have pylons noobs running a site like reddit.
- emmett 16y ago
  I know the reddit guys, and I can tell you it's definitely not that. Hitting the DB is extremely rare - almost everything comes straight out of cache (either memcache, or memcachedb for caches which are more expensive to rebuild). Execution cost is dominated by the python app servers.
- blhack 16y ago
  The overwhelming majority of reddit's pageviews are users that are not logged in, which is very very cacheable.
  - emmett 16y ago
    I am pretty sure this is false. While the overwhelming number of unique visitors to reddit are not logged in, those users are far less engaged than average. Logged in users make up a huge percentage of the pageviews because of their higher engagement level.

cheald 16y ago

That's actually not terribly surprising. Reddit's always been extremely conservative with ad space, and it seems like they could offset a lot of that if they started running more "traditional" ad slots. By their own numbers, they do about 14.3m pageviews/day. Let's assume half of that is AdBlocked. A single 300x250 at an exceptionally low $0.05 CPM would yield $3,575/day in ad revenue, or $107,250/month. I'll guarantee that reddit could pull far, far higher CPMs. Like, an order magnitude or two higher. Edit: I'm off by an order of magnitude, see below. Point stands, though, that there is a lot of ad money they're leaving on the table.

_pi 16y ago
Reddit's actual CPM is 8$. One of the admins said that Conde wants to keep it a premium brand and they don't actually set the CPM. I'll try to dig up a link. It's also why there's tons of cross reddit advertising, and adverts that are paid are only seen for a tiny bit of time.
- endlessvoid94 16y ago
  I'm really interested in hearing more about this. Were you able to find a link?
  - _pi 16y ago
    http://www.reddit.com/r/ads/comments/cpm2d/i_cant_believe_somebody_bought_this_ad/c0ubpto http://www.reddit.com/r/ads/comments/cpm2d/i_cant_believe_so... Here you go. Took a bit. Also it's risen about 1$ in the last year. http://mediamemo.allthingsd.com/20090327/reddits-ad-experiment-is-good-news-for-conde-nast-maybe-for-digg-too/ http://mediamemo.allthingsd.com/20090327/reddits-ad-experime...
    - corin_ 16y ago
      Pretty depressing seeing comments like "Also there is no way a website has a $8CPM" and wondering why people who don't know what they're talking about keep on talking. I sell at double that rate for smaller sites on a daily basis ;/

deleted 16y ago

[deleted]

apower 16y ago

That's just the server instance cost. What about bandwidth and S3 cost? Bandwidth would cost a bundle.

keefe 16y ago
I haven't studied their arch, but I could imagine scenarios where spot instances would reduce the bill some, but yeah. Bandwidth. Ouch.
jedberg 16y ago
The bandwidth is less than 5% of the total cost. It's about $2500 a month.

pbhjpbhj 16y ago

I wonder what the best money-saving optimisation would be? Whilst it sounds like a lot of money remember that the whole sites annual EC2 bill could easily be paid for several years by one of the Newhouse brothers (who own Advance Publications, who own Conde Nast) selling one of their Reubens or probably just giving up their bank interest for a month (est. about ¼-million USD per day for the poor one : 1day of 2% annual interest on $4.5 billion).

ryanwaggoner 16y ago
Of course the Newhouse brothers could do that, but they could do it precisely for the reason they'd be unlikely to: if it's a bad investment, they shouldn't keep throwing money at it. Not saying that it is a bad investment, just that ignoring the problem because it's owned by a rich person who could easily bail it out ignores the way rich people think.
- pbhjpbhj 16y ago
  >Not saying that it is a bad investment, just that ignoring the problem because it's owned by a rich person who could easily bail it out ignores the way rich people think. I don't want to argue that at all. I'd argue that bailing them out as if they're a charity is wrong given that supporting it would be pocket change for the owners - less than they'd pay for a painting that they probably don't even bother to look at (I know that these paintings are not to look at, humour me).

known 16y ago

Would moving reddit to a cloud will make it cheaper?

dgudkov 16y ago
It's already there.

theprodigy 16y ago

People are usually the highest cost for most companies. Reddit for sure needs admins, developers, sales, marketing, etc.

Revisor 16y ago

There is a great talk by an (ex-) developer from Reddit about the technical background, scalability problems, caching, storing redundant data, queued offline computing etc. This might shed light on some questions in the comments here. http://thinkvitamin.com/dev/steve-huffman-on-lessons-learned-at-reddit/ http://thinkvitamin.com/dev/steve-huffman-on-lessons-learned...

eleitl 16y ago

Presumably, they could halve the costs buy racking a bunch of SuperMicro boxes. Renting an empty rack is about 200 EUR/month, and you can fill it with back to back mounted Atoms with SSDs for less than 40 kUSD. 160 cores and 320 GByte RAM with some 16 TByte/s peak throughput for some ~2 kW is not that bad for the price.