9 ms·
Reddit is the only major social media/aggregator I can think of that actually has to go offline for scheduled maintenance still. I can’t imagine the kind of shi
by dymk 7y ago
Reddit is the only major social media/aggregator I can think of that actually has to go offline for scheduled maintenance still. I can’t imagine the kind of shitshow it must be internally.
Imagine if Facebook or Twitter had to go down regularly for “maintenance”. Heads would roll.
- cdiddy2 7y agothey don't make much money so don't have the same redundancy luxuries as other sites
- dymk 7y agoThey’re Alexa rank #16 globally - that they’re having trouble monetizing their userbase just means they have absolutely no idea what they’re doing.
- zbentley 7y agoThat doesn't follow at all. There are all sorts of insanely popular websites which have significant monetization issues (think tumblr, wikipedia). Just because lots of people use something doesn't make them willing to pay for it, and it doesn't make a platform inherently monetizable.
- lazyjones 7y agoIndeed, also not everybody is trying to squeeze money out of their user base regardless of costs (to popularity, principles...).
- dymk 7y agoReddit gold, the new Reddit platinum and bronze, they already show ads, redesign shows ads much more prominently. Literally dozens of ad trackers when you load their site. Reddit is just as much of a profit driven company as Facebook or Twitter, I’m not sure why you’d think otherwise.
- lazyjones 7y ago> Reddit is just as much of a profit driven company as Facebook or Twitter, I’m not sure why you’d think otherwise. Because, among other things, they don't show annoying ad formats and don't demand real names and phone numbers from their users.
- birdyrooster 7y agoI thought Wikipedia doesn’t have monetization issues, but maybe things have changed? https://www.theregister.co.uk/2012/12/20/cash_rich_wikipedia_chugging/ https://www.theregister.co.uk/2012/12/20/cash_rich_wikipedia...
- gburdell3 7y agoAny attempt at monetization through the "normal" means, such as increasing the number of ads or selling user data, would completely undermine the things that make Reddit such a valuable site. We've already seen a few examples of Reddit moving toward a more legitimate, financially viable social media model, and each one of those moves has been met with significant backlash. It seems that they're in a lose-lose situation: either run out of money to keep the site running, or lose all their users.
- lelibertaire 7y agoI think they just don't sell as much data as something like Google, Facebook, Twitter, etc.
- propogandist 7y agoEach outbound link on reddit is tracked, similar to how Google tracks users clicking on SERPs. This can be tied back to a user to build profile on your interests, in addition to sub-reddits you subscribe to etc. which can be valuable if combined with other demographic data about you. I dont know if they started selling this data directly. Last I spoke to ad people there, they had plans to build out their own tracking and analytics infrastructure, to improve things like geo-targeted ads, as their capabilities were lacking signficiantly.
- city41 7y agoYou can turn off outbound link tracking in the privacy section of preferences. Uncheck “allow reddit to log my outbound clicks for personalization”
- propogandist 7y agowhich means your logged in and your behavior and activity is captured, and available for ad-targeting anyway. There's no real opt-out.
- blackhaz 7y agoAnd I wholeheartedly wish Reddit will continue doing what they are doing. As a user, I would vote for scheduled maintenance downtime to many monetization attempts present on other networks.
- dymk 7y agoLooks like they’re having trouble just keeping the lights on without taking VC money. Not exactly a sustainable plan if you value using their service, and they’ve already got early investors pushing them to monetize.
- Silhouette 7y agoThe problem these sites have is that most of us don't particularly value using their service. We might appreciate the facility they offer for free, but what we truly value is the communities that are found there and the information and discussions that originate from those communities. If Reddit died tomorrow, we'd all shrug and find somewhere else to talk, just like the countless forums we used before Reddit that are no longer with us. Such is the danger for any social media: if all you provide is a commodity platform, you're only ever as valuable as the community you can build and whatever that community is willing to contribute, and ultimately you have very little control over those things.
- manigandham 7y agoThey don't make nearly as much as others with that traffic. Brand safety issues with subreddits and discussions, anonymous user profiles, poor mobile experience, mostly link aggregation instead of content, high adblocking, etc. Here's an old answer of mine comparing reddit's $7M to Instagram's $700M in 2014: https://www.quora.com/Why-are-reddits-advertising-revenues-only-8-million-in-2014-compared-to-Instagram-with-700-million/answer/Mani-Gandham https://www.quora.com/Why-are-reddits-advertising-revenues-o...
- tyrust 7y agoI think they have the resources. Recent example: https://www.reuters.com/article/us-reddit-funding/reddit-valued-at-3-billion-after-raising-300-million-in-latest-funding-round-idUSKCN1Q020W https://www.reuters.com/article/us-reddit-funding/reddit-val...
- cdiddy2 7y agoi'm not saying they are completely broke, but they certainly are on the more thrifty side of things compared to most tech companies
- EpicEng 7y agoThey make a lot more than does the little outfit I work for and we manage it.
- ses1984 7y agoHow much does reddit make, I wouldn't be surprised if it operates at a loss.
- EpicEng 7y agoIt definitely does, just like so many VC funded companies, but they could afford to fix their deploys. The reddit back end is notoriously bad and, judging by their new UI, it isn't getting better.
- ses1984 7y agoWhat's the point of making a site reliable if it costs you money to operate? Their strategy is totally different from a Twitter or facebook. All their cycles are busy trying to figure out how to turn a profit. And the new ui is trash.
- jswny 7y agoThey also have very frequent cases where the site throws up an error page or is completely unreachable because it is under heavy load. I can't think of another major social network that has those problems on a weekly basis either.
- fragmede 7y agoNot recently, but Twitter's "fail whale" became famous due to problems Twitter had under heavy load.
- coryrc 7y agoSo what? 24/7 builds up to fragility like the NYC subway system or has huge costs. It's social media, not a hospital.
- paxys 7y agoA website is not a subway system. There are no "huge costs" to having basic rolling deployments set up. In fact it's likely more expensive to have extended outages and maintenance windows, because it shows that there are humans running commands behind the scenes rather than automation.
- stevenpetryk 7y agoJust wondering (sort of—I'm also slightly trying to point out that you're being a bit of an armchair Reddit ops person :) What makes you think that the issue Reddit faced today would be fixed by rolling deploys?
- paxys 7y agoI definitely am (but we are on Hacker News after all). Without knowing anything about this particular issue, at a basic level I'd think any non-trivial site should have a system of rolling out changes to a small percentage of servers and have automated monitoring that fires alerts if it notices any disruptions in functionality. If nothing goes wrong then gradually roll out to 100%. If something does then stop the rollout and go back to the last stable version. Then engineers can look at what went wrong.
- hobofan 7y agoFrom my experience that mostly only applies to the application layer. Especially updates on distributed databases are often not as easy as "stop and rollback" as you have to achieve a stable quorum or other similar dynamic states like that which you can't easily achieve from restoring a snapshot. There your best shot often is to learn how to deal with all the failure scenarios manually and do as many tests on a staging system until you feel comfortable.
- amelius 7y agoCan't they run in read-only mode during maintenance? That would be a relatively simple workaround, I suppose.
- Sendotsh 7y agoThey used to. Not sure why they don’t now.
- disjhshava 7y agoI remember the read-only mode as some presumably cached pre-rendered html, not sure if it ever existed for the app/api. I wonder if the read-only mode didn't mesh well with the single page app they have been pushing - perhaps it wasn't monetized in the same way.
- boobePhuu7iet7i 7y agoDo we know this was scheduled maintenance and not some kind of unexpected outage?
- Deimorz 7y agoThis definitely wasn't scheduled. Reddit does still do scheduled maintenance occasionally though (not often, usually only for something like a database version upgrade), and I think they're talking about that, not this specific incident.
- deleted 7y ago[deleted]
- amoshi 7y agoSteam is worse - weekly "maintenance" on Tuesday evening, where everyone is disconnected for ~10-20 minutes. It's common knowledge not to play at that time if the game uses Steam networking, otherwise you'll be kicked mid-session with a "Lost connection to Steam" error message and lose all your progress. And yes, this is happening in 2019.
- colejohnson66 7y agoAt least it’s scheduled so you can schedule around it
- Shivetya 7y agowell World of Warcraft and other MMOs regularly have outages each week and sometimes more frequently. this isn't just for major content pushes either. heck blizzard has authentication services outages too
- trevor-e 7y agoPUBG regularly goes down for 4+ hour updates during prime NA server time. :/
- Stevvo 7y agoI remember that from 15 years ago, would have thought they had enough time to fix it since then.
- mike_d 7y ago"We are a startup that inherited a 15 year old website" was one of the things that resonated when I worked there. Facebook is Alexa #5 in the US and has ~40,000 employees. Reddit is #6 and has ~500 employees. Twitter is #17 and has ~4,000 employees. (Numbers taken from quick Google searches) They have some super talented engineers that work miracles with band aids and bubble gum. Edit: Yes I understand not all employees work on everything. I was trying to give a sense of scale. Reddit is a tiny company with a massive website and no way to make money. If you have ideas - they are hiring.
- opmac 7y agoMeh... they could have rewritten the site from the ground up, including any sort of data migration, multiple times over by now. I have never worked there, but outside looking in, it seems like there is a significant lack of technical vision from them. Other big players like Facebook, Twitter, etc, are literally inventing pivotal parts of the modern web technical stack. What has come out of reddit?
- joatmon-snoo 7y ago> including any sort of data migration, multiple times over by now Someone's never done a data migration while serving 1M+ qps!
- deleted 7y ago[deleted]
- ses1984 7y agoBut they have had downtime, it's not like downtime for a migration would damage their reputation.
- sethammons 7y agoI've come close to that many, many times with zero downtime. It does take planning, more than many folks would think. But it can be done as long as you think it through.
- teh_klev 7y ago<making salty tears emoty>*10 > I can’t imagine the kind of shitshow it must be internally. In real life, nobody cares, they aren't important, normal people react with "nobody cares". My own reaction as a reddit user and commenter was....well who gives a shit? I'll go do another thing such as reading a book. Rreddit is just a honeypot for shitposters (possibly a good thing for the rest of the internet). I'm in my 50's and hung out on a plethora of (what are to be honest mostly moronic) "boards"...slashdot, digg, reddit etc. When they go away for a few hours or days this doesn't black out my life. If your social life hinges on the availability of Reddit, Facebook et al then I kinda feel you have deeper issues. Especially when you don't pay for this service.
- buboard 7y agois it a big deal? so what if fb goes down, people will just spam twitter about it. reading social media is not some crucial function, it's just a compulsion. and , like drugs, withdrawal symptoms take some time to develop
- tayo42 7y agoFor the world no it doesn't matter. For the company, every minute down is ads not being looked at and income being generated
- coupdetaco 7y agoFacebook goes down all the time