8 ms·

I love write-ups like this because they are such a nice contrast to the too-common comments on Reddit and HN where people claim that they could rebuild FB or Ub

I love write-ups like this because they are such a nice contrast to the too-common comments on Reddit and HN where people claim that they could rebuild FB or Uber as a side project. Something as superficially trivial as the r/Place requires a tremendous amount of effort to run smoothly and there are countless gotchas and issues that you'd never even begin to consider unless you really tried to implement it yourself. Big thanks to Reddit for the fun experiment and for sharing with us some of the engineering lessons behind it.

deleted 10y ago
[deleted]
dstroot 10y ago
I came to write exactly what you wrote. Here's an upvote brother from another mother.
- JTrilogy 10y ago
  I'm I'm I' are] can you 3a1 we 3e 2 2 I'm/e in to in3/'maw3
deleted 10y ago
[deleted]
soup10 10y ago
To me it seems a little over-engineered but it held up so props to them.
- joezydeco 10y ago
  I have this hunch that Reddit engineers know what they're up against when they launch something like this. It can get out of control extremely fast when something catches on.
- dmihal 10y ago
  Reddit is like the 7th most popular website, features like this need to be able to handle significant load
- nojvek 10y ago
  Why do you think it's over engineered. The scaling issues made sense. Cdn was a great move to limit throughput of requests. Redis made sense, javascript canvas and typedarrays made sense. How else would you have done it?
  - soup10 10y ago
    For the core application of writing pixels to the board, I would of used a more efficient monolithic design with less points of failure. One big server could handle 333 pixels/s(their expected volume). Not sure that using cassandra and redis is really necessary or makes things easier. The secondary goal of pushing pixel updates to lots of users would probably want separate hardware. I would probably fake it so its not really realtime. If you send the web client batches of updates to replay on the board it looks realtime and doesn't require persistent connections.
josephg 10y ago
> the too-common comments on Reddit and HN where people claim that they could rebuild FB or Uber as a side project. I do and don't agree with you. Whats really going on here is that development time scales linearly with the number of decisions you need to make. Decisions can take the form of a product questions - "what are we making?" and development questions - "how should we implement that?". There are three reasons why people feel confident saying "I could copy that in a weekend": - When looking at an existing thing (like r/place), most of the product decisions have already been made. You don't need to re-make them. - If you have a lot of development experience in a given domain, you don't need to make as many decisions to come up with a good structure - your intuition (tuned by experience) will do the hard work for you. - For most products, most of the actual work is in little features and polish thats 'below the water', and not at all obvious to an outsider. Check out this 'should we send a notification in slack' flowchart: https://twitter.com/mathowie/status/837807513982525440?lang=en https://twitter.com/mathowie/status/837807513982525440?lang=... . I'm sure Uber and Facebook have hundreds of little features like that. When people say "I could copy that in a weekend", they're usually only thinking about the core functionality. Not all the polish and testing you'd actually need to launch a real product. With all that said, I bet I could implement r/place in less than a weekend with the requirements stated at the top of that blog post, so long as I don't need to think about mobile support and notifications. Thats possible not because I'm special, and not because I'm full of it but because most of the hard decisions have already been made. The product decisions are specified in the post (image size, performance requirements) and for the technical decisions I can rely on experience. (I'd do it on top of kafka. Use CQRS, then have caching nodes I could scale out and strong versions using the event stream numbers. Tie the versions into a rendered image URL and use nginx to cache ... Etc.) But I couldn't implement r/place that quickly if reddit didn't already do all the work deciding on the scope of the problem, and what the end result should look like. In a sense, I think programming is slow and unpredictable because of the learning we need to constantly do. Programming without needing to learn is just typing; and typing is fast. I rule out doing the mobile notifications not because I think it would be hard, but because I haven't done it before. So I'd need to learn the right way to implement it. - And hence, thats where the scope might blowout for me. Thats where I would have to make decisions, and deciding correctly takes time.
- 10y ago
ziikutv 10y ago
Regarding Uber braggers (pun intended): Who cares! If they build it they will not come.
wishinghand 10y ago
> the too-common comments on Reddit and HN where people claim that they could rebuild FB or Uber as a side project. I'm pretty active on both. What threads/subreddits are you reading?
- easytiger 10y ago
  you should try r/unitedkingdom where all posters are pretty sure they could flawlessly redesign modern civilization
  - nathancahill 10y ago
    http://www.theonion.com/article/there-are-no-good-options-syria-sighs-man-who-has--55758 http://www.theonion.com/article/there-are-no-good-options-sy...
  - travmatt 10y ago
    To be fair that's kinda the history of the United Kingdom.
  - angusp 10y ago
    You ain't seen nothing yet - /r/Libertarian
- Ajedi32 10y ago
  Here's one: https://www.reddit.com/r/programming/comments/656z8t/how_we_built_rplace/dg7z99x/ https://www.reddit.com/r/programming/comments/656z8t/how_we_... Pretty funny considering the content of the post he wrote that on.
conradfr 10y ago
With a polished front-end maybe not in a week-end but I was thinking that it was a good use case for a fun Elixir/Phoenix exercise anyway.

deleted 10y ago

[deleted]

dstroot 10y ago

I came to write exactly what you wrote. Here's an upvote brother from another mother.

JTrilogy 10y ago
I'm I'm I' are] can you 3a1 we 3e 2 2 I'm/e in to in3/'maw3

deleted 10y ago

[deleted]

soup10 10y ago

To me it seems a little over-engineered but it held up so props to them.

joezydeco 10y ago
I have this hunch that Reddit engineers know what they're up against when they launch something like this. It can get out of control extremely fast when something catches on.
dmihal 10y ago
Reddit is like the 7th most popular website, features like this need to be able to handle significant load
nojvek 10y ago
Why do you think it's over engineered. The scaling issues made sense. Cdn was a great move to limit throughput of requests. Redis made sense, javascript canvas and typedarrays made sense. How else would you have done it?
- soup10 10y ago
  For the core application of writing pixels to the board, I would of used a more efficient monolithic design with less points of failure. One big server could handle 333 pixels/s(their expected volume). Not sure that using cassandra and redis is really necessary or makes things easier. The secondary goal of pushing pixel updates to lots of users would probably want separate hardware. I would probably fake it so its not really realtime. If you send the web client batches of updates to replay on the board it looks realtime and doesn't require persistent connections.

josephg 10y ago

> the too-common comments on Reddit and HN where people claim that they could rebuild FB or Uber as a side project. I do and don't agree with you. Whats really going on here is that development time scales linearly with the number of decisions you need to make. Decisions can take the form of a product questions - "what are we making?" and development questions - "how should we implement that?". There are three reasons why people feel confident saying "I could copy that in a weekend": - When looking at an existing thing (like r/place), most of the product decisions have already been made. You don't need to re-make them. - If you have a lot of development experience in a given domain, you don't need to make as many decisions to come up with a good structure - your intuition (tuned by experience) will do the hard work for you. - For most products, most of the actual work is in little features and polish thats 'below the water', and not at all obvious to an outsider. Check out this 'should we send a notification in slack' flowchart: https://twitter.com/mathowie/status/837807513982525440?lang=en https://twitter.com/mathowie/status/837807513982525440?lang=... . I'm sure Uber and Facebook have hundreds of little features like that. When people say "I could copy that in a weekend", they're usually only thinking about the core functionality. Not all the polish and testing you'd actually need to launch a real product. With all that said, I bet I could implement r/place in less than a weekend with the requirements stated at the top of that blog post, so long as I don't need to think about mobile support and notifications. Thats possible not because I'm special, and not because I'm full of it but because most of the hard decisions have already been made. The product decisions are specified in the post (image size, performance requirements) and for the technical decisions I can rely on experience. (I'd do it on top of kafka. Use CQRS, then have caching nodes I could scale out and strong versions using the event stream numbers. Tie the versions into a rendered image URL and use nginx to cache ... Etc.) But I couldn't implement r/place that quickly if reddit didn't already do all the work deciding on the scope of the problem, and what the end result should look like. In a sense, I think programming is slow and unpredictable because of the learning we need to constantly do. Programming without needing to learn is just typing; and typing is fast. I rule out doing the mobile notifications not because I think it would be hard, but because I haven't done it before. So I'd need to learn the right way to implement it. - And hence, thats where the scope might blowout for me. Thats where I would have to make decisions, and deciding correctly takes time.

10y ago

ziikutv 10y ago

Regarding Uber braggers (pun intended): Who cares! If they build it they will not come.

wishinghand 10y ago

> the too-common comments on Reddit and HN where people claim that they could rebuild FB or Uber as a side project. I'm pretty active on both. What threads/subreddits are you reading?

easytiger 10y ago
you should try r/unitedkingdom where all posters are pretty sure they could flawlessly redesign modern civilization
- nathancahill 10y ago
  http://www.theonion.com/article/there-are-no-good-options-syria-sighs-man-who-has--55758 http://www.theonion.com/article/there-are-no-good-options-sy...
- travmatt 10y ago
  To be fair that's kinda the history of the United Kingdom.
- angusp 10y ago
  You ain't seen nothing yet - /r/Libertarian
Ajedi32 10y ago
Here's one: https://www.reddit.com/r/programming/comments/656z8t/how_we_built_rplace/dg7z99x/ https://www.reddit.com/r/programming/comments/656z8t/how_we_... Pretty funny considering the content of the post he wrote that on.

conradfr 10y ago

With a polished front-end maybe not in a week-end but I was thinking that it was a good use case for a fun Elixir/Phoenix exercise anyway.