4 ms·

it seem you don't have experience on large scale apps... and the people that make these type of comments almost never have had to work on large applications...

it seem you don't have experience on large scale apps... and the people that make these type of comments almost never have had to work on large applications..... Spotify had about 500k, lines of code in 2015, (not including external libraries). We did make it in a feature based architecture, where there was a main container app (and a major core playback library) and for every major feature they had their own sub-project.... and there were 10+ teams contributing into it. There were about 13 features at the beginning, (think, playlists, radio, shared ui libraries, etc...). Each feature was about 10-20k lines of code in total. So, about 2-3 features equal a small standalone iOS app. When I left, there were more than 80 features in the app. Some were essential, many were a/b tests, and some were probably just legacy/waiting to be removed. 80 features * 15k lines of code = 1.2mil loc, easily, and they are the equivalent of shipping 20 small apps standalone.... What are these features and why is just a stupid music app so large? : video, podcasts/shows, genius, etc.. etc.. etc.... To users just looks like another tab but almost every one of these major features could be its own standalone app for size and complexity.

wodenokoto 7y ago
I would love to read more about why large apps are so big. I currently work on a project that is much larger than I would expect it to be, both in terms of codebase and people, and while I don't know who to jank, I also don't understand why it has to be so big.
- Domenic_S 7y ago
  Short answer: because it's way more feasible to remodel your kitchen than to tear down your entire house and rebuild it. And then your bathroom. And then convert the garage. And then add an addition. And in 10 years you have this cobbled-together monstrosity
  - Aeolun 7y ago
    But your garage (the first part of what is now your house), was built with a weird kind of screw, that was only sold by one shop at the time, but was cheapest. Now your entire house is built with those screws, but they’re not produced anymore so you’ve set up a small smelter next to your home where you’re making them yourself. Unfortunately the quality is a good bit worse, so parts of your house start degrading faster and faster, but meanwhile you’re still building all new parts with your shitty screws. Nobody wants to replace the screws because “It’s a proven method”, ignoring all signs to the contrary. All new parts of the house are only as good as the worst part, the screws, and the moment anyone steps on the wrong plank the whole room comes crashing down. But nobody worries, next time they’ll just use two screws.
- otabdeveloper4 7y ago
  > I would love to read more about why large apps are so big. Because nobody ever got promoted for not hiring people or for not writing code. The problem is 10x as bad when you're an "IT company", i.e., writing code is your primary product.
- ss3000 7y ago
  Part of it also comes from the tendency of large orgs to ship their org structure. It can oftentimes be worthwhile to have individual teams each fully "own" a piece of the product, and that also comes with the autonomy to choose to duplicate stuff that other teams might have already built rather than working with them and tweaking it to match their own specific new requirement. This significantly reduces communication & coordination overhead for delivering new features, at the expense of consistency and code reuse, which can often be a worthwhile tradeoff at scale.
  - archagon 7y ago
    Interesting. I wonder if there's a corollary: "you will ship your org, so be careful how you structure your org."
    - arlo_phoenix 7y ago
      https://en.wikipedia.org/wiki/Conway's_law https://en.wikipedia.org/wiki/Conway's_law
aikah 7y ago
What language(s) does Spotify use for the backend? not the website or the mobile clients, but the backend itself.
- ardit33 7y ago
  Mostly Java.... some scala (some teams might use it), and potentially other languages. It built its own api lightweight java framework around 2014-2015 (called Apollo), which is actually pretty decent/nice to work with. There was a lot of protobuf usage early on, but most teams started transitioning to plain json. So, each team runs its own servers, for the features it is developing/maintaining, and most of the api goes throw a common gateway, etc.. etc... don't want to give out more than this. Some of the simple front end web pages (i.e. marketing websites, analytics) might even be on php/python/ruby. Some teams wanted to use javascript (Node), but those are not the critical main app, as java is pretty much required to be used for anything that affects the main app. Also, if you are doing machine learning, then whatever it gets the job done (usually Python). btw, I left few years ago, so things might have changed.... I know at least one team that wanted to use GO for some of their services....
  - TeMPOraL 7y ago
    Thanks for the insights. I can't shake the feeling that the dominating factor in growth of LOC isn't the number of features per se, but the number of features worked on in parallel by separate teams.
jeswin 7y ago
Thank you for saying this. The overwhelming majority of comments on this thread are pedestrian - as if FBs engineers have been making decisions out of naivety, incompetence or self interest.
- Cthulhu_ 7y ago
  It might look that way in hindsight, but hindsight is easy. As is armchair theorizing about the competence of other developers. Most people (including myself) will probably never see a >500K loc application or a project with >100 developers working on it. Biggest ones I've worked with had around 30-50 odd developers?
- otabdeveloper4 7y ago
  > as if FBs engineers have been making decisions out of naivety, No. > incompetence Yes. > or self interest Yes.
  - Aeolun 7y ago
    Definitely the last one. They’re working for Facebook after all. That said, I’m happy with React.
dackroyd99 7y ago
Turns out there's the same amount of code in the Spotify app as there is in the product I work on... True though, if you want more features you need more code. Sure, loads of people on HN could build a 'chat app' in a lot less but if they kept it going for ten years (same age as our product) and kept on adding new features the lines would start to add up...

wodenokoto 7y ago

I would love to read more about why large apps are so big. I currently work on a project that is much larger than I would expect it to be, both in terms of codebase and people, and while I don't know who to jank, I also don't understand why it has to be so big.

Domenic_S 7y ago
Short answer: because it's way more feasible to remodel your kitchen than to tear down your entire house and rebuild it. And then your bathroom. And then convert the garage. And then add an addition. And in 10 years you have this cobbled-together monstrosity
- Aeolun 7y ago
  But your garage (the first part of what is now your house), was built with a weird kind of screw, that was only sold by one shop at the time, but was cheapest. Now your entire house is built with those screws, but they’re not produced anymore so you’ve set up a small smelter next to your home where you’re making them yourself. Unfortunately the quality is a good bit worse, so parts of your house start degrading faster and faster, but meanwhile you’re still building all new parts with your shitty screws. Nobody wants to replace the screws because “It’s a proven method”, ignoring all signs to the contrary. All new parts of the house are only as good as the worst part, the screws, and the moment anyone steps on the wrong plank the whole room comes crashing down. But nobody worries, next time they’ll just use two screws.
otabdeveloper4 7y ago
> I would love to read more about why large apps are so big. Because nobody ever got promoted for not hiring people or for not writing code. The problem is 10x as bad when you're an "IT company", i.e., writing code is your primary product.
ss3000 7y ago
Part of it also comes from the tendency of large orgs to ship their org structure. It can oftentimes be worthwhile to have individual teams each fully "own" a piece of the product, and that also comes with the autonomy to choose to duplicate stuff that other teams might have already built rather than working with them and tweaking it to match their own specific new requirement. This significantly reduces communication & coordination overhead for delivering new features, at the expense of consistency and code reuse, which can often be a worthwhile tradeoff at scale.
- archagon 7y ago
  Interesting. I wonder if there's a corollary: "you will ship your org, so be careful how you structure your org."
  - arlo_phoenix 7y ago
    https://en.wikipedia.org/wiki/Conway's_law https://en.wikipedia.org/wiki/Conway's_law

aikah 7y ago

What language(s) does Spotify use for the backend? not the website or the mobile clients, but the backend itself.

ardit33 7y ago
Mostly Java.... some scala (some teams might use it), and potentially other languages. It built its own api lightweight java framework around 2014-2015 (called Apollo), which is actually pretty decent/nice to work with. There was a lot of protobuf usage early on, but most teams started transitioning to plain json. So, each team runs its own servers, for the features it is developing/maintaining, and most of the api goes throw a common gateway, etc.. etc... don't want to give out more than this. Some of the simple front end web pages (i.e. marketing websites, analytics) might even be on php/python/ruby. Some teams wanted to use javascript (Node), but those are not the critical main app, as java is pretty much required to be used for anything that affects the main app. Also, if you are doing machine learning, then whatever it gets the job done (usually Python). btw, I left few years ago, so things might have changed.... I know at least one team that wanted to use GO for some of their services....
- TeMPOraL 7y ago
  Thanks for the insights. I can't shake the feeling that the dominating factor in growth of LOC isn't the number of features per se, but the number of features worked on in parallel by separate teams.

jeswin 7y ago

Thank you for saying this. The overwhelming majority of comments on this thread are pedestrian - as if FBs engineers have been making decisions out of naivety, incompetence or self interest.

Cthulhu_ 7y ago
It might look that way in hindsight, but hindsight is easy. As is armchair theorizing about the competence of other developers. Most people (including myself) will probably never see a >500K loc application or a project with >100 developers working on it. Biggest ones I've worked with had around 30-50 odd developers?
otabdeveloper4 7y ago
> as if FBs engineers have been making decisions out of naivety, No. > incompetence Yes. > or self interest Yes.
- Aeolun 7y ago
  Definitely the last one. They’re working for Facebook after all. That said, I’m happy with React.

dackroyd99 7y ago

Turns out there's the same amount of code in the Spotify app as there is in the product I work on... True though, if you want more features you need more code. Sure, loads of people on HN could build a 'chat app' in a lot less but if they kept it going for ten years (same age as our product) and kept on adding new features the lines would start to add up...