7 ms·

Fantastic contribution back to the community; I look forward to trying it out. I must say this is the first time I've been disappointed with the quality of dis

Fantastic contribution back to the community; I look forward to trying it out. I must say this is the first time I've been disappointed with the quality of discussion on HN. For a community that promotes using the right tool for the job at the time, I would have thought people would be more open to the choices the early engineers made. I'm sure that Instagram are using a variety of tech across their stack.

blub 9y ago
Most people would expect software to crash, hang, be slow or somehow leak their personal information. That's normal behaviour for the products of the software industry. For a long time there have been efforts to ensure at least a degree of quality and robustness through processes, practices and verification tools. One such tool is a type system which allows encoding requirements and expectations that will be automatically verified with the help of a checker. This is exactly what Instagram is attempting in an effort to increase quality and ease maintenance. It seems many are wondering why they haven't done this from the beginning. A typical (and probably correct) answer is development speed and flexibility: early stage companies need to be nimble and compromise on quality if they want to survive. Fair enough, I understand that. We can applaud Instagram's success on the markets, but that doesn't have to mean that they're a role model of technical excellence for building a million line Python code base. This tool is the proof that Python has significant problems at scale, which is something the Python community has denied for a long time. They're still doing it in this thread, but the lesson looks pretty clear to me: if you plan on building large scale, don't use Python. Or PHP while we're at it (see Facebook). Definitely not JavaScript (see FirefoxOS). The founders of a start-up can continue to do whatever they want in the name of success. Instagram at their beginnings was basically a different company from today's Instagram, no one would have used them as a technical role-model. Now they're at best a role model for large Python code bases, but many here seem to be drawing the wrong conclusion, namely that it's a good idea to do large-scale Python in the first place.
- kbd 9y ago
  > This tool is the proof that Python has significant problems at scale... This does not prove your point. Annotating a large dynamically-typed codebase with type information is a large amount of work, regardless of the language. This tool makes that easier.
  - blub 9y ago
    I'm bemused by your reply and curious to know why you think Dropbox and Instagram are working on static type analysers for their large Python code bases. Instagram at least gave us a hint: "we’re keen to make our code easier for new developers to read and understand, as well as more amenable to static analysis that shrinks the domain of possible bugs". It seems to me it's so difficult to manage such a code base, that they decided to do that "large amount of work" in addition to the large amount of work required to develop the necessary tools! In which case it seems prudent to avoid the said amount of work by picking another programming language for one's large-scale code base.
    - outsideoflife 9y ago
      Isn't dropbox doing it partially to aid a future migration to Python 3? > In which case it seems prudent to avoid the said amount of work by picking another programming language for one's large-scale code base When I first saw Instagram I thought it was nonsense, I was wrong. When the founders started Instagram, I wonder if they had the amazing foresight to see what it would become. I would suggest getting to market was more valuable to them than worrying about what maintenance they would have to do once they had a billion dollars
    - bonesss 9y ago
      > ... it seems prudent to avoid the said amount of work by picking another programming language for one's large-scale code base. While I'm a type-adherent in my day-to-day (F# represent, wut wuuut), I think this isn't reflective of the chicken and egg dilemma for startups... There are an immense number of things they "should be" doing at scale that they can't do early because they're relying on their early product to scale. Time to market, and windows of opportunity, are critical to startups. Any immature technical decision at that part of the lifecycle needs to be made not with an attempt to make perfect forever from the start, but rather with an eye to transitioning to smarter solutions aggressively as you scale up. The company may be 6 pivots away from success, so better to validate solutions in the market than prognosticate. I don't accept the premise that a lack of typing significantly improves development velocity, per se, but language decisions are about ecosystems, key components, and local talent. Where these companies get into coo-coo land is not integrating those immature components into better systems as they're getting bigger. Next thing you know someone is writing a whole compiler chain for PHP in an attempt to reinvent a sane programming language, or trying to get Python to be Java. Smart, modern, functional languages provide development velocity and pleasure equal to Python with fundamental type safety and guarantees. But in a world where just putting a button on a webpage means multiple dialects in multiple languages I think we should be ok mixing and matching on the backend to scale smartly.
- techdragon 9y ago
  The reality borne out by the evidence [0][1] is that Python is at the very least perfectly suitable for the development of web stacks powering companies worth in excess of hundreds of million dollars. Putting aside concrete technical issues regarding the python runtime’s performance envelope (eg: startup time, FFI inter-op call time, etc) and memory footprint, there is no reason not to use Python. Again, concrete technical issues aside, Python will not be an issue until your business is big enough that having to push the performance of your product is a nice problem to have and you will probably have the money to spend solving it by either optimising your python or rewriting parts of your application, as all these large companies have done. 0 - https://www.linkedin.com/pulse/top-10-sites-built-django-framework-vladimir-bogdanov https://www.linkedin.com/pulse/top-10-sites-built-django-fra... 1 - https://worldwebtech.weebly.com/blog/top-ten-most-popular-website-development-using-python-and-django-platform https://worldwebtech.weebly.com/blog/top-ten-most-popular-we...
  - nothrabannosir 9y ago
    You can substitute Python for assembly, point at the Apollo mission and your argument still holds. Rarely is a programming language chosen because it actually is the best tool for the job. Often, it’s whatever the people at the ground floor were most comfortable getting a prototype out the door with. Prototype working well? Fix this bug, add yonder feature. Before you know it you’ve built Facebook in PHP. “PHP is at the very least suitable for the development etc etc”, yes, that’s why Facebook invested all that effort in Hiphop VM. There’s an irony in saying “there’s no reason not to use X” in reply to an article that is about a company spending a ton of effort working around a problem in X. My point is: as a community, it’s our duty to learn from these mistakes. Let’s admit there is a problem, investigate, adapt, overcome. We can improve whatever will be the next Python so the next Instagram doesn’t have to go through this. But that won’t work if we keep saying “this is a nice problem to have, there’s nothing wrong.” It isn’t. There is. Look at the article.
  - blub 9y ago
    I'm not claiming that one can't be successful by using Python (or pretty much any programming language). Market success or user count are unfortunately not tightly correlated with technical excellence, as many of us have bitterly found out. Development speed, maintainability, error count are also important development issues. Unfortunately we don't have much data to judge, but the little that we have such as this article indicate that dynamic typing has a non-negligible negative impact on the above. ""having to push the performance of your product is a nice problem to have and you will probably have the money to spend solving it by either optimising your python or rewriting parts of your application, as all these large companies have done."" Not considering the topic at all is negligent. I don't understand why you're so sure that the only possible outcomes are either not reaching scale or having the money to optimise or rewrite. First of all, not even Facebook had the money to rewrite their PHP code base, so that's probably out of the question. And it's very well possible that one will reach scale and not have the money (or worse the time) to optimise, if optimising means inventing type checkers. At least one should spend some time thinking about this topic and picking a language that is flexible and can scale at least somewhat. Having to stop writing production code in order to invent a Ruby static type checker or native compiler is decidedly not a good problem to have.
- moreless 9y ago
  > This tool is the proof that Python has significant problems at scale, which is something the Python community has denied for a long time. They're still doing it in this thread, but the lesson looks pretty clear to me: if you plan on building large scale, don't use Python. Or PHP while we're at it (see Facebook). Yeah, absolutely, don't do this! These are two examples of successful companies that did it and look at them now! </s> Being able to move fast and produce a winning product on time is much more important for startups. What does it matter if you used <your_cool_scalable_thingie> for a project, when it never went past 10 users because you were concentrating on wrong side aspect of your business? PHP is fine. Python is great. Use the tools that fit your problem and you know how to use, not the latest toy.
  - zerkten 9y ago
    How do you identify the inflection point and then execute when it hits? There are conservative choices that would scale all the way through, but many startups would avoid them due to the hit on "velocity" (which is itself a very fuzzy topic.) It seems that this same pattern plays out with many tools, and not just languages. When you've built something and you now have a team, processes, etc. built up it becomes difficult to see the forest for the trees, or to make the hard decisions because it might involve replacing people.
  - 1_2__4 9y ago
    HN has a favorite pastime: dismissing tech that has demonstrated incredible utility because it’s lacks some kind of ideological purity they demand.
  - blub 9y ago
    There are two things that are critically incorrect about your argument: 1) That market success implies having quality software. Average seems to be enough in my experience. 2) That start-ups are a good example to follow if one wants to achieve good quality. In fact they should be ignored, because they will absolutely murder quality in order to stay alive. Sometimes the product doesn't even work and is held together with duct tape in order to get past that important demo... It's quite pointless to discuss quality and start-ups. The lesson I mentioned should be heeded by mature companies that are able to do some project planning, complexity estimation, etc.
- nine_k 9y ago
  Use the right tool for the job. "Scale" can be seen under different angles. You can run thousands of boxes with relatively simple code if it's designed to scale horizontally. Twitter used to run on Ruby this way, until they accumulated money and expertise enough to rewrite the whole thing and save on operations and further development costs. Your code can be millions LOCs and run on a few boxes; it's a very different sort of "scale". In one company where I worked they had a crazy mix of codebases, from modern Scala down to ancient PHP code. But since it was architected reasonably, it was possible to replace the PHP code piecemeal, without stopping the system. Do the devs that started it 15 years ago with PHP deserve blame for choosing a poor language, or praise for coming up with a serviceable architecture? You can see the original post as a step in the right direction: in a complex codebase, static typing has a large number of advantages. Barring a wholesale rewrite, how would you gradually transform your code to using it? Yes, by documenting the current state in a formal way, introducing a typecheck build step, then maybe splitting out certain components and rewriting them in other languages, etc. Look at TypeScript. Unfortunately, there's no way around using quick-and-dirty prototypes at the very early stages, if nothing else, for lack of technical expertise among founders and their first employees. They have business ideas first and foremost. So a tool like that would help exactly the step you want done, switching to a nicer and more manageable stack as growth allows / demands it.

blub 9y ago

Most people would expect software to crash, hang, be slow or somehow leak their personal information. That's normal behaviour for the products of the software industry. For a long time there have been efforts to ensure at least a degree of quality and robustness through processes, practices and verification tools. One such tool is a type system which allows encoding requirements and expectations that will be automatically verified with the help of a checker. This is exactly what Instagram is attempting in an effort to increase quality and ease maintenance. It seems many are wondering why they haven't done this from the beginning. A typical (and probably correct) answer is development speed and flexibility: early stage companies need to be nimble and compromise on quality if they want to survive. Fair enough, I understand that. We can applaud Instagram's success on the markets, but that doesn't have to mean that they're a role model of technical excellence for building a million line Python code base. This tool is the proof that Python has significant problems at scale, which is something the Python community has denied for a long time. They're still doing it in this thread, but the lesson looks pretty clear to me: if you plan on building large scale, don't use Python. Or PHP while we're at it (see Facebook). Definitely not JavaScript (see FirefoxOS). The founders of a start-up can continue to do whatever they want in the name of success. Instagram at their beginnings was basically a different company from today's Instagram, no one would have used them as a technical role-model. Now they're at best a role model for large Python code bases, but many here seem to be drawing the wrong conclusion, namely that it's a good idea to do large-scale Python in the first place.

kbd 9y ago
> This tool is the proof that Python has significant problems at scale... This does not prove your point. Annotating a large dynamically-typed codebase with type information is a large amount of work, regardless of the language. This tool makes that easier.
- blub 9y ago
  I'm bemused by your reply and curious to know why you think Dropbox and Instagram are working on static type analysers for their large Python code bases. Instagram at least gave us a hint: "we’re keen to make our code easier for new developers to read and understand, as well as more amenable to static analysis that shrinks the domain of possible bugs". It seems to me it's so difficult to manage such a code base, that they decided to do that "large amount of work" in addition to the large amount of work required to develop the necessary tools! In which case it seems prudent to avoid the said amount of work by picking another programming language for one's large-scale code base.
  - outsideoflife 9y ago
    Isn't dropbox doing it partially to aid a future migration to Python 3? > In which case it seems prudent to avoid the said amount of work by picking another programming language for one's large-scale code base When I first saw Instagram I thought it was nonsense, I was wrong. When the founders started Instagram, I wonder if they had the amazing foresight to see what it would become. I would suggest getting to market was more valuable to them than worrying about what maintenance they would have to do once they had a billion dollars
  - bonesss 9y ago
    > ... it seems prudent to avoid the said amount of work by picking another programming language for one's large-scale code base. While I'm a type-adherent in my day-to-day (F# represent, wut wuuut), I think this isn't reflective of the chicken and egg dilemma for startups... There are an immense number of things they "should be" doing at scale that they can't do early because they're relying on their early product to scale. Time to market, and windows of opportunity, are critical to startups. Any immature technical decision at that part of the lifecycle needs to be made not with an attempt to make perfect forever from the start, but rather with an eye to transitioning to smarter solutions aggressively as you scale up. The company may be 6 pivots away from success, so better to validate solutions in the market than prognosticate. I don't accept the premise that a lack of typing significantly improves development velocity, per se, but language decisions are about ecosystems, key components, and local talent. Where these companies get into coo-coo land is not integrating those immature components into better systems as they're getting bigger. Next thing you know someone is writing a whole compiler chain for PHP in an attempt to reinvent a sane programming language, or trying to get Python to be Java. Smart, modern, functional languages provide development velocity and pleasure equal to Python with fundamental type safety and guarantees. But in a world where just putting a button on a webpage means multiple dialects in multiple languages I think we should be ok mixing and matching on the backend to scale smartly.
techdragon 9y ago
The reality borne out by the evidence [0][1] is that Python is at the very least perfectly suitable for the development of web stacks powering companies worth in excess of hundreds of million dollars. Putting aside concrete technical issues regarding the python runtime’s performance envelope (eg: startup time, FFI inter-op call time, etc) and memory footprint, there is no reason not to use Python. Again, concrete technical issues aside, Python will not be an issue until your business is big enough that having to push the performance of your product is a nice problem to have and you will probably have the money to spend solving it by either optimising your python or rewriting parts of your application, as all these large companies have done. 0 - https://www.linkedin.com/pulse/top-10-sites-built-django-framework-vladimir-bogdanov https://www.linkedin.com/pulse/top-10-sites-built-django-fra... 1 - https://worldwebtech.weebly.com/blog/top-ten-most-popular-website-development-using-python-and-django-platform https://worldwebtech.weebly.com/blog/top-ten-most-popular-we...
- nothrabannosir 9y ago
  You can substitute Python for assembly, point at the Apollo mission and your argument still holds. Rarely is a programming language chosen because it actually is the best tool for the job. Often, it’s whatever the people at the ground floor were most comfortable getting a prototype out the door with. Prototype working well? Fix this bug, add yonder feature. Before you know it you’ve built Facebook in PHP. “PHP is at the very least suitable for the development etc etc”, yes, that’s why Facebook invested all that effort in Hiphop VM. There’s an irony in saying “there’s no reason not to use X” in reply to an article that is about a company spending a ton of effort working around a problem in X. My point is: as a community, it’s our duty to learn from these mistakes. Let’s admit there is a problem, investigate, adapt, overcome. We can improve whatever will be the next Python so the next Instagram doesn’t have to go through this. But that won’t work if we keep saying “this is a nice problem to have, there’s nothing wrong.” It isn’t. There is. Look at the article.
- blub 9y ago
  I'm not claiming that one can't be successful by using Python (or pretty much any programming language). Market success or user count are unfortunately not tightly correlated with technical excellence, as many of us have bitterly found out. Development speed, maintainability, error count are also important development issues. Unfortunately we don't have much data to judge, but the little that we have such as this article indicate that dynamic typing has a non-negligible negative impact on the above. ""having to push the performance of your product is a nice problem to have and you will probably have the money to spend solving it by either optimising your python or rewriting parts of your application, as all these large companies have done."" Not considering the topic at all is negligent. I don't understand why you're so sure that the only possible outcomes are either not reaching scale or having the money to optimise or rewrite. First of all, not even Facebook had the money to rewrite their PHP code base, so that's probably out of the question. And it's very well possible that one will reach scale and not have the money (or worse the time) to optimise, if optimising means inventing type checkers. At least one should spend some time thinking about this topic and picking a language that is flexible and can scale at least somewhat. Having to stop writing production code in order to invent a Ruby static type checker or native compiler is decidedly not a good problem to have.
moreless 9y ago
> This tool is the proof that Python has significant problems at scale, which is something the Python community has denied for a long time. They're still doing it in this thread, but the lesson looks pretty clear to me: if you plan on building large scale, don't use Python. Or PHP while we're at it (see Facebook). Yeah, absolutely, don't do this! These are two examples of successful companies that did it and look at them now! </s> Being able to move fast and produce a winning product on time is much more important for startups. What does it matter if you used <your_cool_scalable_thingie> for a project, when it never went past 10 users because you were concentrating on wrong side aspect of your business? PHP is fine. Python is great. Use the tools that fit your problem and you know how to use, not the latest toy.
- zerkten 9y ago
  How do you identify the inflection point and then execute when it hits? There are conservative choices that would scale all the way through, but many startups would avoid them due to the hit on "velocity" (which is itself a very fuzzy topic.) It seems that this same pattern plays out with many tools, and not just languages. When you've built something and you now have a team, processes, etc. built up it becomes difficult to see the forest for the trees, or to make the hard decisions because it might involve replacing people.
- 1_2__4 9y ago
  HN has a favorite pastime: dismissing tech that has demonstrated incredible utility because it’s lacks some kind of ideological purity they demand.
- blub 9y ago
  There are two things that are critically incorrect about your argument: 1) That market success implies having quality software. Average seems to be enough in my experience. 2) That start-ups are a good example to follow if one wants to achieve good quality. In fact they should be ignored, because they will absolutely murder quality in order to stay alive. Sometimes the product doesn't even work and is held together with duct tape in order to get past that important demo... It's quite pointless to discuss quality and start-ups. The lesson I mentioned should be heeded by mature companies that are able to do some project planning, complexity estimation, etc.
nine_k 9y ago
Use the right tool for the job. "Scale" can be seen under different angles. You can run thousands of boxes with relatively simple code if it's designed to scale horizontally. Twitter used to run on Ruby this way, until they accumulated money and expertise enough to rewrite the whole thing and save on operations and further development costs. Your code can be millions LOCs and run on a few boxes; it's a very different sort of "scale". In one company where I worked they had a crazy mix of codebases, from modern Scala down to ancient PHP code. But since it was architected reasonably, it was possible to replace the PHP code piecemeal, without stopping the system. Do the devs that started it 15 years ago with PHP deserve blame for choosing a poor language, or praise for coming up with a serviceable architecture? You can see the original post as a step in the right direction: in a complex codebase, static typing has a large number of advantages. Barring a wholesale rewrite, how would you gradually transform your code to using it? Yes, by documenting the current state in a formal way, introducing a typecheck build step, then maybe splitting out certain components and rewriting them in other languages, etc. Look at TypeScript. Unfortunately, there's no way around using quick-and-dirty prototypes at the very early stages, if nothing else, for lack of technical expertise among founders and their first employees. They have business ideas first and foremost. So a tool like that would help exactly the step you want done, switching to a nicer and more manageable stack as growth allows / demands it.