13 ms·

But but but HN told me that python is only good for small scale and prototypes, the engineers at meta are wrong

by lagt_t 3y ago

DarkNova6 3y ago
The engineers at meta worte a language + runtime on top of PHP to make it work "at scale". If Facebook wants to make something work they have enough resources to throw at the problem to solve it. Regardless of whether it makes sense or not.
- jerf 3y ago
  I mean, as far as I'm concerned, everything Facebook has done has done nothing but reconfirm how unwise it is to build large-scale infrastructure on dynamic scripting languages. They have the resources to move heaven and earth to do what amounts to turning their dynamic scripting language back into static languages in everything but name, and they have the hole they've dug themselves into that justifies it. I have neither. I don't think that Facebook necessarily made a mistake at the time, though. Static languages have come a long way. I can prototype in them very nearly as quickly as I can in dynamic languages now, with the crossover point for where the static language is simply straight-up an advantage being roughly one to two weeks. Circa 2004 I would not even remotely make that claim. By the time static languages reached that state they were already in deep. But I would consider it a mistake for most cases to start right now, in 2023, with dynamic scripting languages as a base layer.
  - crabbone 3y ago
    > dynamic scripting languages. Why keep repeating this nonsense? "Dynamic" or "scripting" aren't features of languages. When anyone says something like this, it's like talking about square chicken... (i.e. a category error). Obviously, you had some idea in your mind, and you wanted to communicate it somehow, but your readers will not know what it was unless you make an effort to analyze what you want to say and make sure it doesn't have internal contradictions and that readers can within reason understand what you are trying to say. You seem to be complaining about something. My guess is that your problem isn't even with the language. Your problem is that some tools for working with your kind of programs are missing or aren't effective.
    - jerf 3y ago
      Because it a common term used for a category of languages that everybody understands. If you have a problem with it, you can take it up with the aforementioned everybody. Personally I think it gets perilously close to the error that if you argue definitions enough you can change reality. It doesn't matter what labels we slap on clearly related languages Python/Perl/Javascript/Lua/PHP/Javascript, I consider it a mistake to build a large system with them in 2023 because of their deficiencies, and given the extreme efforts being exerted trying to fix those very same deficiencies with things like gradual typing and strongly-typed languages like TypeScript that compile down into them, which includes the effort Facebook has exerted, clearly it is not a terribly heterodox opinion.
      - lijok 3y ago
        > I consider it a mistake to build a large system with them in 2023 A mistake in what sense? What are the risks that you see with using these languages for large scale systems, and how do those risks balance around all the other risks that come with not using one of these languages and using a different one?
        jerf 3y ago
        I think I pretty much laid out the bulk of the argument. Facebook is moving heaven and earth to mitigate their choice of a dynamic language, even if it was the right choice at the time. You can go look at everything they've done, and everything Typescript has done, and all the gradual typing initiatives. I left plenty of threads to pull there.
    - myvoiceismypass 3y ago
      Dynamic Typing is absolutely a feature of languages. Static Typing is, as well. I was not confused by using these terms. I cannot imagine many people on HN were either.
    - dukeyukey 3y ago
      > "Dynamic" or "scripting" aren't features of languages Surely dynamic typing is a language feature? I can't imagine what else someone would refer to with "dynamic".
- tyingq 3y ago
  Out of the box php also scaled pretty damn far for them before they had to do anything significant. Hundreds of millions of users as early as 2009, which seems to be before the original Hiphop existed.
sweettea 3y ago
https://discuss.python.org/t/a-fast-free-threading-python/27903/99 https://discuss.python.org/t/a-fast-free-threading-python/27... In which Meta promises 3 engineer-years for enhancing python.
pjmlp 3y ago
Like PHP, where FB has spent many man years writing AOT compilers, an new VM, and a strong typed variant from PHP?
corethree 3y ago
Python not only has types but it's type system is superior to typeScript. Get this python has sum types and exhaustive pattern matching exactly like rust or haskell. The only problem with python are the libraries are sometimes written with tricks that make static typing ineffective. Other than that it is really really good at scale. Better API then typescript imo which is really it's main competitor. edit: (rate limiter is preventing from replying to everyone... I will respond to everyone.)
- seabrookmx 3y ago
  Yes but the if you want to do ahead-of-time type checking you need to run a tool like mypy, and none of those tools are as comprehensive or performant as Typescript. Also the ecosystem of libraries with type annotations is much smaller (Typescript has the first mover advantage, after all).
  - corethree 3y ago
    I literally said it supports exhaustive pattern matching which adds a level of safety and flexibility superior to that of type script. Many IDEs have real time type checking that highlights the errors so don't even have to run the external checker. Even if you don't use IDEs running the type checker is measured in seconds. Not far off from linters that most people will also use for TS. What you say about the libraries is true though. But you can always place a type "shell" around those libraries such that your code is type safe. The other main problem with the libraries in python is that a lot of people who use python are data scientists who haven't figured out why types are so great. Those guys are the main ones holding the libraries back.
    - chlorion 3y ago
      >I literally said it supports exhaustive pattern matching which adds a level of safety and flexibility superior to that of type script. I don't have an opinion on which language is superior here, I've never written typescript or even javascript before, but I think saying that python's type system/checkers is superior because of this one feature is not correct. I'm also skeptical of the claim that typescript doesn't support exhaustive branching. This very well could be true but it seems hard to believe. I'm a big fan of exhaustive branching, but I think there are other things that are as or more important.
crabbone 3y ago
I've been writing in Python for over ten years, in different roles, for wildly different projects (research, infra, Web, testing, education). I'm yet to find anything Python was good for. On engineering merits alone Python isn't best for anything, nor is it best for combinations of things. It's silly to think that any tool that works with Python does so because Python was the best language for the job, and they only needed that tool to make it even better. Most stuff written around Python is yet another layer of band aids on top of a huge ball of band aids that's already there. So, you may wonder, with all those band aids, didn't they all make it better? And, in a sense, yes, that's what they are for. The band aids improve the experience of the user end. But this isn't how I use the word "better". When I use "better" I mean the quality of execution, not the satisfaction it gives to the user.
- devjab 3y ago
  I’m not really a fan of Python as such, but after a few decades in the industry, I’m beginning to think that being good at being bandaid is “better”. I can’t think of a single tech where we don’t have a bunch of duct tape (as we refer to it), not so much because we want to but because that’s just how things end up in the imperfect world of organisations. I value the techs that fit into this reality more than the ones which don’t, but you’re right, Python isn’t really great for any technology based merits, it’s good because the world is a messy place where being productive with your band aid is often more valuable than using the “better” programming language.
  - Izkata 3y ago
    Old proverb that I think applies here: A jack of all trades is a master of none, but oftentimes better than a master of one.
  - crabbone 3y ago
    Your mistake is going from quantitative claims to categorical without evaluating the quantitative part first. You say "everything is a band aid to some degree", and from that you conclude that "everything is equally bad or good". You conveniently forget the "to some degree" part to further your point. It's similar to saying that all food has some amount of dust in it, so it shouldn't matter whether you just open the vacuum cleaner and eat the stuff collected on the drum, or if you order a meal at a fancy restaurant. There's no evidence that Python "fits into reality" more than any other language. The thing that's going on for it is popularity. Popularity doesn't need to be rooted in technical merits, and in the case of Python it isn't. Finally, Python isn't unique in this sense. Even though the stage for programming language popularity pageant was set relatively recently, the participants learned to abuse the rules of the competition very quickly. There's a "rule" by a statistician whose name I cannot recall at this point which states that once people know the metric they are measured on, they will learn to game it. When we assess language popularity, we most assess how well the language authors or their community was able to game the metric rather than measuring any meaningful aspect of those languages.
dehrmann 3y ago
Disclosure: Meta employee, but these views are my own, and not all the experiences are from Meta. Type annotation have made Python much more scalable in terms of engineers and codebase size. It still has other scale problems, especially if you actually need threads. One project I worked on managed Python worker tasks, and we resorted to subprocesses (within subprocesses!) because what we thought was IO-bound became CPU-bound, and workers started timing out on RPC calls. I also worked on a Python API service that scaled beautifully horizontally, but we had to manage extra logic for spinning up one worker per CPU. At some point, you actually start caring about performance, but you're more likely to hit other issues before you care about the extra hardware cost.

But but but HN told me that python is only good for small scale and prototypes, the engineers at meta are wrong

by lagt_t 3y ago

But but but HN told me that python is only good for small scale and prototypes, the engineers at meta are wrong

The engineers at meta worte a language + runtime on top of PHP to make it work "at scale". If Facebook wants to make something work they have enough resources to throw at the problem to solve it. Regardless of whether it makes sense or not.

jerf 3y ago
I mean, as far as I'm concerned, everything Facebook has done has done nothing but reconfirm how unwise it is to build large-scale infrastructure on dynamic scripting languages. They have the resources to move heaven and earth to do what amounts to turning their dynamic scripting language back into static languages in everything but name, and they have the hole they've dug themselves into that justifies it. I have neither. I don't think that Facebook necessarily made a mistake at the time, though. Static languages have come a long way. I can prototype in them very nearly as quickly as I can in dynamic languages now, with the crossover point for where the static language is simply straight-up an advantage being roughly one to two weeks. Circa 2004 I would not even remotely make that claim. By the time static languages reached that state they were already in deep. But I would consider it a mistake for most cases to start right now, in 2023, with dynamic scripting languages as a base layer.
- crabbone 3y ago
  > dynamic scripting languages. Why keep repeating this nonsense? "Dynamic" or "scripting" aren't features of languages. When anyone says something like this, it's like talking about square chicken... (i.e. a category error). Obviously, you had some idea in your mind, and you wanted to communicate it somehow, but your readers will not know what it was unless you make an effort to analyze what you want to say and make sure it doesn't have internal contradictions and that readers can within reason understand what you are trying to say. You seem to be complaining about something. My guess is that your problem isn't even with the language. Your problem is that some tools for working with your kind of programs are missing or aren't effective.
  - jerf 3y ago
    Because it a common term used for a category of languages that everybody understands. If you have a problem with it, you can take it up with the aforementioned everybody. Personally I think it gets perilously close to the error that if you argue definitions enough you can change reality. It doesn't matter what labels we slap on clearly related languages Python/Perl/Javascript/Lua/PHP/Javascript, I consider it a mistake to build a large system with them in 2023 because of their deficiencies, and given the extreme efforts being exerted trying to fix those very same deficiencies with things like gradual typing and strongly-typed languages like TypeScript that compile down into them, which includes the effort Facebook has exerted, clearly it is not a terribly heterodox opinion.
    - lijok 3y ago
      > I consider it a mistake to build a large system with them in 2023 A mistake in what sense? What are the risks that you see with using these languages for large scale systems, and how do those risks balance around all the other risks that come with not using one of these languages and using a different one?
      - jerf 3y ago
        I think I pretty much laid out the bulk of the argument. Facebook is moving heaven and earth to mitigate their choice of a dynamic language, even if it was the right choice at the time. You can go look at everything they've done, and everything Typescript has done, and all the gradual typing initiatives. I left plenty of threads to pull there.
  - myvoiceismypass 3y ago
    Dynamic Typing is absolutely a feature of languages. Static Typing is, as well. I was not confused by using these terms. I cannot imagine many people on HN were either.
  - dukeyukey 3y ago
    > "Dynamic" or "scripting" aren't features of languages Surely dynamic typing is a language feature? I can't imagine what else someone would refer to with "dynamic".
tyingq 3y ago
Out of the box php also scaled pretty damn far for them before they had to do anything significant. Hundreds of millions of users as early as 2009, which seems to be before the original Hiphop existed.

sweettea 3y ago

https://discuss.python.org/t/a-fast-free-threading-python/27903/99 https://discuss.python.org/t/a-fast-free-threading-python/27... In which Meta promises 3 engineer-years for enhancing python.

pjmlp 3y ago

Like PHP, where FB has spent many man years writing AOT compilers, an new VM, and a strong typed variant from PHP?

corethree 3y ago

Python not only has types but it's type system is superior to typeScript. Get this python has sum types and exhaustive pattern matching exactly like rust or haskell. The only problem with python are the libraries are sometimes written with tricks that make static typing ineffective. Other than that it is really really good at scale. Better API then typescript imo which is really it's main competitor. edit: (rate limiter is preventing from replying to everyone... I will respond to everyone.)

seabrookmx 3y ago
Yes but the if you want to do ahead-of-time type checking you need to run a tool like mypy, and none of those tools are as comprehensive or performant as Typescript. Also the ecosystem of libraries with type annotations is much smaller (Typescript has the first mover advantage, after all).
- corethree 3y ago
  I literally said it supports exhaustive pattern matching which adds a level of safety and flexibility superior to that of type script. Many IDEs have real time type checking that highlights the errors so don't even have to run the external checker. Even if you don't use IDEs running the type checker is measured in seconds. Not far off from linters that most people will also use for TS. What you say about the libraries is true though. But you can always place a type "shell" around those libraries such that your code is type safe. The other main problem with the libraries in python is that a lot of people who use python are data scientists who haven't figured out why types are so great. Those guys are the main ones holding the libraries back.
  - chlorion 3y ago
    >I literally said it supports exhaustive pattern matching which adds a level of safety and flexibility superior to that of type script. I don't have an opinion on which language is superior here, I've never written typescript or even javascript before, but I think saying that python's type system/checkers is superior because of this one feature is not correct. I'm also skeptical of the claim that typescript doesn't support exhaustive branching. This very well could be true but it seems hard to believe. I'm a big fan of exhaustive branching, but I think there are other things that are as or more important.

crabbone 3y ago

I've been writing in Python for over ten years, in different roles, for wildly different projects (research, infra, Web, testing, education). I'm yet to find anything Python was good for. On engineering merits alone Python isn't best for anything, nor is it best for combinations of things. It's silly to think that any tool that works with Python does so because Python was the best language for the job, and they only needed that tool to make it even better. Most stuff written around Python is yet another layer of band aids on top of a huge ball of band aids that's already there. So, you may wonder, with all those band aids, didn't they all make it better? And, in a sense, yes, that's what they are for. The band aids improve the experience of the user end. But this isn't how I use the word "better". When I use "better" I mean the quality of execution, not the satisfaction it gives to the user.

devjab 3y ago
I’m not really a fan of Python as such, but after a few decades in the industry, I’m beginning to think that being good at being bandaid is “better”. I can’t think of a single tech where we don’t have a bunch of duct tape (as we refer to it), not so much because we want to but because that’s just how things end up in the imperfect world of organisations. I value the techs that fit into this reality more than the ones which don’t, but you’re right, Python isn’t really great for any technology based merits, it’s good because the world is a messy place where being productive with your band aid is often more valuable than using the “better” programming language.
- Izkata 3y ago
  Old proverb that I think applies here: A jack of all trades is a master of none, but oftentimes better than a master of one.
- crabbone 3y ago
  Your mistake is going from quantitative claims to categorical without evaluating the quantitative part first. You say "everything is a band aid to some degree", and from that you conclude that "everything is equally bad or good". You conveniently forget the "to some degree" part to further your point. It's similar to saying that all food has some amount of dust in it, so it shouldn't matter whether you just open the vacuum cleaner and eat the stuff collected on the drum, or if you order a meal at a fancy restaurant. There's no evidence that Python "fits into reality" more than any other language. The thing that's going on for it is popularity. Popularity doesn't need to be rooted in technical merits, and in the case of Python it isn't. Finally, Python isn't unique in this sense. Even though the stage for programming language popularity pageant was set relatively recently, the participants learned to abuse the rules of the competition very quickly. There's a "rule" by a statistician whose name I cannot recall at this point which states that once people know the metric they are measured on, they will learn to game it. When we assess language popularity, we most assess how well the language authors or their community was able to game the metric rather than measuring any meaningful aspect of those languages.

dehrmann 3y ago

Disclosure: Meta employee, but these views are my own, and not all the experiences are from Meta. Type annotation have made Python much more scalable in terms of engineers and codebase size. It still has other scale problems, especially if you actually need threads. One project I worked on managed Python worker tasks, and we resorted to subprocesses (within subprocesses!) because what we thought was IO-bound became CPU-bound, and workers started timing out on RPC calls. I also worked on a Python API service that scaled beautifully horizontally, but we had to manage extra logic for spinning up one worker per CPU. At some point, you actually start caring about performance, but you're more likely to hit other issues before you care about the extra hardware cost.