4 ms·

It's another junk format in the sea of junk formats. It sucks for describing data, its structure is too primitive, has wrong primitives, doesn't allow extensio

It's another junk format in the sea of junk formats. It sucks for describing data, its structure is too primitive, has wrong primitives, doesn't allow extension and has no mitigations for that. Essentially, it's used because it's simple to parse and there are existing parsers. But the problems it creates are very complicated. So, here's one example of a popular use-case: pyproject.toml. I mean, PyPA aren't the brightest minds around, so, they've created this joke. The stated goal of pyproject.toml was unification of dependency / requirements specification. But, what they ended up doing is this: [tool.joke] tool.joke.requires=<requirements> [tool.clown_fiesta] tool.clown_fiesta.requires=<requirements> The problem here is that the goal was to unify, to specify requirements only once... but in practice, you still have to write them twice, and, of course, it's possible that "joke" and "clown_fiesta" don't use the same format to specify requirements. So, you have no way to specify requirements for both at once, and no way to process similar structures, to present them differently to different tools. And so, PyPA has created an piece of junk software that today has to be used by hundreds of thousands. Because we live in the world of worse is better.

eviks 3y ago
Could you list a couple of great formats? The various issues are indeed widespread across all the popular ones, and there are too many formats to find the greats (wish there were a comparison site with all the features so you could easily exclude those that fail at a specific task)
jborean93 3y ago
Can you share an actual example where you need to specify requirements twice. With pyproject.toml I know of 3 ways requirements are specified and they are all used by different things: 1. Build requirements - requirements needed to build your package from an sdist 2. Runtime requirements - requirements your library needs at runtime 3. Extra requirements - optional runtime requirements For example my library pyspnego https://github.com/jborean93/pyspnego/blob/main/pyproject.toml https://github.com/jborean93/pyspnego/blob/main/pyproject.to... has all 3: 1. Cython for Win32 needed to build the sdist and setuptools as the general build system - https://github.com/jborean93/pyspnego/blob/c3db058b636fc102fa9f7db9ad02f8b46cd62379/pyproject.toml#L1-L6 https://github.com/jborean93/pyspnego/blob/c3db058b636fc102f... 2. cryptography as a runtime dependency - https://github.com/jborean93/pyspnego/blob/c3db058b636fc102fa9f7db9ad02f8b46cd62379/pyproject.toml#L28-L30 https://github.com/jborean93/pyspnego/blob/c3db058b636fc102f... 3. optional extras kerberos and yaml - https://github.com/jborean93/pyspnego/blob/c3db058b636fc102fa9f7db9ad02f8b46cd62379/pyproject.toml#L33-L38 https://github.com/jborean93/pyspnego/blob/c3db058b636fc102f... Granted the toml format and what is used in pyproject.toml has its warts but I'm curious what your joke and clown_fiesta examples are actually from as from where I am standing each section currently serve different purposes.
- crabbone 3y ago
  Even in your list, there's a problem. I'll describe it first, and then will get to the one I had in mind when I gave my example. It's common that your build requirements and your runtime requirements have a big overlap that has to iterate together. Here's a realistic example: NumPy comes with various build helpers that you need at build time, if you are making a native module. So, you need to depend on a particular version of NumPy both during the build and at runtime. In other words, you would want to be able to reference the same package + version in two places. You could say that this is a problem with "specific" package, where the developers didn't care to separate two kinds of requirements... and in some sense you are right... but: there is a value to having build helpers and the rest of the runtime code in the same package. It's simpler. It prevents those using your package from accidentally messing up. Finally, it's NumPy we are talking about, it's not a "specific" package... it's like 50% of the entire Python ecosystem. --- Now, to what I mean in my example. This is taken verbatim from "real Python" site for the lack of a better example: # pyproject.toml [tool.poetry] name = "rp-poetry" version = "0.1.0" description = "" authors = ["Philipp <philipp@realpython.com>"] [tool.poetry.dependencies] python = "^3.9" [tool.poetry.dev-dependencies] pytest = "^5.2" [build-system] requires = ["poetry-core>=1.0.0"] build-backend = "poetry.core.masonry.api" See how "python = ^3.9" is a dependency of Poetry? But what if I also want to make my project build with setuptools? Setuptools also depends on Python... where would I put that now? Not under [tool.setuptools.dependencies]? What do you think?
Spivak 3y ago
Huh? [project] dependencies = [blah] [project.optional-dependencies] groupname = [blah] Install project with `pip install .` and install optional deps with `pip install .[groupname]`.
localghost3000 3y ago
> It's another junk format in the sea of junk formats. > PyPA aren't the brightest minds around, so, they've created this joke You have some interesting and valid criticisms of the format. Why not just say them without all the harsh insulting language? Is this how you'd want someone to speak about you and/or your work?
- crabbone 3y ago
  I don't care what kind of language people use when they speak about me. Also, I select what to say to say precisely what I want. Junk is something that is likely of poor quality, old and broken and therefore discarded, useless. This perfectly describes what TOML, JSON, YAML and the rest in the sea of junk formats are. PyPA are awful at doing their only job: defining rules for Python packaging. In the post above I gave one illustration of the poor quality of their work. Something that could've been easily prevented had they applied themselves to critically assessing the outcomes of their work. And, if need be, I can find a dozen of examples of the same, and probably more. And they don't work in isolation, even in Python world the problem of packaging Python code is not unique to them. They can be compared oranges-to-oranges with other similar efforts. And, yes, they aren't the brightest by a wide margin. And they should be told about it, because they self-selected to handle the problem plenty of people who didn't nominate them for that role depend on in their daily lives. It's a moral duty of anyone who understands how bad they are to repeatedly remind them about it.
  - localghost3000 3y ago
    [flagged]
3193528104 3y ago
> I mean, PyPA aren't the brightest minds around You don't need to be an asshole to get your message across.
- crabbone 3y ago
  I don't care if I'm an asshole. The subject is PyPA's incompetence. And I don't see any point of tiptoeing around this fact. PyPA self-selected to handle the task they aren't up to. They keep doing it despite repeatedly failing at it. And they would not relinquish the power, nor would they listen to the criticism. Regardless of whether I am an asshole or not, the impact of my actions is negligible compared to the damage PyPA has done and keeps doing to the very thing they are meant to support. They should be called what they are, without hesitation and euphemisms. This is the right thing to do.
  - 3193528104 3y ago
    My claim is that you _don't_ need to be an asshole to get your point across. It sounds like your goal is to change how PyPA works/what it does. That's great! I think you will be less effective in your goals by personally insulting the individuals behind the group; regardless of whether you think that this _ought_ to be the case. > They should be called what they are, without hesitation and euphemisms. You disagree with decisions made by the people at PyPA. I don't think you presented any reason to believe that these people were dumb? Or that they had a low IQ? Or anything else to that extent? You disagree with decisions, and then concluded that this _must_ make them dumb. Is it possible that they are simply making different trade-offs from you? Or that there's some (possibly mutual) misunderstanding of the problem? As another commenter has pointed out, some of your claims about the state of PyPA tooling were spurious, yet nobody would say that "you're not the brightest" because of it! You seem like a passionate person, and I genuinely hope that you come to the conclusion that empathy (especially in technical contexts) is a better route than hurling insults. I hope that you can direct your passion to making the world a better place, rather than putting people down. The world could certainly use the help.

eviks 3y ago

Could you list a couple of great formats? The various issues are indeed widespread across all the popular ones, and there are too many formats to find the greats (wish there were a comparison site with all the features so you could easily exclude those that fail at a specific task)

jborean93 3y ago

Can you share an actual example where you need to specify requirements twice. With pyproject.toml I know of 3 ways requirements are specified and they are all used by different things: 1. Build requirements - requirements needed to build your package from an sdist 2. Runtime requirements - requirements your library needs at runtime 3. Extra requirements - optional runtime requirements For example my library pyspnego https://github.com/jborean93/pyspnego/blob/main/pyproject.toml https://github.com/jborean93/pyspnego/blob/main/pyproject.to... has all 3: 1. Cython for Win32 needed to build the sdist and setuptools as the general build system - https://github.com/jborean93/pyspnego/blob/c3db058b636fc102fa9f7db9ad02f8b46cd62379/pyproject.toml#L1-L6 https://github.com/jborean93/pyspnego/blob/c3db058b636fc102f... 2. cryptography as a runtime dependency - https://github.com/jborean93/pyspnego/blob/c3db058b636fc102fa9f7db9ad02f8b46cd62379/pyproject.toml#L28-L30 https://github.com/jborean93/pyspnego/blob/c3db058b636fc102f... 3. optional extras kerberos and yaml - https://github.com/jborean93/pyspnego/blob/c3db058b636fc102fa9f7db9ad02f8b46cd62379/pyproject.toml#L33-L38 https://github.com/jborean93/pyspnego/blob/c3db058b636fc102f... Granted the toml format and what is used in pyproject.toml has its warts but I'm curious what your joke and clown_fiesta examples are actually from as from where I am standing each section currently serve different purposes.

crabbone 3y ago
Even in your list, there's a problem. I'll describe it first, and then will get to the one I had in mind when I gave my example. It's common that your build requirements and your runtime requirements have a big overlap that has to iterate together. Here's a realistic example: NumPy comes with various build helpers that you need at build time, if you are making a native module. So, you need to depend on a particular version of NumPy both during the build and at runtime. In other words, you would want to be able to reference the same package + version in two places. You could say that this is a problem with "specific" package, where the developers didn't care to separate two kinds of requirements... and in some sense you are right... but: there is a value to having build helpers and the rest of the runtime code in the same package. It's simpler. It prevents those using your package from accidentally messing up. Finally, it's NumPy we are talking about, it's not a "specific" package... it's like 50% of the entire Python ecosystem. --- Now, to what I mean in my example. This is taken verbatim from "real Python" site for the lack of a better example: # pyproject.toml [tool.poetry] name = "rp-poetry" version = "0.1.0" description = "" authors = ["Philipp <philipp@realpython.com>"] [tool.poetry.dependencies] python = "^3.9" [tool.poetry.dev-dependencies] pytest = "^5.2" [build-system] requires = ["poetry-core>=1.0.0"] build-backend = "poetry.core.masonry.api" See how "python = ^3.9" is a dependency of Poetry? But what if I also want to make my project build with setuptools? Setuptools also depends on Python... where would I put that now? Not under [tool.setuptools.dependencies]? What do you think?

Spivak 3y ago

Huh? [project] dependencies = [blah] [project.optional-dependencies] groupname = [blah] Install project with `pip install .` and install optional deps with `pip install .[groupname]`.

localghost3000 3y ago

> It's another junk format in the sea of junk formats. > PyPA aren't the brightest minds around, so, they've created this joke You have some interesting and valid criticisms of the format. Why not just say them without all the harsh insulting language? Is this how you'd want someone to speak about you and/or your work?

crabbone 3y ago
I don't care what kind of language people use when they speak about me. Also, I select what to say to say precisely what I want. Junk is something that is likely of poor quality, old and broken and therefore discarded, useless. This perfectly describes what TOML, JSON, YAML and the rest in the sea of junk formats are. PyPA are awful at doing their only job: defining rules for Python packaging. In the post above I gave one illustration of the poor quality of their work. Something that could've been easily prevented had they applied themselves to critically assessing the outcomes of their work. And, if need be, I can find a dozen of examples of the same, and probably more. And they don't work in isolation, even in Python world the problem of packaging Python code is not unique to them. They can be compared oranges-to-oranges with other similar efforts. And, yes, they aren't the brightest by a wide margin. And they should be told about it, because they self-selected to handle the problem plenty of people who didn't nominate them for that role depend on in their daily lives. It's a moral duty of anyone who understands how bad they are to repeatedly remind them about it.
- localghost3000 3y ago
  [flagged]

3193528104 3y ago

> I mean, PyPA aren't the brightest minds around You don't need to be an asshole to get your message across.

crabbone 3y ago
I don't care if I'm an asshole. The subject is PyPA's incompetence. And I don't see any point of tiptoeing around this fact. PyPA self-selected to handle the task they aren't up to. They keep doing it despite repeatedly failing at it. And they would not relinquish the power, nor would they listen to the criticism. Regardless of whether I am an asshole or not, the impact of my actions is negligible compared to the damage PyPA has done and keeps doing to the very thing they are meant to support. They should be called what they are, without hesitation and euphemisms. This is the right thing to do.
- 3193528104 3y ago
  My claim is that you _don't_ need to be an asshole to get your point across. It sounds like your goal is to change how PyPA works/what it does. That's great! I think you will be less effective in your goals by personally insulting the individuals behind the group; regardless of whether you think that this _ought_ to be the case. > They should be called what they are, without hesitation and euphemisms. You disagree with decisions made by the people at PyPA. I don't think you presented any reason to believe that these people were dumb? Or that they had a low IQ? Or anything else to that extent? You disagree with decisions, and then concluded that this _must_ make them dumb. Is it possible that they are simply making different trade-offs from you? Or that there's some (possibly mutual) misunderstanding of the problem? As another commenter has pointed out, some of your claims about the state of PyPA tooling were spurious, yet nobody would say that "you're not the brightest" because of it! You seem like a passionate person, and I genuinely hope that you come to the conclusion that empathy (especially in technical contexts) is a better route than hurling insults. I hope that you can direct your passion to making the world a better place, rather than putting people down. The world could certainly use the help.