4 ms·

This actually strikes me as a huuuge waste of effort. I work with every one of the different technologies they mention every day. The differences and idiosyncr

by mlthoughts2018 6y ago

This actually strikes me as a huuuge waste of effort. I work with every one of the different technologies they mention every day. The differences and idiosyncrasies are truly, truly not a big deal and really what you want is to allow different library maintainers to do it all differently and just build your own adapters over top of them. This allows library developers to worry about narrow use cases and have their own separate processes to introduce breaking changes and features that deal with extreme specifics like GPU or TPU interop, heterogeneous distributed backed arrays, jagged arrays, etc. etc. Let a thousand flowers blossom and a thousand different Python array and record APIs contend. End users can write their own adapters to mollify irregularities between them, possibly writing different adapters on a case by case basis. If any “standard” adapters gain popularity as open source projects, great - but don’t try to bake that in from an RFC point of view into the array & data structure libraries themselves. Let them be free / whatever they want to be. That diversity of API approach is super valuable and easily mollified by your own custom adapter logic.

xiaodai 6y ago
Personally, I like how the Julia ecosystem coalesced around a Tables.jl package that gives some consistency to tabular data packages that choose to implement it. Row-oriented or column-oriented, doesn't matter! Using Tables.jl they can be interchangeable.
- mlthoughts2018 6y ago
  There’s a huge difference between “I personally like it when some libraries choose to adhere to a convention” vs “let’s bake this into every tool with a wide, bureaucratic shared RFC process.”
  - xiaodai 6y ago
    But I don't think people are forced into following right? There's still choice. But if I was developing something in Python, I would rather join than not join.
    - mlthoughts2018 6y ago
      I’m saying as a consumer of these libraries, I don’t want joiners, I want lots of different APIs that make different trade-offs for each use case, and I will write (or use) adapters if I need to (that’s my responsibility as a consumer). Just because I personally either do or do not get some value out of a shared API standard doesn’t mean you do or do not, so personal feelings of what one likes need to be set aside, so the service boundary is at the level of consumers choosing their own adapters and making their own trade-offs. If lots of libraries sign up, we all lose. Much slower compliance-constrained development cycles, less freedom for libraries to break the shared API contract in ways many users would be super happy to abide for the sake of a faster / better feature. Instead of making these trade-offs library by library, case by case, it’s made for you with standardization compliance as the highest constraint, regardless of the user base for which that does / doesn’t work well.
      - biddlesby 6y ago
        This strikes me as a bit pessimistic. I think you are mostly against _bad_ standards than standards per se. If the consortium does its job well it will produce a good standard that all the major libraries like: from a UX as well as a performance viewpoint. If it produces a bad standard, all that will happen is that nobody will sign up. No libraries are going to be _forced_ to do anything.
        mlthoughts2018 6y ago
        That’s fair except I’d say even good standards can cause a big slowdown in feature releases, and not everyone values the adherence to the standard as much as the features. It still seems better to just spin out the standard into a set of optional adapters and let library maintainers and users pick their own trade-offs in terms of when to adopt breaking changes, how / whether to smooth over idiosyncrasies across multiple libraries. Even a good standard has costs and this particular case does not seem like it has good arguments in favor of a wide standard, but many arguments against it.
        6y ago
- improbable22 6y ago
  Perhaps a more direct Julia analogy would be the built-in AbstractArray interface. There is simply no question of how (to use their example) `mean(::CuArray; dims)` should work, it not only follows Base's notation, but extends the same `Statistics.mean` function, so that exotic arrays can be passed to packages written before the package defining them existed.
travisoliphant 6y ago
Bias warning --- I'm part of the group doing this work. For an end-user, your arguments are fine. A single developer or team can indeed use whatever tool they want and adapt their code without too much difficulty. The real challenge comes, however, as you try to build an ecosystem of libraries on top of multiple different APIs. You end up reinventing the wheel or struggling to maintain multiple versions of the same library. Some cooperation is immensely helpful at the lowest level, and we hope ultimately to be helpful to the different tool builders in providing information and guidance about what is already a standard and agreed upon in the space. This is not about establishing or forcing behavior. It is about documenting and clarifying best practices that have already emerged as well as potentially giving libraries a way to signal to downstream developers of additional libraries that they adhere to the standard.
- BiteCode_dev 6y ago
  Do you have any ball bark value you could estimate for the size of a bare bone common array implementation? I'm dreaming of having basic numpy features in the Python stlib, but numpy is so huge, and I have no idea how much of it is the array itself, and how much are all the tooling around it.
- mlthoughts2018 6y ago
  I don’t really disagree with most of your points, but what you describe does not sound like it should be a committee with governance rules and member libraries tied to an RFC process. What you describe sounds like you could just create your own single library like a Swiss army knife that contains adapters to rationalize the semantics if you want, even at the ABI level if truly needed. Member libraries can do a best effort support for not breaking that adapter library’s way of wrapping them, and/or contribute patches that keeps the adapter in sync with new releases of the library. This makes the adapter layer become totally opt-in and backported rather than required via consortium membership & RFC governance, and even leaves room for competitor adapter tools to coexist that might do things differently. Ultimately my concern with a governance model is that it creates political power to compel the member libraries to do things that might go against the needs of their users for the sake of the consortium. I just don’t see a reason why that’s more valuable than letting all the libraries do their own thing and let users choose or make their own custom adapters. This example of array APIs is nothing like say overall HTTP protocol or IEEE floating point protocol, which are cases where a governing standard makes sense. This is nothing like that in my opinion.
short_sells_poo 6y ago
So what would you have libraries like scikits-learn do? Support half a dozen array APIs until a 7th array implementation comes around? Support one, and force users to litter adapters everywhere and (often poorly) reinvent the wheel? I'd very much welcome a unified array API initiative. It's been accomplished in Julia and it's incredibly empowering. The fact that today you have to special case functions because the various array types are only loosely compatible is a nightmare, not a benefit.
djaychela 6y ago
As someone who's got some experience with this (but nowhere near as much as you have - I am an amateur), I'd like to strongly disagree with you here. I find that slightly different APIs between libraries are a real energy-sapper, and having to do the mental gymnastics around them makes little sense to me. Yes, as a super-experienced programmer, you can remember each subtle difference in the syntax of each API, but I think many people find this difficult (me included), and it also leads to problems with code that's written when you've missed something or not used the appropriate part of the API. One of the strengths of using SciKitLearn for me has been the interoperability across different ML models - by having the same API it means you can try something quickly without having to rewrite code, and can concentrate on the end results, not whether or not you're really telling it what you think you are because you've swapped an element of your pipeline out. Learning the standard library of Python is a big task; there have been countless times when I've created something which is actually present, but I didn't know it was there, or how to look for it, and then 3 months later seen that utilised in some other code and had a "Why didn't I know that?" moment. I think having non-standard APIs across libraries will lead to this sort of situation, and a lot of reading documentation which would be obviated by the approach outlined in the article. >Forcing the same call semantics on all these users doesn’t make sense. I don't understand why not (providing, of course, they are all capable of having whatever is relevant conveyed to them appropriately? What are the trade-offs that following this standard would involve (other than the delay due to bureaucracy that you mention)? I don't see what is gained by the use of different method/function calls for what are the same method/function, as outlined in the table [1] [1] - https://data-apis.org/images/array_API_comparison_output.png https://data-apis.org/images/array_API_comparison_output.png
- mlthoughts2018 6y ago
  I think a scary part of your comment is that since you don’t feel it would impact others use cases and you don’t see why some library might want different semantics, you leap to the conclusion that the burden of proof to prove these things rests with other people.

xiaodai 6y ago

Personally, I like how the Julia ecosystem coalesced around a Tables.jl package that gives some consistency to tabular data packages that choose to implement it. Row-oriented or column-oriented, doesn't matter! Using Tables.jl they can be interchangeable.

mlthoughts2018 6y ago
There’s a huge difference between “I personally like it when some libraries choose to adhere to a convention” vs “let’s bake this into every tool with a wide, bureaucratic shared RFC process.”
- xiaodai 6y ago
  But I don't think people are forced into following right? There's still choice. But if I was developing something in Python, I would rather join than not join.
  - mlthoughts2018 6y ago
    I’m saying as a consumer of these libraries, I don’t want joiners, I want lots of different APIs that make different trade-offs for each use case, and I will write (or use) adapters if I need to (that’s my responsibility as a consumer). Just because I personally either do or do not get some value out of a shared API standard doesn’t mean you do or do not, so personal feelings of what one likes need to be set aside, so the service boundary is at the level of consumers choosing their own adapters and making their own trade-offs. If lots of libraries sign up, we all lose. Much slower compliance-constrained development cycles, less freedom for libraries to break the shared API contract in ways many users would be super happy to abide for the sake of a faster / better feature. Instead of making these trade-offs library by library, case by case, it’s made for you with standardization compliance as the highest constraint, regardless of the user base for which that does / doesn’t work well.
    - biddlesby 6y ago
      This strikes me as a bit pessimistic. I think you are mostly against _bad_ standards than standards per se. If the consortium does its job well it will produce a good standard that all the major libraries like: from a UX as well as a performance viewpoint. If it produces a bad standard, all that will happen is that nobody will sign up. No libraries are going to be _forced_ to do anything.
      - mlthoughts2018 6y ago
        That’s fair except I’d say even good standards can cause a big slowdown in feature releases, and not everyone values the adherence to the standard as much as the features. It still seems better to just spin out the standard into a set of optional adapters and let library maintainers and users pick their own trade-offs in terms of when to adopt breaking changes, how / whether to smooth over idiosyncrasies across multiple libraries. Even a good standard has costs and this particular case does not seem like it has good arguments in favor of a wide standard, but many arguments against it.
        6y ago
improbable22 6y ago
Perhaps a more direct Julia analogy would be the built-in AbstractArray interface. There is simply no question of how (to use their example) `mean(::CuArray; dims)` should work, it not only follows Base's notation, but extends the same `Statistics.mean` function, so that exotic arrays can be passed to packages written before the package defining them existed.

travisoliphant 6y ago

Bias warning --- I'm part of the group doing this work. For an end-user, your arguments are fine. A single developer or team can indeed use whatever tool they want and adapt their code without too much difficulty. The real challenge comes, however, as you try to build an ecosystem of libraries on top of multiple different APIs. You end up reinventing the wheel or struggling to maintain multiple versions of the same library. Some cooperation is immensely helpful at the lowest level, and we hope ultimately to be helpful to the different tool builders in providing information and guidance about what is already a standard and agreed upon in the space. This is not about establishing or forcing behavior. It is about documenting and clarifying best practices that have already emerged as well as potentially giving libraries a way to signal to downstream developers of additional libraries that they adhere to the standard.

BiteCode_dev 6y ago
Do you have any ball bark value you could estimate for the size of a bare bone common array implementation? I'm dreaming of having basic numpy features in the Python stlib, but numpy is so huge, and I have no idea how much of it is the array itself, and how much are all the tooling around it.
mlthoughts2018 6y ago
I don’t really disagree with most of your points, but what you describe does not sound like it should be a committee with governance rules and member libraries tied to an RFC process. What you describe sounds like you could just create your own single library like a Swiss army knife that contains adapters to rationalize the semantics if you want, even at the ABI level if truly needed. Member libraries can do a best effort support for not breaking that adapter library’s way of wrapping them, and/or contribute patches that keeps the adapter in sync with new releases of the library. This makes the adapter layer become totally opt-in and backported rather than required via consortium membership & RFC governance, and even leaves room for competitor adapter tools to coexist that might do things differently. Ultimately my concern with a governance model is that it creates political power to compel the member libraries to do things that might go against the needs of their users for the sake of the consortium. I just don’t see a reason why that’s more valuable than letting all the libraries do their own thing and let users choose or make their own custom adapters. This example of array APIs is nothing like say overall HTTP protocol or IEEE floating point protocol, which are cases where a governing standard makes sense. This is nothing like that in my opinion.

short_sells_poo 6y ago

So what would you have libraries like scikits-learn do? Support half a dozen array APIs until a 7th array implementation comes around? Support one, and force users to litter adapters everywhere and (often poorly) reinvent the wheel? I'd very much welcome a unified array API initiative. It's been accomplished in Julia and it's incredibly empowering. The fact that today you have to special case functions because the various array types are only loosely compatible is a nightmare, not a benefit.

djaychela 6y ago

As someone who's got some experience with this (but nowhere near as much as you have - I am an amateur), I'd like to strongly disagree with you here. I find that slightly different APIs between libraries are a real energy-sapper, and having to do the mental gymnastics around them makes little sense to me. Yes, as a super-experienced programmer, you can remember each subtle difference in the syntax of each API, but I think many people find this difficult (me included), and it also leads to problems with code that's written when you've missed something or not used the appropriate part of the API. One of the strengths of using SciKitLearn for me has been the interoperability across different ML models - by having the same API it means you can try something quickly without having to rewrite code, and can concentrate on the end results, not whether or not you're really telling it what you think you are because you've swapped an element of your pipeline out. Learning the standard library of Python is a big task; there have been countless times when I've created something which is actually present, but I didn't know it was there, or how to look for it, and then 3 months later seen that utilised in some other code and had a "Why didn't I know that?" moment. I think having non-standard APIs across libraries will lead to this sort of situation, and a lot of reading documentation which would be obviated by the approach outlined in the article. >Forcing the same call semantics on all these users doesn’t make sense. I don't understand why not (providing, of course, they are all capable of having whatever is relevant conveyed to them appropriately? What are the trade-offs that following this standard would involve (other than the delay due to bureaucracy that you mention)? I don't see what is gained by the use of different method/function calls for what are the same method/function, as outlined in the table [1] [1] - https://data-apis.org/images/array_API_comparison_output.png https://data-apis.org/images/array_API_comparison_output.png

mlthoughts2018 6y ago
I think a scary part of your comment is that since you don’t feel it would impact others use cases and you don’t see why some library might want different semantics, you leap to the conclusion that the burden of proof to prove these things rests with other people.