4 ms·

Personally, I like how the Julia ecosystem coalesced around a Tables.jl package that gives some consistency to tabular data packages that choose to implement it

Personally, I like how the Julia ecosystem coalesced around a Tables.jl package that gives some consistency to tabular data packages that choose to implement it. Row-oriented or column-oriented, doesn't matter! Using Tables.jl they can be interchangeable.

mlthoughts2018 6y ago
There’s a huge difference between “I personally like it when some libraries choose to adhere to a convention” vs “let’s bake this into every tool with a wide, bureaucratic shared RFC process.”
- xiaodai 6y ago
  But I don't think people are forced into following right? There's still choice. But if I was developing something in Python, I would rather join than not join.
  - mlthoughts2018 6y ago
    I’m saying as a consumer of these libraries, I don’t want joiners, I want lots of different APIs that make different trade-offs for each use case, and I will write (or use) adapters if I need to (that’s my responsibility as a consumer). Just because I personally either do or do not get some value out of a shared API standard doesn’t mean you do or do not, so personal feelings of what one likes need to be set aside, so the service boundary is at the level of consumers choosing their own adapters and making their own trade-offs. If lots of libraries sign up, we all lose. Much slower compliance-constrained development cycles, less freedom for libraries to break the shared API contract in ways many users would be super happy to abide for the sake of a faster / better feature. Instead of making these trade-offs library by library, case by case, it’s made for you with standardization compliance as the highest constraint, regardless of the user base for which that does / doesn’t work well.
    - biddlesby 6y ago
      This strikes me as a bit pessimistic. I think you are mostly against _bad_ standards than standards per se. If the consortium does its job well it will produce a good standard that all the major libraries like: from a UX as well as a performance viewpoint. If it produces a bad standard, all that will happen is that nobody will sign up. No libraries are going to be _forced_ to do anything.
      - mlthoughts2018 6y ago
        That’s fair except I’d say even good standards can cause a big slowdown in feature releases, and not everyone values the adherence to the standard as much as the features. It still seems better to just spin out the standard into a set of optional adapters and let library maintainers and users pick their own trade-offs in terms of when to adopt breaking changes, how / whether to smooth over idiosyncrasies across multiple libraries. Even a good standard has costs and this particular case does not seem like it has good arguments in favor of a wide standard, but many arguments against it.
        westurner 6y ago
        No, it's easy for library maintainers to offer a compat API in addition to however else they feel they need to differentiate and optimize the interfaces for array operations. People can contribute such APIs directly to libraries once instead of creating many conditionals in every library-utilizing project or requiring yet another dependency on an adapter / facade package that's not kept in sync with the libraries it abstracts. If a library chooses to implement a spec compatability API, they do that once (optimally, as compared with somebody's hackish adapter facade which has very little comprehension of each library's internals) and everyone else's code doesn't need to have conditionals. Each of L libraries implements a compat API: O(L) Each of U library utilizers implements conditionals for every N places arrays are utilized: O(U x N_) Each of U library utilizers uses the common denominator compat API: O(U) L < U < (L + U) < (U x N_)
    - rabidrat 6y ago
      Where the APIs are compatible anyway, but only the names or order of arguments are different, it sure would be nice to have a standard so you didn't have to look up whether it was count or len or length or size.
improbable22 6y ago
Perhaps a more direct Julia analogy would be the built-in AbstractArray interface. There is simply no question of how (to use their example) `mean(::CuArray; dims)` should work, it not only follows Base's notation, but extends the same `Statistics.mean` function, so that exotic arrays can be passed to packages written before the package defining them existed.

mlthoughts2018 6y ago

There’s a huge difference between “I personally like it when some libraries choose to adhere to a convention” vs “let’s bake this into every tool with a wide, bureaucratic shared RFC process.”

xiaodai 6y ago
But I don't think people are forced into following right? There's still choice. But if I was developing something in Python, I would rather join than not join.
- mlthoughts2018 6y ago
  I’m saying as a consumer of these libraries, I don’t want joiners, I want lots of different APIs that make different trade-offs for each use case, and I will write (or use) adapters if I need to (that’s my responsibility as a consumer). Just because I personally either do or do not get some value out of a shared API standard doesn’t mean you do or do not, so personal feelings of what one likes need to be set aside, so the service boundary is at the level of consumers choosing their own adapters and making their own trade-offs. If lots of libraries sign up, we all lose. Much slower compliance-constrained development cycles, less freedom for libraries to break the shared API contract in ways many users would be super happy to abide for the sake of a faster / better feature. Instead of making these trade-offs library by library, case by case, it’s made for you with standardization compliance as the highest constraint, regardless of the user base for which that does / doesn’t work well.
  - biddlesby 6y ago
    This strikes me as a bit pessimistic. I think you are mostly against _bad_ standards than standards per se. If the consortium does its job well it will produce a good standard that all the major libraries like: from a UX as well as a performance viewpoint. If it produces a bad standard, all that will happen is that nobody will sign up. No libraries are going to be _forced_ to do anything.
    - mlthoughts2018 6y ago
      That’s fair except I’d say even good standards can cause a big slowdown in feature releases, and not everyone values the adherence to the standard as much as the features. It still seems better to just spin out the standard into a set of optional adapters and let library maintainers and users pick their own trade-offs in terms of when to adopt breaking changes, how / whether to smooth over idiosyncrasies across multiple libraries. Even a good standard has costs and this particular case does not seem like it has good arguments in favor of a wide standard, but many arguments against it.
      - westurner 6y ago
        No, it's easy for library maintainers to offer a compat API in addition to however else they feel they need to differentiate and optimize the interfaces for array operations. People can contribute such APIs directly to libraries once instead of creating many conditionals in every library-utilizing project or requiring yet another dependency on an adapter / facade package that's not kept in sync with the libraries it abstracts. If a library chooses to implement a spec compatability API, they do that once (optimally, as compared with somebody's hackish adapter facade which has very little comprehension of each library's internals) and everyone else's code doesn't need to have conditionals. Each of L libraries implements a compat API: O(L) Each of U library utilizers implements conditionals for every N places arrays are utilized: O(U x N_) Each of U library utilizers uses the common denominator compat API: O(U) L < U < (L + U) < (U x N_)
  - rabidrat 6y ago
    Where the APIs are compatible anyway, but only the names or order of arguments are different, it sure would be nice to have a standard so you didn't have to look up whether it was count or len or length or size.

improbable22 6y ago

Perhaps a more direct Julia analogy would be the built-in AbstractArray interface. There is simply no question of how (to use their example) `mean(::CuArray; dims)` should work, it not only follows Base's notation, but extends the same `Statistics.mean` function, so that exotic arrays can be passed to packages written before the package defining them existed.