4 ms·

I know nothing about Lisp, so at the risk of talking past eachother.... I never said macros were required. I said implementing this type of code without OOP re

I know nothing about Lisp, so at the risk of talking past eachother.... I never said macros were required. I said implementing this type of code without OOP required more boilerplate, and MLJ uses macros to reduce that boilerplate. As I understand module imports in Julia: Each module developer exports a list of publicly facing objects. Obviously "fit" and "model" would be among them. If you import two modules that both export a new "nn" subtype of shared parent type "model", and both extend "fit" and "predict" and etc to accept their own subtype "nn", then you have to manually specify which module you are referring to every time you call nn, or fit, or predict, or whatever. Is that wrong? If I just import PyTorch, import TensorFlow, and then call "mymodel = TensorFlow.nn; fit(mymodel, mydata)" then Julia doesn't know that the "fit" I am calling is the TensorFlow implementation and not the PyTorch implementation; what if I had WANTED to use module A's "fit" on module B's model, and they intentionally adopted the same abstract type system to enable this interoperability? So instead I have to write "mymodel = TensorFlow.nn; TensorFlow.fit(mymodel, mydata); TensorFlow.predict(mymodel, mynewdata)". Obviously the extra typing is mildly annoying but the bigger problem is potentially introducing bugs by mismatching modules, and the developer's cognitive overload of having to keep track of modules. Python style OOP is a more elegant solution to the namespace problem and results in more readable, maintainble code, at least in my opinion. Anyways, maybe Julia has a more elegant solution I'm not aware of, if so I'd love to hear it.

cbkeller 5y ago
In Julia it will just dispatch to the correct function. In other words, one package would define `fit(mymodel::TensorFlowModel)` and the other would define `fit(mymodel:PyTorchModel)`, and then when you call `fit` it'll just dispatch to the appropriate one depending on the type of `mymodel`. This dispatch-oriented style also allows a shocking degree of composability, e.g. [1], where a lot of packages will just work together, such that you could for example just use the equivalent of PyTorch or TensorFlow on the equivalent of (e.g.) NumPy arrays without having to convert anything. If you mean "what about the case where both packages just call their model type `Model`", while I've never run into that, the worst case scenario is just that you have to fall back to the Python style explicit Module.function usage (which was always allowed anyways...). And if you if you don't like names being exported, you can always just `import` a package instead of `using` it: help?> import search: import import import Foo will load the module or package Foo. Names from the imported Foo module can be accessed with dot syntax (e.g. Foo.foo to access the name foo). See the manual section about modules for details. [1] https://www.youtube.com/watch?v=kc9HwsxE1OY https://www.youtube.com/watch?v=kc9HwsxE1OY
- deleted 5y ago
  [deleted]
- awaythrowact 5y ago
  I very frequently run into namespace collisions like that. I think they are quite common in large codebases. I am aware of the ability to do eg "import TensorFlow; model = TensorFlow.model; TensorFlow.fit(model,data)" As I mentioned previously, I find Python's OOP "model.fit" syntax to be better, for a variety of reasons. Thank you for your engagement. Have a nice day.
  - cbkeller 5y ago
    Do you have an example of a case where you ran into this in Julia with two packages that you wanted to use together? If the packages are still actively developed, I suspect the developers would be interested to resolve the situation to allow interop.
    - awaythrowact 5y ago
      Never made it that far. This was a feature I use all the time in Python ML development (both consuming open source packages via an OOP interface and also writing in-house model classes) that I consider essential for my productivity and that Julia was missing. <edit>I'm also nervous of relying on the recourse of asking package maintainers to edit their variable names to improve compatibility with the random third package I want to use; maybe the culture is different in Julia but in Python that's a good way to get laughed out of the issue tracker :) </edit> If I were you I would maybe ask people like @systemvoltage who took the plunge and wrote a big project in Julia only to find they had trouble maintaining the project. Maybe one reason he can't upgrade without breaking everything is because of namespace collisions amongst his many dependencies? If not that, it's something like that.
      - cbkeller 5y ago
        Composability is one of the things the Julia community generally Takes Seriouslyᵀᴹ so definitely don't hesitate to ask if there are two packages that don't play as nicely as you would like! I'm a bit confused still though why you say it's a "missing" feature, given that as we discussed above, there is absolutely nothing to stop anyone who wants to use the "Python OOP" style of namespacing in Julia from doing so? Most of us don't seem to find it necessary or to prefer it personally, but that doesn't restrict anyone else from choosing it.
deleted 5y ago
[deleted]

cbkeller 5y ago

In Julia it will just dispatch to the correct function. In other words, one package would define `fit(mymodel::TensorFlowModel)` and the other would define `fit(mymodel:PyTorchModel)`, and then when you call `fit` it'll just dispatch to the appropriate one depending on the type of `mymodel`. This dispatch-oriented style also allows a shocking degree of composability, e.g. [1], where a lot of packages will just work together, such that you could for example just use the equivalent of PyTorch or TensorFlow on the equivalent of (e.g.) NumPy arrays without having to convert anything. If you mean "what about the case where both packages just call their model type `Model`", while I've never run into that, the worst case scenario is just that you have to fall back to the Python style explicit Module.function usage (which was always allowed anyways...). And if you if you don't like names being exported, you can always just `import` a package instead of `using` it: help?> import search: import import import Foo will load the module or package Foo. Names from the imported Foo module can be accessed with dot syntax (e.g. Foo.foo to access the name foo). See the manual section about modules for details. [1] https://www.youtube.com/watch?v=kc9HwsxE1OY https://www.youtube.com/watch?v=kc9HwsxE1OY

deleted 5y ago
[deleted]
awaythrowact 5y ago
I very frequently run into namespace collisions like that. I think they are quite common in large codebases. I am aware of the ability to do eg "import TensorFlow; model = TensorFlow.model; TensorFlow.fit(model,data)" As I mentioned previously, I find Python's OOP "model.fit" syntax to be better, for a variety of reasons. Thank you for your engagement. Have a nice day.
- cbkeller 5y ago
  Do you have an example of a case where you ran into this in Julia with two packages that you wanted to use together? If the packages are still actively developed, I suspect the developers would be interested to resolve the situation to allow interop.
  - awaythrowact 5y ago
    Never made it that far. This was a feature I use all the time in Python ML development (both consuming open source packages via an OOP interface and also writing in-house model classes) that I consider essential for my productivity and that Julia was missing. <edit>I'm also nervous of relying on the recourse of asking package maintainers to edit their variable names to improve compatibility with the random third package I want to use; maybe the culture is different in Julia but in Python that's a good way to get laughed out of the issue tracker :) </edit> If I were you I would maybe ask people like @systemvoltage who took the plunge and wrote a big project in Julia only to find they had trouble maintaining the project. Maybe one reason he can't upgrade without breaking everything is because of namespace collisions amongst his many dependencies? If not that, it's something like that.
    - cbkeller 5y ago
      Composability is one of the things the Julia community generally Takes Seriouslyᵀᴹ so definitely don't hesitate to ask if there are two packages that don't play as nicely as you would like! I'm a bit confused still though why you say it's a "missing" feature, given that as we discussed above, there is absolutely nothing to stop anyone who wants to use the "Python OOP" style of namespacing in Julia from doing so? Most of us don't seem to find it necessary or to prefer it personally, but that doesn't restrict anyone else from choosing it.

deleted 5y ago

[deleted]