3 ms·

I do most of my work with Perl, rather than Python, but in Perl there are several kinds of stock import-level side-effects which are actually quite helpful. Th

I do most of my work with Perl, rather than Python, but in Perl there are several kinds of stock import-level side-effects which are actually quite helpful. These are pretty light-weight though. They boil down to: 1. Global lexical changes to Perl. Sometimes this is the whole point of an import (for example Carp::Always, which typically turns any warning or exception string into a full stack dump). I can't imagine doing something like this in Python. However making soemthing like this work properly without breaking too many things requires a heck of a lot of forethought Yes, even Carp::Always may break something. 2. Manipulation of the importing module's symbol table. This is important for lexical extensions to Perl that you don't want to be global (for example Moo, Moose, and PGObject::Util::DBMethod). Among other things this allows MOPs to be added with greater sophistication than the language typically allows. I am not a Python guru but I could imagine metaprogramming side effects to be useful in setting up a consistent and powerful environment. The problem the author describes is something which is different though, which not only is a side effect issue but also a violation of separation of concerns. There are certain problems you do not want to solve at import time, and connecting with/configuring external components is almost always one of them. Why? Because integration with external components is almost always something you want the fine-tuning and decision-making to reside with the application developer. That's very different than setting up a consistent lexical programming environment for use (which is what the acceptable side effects do).

chrismorgan 13y ago
In Python, side-effects are simply not acceptable at all. One should instead put the code with the side-effect in a function and call it. This is the approach taken by gevent, which allows you to replace the entire I/O stack. But importing it does not effect the change; you must explicitly call code to do that. This is done thus: from gevent.monkey import patch_all patch_all() c.f. http://www.gevent.org/gevent.monkey.html http://www.gevent.org/gevent.monkey.html
- Mithaldu 13y ago
  In Perl the act of simply importing that patch_all function would be a side-effect, as it's implemented as code in pure perl that changes the caller's symbol table. I suspect it would be fair to call your import up there also a side-effect, but that it slips the mind since it's implemented in the python core. :)
  - einhverfr 13y ago
    Indeed. Also there are times (Moose for example) where you want to do more than import a function as written. For example Moo and Moose create a custom function "has" at import time that is them attached to the caller's namespace. The reason for this is that you want to fully attach the function in the space as a native method and this requires some closures to make work sanely. In short you want: package foo; use Moose; has bar => (is => 'rw'); to behave identically as: package notfoo; use foo; foo::has(bar => (is=>'rw')); If you rely on the caller the second example would add a bar accessor to the caller's namespace (notfoo) rather than to foo. In short you on't want to use "has" in the Moose namespace. You want to add a custom function to the importing namespace. As I understand it you can't do this in Python not because it is a bad idea (it is a very good idea sometimes) but rather because of limitations in the language (no multiline lambdas, which you need in order to do useful stuff with the symbol table in this regard unless there is an equivalent in terms of defining a method the calling class inside a closure in the imported package). But again, my knowledge of Perl is much better than my knowledge of Python, so I could be wrong. When you get into this sort of metaprogramming, custom symbol table manipulations are not only helpful but downright necessary. It is what allows you to add a sophisticated MOP when such is not included in core.
    - marcosdumay 13y ago
      Technically, you can do this with Python, but you'll have to use some dark magic to get the importing module namespace, in order to manipulate it. It'll not be pretty. Instead, in normal Python it is the importing module that must decide who has access to its own namespace, and who must live within a separated one. It has a completely different set of costs and benefits. (I sometimes wonder if Guido used Perl as a counterexample when creating Python - its principles are almost completely opposite.)
- ta0967 13y ago
  from __future__ import print_function
  - einhverfr 13y ago
    That's a side effect, right? Your module after the import is not the same as it was before, or am I missing something?
    - chrismorgan 13y ago
      It only changes the behaviour for the current file, so it's not a side-effect.
      - einhverfr 13y ago
        That seems too narrow a view of side effect. Which of the following are side effects? 1. Initializing a point of sale printer, checking for errors, and raising exceptions if, say, it is out of paper? Let's say this is a cash drawer driver and the cash drawer connects through the printer, and if the printer is out of paper, the drawer won't open properly (this happens btw). I would call this a side effect as well as a separation of concerns violation btw. However it is not likely to be a visible change to other modules. 2. Check for the presence of a binary and if found, cache the path to it, perhaps instantiating another object to do so? Definitely a side effect there, but not publicly visible. 3. Initializing an external library's environment (as happened in this case)? Done wrong it crashes the system but I suspect the segfault was not intended. Again it isnt clear to me you have a publically visible side effect intended.
    - ak217 13y ago
      Future imports are not imports at all (the fact that they use the same syntax as imports is misleading IMO). They are tags that instruct the interpreter to turn on certain features only when interpreting the given file. They must appear before any statements in the file, and they take effect before interpretation of the file starts. You could argue that this constitutes an import side effect, but that's neither here nor there, and not helpful.
jerf 13y ago
Technically in Perl, a "use" statement is asking for those manipulations, though. If you want to just load the library but not ask for any of those things, you would require the library, either in or not in a BEGIN block depending on your desires. If, however, merely "require"ing the library performed any of those manipulations, that would be a violation of what I'd expect. (Of course, if the sole purpose of your module is to perform those manipulations, "require" may not do anything useful, but I'd still expect it not to do anything to my local namespace, either.)

chrismorgan 13y ago

In Python, side-effects are simply not acceptable at all. One should instead put the code with the side-effect in a function and call it. This is the approach taken by gevent, which allows you to replace the entire I/O stack. But importing it does not effect the change; you must explicitly call code to do that. This is done thus: from gevent.monkey import patch_all patch_all() c.f. http://www.gevent.org/gevent.monkey.html http://www.gevent.org/gevent.monkey.html

Mithaldu 13y ago
In Perl the act of simply importing that patch_all function would be a side-effect, as it's implemented as code in pure perl that changes the caller's symbol table. I suspect it would be fair to call your import up there also a side-effect, but that it slips the mind since it's implemented in the python core. :)
- einhverfr 13y ago
  Indeed. Also there are times (Moose for example) where you want to do more than import a function as written. For example Moo and Moose create a custom function "has" at import time that is them attached to the caller's namespace. The reason for this is that you want to fully attach the function in the space as a native method and this requires some closures to make work sanely. In short you want: package foo; use Moose; has bar => (is => 'rw'); to behave identically as: package notfoo; use foo; foo::has(bar => (is=>'rw')); If you rely on the caller the second example would add a bar accessor to the caller's namespace (notfoo) rather than to foo. In short you on't want to use "has" in the Moose namespace. You want to add a custom function to the importing namespace. As I understand it you can't do this in Python not because it is a bad idea (it is a very good idea sometimes) but rather because of limitations in the language (no multiline lambdas, which you need in order to do useful stuff with the symbol table in this regard unless there is an equivalent in terms of defining a method the calling class inside a closure in the imported package). But again, my knowledge of Perl is much better than my knowledge of Python, so I could be wrong. When you get into this sort of metaprogramming, custom symbol table manipulations are not only helpful but downright necessary. It is what allows you to add a sophisticated MOP when such is not included in core.
  - marcosdumay 13y ago
    Technically, you can do this with Python, but you'll have to use some dark magic to get the importing module namespace, in order to manipulate it. It'll not be pretty. Instead, in normal Python it is the importing module that must decide who has access to its own namespace, and who must live within a separated one. It has a completely different set of costs and benefits. (I sometimes wonder if Guido used Perl as a counterexample when creating Python - its principles are almost completely opposite.)
ta0967 13y ago
from __future__ import print_function
- einhverfr 13y ago
  That's a side effect, right? Your module after the import is not the same as it was before, or am I missing something?
  - chrismorgan 13y ago
    It only changes the behaviour for the current file, so it's not a side-effect.
    - einhverfr 13y ago
      That seems too narrow a view of side effect. Which of the following are side effects? 1. Initializing a point of sale printer, checking for errors, and raising exceptions if, say, it is out of paper? Let's say this is a cash drawer driver and the cash drawer connects through the printer, and if the printer is out of paper, the drawer won't open properly (this happens btw). I would call this a side effect as well as a separation of concerns violation btw. However it is not likely to be a visible change to other modules. 2. Check for the presence of a binary and if found, cache the path to it, perhaps instantiating another object to do so? Definitely a side effect there, but not publicly visible. 3. Initializing an external library's environment (as happened in this case)? Done wrong it crashes the system but I suspect the segfault was not intended. Again it isnt clear to me you have a publically visible side effect intended.
  - ak217 13y ago
    Future imports are not imports at all (the fact that they use the same syntax as imports is misleading IMO). They are tags that instruct the interpreter to turn on certain features only when interpreting the given file. They must appear before any statements in the file, and they take effect before interpretation of the file starts. You could argue that this constitutes an import side effect, but that's neither here nor there, and not helpful.

jerf 13y ago

Technically in Perl, a "use" statement is asking for those manipulations, though. If you want to just load the library but not ask for any of those things, you would require the library, either in or not in a BEGIN block depending on your desires. If, however, merely "require"ing the library performed any of those manipulations, that would be a violation of what I'd expect. (Of course, if the sole purpose of your module is to perform those manipulations, "require" may not do anything useful, but I'd still expect it not to do anything to my local namespace, either.)