5 ms·

Its notable, perhaps, that the if name == "main": block allows you to do this for a _module_, i.e. a single *.py file. If you want to do this for a

by lizard 3y ago

Its notable, perhaps, that the if __name__ == "__main__": block allows you to do this for a _module_, i.e. a single *.py file. If you want to do this for a package, add a `__main__.py` You can also use either of these throughout your code so that you can have python -m foo python -m foo.bar python -m foo.bar.baz each doing different, but hopefully somewhat related, things.

BiteCode_dev 3y ago
It also uses the file as a module, not a script, which means suddenly relative imports works the root dir and the cwd are the same, and it is added to sys.path. This prevents a ton of import problems, albeit for the price of more verbose typing, especially since you don't have completion on dotted path. It is my favorite way of running my projects. Unfortunalty it means you can't use "-m pdb", and that's a big loss.
- sharikous 3y ago
  You can use pdb Just python -m pdb -m module
  - BiteCode_dev 3y ago
    Damn, 15 years of python and I learn you can use -m twice. I've never even tried, didn't occur to me it would be supported. EDIT: it support other options as well, like -c. That deserves an alias: debug_module() { if python -c "import ipdb" &>/dev/null; then python -m ipdb -c c -m "$@" else python -m pdb -c c -m "$@" fi }
    - abhishekjha 3y ago
      Like whats the general usage though? python -m http.server is the most I have done.
      - BiteCode_dev 3y ago
        -m pdb is post mortem debugging, it drops you in the debugger when it encounters the first unhandled exception. This is much easier than trying to pin point where to put a breakpoint.
    - kzrdude 3y ago
      There's no magic, only layers. `python -m pdb <args>` runs pdb with the rest of the arguments. pdb handles the second `-m`. If you have a fancy IDE feature, open a new python file, type "import pdb", use go to definition on pdb to jump to that file in the standard library, and read its main function - it handles -m explicitly :)
      - sharikous 3y ago
        Speaking of pdb.. maybe someone knows why pdb has some issues with scope in its REPL that are resolved in VSCode's debugger and PyCharm? Multiline statements are not accepted, nor things like if/for Even list comprehensions and lambda expressions have trouble loading local variables defined via the REPL Are there workarounds? It would reduce the need for using IDEs. People who have experience with Julia and Matlab are very used to a trial and error programming style in a console and bare python does not address this need
        BiteCode_dev 3y ago
        There are not workaround, pdb is a rustic tool. Since I still enjoy a cmdline debugger more than a graphical one, I use ipdb, which doesn't suffer from the multiline limitation. However, scoping issues with lambda and comprehension are actually a Python problem, not a pdb problem.
        johnnymellor 3y ago
        Not supporting multi-line statements is just because pdb doesn't bother to parse the statement to work out if it is an incomplete multi-line statement. That could be easily fixed (I have a prototype patch for that using `code.compile_command`). The scope problems are more fundamental: The pdb REPL calls[1] the exec builtin as `exec(code, frame.f_globals, frame.f_locals)`, which https://docs.python.org/3/library/functions.html#exec https://docs.python.org/3/library/functions.html#exec documents as: "If exec gets two separate objects as globals and locals, the code will be executed as if it were embedded in a class definition." And https://docs.python.org/3/reference/executionmodel.html#resolution-of-names https://docs.python.org/3/reference/executionmodel.html#reso... documents that: "The scope of names defined in a class block is limited to the class block; it does not extend to the code blocks of methods - this includes comprehensions and generator expressions since they are implemented using a function scope." This is a fundamental limitation of `exec`. You can workaround it by only passing a single namespace dictionary to exec instead of passing separate globals and locals, which is what pdb's interact command does[2], but then it's ambiguous how to map changes to that dictionary back to the separate globals and locals dictionaries (pdb's interact command just discards any changes you make to the namespace). This too could be solved, but requires either brittle ast parsing or probably a PEP to add new functionality to exec. I'll file a bug against Python soon. [1]: https://github.com/python/cpython/blob/25a64fd28aaaaf2d21fae23212a0554c24fc7b20/Lib/pdb.py#L479 https://github.com/python/cpython/blob/25a64fd28aaaaf2d21fae... [2]: https://github.com/python/cpython/blob/25a64fd28aaaaf2d21fae23212a0554c24fc7b20/Lib/pdb.py#L1584 https://github.com/python/cpython/blob/25a64fd28aaaaf2d21fae...
    - misnome 3y ago
      to be fair, it was only added in 3.7
- scruple 3y ago
  I also develop and run projects this way. I really, really enjoy it. It's a very pleasant experience, on both the development side and execution side. I'm relatively new to Python (used it for ~1 year in 2007/2008, again briefly in 2014 -- which is when I believe I picked this module trick up -- and then didn't touch it again until March of this year). It's made an impression on my team and we're all having a good time developing code this way. I do wonder, though, what other shortcomings might exist with this approach.
- andreareina 3y ago
  I've taken to adding a --pdb option to my scripts.
  - maleldil 3y ago
    What does it do? Just `import pdb; pdb.set_trace()`? Or `breakpoint()` after 3.7 (better because the user can override pdb with ipdb or other with `PYTHONBREAKPOINT`)?
    - andreareina 3y ago
      Either breakpoint() or setting sys.excepthook to call the postmortem debugger.
- quietbritishjim 3y ago
  > which means suddenly relative imports works Not just relative imports but also (properly formed) absolute imports. For example, if you have a directory my_pkg/ with files mod1.py and mod2.py, then # In my_pkg/mod2.py import my_pkg.mod1 will work if you run `python -m my_pkg.mod2` but will fail if you run `python my_pkg/mod2.py` However, the script syntax does work properly with absolute paths if if you set the enviornment variable `PYTHONPATH=.` (I don't know about relative paths - I don't use those). That would presumably allow pdb to work (but, shame on me, I've never tried it).
- jxramos 3y ago
  I've always been curious how that mechanism works exactly what is it about that invocation technique that satisfies the relative imports? I think it changes the pythonpath somehow right in a way related to the module being ran, something like appending the basedir where the module is saved to the PYTHONPATH?

Its notable, perhaps, that the if __name__ == "__main__": block allows you to do this for a _module_, i.e. a single *.py file. If you want to do this for a

by lizard 3y ago

BiteCode_dev 3y ago

It also uses the file as a module, not a script, which means suddenly relative imports works the root dir and the cwd are the same, and it is added to sys.path. This prevents a ton of import problems, albeit for the price of more verbose typing, especially since you don't have completion on dotted path. It is my favorite way of running my projects. Unfortunalty it means you can't use "-m pdb", and that's a big loss.

sharikous 3y ago
You can use pdb Just python -m pdb -m module
- BiteCode_dev 3y ago
  Damn, 15 years of python and I learn you can use -m twice. I've never even tried, didn't occur to me it would be supported. EDIT: it support other options as well, like -c. That deserves an alias: debug_module() { if python -c "import ipdb" &>/dev/null; then python -m ipdb -c c -m "$@" else python -m pdb -c c -m "$@" fi }
  - abhishekjha 3y ago
    Like whats the general usage though? python -m http.server is the most I have done.
    - BiteCode_dev 3y ago
      -m pdb is post mortem debugging, it drops you in the debugger when it encounters the first unhandled exception. This is much easier than trying to pin point where to put a breakpoint.
  - kzrdude 3y ago
    There's no magic, only layers. `python -m pdb <args>` runs pdb with the rest of the arguments. pdb handles the second `-m`. If you have a fancy IDE feature, open a new python file, type "import pdb", use go to definition on pdb to jump to that file in the standard library, and read its main function - it handles -m explicitly :)
    - sharikous 3y ago
      Speaking of pdb.. maybe someone knows why pdb has some issues with scope in its REPL that are resolved in VSCode's debugger and PyCharm? Multiline statements are not accepted, nor things like if/for Even list comprehensions and lambda expressions have trouble loading local variables defined via the REPL Are there workarounds? It would reduce the need for using IDEs. People who have experience with Julia and Matlab are very used to a trial and error programming style in a console and bare python does not address this need
      - BiteCode_dev 3y ago
        There are not workaround, pdb is a rustic tool. Since I still enjoy a cmdline debugger more than a graphical one, I use ipdb, which doesn't suffer from the multiline limitation. However, scoping issues with lambda and comprehension are actually a Python problem, not a pdb problem.
      - johnnymellor 3y ago
        Not supporting multi-line statements is just because pdb doesn't bother to parse the statement to work out if it is an incomplete multi-line statement. That could be easily fixed (I have a prototype patch for that using `code.compile_command`). The scope problems are more fundamental: The pdb REPL calls[1] the exec builtin as `exec(code, frame.f_globals, frame.f_locals)`, which https://docs.python.org/3/library/functions.html#exec https://docs.python.org/3/library/functions.html#exec documents as: "If exec gets two separate objects as globals and locals, the code will be executed as if it were embedded in a class definition." And https://docs.python.org/3/reference/executionmodel.html#resolution-of-names https://docs.python.org/3/reference/executionmodel.html#reso... documents that: "The scope of names defined in a class block is limited to the class block; it does not extend to the code blocks of methods - this includes comprehensions and generator expressions since they are implemented using a function scope." This is a fundamental limitation of `exec`. You can workaround it by only passing a single namespace dictionary to exec instead of passing separate globals and locals, which is what pdb's interact command does[2], but then it's ambiguous how to map changes to that dictionary back to the separate globals and locals dictionaries (pdb's interact command just discards any changes you make to the namespace). This too could be solved, but requires either brittle ast parsing or probably a PEP to add new functionality to exec. I'll file a bug against Python soon. [1]: https://github.com/python/cpython/blob/25a64fd28aaaaf2d21fae23212a0554c24fc7b20/Lib/pdb.py#L479 https://github.com/python/cpython/blob/25a64fd28aaaaf2d21fae... [2]: https://github.com/python/cpython/blob/25a64fd28aaaaf2d21fae23212a0554c24fc7b20/Lib/pdb.py#L1584 https://github.com/python/cpython/blob/25a64fd28aaaaf2d21fae...
  - misnome 3y ago
    to be fair, it was only added in 3.7
scruple 3y ago
I also develop and run projects this way. I really, really enjoy it. It's a very pleasant experience, on both the development side and execution side. I'm relatively new to Python (used it for ~1 year in 2007/2008, again briefly in 2014 -- which is when I believe I picked this module trick up -- and then didn't touch it again until March of this year). It's made an impression on my team and we're all having a good time developing code this way. I do wonder, though, what other shortcomings might exist with this approach.
andreareina 3y ago
I've taken to adding a --pdb option to my scripts.
- maleldil 3y ago
  What does it do? Just `import pdb; pdb.set_trace()`? Or `breakpoint()` after 3.7 (better because the user can override pdb with ipdb or other with `PYTHONBREAKPOINT`)?
  - andreareina 3y ago
    Either breakpoint() or setting sys.excepthook to call the postmortem debugger.
quietbritishjim 3y ago
> which means suddenly relative imports works Not just relative imports but also (properly formed) absolute imports. For example, if you have a directory my_pkg/ with files mod1.py and mod2.py, then # In my_pkg/mod2.py import my_pkg.mod1 will work if you run `python -m my_pkg.mod2` but will fail if you run `python my_pkg/mod2.py` However, the script syntax does work properly with absolute paths if if you set the enviornment variable `PYTHONPATH=.` (I don't know about relative paths - I don't use those). That would presumably allow pdb to work (but, shame on me, I've never tried it).
jxramos 3y ago
I've always been curious how that mechanism works exactly what is it about that invocation technique that satisfies the relative imports? I think it changes the pythonpath somehow right in a way related to the module being ran, something like appending the basedir where the module is saved to the PYTHONPATH?