5 ms·

A debugging switch that, when enabled, turns a valid program into an invalid one is madness. Python has decided to place very conservative requirements on what

by bcoates 14y ago

A debugging switch that, when enabled, turns a valid program into an invalid one is madness. Python has decided to place very conservative requirements on what an implementation's stack must be capable of. This might indicate a certain lack of ambition but it's a justifiable engineering tradeoff.

andolanra 14y ago
There are better alternatives to said debugging switch for examining the stack of tail-call-optimized programs, c.f. Clements and Felleisen's 2004 paper, "A Tail-Recursive Machine with Stack Inspection." The debugging switch is merely the simplest naïve option. Essentially, your engineering tradeoff boils down to, "It's easier (for implementers) to omit TCO." That is actually a very reasonable and rational position, as simpler implementations are less likely to have bugs and have a lower creation cost. However, I still hold that the advantages of TCO outweigh the concern about its implementation. (Also, I do not believe that Python does place conservative requirements on an implementation's stack, as a conformant Python implementation must also implement generators; I regard this as roughly on par with TCO's complexity.)
- kragen 14y ago
  Complexity and stack space usage are two very different things. CPython does indeed place (very!) conservative limits on stack depth, and it also implements generators. I haven't looked at the generator implementation, but it has a number of limits that make it easy to implement — the main one is that generators don't have their own stacks, so can't yield from inside a function they call, the way e.g. Lua coroutines or (effectively) Icon generators can.
  - andolanra 14y ago
    I'm sorry, I made something of a terminological error. When I said 'complexity', I was referring to complexity of implementation, not algorithmic time or space complexity. My intention was to assert that implementing generators and implementing tail-call optimization are both endeavors of roughly the same order of engineering difficulty (as the mechanism used to implement TCO is quite simple) and not to compare the stack space required by each. I also should clarify that at this point, it's likely not reasonable to expect Guido or the Python community to ever turn around and decide to implement TCO, as the current engineering effort required to add TCO to all active implementations of Python is much larger in comparison to the engineering required to add it at an earlier stage. I regard this as a failure of the langauge design from the start, but having made their decision I don't fault the Python community for sticking to it.
    - kragen 14y ago
      I was responding to your "I do not believe that Python does place conservative requirements on an implementation's stack", in response to bcoates's statement that it does, nothing about complexity. I was just trying to clarify (my interpretation of) what he said.
  - jacobolus 14y ago
    Actually, in Python 3.3 they can (sort of). See PEP 380: http://www.python.org/dev/peps/pep-0380/ http://www.python.org/dev/peps/pep-0380/
    - kragen 14y ago
      Thanks! I didn't know this, and it should remove a log-N factor from a lot of algorithms :)
- bcoates 14y ago
  I believe that both C# iterator-functions and python generators are designed with enough limits that they can be implemented as syntax sugar: you lift the generator function's locals into an object and decompose the function body into a state-machine.
derefr 14y ago
> A debugging switch that, when enabled, turns a valid program into an invalid one is madness. You mean like gcc's -Werror? Or enabling asserts? Both of those seem useful to me.
- Locke1689 14y ago
  Since the compiler didn't succeed at compilation, no program was generated at all. Removing asserts is really compiling a different program.
- PommeDeTerre 14y ago
  -Werror isn't a "debugging switch". It should be enabled at all times, for development builds and production builds. If compilation warnings are given, then the responsible thing to do is to assume the program is invalid, at least in certain situations. Fix the code, avoid the warnings completely, and that's that. There are remarkably few cases where compilation warnings can or should be ignored.

by bcoates 14y ago

andolanra 14y ago

There are better alternatives to said debugging switch for examining the stack of tail-call-optimized programs, c.f. Clements and Felleisen's 2004 paper, "A Tail-Recursive Machine with Stack Inspection." The debugging switch is merely the simplest naïve option. Essentially, your engineering tradeoff boils down to, "It's easier (for implementers) to omit TCO." That is actually a very reasonable and rational position, as simpler implementations are less likely to have bugs and have a lower creation cost. However, I still hold that the advantages of TCO outweigh the concern about its implementation. (Also, I do not believe that Python does place conservative requirements on an implementation's stack, as a conformant Python implementation must also implement generators; I regard this as roughly on par with TCO's complexity.)

kragen 14y ago
Complexity and stack space usage are two very different things. CPython does indeed place (very!) conservative limits on stack depth, and it also implements generators. I haven't looked at the generator implementation, but it has a number of limits that make it easy to implement — the main one is that generators don't have their own stacks, so can't yield from inside a function they call, the way e.g. Lua coroutines or (effectively) Icon generators can.
- andolanra 14y ago
  I'm sorry, I made something of a terminological error. When I said 'complexity', I was referring to complexity of implementation, not algorithmic time or space complexity. My intention was to assert that implementing generators and implementing tail-call optimization are both endeavors of roughly the same order of engineering difficulty (as the mechanism used to implement TCO is quite simple) and not to compare the stack space required by each. I also should clarify that at this point, it's likely not reasonable to expect Guido or the Python community to ever turn around and decide to implement TCO, as the current engineering effort required to add TCO to all active implementations of Python is much larger in comparison to the engineering required to add it at an earlier stage. I regard this as a failure of the langauge design from the start, but having made their decision I don't fault the Python community for sticking to it.
  - kragen 14y ago
    I was responding to your "I do not believe that Python does place conservative requirements on an implementation's stack", in response to bcoates's statement that it does, nothing about complexity. I was just trying to clarify (my interpretation of) what he said.
- jacobolus 14y ago
  Actually, in Python 3.3 they can (sort of). See PEP 380: http://www.python.org/dev/peps/pep-0380/ http://www.python.org/dev/peps/pep-0380/
  - kragen 14y ago
    Thanks! I didn't know this, and it should remove a log-N factor from a lot of algorithms :)
bcoates 14y ago
I believe that both C# iterator-functions and python generators are designed with enough limits that they can be implemented as syntax sugar: you lift the generator function's locals into an object and decompose the function body into a state-machine.

derefr 14y ago

> A debugging switch that, when enabled, turns a valid program into an invalid one is madness. You mean like gcc's -Werror? Or enabling asserts? Both of those seem useful to me.

Locke1689 14y ago
Since the compiler didn't succeed at compilation, no program was generated at all. Removing asserts is really compiling a different program.
PommeDeTerre 14y ago
-Werror isn't a "debugging switch". It should be enabled at all times, for development builds and production builds. If compilation warnings are given, then the responsible thing to do is to assume the program is invalid, at least in certain situations. Fix the code, avoid the warnings completely, and that's that. There are remarkably few cases where compilation warnings can or should be ignored.