4 ms·

Type-checking _via_ those annotations, are, like Guido said, basically a souped-up linter. With a dynamic language like Python, there is _never_ a guarantee tha

Type-checking _via_ those annotations, are, like Guido said, basically a souped-up linter. With a dynamic language like Python, there is _never_ a guarantee that the types that type-checked during compilation are the types that will be observed during runtime. Type-checking Python code via function annotations is nothing like type checking a Haskell program, where you can be reasonably certain that a program that type-checks will run correctly. Even if you write a program that you can provably show will never have any runtime types that differ from compile time types, Pypy can't assume that will be the case with every program that it tries to run. I don't have any specific knowledge of the optimizations that Pypy runs, but if one of the core contributors of Pypy says it can't be done, I'm inclined to believe him.

andreasvc 12y ago
It is trivial to write a function that always returns a specific type, e.g., def f(): return 1 Therefore your claim that there is "_never_ a guarantee" is simply false. I was asking: what if you DO want PyPy to assume that your types are correct, why couldn't that help?
- bjt 12y ago
  In Python you have no guarantee that that function hasn't been monkeypatched over and replaced by some other library you imported.
  - andreasvc 12y ago
    From what I read those things are problematic for PyPy anyway, causing it to fallback to an unoptimized interpreter. Also, the type annotations could be interpreted as a promise that what is being annotated will not be monkeypatched at runtime. What is a type annotation, if not a promise that something is not as dynamic as it could otherwise be?
- untothebreach 12y ago
  Monkeypatching means that `def f(): return 1` could be turned into `def f(): return "i am now a string!"` at runtime, so I stand by my original statement. EDIT: Here is a trivial example of a program that type checks at compile time but doesn't present the same types at runtime. Obviously you would never do this in production code, but the same effect could be accomplished simply by doing an `import somemodule`. http://codepad.org/OarBYXfZ http://codepad.org/OarBYXfZ
  - andreasvc 12y ago
    Your replacing what 'f' is bound to; this would include replacing any type annotations, and triggering any system that relies on type annotations to drop/reset its optimizations for 'f'. You're not actually rewriting the original function to return something different; this would probably also be possible, but would be considered as even more of a dirty trick. It is unfortunate that such dynamicity is the default in Python, when in reality it is rarely needed. So it would be advantageous to be able to tell the optimization system: this function is not going to change dynamically (on an opt-in basis).
- DasIch 12y ago
  You could replace the code object at runtime changing what the function does and what the type of the returned object is.
bsaul 12y ago
But wouldn't annotatin at least help pypy choose a better suited runtime type ?
- untothebreach 12y ago
  We are out of my depth here, I have no experience or knowledge of the optimizations that Pypy does. So, I am trusting Alex Gaynor's word that it wouldn't help, since if anyone would be an expert on Pypy's optimizations, it would be him.
- davesque 12y ago
  They wouldn't, because there is no runtime specification for enforcement of types based on type annotations. PyPy still can't assume anything about what type a value will be because a value `v` annotated as `str` could still be changed as follows: v = 1234 Then, PyPy would crash or behave erratically since its assumptions about the value's type are false.
  - rwl 12y ago
    > a value `v` annotated as `str` could still be changed as follows: > v = 1234 Surely at least this could be caught by a static type checker? If `v` is annotated to be of type `str`, assigning it a constant int should be a type error, though I'm sure Python provides more indirect ways of setting `v` that would be harder to catch statically (like globals()['v'] = ...). But your comment raises an interesting question: are the annotations MyPy is relying on (static) type annotations, or value annotations? I tend to think of the difference between static and dynamic typing as whether types attach to expressions or to values. If annotations only attach to the value to which a variable is initially bound in some context, I can see how they really wouldn't be much help in this scenario, since `v = 1234` is simply rebinding `v` to a value of a different type, and a previous annotation wouldn't preclude that. But if annotations attach to expressions, then it would be inconsistent to bind `v` to a value like 1234, since the constant "1234" is of a different type than `v` is.
  - bsaul 12y ago
    "there is no runtime specification for enforcement of types based on type annotations" Indeed, if you consider type annotation as just annotation and nothing else. But at least you "could" imagine an option to have pypy optimize based on them.

andreasvc 12y ago

It is trivial to write a function that always returns a specific type, e.g., def f(): return 1 Therefore your claim that there is "_never_ a guarantee" is simply false. I was asking: what if you DO want PyPy to assume that your types are correct, why couldn't that help?

bjt 12y ago
In Python you have no guarantee that that function hasn't been monkeypatched over and replaced by some other library you imported.
- andreasvc 12y ago
  From what I read those things are problematic for PyPy anyway, causing it to fallback to an unoptimized interpreter. Also, the type annotations could be interpreted as a promise that what is being annotated will not be monkeypatched at runtime. What is a type annotation, if not a promise that something is not as dynamic as it could otherwise be?
untothebreach 12y ago
Monkeypatching means that `def f(): return 1` could be turned into `def f(): return "i am now a string!"` at runtime, so I stand by my original statement. EDIT: Here is a trivial example of a program that type checks at compile time but doesn't present the same types at runtime. Obviously you would never do this in production code, but the same effect could be accomplished simply by doing an `import somemodule`. http://codepad.org/OarBYXfZ http://codepad.org/OarBYXfZ
- andreasvc 12y ago
  Your replacing what 'f' is bound to; this would include replacing any type annotations, and triggering any system that relies on type annotations to drop/reset its optimizations for 'f'. You're not actually rewriting the original function to return something different; this would probably also be possible, but would be considered as even more of a dirty trick. It is unfortunate that such dynamicity is the default in Python, when in reality it is rarely needed. So it would be advantageous to be able to tell the optimization system: this function is not going to change dynamically (on an opt-in basis).
DasIch 12y ago
You could replace the code object at runtime changing what the function does and what the type of the returned object is.

bsaul 12y ago

But wouldn't annotatin at least help pypy choose a better suited runtime type ?

untothebreach 12y ago
We are out of my depth here, I have no experience or knowledge of the optimizations that Pypy does. So, I am trusting Alex Gaynor's word that it wouldn't help, since if anyone would be an expert on Pypy's optimizations, it would be him.
davesque 12y ago
They wouldn't, because there is no runtime specification for enforcement of types based on type annotations. PyPy still can't assume anything about what type a value will be because a value `v` annotated as `str` could still be changed as follows: v = 1234 Then, PyPy would crash or behave erratically since its assumptions about the value's type are false.
- rwl 12y ago
  > a value `v` annotated as `str` could still be changed as follows: > v = 1234 Surely at least this could be caught by a static type checker? If `v` is annotated to be of type `str`, assigning it a constant int should be a type error, though I'm sure Python provides more indirect ways of setting `v` that would be harder to catch statically (like globals()['v'] = ...). But your comment raises an interesting question: are the annotations MyPy is relying on (static) type annotations, or value annotations? I tend to think of the difference between static and dynamic typing as whether types attach to expressions or to values. If annotations only attach to the value to which a variable is initially bound in some context, I can see how they really wouldn't be much help in this scenario, since `v = 1234` is simply rebinding `v` to a value of a different type, and a previous annotation wouldn't preclude that. But if annotations attach to expressions, then it would be inconsistent to bind `v` to a value like 1234, since the constant "1234" is of a different type than `v` is.
- bsaul 12y ago
  "there is no runtime specification for enforcement of types based on type annotations" Indeed, if you consider type annotation as just annotation and nothing else. But at least you "could" imagine an option to have pypy optimize based on them.