6 ms·

I'm really glad to see some of the Python committers taking a serious look at the GIL. Python is either posed for great victory (given its rapid rate of adopti

by jnbiche 12y ago

I'm really glad to see some of the Python committers taking a serious look at the GIL. Python is either posed for great victory (given its rapid rate of adoption in academia) or slow failure (given the rapid rate at which server apps are starting to migrate from Python to Go). However, between accomplishments like Micropython (huge potential for Python on mobile/resource constrained devices) , PyPy's slow but steady gains, and projects like this, it's at least an interesting time for Pythonistas. Now, if we could only get an optional static type checker... (heresy, I know). Dynamic typing is great for quick prototyping, and I would never want to lose that in Python, but I'm very uneasy now taking on any large projects or long-term projects without static typing. Mypy holds some promise here, but I think it will take sponsorship from a big company to push something like this to a mature state.

unethical_ban 12y ago
I wrote a small XML parsing library for a firewall API recently. Just writing about 15 functions handing data off to each other made me nervous about not enforcing types. The best I could do was write function signatures and descriptions.
- calpaterson 12y ago
  Types are not the only way to ensure correctness (and they aren't even the only thing you should use, even when they're available) - did you consider using tests?
  - claudius 12y ago
    Tests are not the only way to ensure correctness (and they aren’t even the only thing you should use, even when they’re mandated by policy) – did you consider using strong types encoding exactly the assumptions you spot-check with tests?
    - klibertp 12y ago
      > did you consider using strong types encoding exactly the assumptions you spot-check with tests? Good luck encoding this simple contract (in Racket) exactly in your type system: (define/contract (fun numbers) (-> (listof (and/c (integer-in 0 255) even?)) any) ;; do something with a list of even integers in a given range null) I hope you like your Zero and Succ n and are proficient with Agda or ATS or something like this.
      - Shish2k 12y ago
        I have no idea what you just said. Given that we're talking about type checking in python, care to give an example of a weakness of type checking in python?
        klibertp 12y ago
        We're even, because I don't know about what "weakness" you're talking and why the heck you're bringing Python into abstract discussion on static typing. Basically, my parent said that static typing can express many things that testing (which is not static, because it runs the code) can. This is true, but there are things you can't express in any of the commonly used[1] static type systems and I just gave an example of such thing. Other than this there's nothing wrong with static typing in general and I'd be happy if Python got some kind of (optional) static type system, maybe in the vein of Typed Clojure and Typed Racket or Erlang's Dialyzer. Still, static type systems are not silver bullets and have their drawbacks - and that's all I wanted to convey in my comment. Edit: [1] the type system feature which would let you encode the example contract I posted is called "dependent typing" and is available only in a very few languages.
      - marcosdumay 12y ago
        Pycontracts does that. Racket is more powerful, but that is a bad example.
        klibertp 12y ago
  - coldtea 12y ago
    Yes, hate them. Gimme types.
    - pyre 12y ago
      You hate tests, therefore want static-typing. You do realize that static-typing doesn't mean error-free code, right? Even with static-typing, it's probably a good idea to have tests.
      - coldtea 12y ago
        >You hate tests, therefore want static-typing. You do realize that static-typing doesn't mean error-free code, right? As much as you realize that having tests also doesn't mean error-free code. >Even with static-typing, it's probably a good idea to have tests. For much less stuff. With a proper compiler, half of the kind of tests people do in Ruby land are totally needless. You can refactor and you know from the first recompile what has broken and where.
Glide 12y ago
That's not really heresy considering the existence of Typescript. You're probably right about it needing sponsorship from a big company. Adding type checking to a language is difficult enough, but the tooling around it is what really makes it powerful.
- jnbiche 12y ago
  Yeah, Typescript is great. But by heresy I meant that's the concept is pretty rejected among many (most?) core Python developers, not that it's not feasible. And I agree that the tooling is part of what makes static types so powerful, but I actually think the tooling might emerge organically if there's a well-designed, effective, and open-source static type checker available.
ak217 12y ago
There are a lot of ideas out there on how to do optional runtime type checking in a Pythonic way. My contribution is ensure (https://github.com/kislyuk/ensure https://github.com/kislyuk/ensure), for example check out the ensure_annotations decorator (https://ensure.readthedocs.org/en/latest/#ensure.ensure_annotations https://ensure.readthedocs.org/en/latest/#ensure.ensure_anno...).
- RodericDay 12y ago
  I kinda wish I could keep all the type definitions in a separate file, myself. Like a map of the project. That would probably make it easier to decouple from "Normal" python code too. I'm not a very senior software dev or anything, but I imagine myself liking having my well-written, compact .py file detailing logic, and a separate 'doc'(?) file containing long-winded explanations of what some key functions do, and listing all the type consistencies I absolutely wish to guarantee.
  - ak217 12y ago
    Have you used Sphinx for documenting your project? (You can also use RtD - for example, https://ensure.readthedocs.org/ https://ensure.readthedocs.org/ has the docs for the project linked above). Sphinx makes it fairly straightforward to inline documentation and code while maintaining highly readable docs. Also, you can collapse and otherwise manage doctrings in IDEs like PyCharm. So, if I were to implement what you said, I'd go with a file containing interface classes, i.e. classes that have getters/setters and type checking built in and documented, and another file defining logic operating on them. Another pattern is to use something like: class Widget(object): ... define key constraints... """ Long-winded docstring about do_stuff """ from .impl import do_stuff So all the implementation complexity of do_stuff is hidden in .impl (which also avoids stray exports).
  - jnbiche 12y ago
    >I kinda wish I could keep all the type definitions in a separate file, myself. Like a map of the project. In fact, this is exactly what OCaml does (or can do)[0]. The interface file is not always required, but people usually include it for clarity, to make sure the compiler's type inference works according to their wishes, or to limit visibility of certain types and functions. 0. http://stackoverflow.com/a/3268836 http://stackoverflow.com/a/3268836
masklinn 12y ago
> I'm really glad to see some of the Python committers taking a serious look at the GIL. People have been taking serious looks at the GIL for a long time (there were technically working GIL-less versions of 1.5). The issue has never been that no one wanted to work on it (let alone that it was impossible), but that so far the single-threaded performance hit has been too large to be acceptable to the core team. > Now, if we could only get an optional static type checker... (heresy, I know). Er… how would that be heretic when Python 3's annotations were introduced to support exactly this?
- jnbiche 12y ago
  >Er… how would that be heretic when Python 3's annotations were introduced to support exactly this? I know well about Python 3 annotations, since that's what gave me hope to begin with. But I had the impression that every serious proposal I've seen on the Python Dev mailing list to integrate a static type checker in CPython had been met with rejection. However, I just checked the archives and see now I was wrong. There have been multiple discussions about this, and even a working group. GvR has given static types in Python the cold shoulder, but that attitude is clearly not shared by everyone in the community. Glad to have been wrong in this case.
  - maximilian 12y ago
    There is also the mypy project (http://mypy-lang.org/ http://mypy-lang.org/). They are working on a type-checked variant of python. Currently it's just a pre-processing stage, but they eventually want to make it possible to AOT compile using the type information to get a faster language.
    - freyrs3 12y ago
      It was originally part of the authors PhD work, it hasn't been touched since he graduated.
urza 12y ago
If you like static typing and dynamic features, take a look at C# .. it has features inspired from dynamic and functional languages. Plus excellent support for parallelism and concurrency. Gone are times when C# was only object oriented java alternative.
hcarvalhoalves 12y ago
> (given the rapid rate at which server apps are starting to migrate from Python to Go). Do you have data for this claim or is it a hunch? I've seen this meme repeated a lot in HN.
- zo1 12y ago
  It's a case of "keep repeating it and maybe it'll be true, someday, err, I hope". It's something you often see when people make mental investments into certain paths/things/ideas. And then they have to keep repeating it, both in order to try make it true, and to shield themselves from reality. Sort of like node.js
  - jnbiche 12y ago
    Wow, that's so full of wrong assertions and ad hominem I don't even know where to start. First of all, I don't even particularly like Golang as a language, so you're incredibly wrong about my motivations. Secondly, if you don't think Python to Golang adoption is on the rise among SV start-ups, you're living in a bubble. Look up HN Golang posts and count how many of them are from Python backgrounds. This may not represent the entire global Python programming community, but it's without doubt an important and influential segment of it. Finally, I clearly struck some kind of nerve, since my comment is at the top of the page. You may not like my opinion, but it's evidently a common one.
- jnbiche 12y ago
  Thank you for not asking in a rude and confrontational manner. No, I don't have any solid data, but I do think the majority of people posting about Go on HN have either Python or Ruby backgrounds. I 've also found a lot of Python people in the Rust community (which I personally prefer vastly to Go). People need more performance, particularly multicore performance. Traditional Python supporters can put their head in the sand about this if they want, but these highly-performant new languages have clearly found a niche among Python folks tired of trying to optimize all the time. And yes, there's a fine line between conducting needed optimizations and wasting time prematurely optimizing, but people would clearly rather spend a little more time up front in exchange for a big speedup. The choice is no longer between C and Python -- there's a nice middle ground.
meowface 12y ago
Personally I don't see why someone would switch from Python to Go. Go feels like a major step down from Python, in everything except for speed and parallelism.
- runeks 12y ago
  Well then the reason for someone switching would probably be to gain speed and parallelism. :) I also think static typing is a big reason, for large projects at least.
Demiurge 12y ago
>I'm really glad to see some of the Python committers taking a serious look at the GIL. Python is either posed for great victory (given its rapid rate of adoption in academia) or slow failure (given the rapid rate at which server apps are starting to migrate from Python to Go) Or neither, given that it's okey to have specialized tools for different jobs. Speed/networking was never supposed to be Pythons moto.

by jnbiche 12y ago

unethical_ban 12y ago

I wrote a small XML parsing library for a firewall API recently. Just writing about 15 functions handing data off to each other made me nervous about not enforcing types. The best I could do was write function signatures and descriptions.

calpaterson 12y ago
Types are not the only way to ensure correctness (and they aren't even the only thing you should use, even when they're available) - did you consider using tests?
- claudius 12y ago
  Tests are not the only way to ensure correctness (and they aren’t even the only thing you should use, even when they’re mandated by policy) – did you consider using strong types encoding exactly the assumptions you spot-check with tests?
  - klibertp 12y ago
    > did you consider using strong types encoding exactly the assumptions you spot-check with tests? Good luck encoding this simple contract (in Racket) exactly in your type system: (define/contract (fun numbers) (-> (listof (and/c (integer-in 0 255) even?)) any) ;; do something with a list of even integers in a given range null) I hope you like your Zero and Succ n and are proficient with Agda or ATS or something like this.
    - Shish2k 12y ago
      I have no idea what you just said. Given that we're talking about type checking in python, care to give an example of a weakness of type checking in python?
      - klibertp 12y ago
        We're even, because I don't know about what "weakness" you're talking and why the heck you're bringing Python into abstract discussion on static typing. Basically, my parent said that static typing can express many things that testing (which is not static, because it runs the code) can. This is true, but there are things you can't express in any of the commonly used[1] static type systems and I just gave an example of such thing. Other than this there's nothing wrong with static typing in general and I'd be happy if Python got some kind of (optional) static type system, maybe in the vein of Typed Clojure and Typed Racket or Erlang's Dialyzer. Still, static type systems are not silver bullets and have their drawbacks - and that's all I wanted to convey in my comment. Edit: [1] the type system feature which would let you encode the example contract I posted is called "dependent typing" and is available only in a very few languages.
    - marcosdumay 12y ago
      Pycontracts does that. Racket is more powerful, but that is a bad example.
      - klibertp 12y ago
- coldtea 12y ago
  Yes, hate them. Gimme types.
  - pyre 12y ago
    You hate tests, therefore want static-typing. You do realize that static-typing doesn't mean error-free code, right? Even with static-typing, it's probably a good idea to have tests.
    - coldtea 12y ago
      >You hate tests, therefore want static-typing. You do realize that static-typing doesn't mean error-free code, right? As much as you realize that having tests also doesn't mean error-free code. >Even with static-typing, it's probably a good idea to have tests. For much less stuff. With a proper compiler, half of the kind of tests people do in Ruby land are totally needless. You can refactor and you know from the first recompile what has broken and where.

Glide 12y ago

That's not really heresy considering the existence of Typescript. You're probably right about it needing sponsorship from a big company. Adding type checking to a language is difficult enough, but the tooling around it is what really makes it powerful.

jnbiche 12y ago
Yeah, Typescript is great. But by heresy I meant that's the concept is pretty rejected among many (most?) core Python developers, not that it's not feasible. And I agree that the tooling is part of what makes static types so powerful, but I actually think the tooling might emerge organically if there's a well-designed, effective, and open-source static type checker available.

ak217 12y ago

There are a lot of ideas out there on how to do optional runtime type checking in a Pythonic way. My contribution is ensure (https://github.com/kislyuk/ensure https://github.com/kislyuk/ensure), for example check out the ensure_annotations decorator (https://ensure.readthedocs.org/en/latest/#ensure.ensure_annotations https://ensure.readthedocs.org/en/latest/#ensure.ensure_anno...).

RodericDay 12y ago
I kinda wish I could keep all the type definitions in a separate file, myself. Like a map of the project. That would probably make it easier to decouple from "Normal" python code too. I'm not a very senior software dev or anything, but I imagine myself liking having my well-written, compact .py file detailing logic, and a separate 'doc'(?) file containing long-winded explanations of what some key functions do, and listing all the type consistencies I absolutely wish to guarantee.
- ak217 12y ago
  Have you used Sphinx for documenting your project? (You can also use RtD - for example, https://ensure.readthedocs.org/ https://ensure.readthedocs.org/ has the docs for the project linked above). Sphinx makes it fairly straightforward to inline documentation and code while maintaining highly readable docs. Also, you can collapse and otherwise manage doctrings in IDEs like PyCharm. So, if I were to implement what you said, I'd go with a file containing interface classes, i.e. classes that have getters/setters and type checking built in and documented, and another file defining logic operating on them. Another pattern is to use something like: class Widget(object): ... define key constraints... """ Long-winded docstring about do_stuff """ from .impl import do_stuff So all the implementation complexity of do_stuff is hidden in .impl (which also avoids stray exports).
- jnbiche 12y ago
  >I kinda wish I could keep all the type definitions in a separate file, myself. Like a map of the project. In fact, this is exactly what OCaml does (or can do)[0]. The interface file is not always required, but people usually include it for clarity, to make sure the compiler's type inference works according to their wishes, or to limit visibility of certain types and functions. 0. http://stackoverflow.com/a/3268836 http://stackoverflow.com/a/3268836

masklinn 12y ago

> I'm really glad to see some of the Python committers taking a serious look at the GIL. People have been taking serious looks at the GIL for a long time (there were technically working GIL-less versions of 1.5). The issue has never been that no one wanted to work on it (let alone that it was impossible), but that so far the single-threaded performance hit has been too large to be acceptable to the core team. > Now, if we could only get an optional static type checker... (heresy, I know). Er… how would that be heretic when Python 3's annotations were introduced to support exactly this?

jnbiche 12y ago
>Er… how would that be heretic when Python 3's annotations were introduced to support exactly this? I know well about Python 3 annotations, since that's what gave me hope to begin with. But I had the impression that every serious proposal I've seen on the Python Dev mailing list to integrate a static type checker in CPython had been met with rejection. However, I just checked the archives and see now I was wrong. There have been multiple discussions about this, and even a working group. GvR has given static types in Python the cold shoulder, but that attitude is clearly not shared by everyone in the community. Glad to have been wrong in this case.
- maximilian 12y ago
  There is also the mypy project (http://mypy-lang.org/ http://mypy-lang.org/). They are working on a type-checked variant of python. Currently it's just a pre-processing stage, but they eventually want to make it possible to AOT compile using the type information to get a faster language.
  - freyrs3 12y ago
    It was originally part of the authors PhD work, it hasn't been touched since he graduated.

urza 12y ago

If you like static typing and dynamic features, take a look at C# .. it has features inspired from dynamic and functional languages. Plus excellent support for parallelism and concurrency. Gone are times when C# was only object oriented java alternative.

hcarvalhoalves 12y ago

> (given the rapid rate at which server apps are starting to migrate from Python to Go). Do you have data for this claim or is it a hunch? I've seen this meme repeated a lot in HN.

zo1 12y ago
It's a case of "keep repeating it and maybe it'll be true, someday, err, I hope". It's something you often see when people make mental investments into certain paths/things/ideas. And then they have to keep repeating it, both in order to try make it true, and to shield themselves from reality. Sort of like node.js
- jnbiche 12y ago
  Wow, that's so full of wrong assertions and ad hominem I don't even know where to start. First of all, I don't even particularly like Golang as a language, so you're incredibly wrong about my motivations. Secondly, if you don't think Python to Golang adoption is on the rise among SV start-ups, you're living in a bubble. Look up HN Golang posts and count how many of them are from Python backgrounds. This may not represent the entire global Python programming community, but it's without doubt an important and influential segment of it. Finally, I clearly struck some kind of nerve, since my comment is at the top of the page. You may not like my opinion, but it's evidently a common one.
jnbiche 12y ago
Thank you for not asking in a rude and confrontational manner. No, I don't have any solid data, but I do think the majority of people posting about Go on HN have either Python or Ruby backgrounds. I 've also found a lot of Python people in the Rust community (which I personally prefer vastly to Go). People need more performance, particularly multicore performance. Traditional Python supporters can put their head in the sand about this if they want, but these highly-performant new languages have clearly found a niche among Python folks tired of trying to optimize all the time. And yes, there's a fine line between conducting needed optimizations and wasting time prematurely optimizing, but people would clearly rather spend a little more time up front in exchange for a big speedup. The choice is no longer between C and Python -- there's a nice middle ground.

meowface 12y ago

Personally I don't see why someone would switch from Python to Go. Go feels like a major step down from Python, in everything except for speed and parallelism.

runeks 12y ago
Well then the reason for someone switching would probably be to gain speed and parallelism. :) I also think static typing is a big reason, for large projects at least.

Demiurge 12y ago

>I'm really glad to see some of the Python committers taking a serious look at the GIL. Python is either posed for great victory (given its rapid rate of adoption in academia) or slow failure (given the rapid rate at which server apps are starting to migrate from Python to Go) Or neither, given that it's okey to have specialized tools for different jobs. Speed/networking was never supposed to be Pythons moto.