7 ms·

As a pythonista, the only issue I had is with types. A lot of the time, I don't know what to expect from a function, it could be None or a int (legacy code). In

As a pythonista, the only issue I had is with types. A lot of the time, I don't know what to expect from a function, it could be None or a int (legacy code). In a typed language like go, you would not have the ability to do this. So, although powerful, it opens the door for potentially bad decisions. Python's proposal for type hinting will dampen the effect of type inconsistency I feel.

wyldfire 12y ago
I can relate, on some level. Sometimes I feel like writing test cases that increase coverage is not enough. There are days where I think I need a sat solver in order to get confidence that I'll never see incorrect behavior from my code. > type hinting will dampen the effect of type inconsistency I feel I'm not sure that I see type hinting being that advantageous. I kinda hope that it could allow PyPy and other VMs to make assumptions that would otherwise require heuristics. Perhaps type hinting could make static checkers/linters for Python more effective, though.
go1979 12y ago
Agreed on the "lack of declared types" thing. While I'm exceptionally fast writing new Python code, reading other people's Python code is a different story - particularly, if they are new to Python. While there are a lot of concurrency options to not get you in trouble with the GIL, it is annoying that there is no canonical way or pythonic way of doing it. I saw people using things like monkey patching, which made my jaw drop. I had to implement Websockets recently and was totally confused as to the right way of doing it. I ended up evaluating NodeJS and Go for this function, and went with Go. Overall ... I love Python. It is perfect for small projects. For larger codebases (3KLOC+), I am slightly skeptical. It can be done but developers need to have quality code and strong test coverage.
- goostavos 12y ago
  My hobby project is something like 20k lines of Python. As long as you've got good tools (e.g. PyCharm), large codebases in Python are just about as easy to navigate as they would be in something like Java. Our codebase at work is.. phew, I don't even know, a couple hundred thousand lines of Python? Still, easy peasy to dig into. I will 100% agree on the websockets thing. Python is really failing in that area.
  - tarentel 12y ago
    I code in Python from time to time but I am by no means an experts. For a small game server I used http://autobahn.ws http://autobahn.ws. Is there something wrong with this library? It was the easiest websocket library I found for pretty much any language. I plan to use it again for a larger project, more serious project. Is there any reason I should avoid using this and Python in particular for websocket stuff?
  - reinhardt 12y ago
    A couple 100KLOC of Python, excluding 3rd party libraries? Other than Google or maybe NASA, I can't think of many places with so large python codebases.
    - goostavos 12y ago
      Medical industry.
    - gknoy 12y ago
      A 100k Python project doesn't sound that bad, especially with a modular design and a great IDE (PyCharm). Good tools make reading and navigating the codebase easy. At that point, it matters a lot less how big it is, because you can trust that your tools enable you to find the answers you seek. The project that I help maintain is a collection of Django apps totalling more than 700k lines. Thankfully I do not maintain all of it, but I am the primary maintainer for several sub-apps, totalling about 180k lines of Python. (This doesn't count the UI code, which is not in Python.) I'm the sole author for about 12% of the part I maintain (since I wrote some of the individual sub-apps), and am familiar enough to describe probably half of the rest. There are still some parts that I really have to puzzle out ("What happens on the codepath to generate those reports?"), but that part shrinks over time. The modular design that our original engineers used when they first built this system really helps, and the fantastic tooling in PyCharm is critical to my being able to navigate the codebase. Between grepping the python sources (thank god for shell aliases wrapping grep!) and my base familiarity with the system, I can start tracing with PyCharm's "find definition" to investigate things. Being able to run my own copy of the system, and put debugging breakpoints in (even in libraries!) is __amazing__.
- zo1 12y ago
  >"I had to implement Websockets recently and was totally confused as to the right way of doing it. I ended up evaluating NodeJS and Go for this function, and went with Go." You had to implement Websockets in Python? Instead of using an already-functioning, tested library?
  - xorcist 12y ago
    At $work, we call these people "problem solvers".
  - pc2g4d 12y ago
    I'm guessing they meant they implemented a websockets application, not the actual protocol.
    - tim333 12y ago
      I'm guessing that too as I had a similar issue in a hobby web project in Python. I wanted instant messaging between users and the time required to read the docs for twisted and port the whole thing over was off putting. I ended up making a message server with Node and Socket IO that's like 50 lines and sends messages to and from clients and the server. It works but it seems a bit of a kludge.
      - zo1 12y ago
        It's this simple if you had come across Tornado: class WebSocketHandler(tornado.websocket.WebSocketHandler): def on_message(self, message): self.write_message("Echo: %s" % message) I think the problem is that you chose a pretty heavyweight / complicated library.
        tim333 12y ago
        Ta - maybe I'll give that a go. I didn't really try it. Actually I did glance at some docs for the framework I was using, web2py and it suggested some stuff with Tornado but one version only worked on chrome and another needed flash so presumably wouldn't work with iOS so I thought sod it - node/socket IO is easy and works anywhere.
s_kilk 12y ago
I agree. I know we can get picky about the exact characteristics of a languages type-system, but for me they fall into two camps: "Typeful" languages and "(effectively) Typeless" languages. The former have a "good" typesystem, and expose it as a tool the programmer can use to help themselves solve problems. In the latter case, there may or may not be a type system lurking in the language somewhere, but the language either doesn't expose that system as a tool for the programmer to use, or it does so inconsistently. Personally I think python is in the later camp, it has types but it doesn't really give the programmer much help when solving problems. Clojure is another example. The Java type-system is down there somewhere in the murk, but idiomatic Clojure code doesn't use it, and it's not really a tool for the programmer to use, more an accident of the platform.
mox1 12y ago
Yes, in theory this is a problem in lots of languages which are weakly typed. There are lots of ways of handling this, some generic and some language specific. In python I would suggest wrapping your call in a try / expect block or using "isinstance()". If you are using a publicly available library or popular piece of code that can return None or an Integer, I would argue that piece of code was written incorrectly. Newbie python users might do this, but I think experienced python devs would see the problem. Finally, some documentation or justification for the reasoning behind this decision at the top of the function or on a web page somewhere would help as well. As with most powerful , full featured languages it is pretty easy to shoot yourself in the foot if you are not careful, I don't think this is a python specific problem...
- leephillips 12y ago
  Python is strongly typed.
  - g-clef 12y ago
    True. What most people are complaining about is that it's dynamically typed, not weakly typed. That's where most of the confusion sets in.
- chucksmash 12y ago
  I would counter that isinstance() checking is - at least in the general case - an antipattern. Better to check that the object you've been passed implements the interface you need rather than reasoning about its inheritance chain. Duck typing!
  - Roboprog 12y ago
    As much as I hate the Java language, "isinstance" does allow you to check for an interface, not just a class.
  - marcosdumay 12y ago
    Some times one wants classes to be semantic, not only structural. That means that when the developer writes "class C(Foo)" he means something different than writting "class C(Bar)" even if both classes implement the same methods.
deleted 12y ago
[deleted]
lmm 12y ago
Go is not a good example of a type system - its types are limited and restrictive and really will stop you writing some good code that you wanted to write in Python. If you want a Pythonic language with static types, maybe take a look at OCaml (or my personal favourite, Scala - though that language suffers from a bunch of warts for the sake of JVM compatibility). http://roscidus.com/blog/blog/2014/06/06/python-to-ocaml-retrospective/ http://roscidus.com/blog/blog/2014/06/06/python-to-ocaml-ret... is one example of someone making the move.
- Cyther606 12y ago
  I see Go and OCaml recommended to people fed up with Python's lack of static typing, and I just cringe. I love experimenting with programming languages, and Go seems to be picking up momentum but unfortunately for both Go and OCaml, they have this ridiculously restrictive syntax to them respectively which it makes it deeply unattractive for prototyping anything. If you're looking for a statically typed Python that can be grokked in a matter of minutes, and compiles down to a static binary like Go does, and runs as fast as C in benchmarks, you should definitely check out the Nim compiler [1]. Code example: import rdstdin, strutils let time24 = readLineFromStdin("Enter a 24-hour time: ").split(':').map(parseInt) hours24 = time24[0] minutes24 = time24[1] flights: array[8, tuple[since: int, depart: string, arrive: string]] = [(480, "8:00 a.m.", "10:16 a.m."), (583, "9:43 a.m.", "11:52 a.m."), (679, "11:19 a.m.", "1:31 p.m."), (767, "12:47 p.m.", "3:00 p.m."), (840, "2:00 p.m.", "4:08 p.m."), (945, "3:45 p.m.", "5:55 p.m."), (1140, "7:00 p.m.", "9:20 p.m."), (1305, "9:45 p.m.", "11:58 p.m.")] proc minutesSinceMidnight(hours: int = hours24, minutes: int = minutes24): int = hours * 60 + minutes proc cmpFlights(m = minutesSinceMidnight()): seq[int] = result = newSeq[int](flights.len) for i in 0 .. <flights.len: result[i] = abs(m - flights[i].since) proc getClosest(): int = for k,v in cmpFlights(): if v == cmpFlights().min: return k echo "Closest departure time is ", flights[getClosest()].depart, ", arriving at ", flights[getClosest()].arrive Statistics (on an x86_64 Intel Core2Quad Q9300): Lang Time [ms] Memory [KB] Compile Time [ms] Compressed Code [B] Nim 1400 1460 893 486 C++ 1478 2717 774 728 D 1518 2388 1614 669 Rust 1623 2632 6735 934 Java 1874 24428 812 778 OCaml 2384 4496 125 782 Go 3116 1664 596 618 Haskell 3329 5268 3002 1091 LuaJit 3857 2368 - 519 Lisp 8219 15876 1043 1007 Racket 8503 130284 24793 741 This language deserves your attention. [1]: http://goran.krampe.se/2014/10/20/i-missed-nim/ http://goran.krampe.se/2014/10/20/i-missed-nim/ [2]: https://github.com/Araq/Nim/wiki/Nim-for-C-programmers https://github.com/Araq/Nim/wiki/Nim-for-C-programmers
eliben 12y ago
Indeed, this is one of the biggest problems of Python for large-scale projects (saying this as an avid Pythonista myself). This is due to dynamic typing, though, not weak typing (Python's typing is actually strong). Weak typing has its merits but sometimes static typing is very useful for reading and understanding code.
kyllo 12y ago
For me it's not so much the dynamic typing that makes the code hard to reason about--it's the object system, with its inheritance and mutable state. This is why I like dynamic languages that have immutable-by-default data structures, like Clojure.

wyldfire 12y ago

I can relate, on some level. Sometimes I feel like writing test cases that increase coverage is not enough. There are days where I think I need a sat solver in order to get confidence that I'll never see incorrect behavior from my code. > type hinting will dampen the effect of type inconsistency I feel I'm not sure that I see type hinting being that advantageous. I kinda hope that it could allow PyPy and other VMs to make assumptions that would otherwise require heuristics. Perhaps type hinting could make static checkers/linters for Python more effective, though.

go1979 12y ago

Agreed on the "lack of declared types" thing. While I'm exceptionally fast writing new Python code, reading other people's Python code is a different story - particularly, if they are new to Python. While there are a lot of concurrency options to not get you in trouble with the GIL, it is annoying that there is no canonical way or pythonic way of doing it. I saw people using things like monkey patching, which made my jaw drop. I had to implement Websockets recently and was totally confused as to the right way of doing it. I ended up evaluating NodeJS and Go for this function, and went with Go. Overall ... I love Python. It is perfect for small projects. For larger codebases (3KLOC+), I am slightly skeptical. It can be done but developers need to have quality code and strong test coverage.

goostavos 12y ago
My hobby project is something like 20k lines of Python. As long as you've got good tools (e.g. PyCharm), large codebases in Python are just about as easy to navigate as they would be in something like Java. Our codebase at work is.. phew, I don't even know, a couple hundred thousand lines of Python? Still, easy peasy to dig into. I will 100% agree on the websockets thing. Python is really failing in that area.
- tarentel 12y ago
  I code in Python from time to time but I am by no means an experts. For a small game server I used http://autobahn.ws http://autobahn.ws. Is there something wrong with this library? It was the easiest websocket library I found for pretty much any language. I plan to use it again for a larger project, more serious project. Is there any reason I should avoid using this and Python in particular for websocket stuff?
- reinhardt 12y ago
  A couple 100KLOC of Python, excluding 3rd party libraries? Other than Google or maybe NASA, I can't think of many places with so large python codebases.
  - goostavos 12y ago
    Medical industry.
  - gknoy 12y ago
    A 100k Python project doesn't sound that bad, especially with a modular design and a great IDE (PyCharm). Good tools make reading and navigating the codebase easy. At that point, it matters a lot less how big it is, because you can trust that your tools enable you to find the answers you seek. The project that I help maintain is a collection of Django apps totalling more than 700k lines. Thankfully I do not maintain all of it, but I am the primary maintainer for several sub-apps, totalling about 180k lines of Python. (This doesn't count the UI code, which is not in Python.) I'm the sole author for about 12% of the part I maintain (since I wrote some of the individual sub-apps), and am familiar enough to describe probably half of the rest. There are still some parts that I really have to puzzle out ("What happens on the codepath to generate those reports?"), but that part shrinks over time. The modular design that our original engineers used when they first built this system really helps, and the fantastic tooling in PyCharm is critical to my being able to navigate the codebase. Between grepping the python sources (thank god for shell aliases wrapping grep!) and my base familiarity with the system, I can start tracing with PyCharm's "find definition" to investigate things. Being able to run my own copy of the system, and put debugging breakpoints in (even in libraries!) is __amazing__.
zo1 12y ago
>"I had to implement Websockets recently and was totally confused as to the right way of doing it. I ended up evaluating NodeJS and Go for this function, and went with Go." You had to implement Websockets in Python? Instead of using an already-functioning, tested library?
- xorcist 12y ago
  At $work, we call these people "problem solvers".
- pc2g4d 12y ago
  I'm guessing they meant they implemented a websockets application, not the actual protocol.
  - tim333 12y ago
    I'm guessing that too as I had a similar issue in a hobby web project in Python. I wanted instant messaging between users and the time required to read the docs for twisted and port the whole thing over was off putting. I ended up making a message server with Node and Socket IO that's like 50 lines and sends messages to and from clients and the server. It works but it seems a bit of a kludge.
    - zo1 12y ago
      It's this simple if you had come across Tornado: class WebSocketHandler(tornado.websocket.WebSocketHandler): def on_message(self, message): self.write_message("Echo: %s" % message) I think the problem is that you chose a pretty heavyweight / complicated library.
      - tim333 12y ago
        Ta - maybe I'll give that a go. I didn't really try it. Actually I did glance at some docs for the framework I was using, web2py and it suggested some stuff with Tornado but one version only worked on chrome and another needed flash so presumably wouldn't work with iOS so I thought sod it - node/socket IO is easy and works anywhere.

s_kilk 12y ago

I agree. I know we can get picky about the exact characteristics of a languages type-system, but for me they fall into two camps: "Typeful" languages and "(effectively) Typeless" languages. The former have a "good" typesystem, and expose it as a tool the programmer can use to help themselves solve problems. In the latter case, there may or may not be a type system lurking in the language somewhere, but the language either doesn't expose that system as a tool for the programmer to use, or it does so inconsistently. Personally I think python is in the later camp, it has types but it doesn't really give the programmer much help when solving problems. Clojure is another example. The Java type-system is down there somewhere in the murk, but idiomatic Clojure code doesn't use it, and it's not really a tool for the programmer to use, more an accident of the platform.

mox1 12y ago

Yes, in theory this is a problem in lots of languages which are weakly typed. There are lots of ways of handling this, some generic and some language specific. In python I would suggest wrapping your call in a try / expect block or using "isinstance()". If you are using a publicly available library or popular piece of code that can return None or an Integer, I would argue that piece of code was written incorrectly. Newbie python users might do this, but I think experienced python devs would see the problem. Finally, some documentation or justification for the reasoning behind this decision at the top of the function or on a web page somewhere would help as well. As with most powerful , full featured languages it is pretty easy to shoot yourself in the foot if you are not careful, I don't think this is a python specific problem...

leephillips 12y ago
Python is strongly typed.
- g-clef 12y ago
  True. What most people are complaining about is that it's dynamically typed, not weakly typed. That's where most of the confusion sets in.
chucksmash 12y ago
I would counter that isinstance() checking is - at least in the general case - an antipattern. Better to check that the object you've been passed implements the interface you need rather than reasoning about its inheritance chain. Duck typing!
- Roboprog 12y ago
  As much as I hate the Java language, "isinstance" does allow you to check for an interface, not just a class.
- marcosdumay 12y ago
  Some times one wants classes to be semantic, not only structural. That means that when the developer writes "class C(Foo)" he means something different than writting "class C(Bar)" even if both classes implement the same methods.

deleted 12y ago

[deleted]

lmm 12y ago

Go is not a good example of a type system - its types are limited and restrictive and really will stop you writing some good code that you wanted to write in Python. If you want a Pythonic language with static types, maybe take a look at OCaml (or my personal favourite, Scala - though that language suffers from a bunch of warts for the sake of JVM compatibility). http://roscidus.com/blog/blog/2014/06/06/python-to-ocaml-retrospective/ http://roscidus.com/blog/blog/2014/06/06/python-to-ocaml-ret... is one example of someone making the move.

Cyther606 12y ago
I see Go and OCaml recommended to people fed up with Python's lack of static typing, and I just cringe. I love experimenting with programming languages, and Go seems to be picking up momentum but unfortunately for both Go and OCaml, they have this ridiculously restrictive syntax to them respectively which it makes it deeply unattractive for prototyping anything. If you're looking for a statically typed Python that can be grokked in a matter of minutes, and compiles down to a static binary like Go does, and runs as fast as C in benchmarks, you should definitely check out the Nim compiler [1]. Code example: import rdstdin, strutils let time24 = readLineFromStdin("Enter a 24-hour time: ").split(':').map(parseInt) hours24 = time24[0] minutes24 = time24[1] flights: array[8, tuple[since: int, depart: string, arrive: string]] = [(480, "8:00 a.m.", "10:16 a.m."), (583, "9:43 a.m.", "11:52 a.m."), (679, "11:19 a.m.", "1:31 p.m."), (767, "12:47 p.m.", "3:00 p.m."), (840, "2:00 p.m.", "4:08 p.m."), (945, "3:45 p.m.", "5:55 p.m."), (1140, "7:00 p.m.", "9:20 p.m."), (1305, "9:45 p.m.", "11:58 p.m.")] proc minutesSinceMidnight(hours: int = hours24, minutes: int = minutes24): int = hours * 60 + minutes proc cmpFlights(m = minutesSinceMidnight()): seq[int] = result = newSeq[int](flights.len) for i in 0 .. <flights.len: result[i] = abs(m - flights[i].since) proc getClosest(): int = for k,v in cmpFlights(): if v == cmpFlights().min: return k echo "Closest departure time is ", flights[getClosest()].depart, ", arriving at ", flights[getClosest()].arrive Statistics (on an x86_64 Intel Core2Quad Q9300): Lang Time [ms] Memory [KB] Compile Time [ms] Compressed Code [B] Nim 1400 1460 893 486 C++ 1478 2717 774 728 D 1518 2388 1614 669 Rust 1623 2632 6735 934 Java 1874 24428 812 778 OCaml 2384 4496 125 782 Go 3116 1664 596 618 Haskell 3329 5268 3002 1091 LuaJit 3857 2368 - 519 Lisp 8219 15876 1043 1007 Racket 8503 130284 24793 741 This language deserves your attention. [1]: http://goran.krampe.se/2014/10/20/i-missed-nim/ http://goran.krampe.se/2014/10/20/i-missed-nim/ [2]: https://github.com/Araq/Nim/wiki/Nim-for-C-programmers https://github.com/Araq/Nim/wiki/Nim-for-C-programmers

eliben 12y ago

Indeed, this is one of the biggest problems of Python for large-scale projects (saying this as an avid Pythonista myself). This is due to dynamic typing, though, not weak typing (Python's typing is actually strong). Weak typing has its merits but sometimes static typing is very useful for reading and understanding code.

kyllo 12y ago

For me it's not so much the dynamic typing that makes the code hard to reason about--it's the object system, with its inheritance and mutable state. This is why I like dynamic languages that have immutable-by-default data structures, like Clojure.