5 ms·

I've started to wonder what the place of Python is at all. Machine Learning has become deeply coupled with Python, so you have that side of things, but that's n

I've started to wonder what the place of Python is at all. Machine Learning has become deeply coupled with Python, so you have that side of things, but that's not my personal area of interest. People say that it's good for "short scripts", but whenever I decide to write something in Python instead of TS, I'm instantly met with so many runtime type errors that I wonder how people can honestly believe lack of static typing increases productivity in these "short scripts". For instance, the second I decide to refactor, I know that even once I think I've cleaned everything up, the next few minutes will be spent running the code a few times to flush out all the type errors. Python is not my primary language, so that could play a part in the issues I have, but if I am allowed to lose humility for a second or two: even though I primarily use TS, my Python is still, in my opinion, stronger than many of my (college student) peers. I hate to imagine all the issues a novice would face. To be fair, I haven't used mypy in a while. I remember that being decent, but nowhere near the level of TS, in terms of both the power of the type system and the editor support.

basil-rash 8y ago
In further reflection, it is perhaps only because I am so used to TS that I try to make as intense of refactorings as I do. A programmer who does not have prior experience working with as well tooled of a language would not be attempting the kinds of transformations I do, and thus may not experience as many issues.
- jnwatson 8y ago
  You need `import typing`, and an editor with integrated mypy (e.g. vim). It is almost as good as static typing.
  - basil-rash 8y ago
    Initial thoughts from looking through https://docs.python.org/3/library/typing.html# https://docs.python.org/3/library/typing.html#: 1. Love that it doesn't use structural typing, NewType seems great. 2. The syntax is bad. Maybe this is a "it just takes getting used to" thing, but I actually find it really bad. In TS, the syntax for typing almost always directly matches the syntax for the rest of the language. In Python, its a weird sort of LISPy DSL think that they made... compare: Py: Callable[[List[Tuple[int, string]], Dict[string, string]], int] TS: ([number, string][], { [key: string]: string }) => int This only gets worse as you chain callables together, whereas in TS everything left-associates as you'd expect and it all works out nicely. 3. Admittedly, the TS dict syntax isn't beautiful, but it becomes very helpful for things like `{ [K in keyof T]: K extends number ? T[K] : never }`, which does not seem to be possible in Py. 4. Related to 3, but no never type? I see NoReturn, but it does not seem to actually cause any errors when you try to assign it to a variable. See below. 5. Type narrowing... does it exist? Seemingly not, see below. 6. Literals as types (enums)? 7. Generics, do I really need to pass in the internal representation of the type I'd like to use? That seems absurd. Will bad things happen if these internal identifiers collide? (Reference: `T = Generic('T')` creates a generic type) Demo code that should throw an error at the assertUnreachable and nowhere else, but actually throws errors everywhere but the assert unreachable (types seemingly aren't narrowed by `type() == ...` checks): def foo(x: Union[str, int, float]): if (type(x) == int): return x / 3 if (type(x) == str): return x.upper() return assertUnreachable() def assertUnreachable() -> NoReturn: raise RuntimeError('no way') (this is all checked using mypy, v. 0.641)
    - a1369209993 8y ago
      Regarding #2, the python code has the arguable advantage that (with appropriate variables in scope) it's valid standard python code[0] that could evaluate to (a representation of) the desired type, whereas the typescript isn't valid javascript code, and I don't think it's a valid (value-level) typescript expression either. 3 and 4 are disappointing though. 0: I'm assuming the extra [ after Callable is a typo.
      - basil-rash 8y ago
        Is there a utility to having your types and your values in the same namespace? The extra [ is not a typo, the syntax is: `Callable[[Arg1Type, Arg2Type], ReturnType]`. Or, if the arguments don't matter, `Callable[..., ReturnType]`, but this does not mean that the type is not a valid expression. Edit: I was missing a ] actually, separating the arguments from the return value.
        a1369209993 8y ago
        > Is there a utility to having your types and your values in the same namespace? Well somewhat[0], but the main putative benefit would be having types be valid expressions in the target language, since you can do type-checking with function decorators, and generally interact with types useing the normal language mechanism for interacting with values, rather than some horrid bolted-on piece of crap like C++ templates. Server = Tuple[Address,ConnectionOptions] # array subscipt might not the best choice here, but it works UserId = NewType('UserId',int) # ordinary function call Scalar = int | str # could be equivalent to Union[int,str] with appropriate value of Type.__or__ 0: for example: def intBit(N): if N==0: return type(None) if N==1: return Bool if N>INT_WIDTH: return long return int
        basil-rash 8y ago
        mypy's response to that code: error:invalid type comment or annotation note:Suggestion: use intBit[...] instead of intBit(...) So what's really happening is the type expressions are pretending to be "just everyday python", but actually they have arbitrary restrictions (cannot be functions? need to work via overriding `__getitem__`?) that neither you nor I were aware of. This is probably the least "pythonic" implementation possible. And even if the code were valid, it's relying on N being a statically known value, which is a bit off because sure, you could have N be some global const config variable, but it would be very weird for configuring the value of the variable to require you to also go into the code and change things around to work with bool's or None's instead of int's.
- talltimtom 8y ago
  It sounds like you should invest some time in learning to program python in a pythonic way. If you are trying to build stuff using tons of design patterns factory patterns, you are doing it wrong. Most of those where made to work around limitations of type-systems which aren’t there in python, and naturally you end up having issues solving things like that when you don’t have typing. (Which you do still have with mypy, but as you say you’re not using that). At the end of the day you are using python and there must be a better way! though just like anything else you need to dedicate time to lean not just the syntax but also how to effectively use the language without needlessly over complicating things.
  - basil-rash 8y ago
    I'm not trying to build using tons of factory patterns or the like, quite the opposite actually. I try to code as close to the bare language as possible, and work under the assumption that if I at some point would like to pivot my reasoning or approach to a program, the language and the tooling available will help me do so. This is simply not true without types. I'm not alone in thinking that, the creator of Python himself has devoted his recent work to adding types to Python. Is that just because he doesn't write his code "the correct way"? Does he simply know the syntax but not hoe to effectively use the language? I don't think so.
deleted 8y ago
[deleted]
diminoten 8y ago
Coming at it from the other side (Python has put food on my table for ~7 years now), it's not the "short scripts" that make Python great to develop in, it's that it can handle the growth of your app. Writing scientific/academic code and writing business application code is somewhat different, but the speed at which you can develop and maintain Python code is very high, as is the volume of aid available should you run into trouble. I don't think there's a language out there that's both easy to initially pick up and as well supported by its community as Python. The speed at which you can develop application code is astounding, once you get the hang of it and the noise out of the way (CI/CD, code coverage, unit and integration testing, a general familiarity with the major frameworks, etc.), and the maintenance is... tolerable.
darpa_escapee 8y ago
Python has an excellent ecosystem for those in the sciences, finance or those working with data. A few popular and mature web frameworks are written in Python. It has a place next to Bash et al when it comes to system scripting and acting as glue between different streams, utilities and resources. Static analysis tooling is excellent and type annotations only aid their accuracy. It's my impression that developers tend to learn to love static typing if they didn't cut their teeth on a statically typed platform. It takes experience to know that certain errors can be mitigated via language features and to know how annoying those errors are when they needlessly pop up. Unenforced explicit typing was and is still a boon for Python. Typing is often lost on developers who are still in "Hello, world!" territory and scientists who are using it as an adjunct to MATLAB. It can be a hindrance to rapid development, as well.
paultopia 8y ago
I wonder if you have a different definition of "short scripts" than many others. It seems to me that a genuinely short script wouldn't require big refactors, or types---that would be overkill for a short script. Personally, I use Python as a Bash replacement a lot, that's what I think of when I hear "short scripts." Sort of what people used to use Perl for. Pushing strings around, complicated repetitive filesystem manipulations. (And data sciencey stuff of course.)
- basil-rash 8y ago
  I use "short scripts" to mean: I have an idea I want to play around with, usually involving data analysis or some data structure or algorithm sketch, and I'd like to devote the next 2-3 hours to writing a 100-200 line program that evolves with me as I refine what exactly it is that I would like to be examining.
- kungtotte 8y ago
  I agree with you. Automate some task? Scrape some web content? Glue some programs together by translating the output of one to fit the input of the other? Python it is! If it doesn't already have something in the standard library for your task, it's definitely in the PyPI. This reduces your work to importing a library and writing 5-20 lines of code.
- ryl00 8y ago
  > Sort of what people used to use Perl for. Used to? People still do this (I should know; I'm one of them).
  - paultopia 8y ago
    Heh, fair enough! I think I still have an open-ended bet with a friend about using Perl 6 for something...
  - int0x80 8y ago
    And bash/zsh. Bash can be really powerful. Slow for some tasks though (much slower than perl for example), but little times I've had that problem.
keredson 8y ago
it's the quality of the libraries above all else that keeps me coming back to python. truly world class.
- deusofnull 8y ago
  Shout out to Django!
- stevenjohns 8y ago
  Don't forget the community and the support for those libraries. It's extremely welcoming and newbie-friendly.

basil-rash 8y ago

In further reflection, it is perhaps only because I am so used to TS that I try to make as intense of refactorings as I do. A programmer who does not have prior experience working with as well tooled of a language would not be attempting the kinds of transformations I do, and thus may not experience as many issues.

jnwatson 8y ago
You need `import typing`, and an editor with integrated mypy (e.g. vim). It is almost as good as static typing.
- basil-rash 8y ago
  Initial thoughts from looking through https://docs.python.org/3/library/typing.html# https://docs.python.org/3/library/typing.html#: 1. Love that it doesn't use structural typing, NewType seems great. 2. The syntax is bad. Maybe this is a "it just takes getting used to" thing, but I actually find it really bad. In TS, the syntax for typing almost always directly matches the syntax for the rest of the language. In Python, its a weird sort of LISPy DSL think that they made... compare: Py: Callable[[List[Tuple[int, string]], Dict[string, string]], int] TS: ([number, string][], { [key: string]: string }) => int This only gets worse as you chain callables together, whereas in TS everything left-associates as you'd expect and it all works out nicely. 3. Admittedly, the TS dict syntax isn't beautiful, but it becomes very helpful for things like `{ [K in keyof T]: K extends number ? T[K] : never }`, which does not seem to be possible in Py. 4. Related to 3, but no never type? I see NoReturn, but it does not seem to actually cause any errors when you try to assign it to a variable. See below. 5. Type narrowing... does it exist? Seemingly not, see below. 6. Literals as types (enums)? 7. Generics, do I really need to pass in the internal representation of the type I'd like to use? That seems absurd. Will bad things happen if these internal identifiers collide? (Reference: `T = Generic('T')` creates a generic type) Demo code that should throw an error at the assertUnreachable and nowhere else, but actually throws errors everywhere but the assert unreachable (types seemingly aren't narrowed by `type() == ...` checks): def foo(x: Union[str, int, float]): if (type(x) == int): return x / 3 if (type(x) == str): return x.upper() return assertUnreachable() def assertUnreachable() -> NoReturn: raise RuntimeError('no way') (this is all checked using mypy, v. 0.641)
  - a1369209993 8y ago
    Regarding #2, the python code has the arguable advantage that (with appropriate variables in scope) it's valid standard python code[0] that could evaluate to (a representation of) the desired type, whereas the typescript isn't valid javascript code, and I don't think it's a valid (value-level) typescript expression either. 3 and 4 are disappointing though. 0: I'm assuming the extra [ after Callable is a typo.
    - basil-rash 8y ago
      Is there a utility to having your types and your values in the same namespace? The extra [ is not a typo, the syntax is: `Callable[[Arg1Type, Arg2Type], ReturnType]`. Or, if the arguments don't matter, `Callable[..., ReturnType]`, but this does not mean that the type is not a valid expression. Edit: I was missing a ] actually, separating the arguments from the return value.
      - a1369209993 8y ago
        > Is there a utility to having your types and your values in the same namespace? Well somewhat[0], but the main putative benefit would be having types be valid expressions in the target language, since you can do type-checking with function decorators, and generally interact with types useing the normal language mechanism for interacting with values, rather than some horrid bolted-on piece of crap like C++ templates. Server = Tuple[Address,ConnectionOptions] # array subscipt might not the best choice here, but it works UserId = NewType('UserId',int) # ordinary function call Scalar = int | str # could be equivalent to Union[int,str] with appropriate value of Type.__or__ 0: for example: def intBit(N): if N==0: return type(None) if N==1: return Bool if N>INT_WIDTH: return long return int
        basil-rash 8y ago
        mypy's response to that code: error:invalid type comment or annotation note:Suggestion: use intBit[...] instead of intBit(...) So what's really happening is the type expressions are pretending to be "just everyday python", but actually they have arbitrary restrictions (cannot be functions? need to work via overriding `__getitem__`?) that neither you nor I were aware of. This is probably the least "pythonic" implementation possible. And even if the code were valid, it's relying on N being a statically known value, which is a bit off because sure, you could have N be some global const config variable, but it would be very weird for configuring the value of the variable to require you to also go into the code and change things around to work with bool's or None's instead of int's.
talltimtom 8y ago
It sounds like you should invest some time in learning to program python in a pythonic way. If you are trying to build stuff using tons of design patterns factory patterns, you are doing it wrong. Most of those where made to work around limitations of type-systems which aren’t there in python, and naturally you end up having issues solving things like that when you don’t have typing. (Which you do still have with mypy, but as you say you’re not using that). At the end of the day you are using python and there must be a better way! though just like anything else you need to dedicate time to lean not just the syntax but also how to effectively use the language without needlessly over complicating things.
- basil-rash 8y ago
  I'm not trying to build using tons of factory patterns or the like, quite the opposite actually. I try to code as close to the bare language as possible, and work under the assumption that if I at some point would like to pivot my reasoning or approach to a program, the language and the tooling available will help me do so. This is simply not true without types. I'm not alone in thinking that, the creator of Python himself has devoted his recent work to adding types to Python. Is that just because he doesn't write his code "the correct way"? Does he simply know the syntax but not hoe to effectively use the language? I don't think so.

deleted 8y ago

[deleted]

diminoten 8y ago

Coming at it from the other side (Python has put food on my table for ~7 years now), it's not the "short scripts" that make Python great to develop in, it's that it can handle the growth of your app. Writing scientific/academic code and writing business application code is somewhat different, but the speed at which you can develop and maintain Python code is very high, as is the volume of aid available should you run into trouble. I don't think there's a language out there that's both easy to initially pick up and as well supported by its community as Python. The speed at which you can develop application code is astounding, once you get the hang of it and the noise out of the way (CI/CD, code coverage, unit and integration testing, a general familiarity with the major frameworks, etc.), and the maintenance is... tolerable.

darpa_escapee 8y ago

Python has an excellent ecosystem for those in the sciences, finance or those working with data. A few popular and mature web frameworks are written in Python. It has a place next to Bash et al when it comes to system scripting and acting as glue between different streams, utilities and resources. Static analysis tooling is excellent and type annotations only aid their accuracy. It's my impression that developers tend to learn to love static typing if they didn't cut their teeth on a statically typed platform. It takes experience to know that certain errors can be mitigated via language features and to know how annoying those errors are when they needlessly pop up. Unenforced explicit typing was and is still a boon for Python. Typing is often lost on developers who are still in "Hello, world!" territory and scientists who are using it as an adjunct to MATLAB. It can be a hindrance to rapid development, as well.

paultopia 8y ago

I wonder if you have a different definition of "short scripts" than many others. It seems to me that a genuinely short script wouldn't require big refactors, or types---that would be overkill for a short script. Personally, I use Python as a Bash replacement a lot, that's what I think of when I hear "short scripts." Sort of what people used to use Perl for. Pushing strings around, complicated repetitive filesystem manipulations. (And data sciencey stuff of course.)

basil-rash 8y ago
I use "short scripts" to mean: I have an idea I want to play around with, usually involving data analysis or some data structure or algorithm sketch, and I'd like to devote the next 2-3 hours to writing a 100-200 line program that evolves with me as I refine what exactly it is that I would like to be examining.
kungtotte 8y ago
I agree with you. Automate some task? Scrape some web content? Glue some programs together by translating the output of one to fit the input of the other? Python it is! If it doesn't already have something in the standard library for your task, it's definitely in the PyPI. This reduces your work to importing a library and writing 5-20 lines of code.
ryl00 8y ago
> Sort of what people used to use Perl for. Used to? People still do this (I should know; I'm one of them).
- paultopia 8y ago
  Heh, fair enough! I think I still have an open-ended bet with a friend about using Perl 6 for something...
- int0x80 8y ago
  And bash/zsh. Bash can be really powerful. Slow for some tasks though (much slower than perl for example), but little times I've had that problem.

keredson 8y ago

it's the quality of the libraries above all else that keeps me coming back to python. truly world class.

deusofnull 8y ago
Shout out to Django!
stevenjohns 8y ago
Don't forget the community and the support for those libraries. It's extremely welcoming and newbie-friendly.