20 ms·

Python's “disappointing” superpowers

164 points by nalgeon 4y ago

pingiun 4y ago
A fun piece of metaprogramming I did for a Django project was checking whether our models had overwritten str() methods: https://github.com/svthalia/concrexit/blob/4dc7691c2613b268d0922aa25a20b72d1db7ffa1/website/thaliawebsite/tests/test_models.py#L7-L55 https://github.com/svthalia/concrexit/blob/4dc7691c2613b268d...
sargstuff 4y ago
Goal directed-execution programming languages, such as 5th generation languages icon[0] & unicorn[1], would get the 100k (typed) / 10k(untyped) loc's down another factor of ten. ~1-2k loc around the range of being easy to get up to speed on what's going on than equivalent 100k/10k LOC program with less 'typing' issues. (compiler/interpeter & keyboard too!) Haven't run across a uni-py language yet (python & unicorn mash up) [0] : https://en.wikipedia.org/wiki/Icon_(programming_language) https://en.wikipedia.org/wiki/Icon_(programming_language) [1] : https://en.wikipedia.org/wiki/Unicon_(programming_language) https://en.wikipedia.org/wiki/Unicon_(programming_language)
DarkNova6 4y ago
Before I read this article I was not aware of exclusive advantages to dynamic typing. At least now I know what is achievable despite its presence.
- ImprobableTruth 4y ago
  Which of these do you think require dynamic typing?
  - joshuamorton 4y ago
    A lot of the orm style things are not really feasible in static-er languages without clunky approaches like codegen.
    - ImprobableTruth 4y ago
      What part specifically? Java is (in)famous for its orms after all. Unless you want to count that as clunky codegen? I'd argue that all metaprogramming boils down to that though.
      - david2ndaccount 4y ago
        Java is actually a very dynamic runtime with a very static language front end.
        arcturus17 4y ago
        The thread is about programming semantics though, so what does that have to do with anything?
        DarkNova6 4y ago
        cough Reflection. This also includes Annotations which can give clear semantics to code generation.
      - jononomo 4y ago
        Unless you want to count Java as clunky? Well, I definitely think Java is clunky. This article by Steve Yegge, titled "Execution in the Kingdom of Nouns" sums up Java for me: https://www.eecis.udel.edu/~decker/courses/280f07/paper/KingJava.pdf https://www.eecis.udel.edu/~decker/courses/280f07/paper/King...
        DarkNova6 4y ago
        ... this PDF is from 2006, the same year that Java 6 was released. Java 7 and Java 8 were game-changers at their time. I doubt there is much relevance left, especially in light of Java's excellent functional integration into its type system and JVM. Edit: ``` There's no reason Java couldn't simply add first-class functions and finally enter the grown-up, non-skewed world that allows people to use verbs as part of their thought processes ``` Ironically, this is exactly what happened. With exception that proper technological implementation took many years and might have one of the best implementations for anonymous functions out of any programming language (thank you, dynamic JVM).
  - DarkNova6 4y ago
    that's the point, none of these examples are intrinsically linked to dynamic typing.
amelius 4y ago
Is there any language out there that improves on Python in various ways but still allows convenient access to literally the entirety of Python's ecosystem of libraries?
- bayesian_horse 4y ago
  Coconut, F# (Fable)... https://github.com/vindarel/languages-that-compile-to-python https://github.com/vindarel/languages-that-compile-to-python
  - dunefox 4y ago
    F# is a great language... I wish I could use it at work.
    - bayesian_horse 4y ago
      One of the selling problems is that it can be hard to bring coworkers up to speed on it. Management don't like that kind of thing. What management like very much is that using F# attracts better talent, but they no can believe.
      - dunefox 4y ago
        It's actually very easy. If developers can't learn a slightly different language then they're getting paid too much.
        bayesian_horse 4y ago
        F# is a highly different language (at least from Python and Typescript) with additional concepts and I don't think you appreciate the problem for most companies to hire developers at all, let alone the kind of people who can hit a new language without a speed bump.
        dunefox 4y ago
paganel 4y ago
> I’m worried that a de-facto move away from dynamic stuff in the Python ecosystem, possibly motivated by those who use Python only because they have to, and just want to make it more like the C# or Java they are comfortable with, could leave us with the very worst of all worlds. It is certainly happening, and I'm not sure Python the language is all the better for it. I say that as a guy who explicitly mentioned this "Python Is Not Java" blog-post during my first job interview as a professional programmer, more than 15 years ago. [1] https://dirtsimple.org/2004/12/python-is-not-java.html https://dirtsimple.org/2004/12/python-is-not-java.html
- bayesian_horse 4y ago
  Over the years C# (and probably Java, but I'm not so well versed in that) gained quite a few features to improve "Meta-Programming", one could argue in an attempt to compete with dynamic languages. In C# that is mainly possible through introspection, generics and maybe dependency injection. Entity Framework Core can mostly figure out an SQL Table and its queries by looking at a simple class. It's not as good as the Django ORM or SQLAlchemy, but it tries to achieve similar things... So maybe some stuff in the Dotnet ecosystem got started because Python programmers were forced to use C# and tried to make that more dynamic.
  - acedTrex 4y ago
    How is SQLAlchemy or django better at any of that? in my experience EF is far far superior
    - bayesian_horse 4y ago
      Migrations have been painful in EF. No easy way to separate models/migrations for the same db context like with Django apps. Metadata for the models is hard to come by (probably there is a way?). Support for advanced Postgres datatypes slowly becomes ready for primetime, but currently isn't, and it's certainly about ten years behind Django in that regard. In Django, a model alreay contains all the meta data and you need almost no code to generate CRUD views and APIs from it. Can't do that in Dotnet. I think I could make something like that but why bother and take the pain? When dotnet people say "my experience" then they mostly mean no experience beyond dotnet (in my experience). I hope you are basing your comparison on more than a glance or a weekend project trying out Django...
fallingfrog 4y ago
Don't fool yourself- if your programming language has an eval() function, you are writing in Lisp and you are a Lisp programmer now. Embrace it! All joking aside, I do think Python is pretty handy.
- tmtvl 4y ago
  Oh no, Lisp is so much more powerful. For example you can update the definition of a class in a running system and update all existing instances of that class. When an error occurs you can inspect local variables in any of the stack frames and execute commands in them. And that is only the tip of the iceberg, Lisp is, as far as I know, without equal in terms of power.
  - progval 4y ago
    > you can update the definition of a class in a running system and update all existing instances of that class Like this? >>> class MyClass: ... attr = 0 ... >>> my_object = MyClass() >>> MyClass.attr = 42 >>> my_object.attr 42 >>> MyClass.method = lambda self: self.attr >>> my_object.method() 42 > When an error occurs you can inspect local variables in any of the stack frames and execute commands in them What is the difference with https://lukeplant.me.uk/blog/posts/pythons-disappointing-superpowers/#werkzeugs-interactive-debugger https://lukeplant.me.uk/blog/posts/pythons-disappointing-sup... ?
    - troutpanda 4y ago
      To Lispers, Python's hyperpowers (reflection and dynamic typing) are just regular Lisp business. Lisp's strength lies exactly in how convenient doing metaprogramming is in Lisp. There are probably some subtleties and differences in how powerful both systems are (can't say more, I don't know how powerful python's metaprogramming is), but just know that Lisp has practically no limit in how you can extend the language.
    - lispm 4y ago
      Adding/changing/removing slots & methods. But also adding/removing superclasses -> which causes changes in existing objects. Having functions called when an object changes, etc. That's old in CLOS, end 80s. No wonder if some other dynamic languages by now have similar features.
lordgroff 4y ago
I maintain Python code bases for a living, and feel that the language has simply been pushed too far. Static typing in Python doesn't give you the advantage of actually static typing and even IDE support is -- well it's not terrible, just not great. The thing is, once we go through all this static typing exercise in Python, we get no performance advantages, and the whole thing seems bolted on, with worse semantics than most modern typed languages. And yes, it can stifle undeniable advantages a dynamic language like Python can provide. Python remains a fantastic prototyping and scripting language, but my feeling is that it now handles more than it should.
- yummypaint 4y ago
  I routinely use cython to compile python for heavy workloads. A big part of the ~10x speedup i usually see comes from strategically assigning true static types to certain variables that get frequently iterated or compared. Most variables are left as standard python as it isn't necessary to change them for performance. My experience has been that cython "just works" even with lots of external libraries etc, and the code ends up performing as well as any c++ code i could realistically write while being much more understandable.
  - nextaccountic 4y ago
    > I routinely use cython to compile python for heavy workloads. an alternative is rust + pyo3 https://pyo3.rs https://pyo3.rs here's a web framework written with it, https://robyn.tech https://robyn.tech the other poster child for pyo3 is polars, https://www.pola.rs https://www.pola.rs it's simply, amazing.
    - samstave 4y ago
      which of these links should I start with with zero xp?
      - nextaccountic 4y ago
        If you don't know Rust, but know Python, you can install Python libraries written in Rust with pip. Like, pip install polars or pip install robyn. In this case you follow the two bottom links. But then you don't write your own libraries and stuff so.. I guess that's not what you want. If creating your own Python libraries in Rust is what you want, you would check out the first link I sent only, the one for pyo3. But, if you want to learn Rust, you probably wouldn't start out with pyo3. You first install Rust with https://rustup.rs/ https://rustup.rs/ and then check out the official book, and the book rust by example, that you can find here https://www.rust-lang.org/learn https://www.rust-lang.org/learn - and maybe write some code on the Rust playground https://play.rust-lang.org/ https://play.rust-lang.org/ - then, you use pyo3 to build Python libraries in Rust, and then use maturin https://www.maturin.rs/ https://www.maturin.rs/ to build and publish them to Pypi. But if you still prefer to begin with Rust by writing Python libraries (it's a valid strategy if you are very comfortable with working with multiple stacks), the Maturin link has a tutorial that setups a program that is half written in python, half written in Rust, https://www.maturin.rs/tutorial.html https://www.maturin.rs/tutorial.html (well the pyo3 link I sent also has one too. You should refer to the documentation of both, because you will use the two together) After learning Rust and building some stuff with pyo3, the next step is looking for libraries that you could leverage to make Python programs ultra fast. Here https://github.com/rayon-rs/rayon https://github.com/rayon-rs/rayon is an obvious choice, see some examples from the Rust cookbook https://rust-lang-nursery.github.io/rust-cookbook/concurrency/parallel.html https://rust-lang-nursery.github.io/rust-cookbook/concurrenc... - when you create a parallel iterator, it will distribute the processing to many threads (by default, one per core). The rust cookbook, by the way, is a nice reference to see some of the most used crates (Rust libraries) in the Rust ecosystem. If you are doing async stuff in Python, you probably need to look up https://github.com/awestlake87/pyo3-asyncio https://github.com/awestlake87/pyo3-asyncio and https://tokio.rs/ https://tokio.rs/ - that's two libraries that Robyn uses (see Robyn dependencies here https://github.com/sansyrox/robyn/blob/main/Cargo.toml https://github.com/sansyrox/robyn/blob/main/Cargo.toml) Anyway there are some posts about pyo3 on the web, like this blog post https://boring-guy.sh/posts/river-rust/ https://boring-guy.sh/posts/river-rust/ (note: it uses an outdated version of pyo3, and doesn't seem to use maturin which is a newer tool). This post was written by the developers of https://github.com/online-ml/river https://github.com/online-ml/river - another Python library written in Rust
bayesian_horse 4y ago
When I did most of my work in Python, typehints weren't a thing. I still feel like static typechecking is less useful in Python since its datamodel is overally less of a hot mess compared to Javascript for example. And Python modules/frameworks can be a lot more "typed" than you'd expect. Django models, forms etc offer more runtime validation than typesystems can easily achieve. Yes, at runtime. But between not having a build process and in general being more readable and concise, I feel like Python lets you run into the runtime error long before you get the same thing done "error free" in C#. Not that C# doesn't have runtime errors...
- phailhaus 4y ago
  > its datamodel is overally less of a hot mess compared to Javascript for example. I don't understand this, what do you mean? TypeScript is such a success partly because everything is an object in Javascript. There's no need to distinguish between classes and "dicts", as you have to with Python. Another example: Python functions have args and kwargs, which make typing significantly more complex (as the article points out, Callable doesn't even support kwargs). Javascript doesn't, so typing functions is trivial. Part of the point of static typechecking is documentation that can be verified by tooling. If you have no types in your codebase, and you decide to change something from a dict to a class, good luck! You're going to have to manually trace all the data flows through your entire application to figure out what's affected.
  - pdonis 4y ago
    > everything is an object in Javascript Everything is an object in Python too. > There's no need to distinguish between classes and "dicts", as you have to with Python. "dict" in Python (or at least Python 3, but Python 2 is EOL now so Python 3 is the only active Python there is) is a class. You can even subclass it, which if you want a customized dict for some reason in your particular application is often the best way to do it. Or you can subclass MutableMapping, which dict itself is a subclass of, and which is what is supposed to be used for isinstance checks now that Python has established abstract base classes for all of its standard "built-in type" interfaces.
wheelerof4te 4y ago
Either embrace dynamic typing and provide good error guards...or try to use type hints and still make good error guards. We had an entire history of Python 2 without type hints. Why use them now?
- ok123456 4y ago
  And most of that python2 code is completely rotted.
  - markrages 4y ago
    By definition, since it won't run on a supported runtime.
- 9dev 4y ago
  How come everyone realised the benefits of typing by now, except the Python community?
  - pdonis 4y ago
    The benefits of typing for certain kinds of applications. Many people in the Python community simply aren't writing those kinds of applications. They're writing applications where typing is not a benefit, it's a hindrance, so they don't use it, and Python makes that easy.
    - zerodensity 4y ago
      Could you give an example of a type of application where typing is a hindrance?
      - randomluck040 4y ago
        Not a hindrance but I work with image data a lot. Usually what comes in is any kind of Numpy array and what comes out at the end is either statistics or a Numpy array. I try to type hint everywhere I can but I feel stupid doing so because all I do is work with arrays. Maybe I’m not proficient/knowledgeable enough though so take it with a grain of salt.
teaearlgraycold 4y ago
TypeScript is the language Python wants to be. If only JS came with a Go-level stdlib there’d be nothing to complain about.
- deleted 4y ago
  [deleted]
- revskill 4y ago
  Seems valid, most modern Python code is almost the same as Typescript without {}
KurtMueller 4y ago
Python the language doesn't give you super powers and I would say that its clunkier and less elegant than Ruby and obviously not as powerful as languages like F#. Python's ecosystem and contributors are its biggest benefits and they help you succeed in spite of the language itself.
- jgeada 4y ago
  The ecosystem exists because Python does make it possible. That was one of the main thrusts of the article!
  - pdimitar 4y ago
    > The ecosystem exists because Python does make it possible. Only to a certain point, after which the network effects override everything else.
- deleted 4y ago
  [deleted]
ImprobableTruth 4y ago
Metaprogramming does not require dynamic types, but this post seems to equate them. As far as I can see all this could be done in e.g. Java (and probably is). IME this kind of "magic" very quickly loses its appeal when you have to debug it. The author's idea of libraries wrapping this stuff up so users don't have to care about it just doesn't pan out at all in my experience.
- ajuc 4y ago
  LISPs are the quintessential metaprogramming languages, and they learnt the lesson of "use macros sparingly and only as the last resort" very early on.
  - taeric 4y ago
    I mean, you aren't wrong. But most of what they learned was "step debugging is hard in the presence of macros." And step debugging has largely been tossed out of the window in many modern setups. Just look at Java's "stream" apis. I swear they did what they could to replicate the LOOP macro.
    - vkou 4y ago
      I am very rarely concerned about debugging my usage of Java streams, and step debugging works fine in the context surrounding them.
      - taeric 4y ago
        That is also the case for macros in lisp, at large. Similarly, any heavy use of lambdas in java will make step debugging confusing. As will any annotations you may use. Which is largely why many people grow to hate annotations. Which is all a fancy way of saying we learn the same lessons again and again. Used smartly, all of these are great tools. Defining what is "smartly" is a place of dragons.
    - ajuc 4y ago
      My main problem with metaprogramming in Java is that it breaks automatic IDE refactors and static code analysis. At least the metaprogramming we did with java back in the day (using reflections). If you use common java libraries made with metaprogramming - IDEs usually have plugins to work with that.
revskill 4y ago
Let's stop calling "dynamic" and "static" versus "runtime" and "comptime" instead ? Also, "dynamic code generation" for "meta programming" ?
- deleted 4y ago
  [deleted]
- ablob 4y ago
  static and dynamic refers to where you find an object, not when it lives. Some people like to know which type of meta programming is being used at a given point. This is something akin to "all dynamic code generation is meta programming, but not all meta programming is dynamic code generation". One should be careful to not merge or eliminate words with subtle differences. They usually exist for a reason.
fasttriggerfish 4y ago
I think type hints have mostly changed Python for the better but I still get frustrated by the number of half baked features and inconsistencies in the language. You end up fighting quirks ( like isinstance not working properly with generics ) all the time and it can get pretty tedious.
- packetlost 4y ago
  ... what? You're not supposed to use isinstance with generics??? It's a type-hinting only feature afaik
Spivak 4y ago
> But typing.Callable has zero support for them, meaning they can’t be typed in a higher-order context. https://peps.python.org/pep-0612/ https://peps.python.org/pep-0612/ from typing import Awaitable, Callable, ParamSpec, TypeVar P = ParamSpec("P") R = TypeVar("R") def add_logging(f: Callable[P, R]) -> Callable[P, Awaitable[R]]: async def inner(*args: P.args, **kwargs: P.kwargs) -> R: await log_to_database() return f(*args, **kwargs) return inner @add_logging def takes_int_str(x: int, y: str) -> int: return x + 7 await takes_int_str(1, "A") # Accepted await takes_int_str("B", 2) # Correctly rejected by the type checker I think the author and the people working on Python's type annotations actually agree here. The goal is to have type annotations powerful enough to statically describe all behavior generically without library specific extensions. There hasn't been much push for it but I expect a lot of the author's concerns would be alleviated by a type annotation for types themselves. Being able to say, "alright I'm gonna so some dynamic metaprogramming magic bullshit, but at the end will pop out a class of this shape" probably gets you 80% of the way where.
thereisnospork 4y ago
While this might be about my most unpopular opinion, it feels like it is time to start putting together a Python 4.0. I don't particularly have skin in the game but there is(?) enough meat on the bone around things like improving the GIL status quo, JIT compiling, static typing, and presumably etc. to be worth a breaking change.
- JonChesterfield 4y ago
  That's definitely an unpopular opinion. The 2 to 3 change caused a lot of pain. I can't see the python community attempting that again.
- deleted 4y ago
  [deleted]
- codexb 4y ago
  The GIL is already set to be optionally removable via a compile-time flag in an upcoming release. That's about as meaty a change as you can get, and it's still not going to require a 4.x release.
- adamj9431 4y ago
  Maybe string can be ebcdic this time around?
TeeWEE 4y ago
All these features are nice and stuff... But they are often runtime features... At compile time Python doesnt tell you wether a program is correct. Which is fine for small programs or small services. But any big system is written in python is really hard to maintain without LOTS of unit tests... Static typing, compile time checks just win in the long run. And with languages like kotlin you still have all the advantages, and you have nice tools like python has which execute at COMPILE time using DSL, which are type safe themselve.. Python has become mostly a toy language for me to write scripts in.
- chatmasta 4y ago
  What does "compile time" even mean for Python?
  - RedTachyon 4y ago
    The moment you run the script, before it actually gets executed.
- CoastalCoder 4y ago
  > At compile time Python doesnt tell you wether a program is correct. This is a general limitation of type systems, IIUC. If you want a type system to guarantee 100% program correctness, I think it can't handle an arbitrary Turing-complete program. Disclaimer: I'm not sure I really understand this topic, so take this with a grain of salt.
- ReflectedImage 4y ago
  No programming language tells you if a program is correct at compile time. Type errors aren't a very common type of bug either.
  - danpla 4y ago
    As I know, SPARK can do this (in particular, thanks to the strong type system). https://learn.adacore.com/courses/intro-to-spark/chapters/01_Overview.html https://learn.adacore.com/courses/intro-to-spark/chapters/01...
deleted 4y ago
[deleted]
benreesman 4y ago
I kinda hope we can just let Python be Python: it’s in a global-ish maximum for what it’s for, and I suspect any big move away from that will make it worse overall. Obviously things like UTF-8 support, or maybe getting a clean lexical scope option, or other “fixes” are good. Performance improvements are good. But there are lots of mainstream languages that have mature static type systems, strong metaprogramming facilities, higher performance, or all of the above. I’d way rather see effort go into getting the package management thing figured out, or the static analysis stuff improved further than any big movement in the language proper. Just my 2c.
- nine_k 4y ago
  Nit: Python has strong metaprogramming facilities since version 2.2, when "types" and "classes" were made equal, and it became trivial to create a class using the `type` function. It's so easy exactly because Python is so dynamic.
- codexb 4y ago
  I still think python has a ton of headroom to grow. It's maxing out the data science space, and it does reasonably well in the web service space, but they really have not pushed very far into the distributable GUI and CLI space. Dependency, packaging, and distribution have been improving in leaps and bounds still for python, and type-hinting makes large projects manageable.
analog31 4y ago
I'm a so called "scientific" programmer, not a software developer. My work is upstream from turning anything I make into a product by the real devs, who have their own languages. Maybe I'm in a minority, but I use dynamic typing. I can haul data around in flexible formats such as JSON, asdf, pickle, and so forth, and manipulate them as dict structures. Python builds the structure automatically. One of my first languages was Pascal, and so I understand the principle of letting your data structures and type checking do your work, and I still live that way when programming in C for embedded stuff. There may be better ways of doing what I'm doing, but there are also worse ways. From my own experience with getting software changed to suit my own development work, data structures and file formats are right up there in terms of why software is so inflexible.
spoils19 4y ago
I used to be a huge dynamic languages fan, with python being my favorite language over the majority of my career. Then I worked on a large python project of ~100k LOC with a team of ten. That's when I realized that writing code faster isn't the problem. Reading it is, making changes to code someone else wrote is, and refactoring across dozens of modules is a problem. Static languages help a lot with all three. I still love dynamic languages for small tasks, but I'd rather use a static language like Go which still keeps much of the dynamic feel, and helps catch my mistakes at compile time. I actually just set it to watch the project directory for changes and recompile automatically in a terminal on another screen (actually to run the unit tests, which includes compiling.) That's constant feedback and comparable to dynamic languages speed for edit-compile-test runs. But the best part is the unit tests. They run so much faster with Go that I don't get distracted waiting for them to finish, and that keeps me in the zone and much more productive.
- taeric 4y ago
  This is almost always framed in a way that shows large systems need the extra rigidity of some static systems. And that dynamic systems are only about rapid prototyping. I think that is a bit of a false framing. To wit, any system of 100k LOC will be hard to get into. Even harder to make changes in. There is no getting around that. None. Even worse if you have many entities flying about these 100k LOC. Each change to an entity will be dangerous. Static or dynamic. Especially if they are persisted anywhere, as at that point it is all too easy to get static guarantees that aren't reflecting actual data.
  - reissbaker 4y ago
    Anecdotal but — I've worked on many-million-LOC codebases in Ruby with large teams (pre-Sorbet and other type checkers, at Airbnb), and many-million-LOC codebases in typechecker-checked Python with large teams (Instagram). The typechecker-checked Python was way easier to reason about and refactor.
  - cageface 4y ago
    This is a false equivalence. You need all the help you can get maintaining large codebases and static typing is a huge help.
codexb 4y ago
His first example of python's Callable type-hinting not supporting keyword only arguments was true 5 years ago when type hinting was first catching on, but Protocols have existed since python 3.8 and they support keyword only arguments.
fithisux 4y ago
My personal experience is that static typing not only eliminates a huge number of unit tests but increases documentation expression in combinationn with good names. Dynamic typing is excellent for experimentation though. Python hits the sweet spot. But I can definitely recommend Groovy as an excellent alternative having all the strengths of the Java ecosystem. Scala and Kotlin are also quickly catching up.

164 points by nalgeon 4y ago

A fun piece of metaprogramming I did for a Django project was checking whether our models had overwritten str() methods: https://github.com/svthalia/concrexit/blob/4dc7691c2613b268d0922aa25a20b72d1db7ffa1/website/thaliawebsite/tests/test_models.py#L7-L55 https://github.com/svthalia/concrexit/blob/4dc7691c2613b268d...

sargstuff 4y ago

Goal directed-execution programming languages, such as 5th generation languages icon[0] & unicorn[1], would get the 100k (typed) / 10k(untyped) loc's down another factor of ten. ~1-2k loc around the range of being easy to get up to speed on what's going on than equivalent 100k/10k LOC program with less 'typing' issues. (compiler/interpeter & keyboard too!) Haven't run across a uni-py language yet (python & unicorn mash up) [0] : https://en.wikipedia.org/wiki/Icon_(programming_language) https://en.wikipedia.org/wiki/Icon_(programming_language) [1] : https://en.wikipedia.org/wiki/Unicon_(programming_language) https://en.wikipedia.org/wiki/Unicon_(programming_language)

DarkNova6 4y ago

Before I read this article I was not aware of exclusive advantages to dynamic typing. At least now I know what is achievable despite its presence.

ImprobableTruth 4y ago
Which of these do you think require dynamic typing?
- joshuamorton 4y ago
  A lot of the orm style things are not really feasible in static-er languages without clunky approaches like codegen.
  - ImprobableTruth 4y ago
    What part specifically? Java is (in)famous for its orms after all. Unless you want to count that as clunky codegen? I'd argue that all metaprogramming boils down to that though.
    - david2ndaccount 4y ago
      Java is actually a very dynamic runtime with a very static language front end.
      - arcturus17 4y ago
        The thread is about programming semantics though, so what does that have to do with anything?
        DarkNova6 4y ago
        cough Reflection. This also includes Annotations which can give clear semantics to code generation.
    - jononomo 4y ago
      Unless you want to count Java as clunky? Well, I definitely think Java is clunky. This article by Steve Yegge, titled "Execution in the Kingdom of Nouns" sums up Java for me: https://www.eecis.udel.edu/~decker/courses/280f07/paper/KingJava.pdf https://www.eecis.udel.edu/~decker/courses/280f07/paper/King...
      - DarkNova6 4y ago
        ... this PDF is from 2006, the same year that Java 6 was released. Java 7 and Java 8 were game-changers at their time. I doubt there is much relevance left, especially in light of Java's excellent functional integration into its type system and JVM. Edit: ``` There's no reason Java couldn't simply add first-class functions and finally enter the grown-up, non-skewed world that allows people to use verbs as part of their thought processes ``` Ironically, this is exactly what happened. With exception that proper technological implementation took many years and might have one of the best implementations for anonymous functions out of any programming language (thank you, dynamic JVM).
- DarkNova6 4y ago
  that's the point, none of these examples are intrinsically linked to dynamic typing.

amelius 4y ago

Is there any language out there that improves on Python in various ways but still allows convenient access to literally the entirety of Python's ecosystem of libraries?

bayesian_horse 4y ago
Coconut, F# (Fable)... https://github.com/vindarel/languages-that-compile-to-python https://github.com/vindarel/languages-that-compile-to-python
- dunefox 4y ago
  F# is a great language... I wish I could use it at work.
  - bayesian_horse 4y ago
    One of the selling problems is that it can be hard to bring coworkers up to speed on it. Management don't like that kind of thing. What management like very much is that using F# attracts better talent, but they no can believe.
    - dunefox 4y ago
      It's actually very easy. If developers can't learn a slightly different language then they're getting paid too much.
      - bayesian_horse 4y ago
        F# is a highly different language (at least from Python and Typescript) with additional concepts and I don't think you appreciate the problem for most companies to hire developers at all, let alone the kind of people who can hit a new language without a speed bump.
        dunefox 4y ago

paganel 4y ago

> I’m worried that a de-facto move away from dynamic stuff in the Python ecosystem, possibly motivated by those who use Python only because they have to, and just want to make it more like the C# or Java they are comfortable with, could leave us with the very worst of all worlds. It is certainly happening, and I'm not sure Python the language is all the better for it. I say that as a guy who explicitly mentioned this "Python Is Not Java" blog-post during my first job interview as a professional programmer, more than 15 years ago. [1] https://dirtsimple.org/2004/12/python-is-not-java.html https://dirtsimple.org/2004/12/python-is-not-java.html

bayesian_horse 4y ago
Over the years C# (and probably Java, but I'm not so well versed in that) gained quite a few features to improve "Meta-Programming", one could argue in an attempt to compete with dynamic languages. In C# that is mainly possible through introspection, generics and maybe dependency injection. Entity Framework Core can mostly figure out an SQL Table and its queries by looking at a simple class. It's not as good as the Django ORM or SQLAlchemy, but it tries to achieve similar things... So maybe some stuff in the Dotnet ecosystem got started because Python programmers were forced to use C# and tried to make that more dynamic.
- acedTrex 4y ago
  How is SQLAlchemy or django better at any of that? in my experience EF is far far superior
  - bayesian_horse 4y ago
    Migrations have been painful in EF. No easy way to separate models/migrations for the same db context like with Django apps. Metadata for the models is hard to come by (probably there is a way?). Support for advanced Postgres datatypes slowly becomes ready for primetime, but currently isn't, and it's certainly about ten years behind Django in that regard. In Django, a model alreay contains all the meta data and you need almost no code to generate CRUD views and APIs from it. Can't do that in Dotnet. I think I could make something like that but why bother and take the pain? When dotnet people say "my experience" then they mostly mean no experience beyond dotnet (in my experience). I hope you are basing your comparison on more than a glance or a weekend project trying out Django...

fallingfrog 4y ago

Don't fool yourself- if your programming language has an eval() function, you are writing in Lisp and you are a Lisp programmer now. Embrace it! All joking aside, I do think Python is pretty handy.

tmtvl 4y ago
Oh no, Lisp is so much more powerful. For example you can update the definition of a class in a running system and update all existing instances of that class. When an error occurs you can inspect local variables in any of the stack frames and execute commands in them. And that is only the tip of the iceberg, Lisp is, as far as I know, without equal in terms of power.
- progval 4y ago
  > you can update the definition of a class in a running system and update all existing instances of that class Like this? >>> class MyClass: ... attr = 0 ... >>> my_object = MyClass() >>> MyClass.attr = 42 >>> my_object.attr 42 >>> MyClass.method = lambda self: self.attr >>> my_object.method() 42 > When an error occurs you can inspect local variables in any of the stack frames and execute commands in them What is the difference with https://lukeplant.me.uk/blog/posts/pythons-disappointing-superpowers/#werkzeugs-interactive-debugger https://lukeplant.me.uk/blog/posts/pythons-disappointing-sup... ?
  - troutpanda 4y ago
    To Lispers, Python's hyperpowers (reflection and dynamic typing) are just regular Lisp business. Lisp's strength lies exactly in how convenient doing metaprogramming is in Lisp. There are probably some subtleties and differences in how powerful both systems are (can't say more, I don't know how powerful python's metaprogramming is), but just know that Lisp has practically no limit in how you can extend the language.
  - lispm 4y ago
    Adding/changing/removing slots & methods. But also adding/removing superclasses -> which causes changes in existing objects. Having functions called when an object changes, etc. That's old in CLOS, end 80s. No wonder if some other dynamic languages by now have similar features.

lordgroff 4y ago

I maintain Python code bases for a living, and feel that the language has simply been pushed too far. Static typing in Python doesn't give you the advantage of actually static typing and even IDE support is -- well it's not terrible, just not great. The thing is, once we go through all this static typing exercise in Python, we get no performance advantages, and the whole thing seems bolted on, with worse semantics than most modern typed languages. And yes, it can stifle undeniable advantages a dynamic language like Python can provide. Python remains a fantastic prototyping and scripting language, but my feeling is that it now handles more than it should.

yummypaint 4y ago
I routinely use cython to compile python for heavy workloads. A big part of the ~10x speedup i usually see comes from strategically assigning true static types to certain variables that get frequently iterated or compared. Most variables are left as standard python as it isn't necessary to change them for performance. My experience has been that cython "just works" even with lots of external libraries etc, and the code ends up performing as well as any c++ code i could realistically write while being much more understandable.
- nextaccountic 4y ago
  > I routinely use cython to compile python for heavy workloads. an alternative is rust + pyo3 https://pyo3.rs https://pyo3.rs here's a web framework written with it, https://robyn.tech https://robyn.tech the other poster child for pyo3 is polars, https://www.pola.rs https://www.pola.rs it's simply, amazing.
  - samstave 4y ago
    which of these links should I start with with zero xp?
    - nextaccountic 4y ago
      If you don't know Rust, but know Python, you can install Python libraries written in Rust with pip. Like, pip install polars or pip install robyn. In this case you follow the two bottom links. But then you don't write your own libraries and stuff so.. I guess that's not what you want. If creating your own Python libraries in Rust is what you want, you would check out the first link I sent only, the one for pyo3. But, if you want to learn Rust, you probably wouldn't start out with pyo3. You first install Rust with https://rustup.rs/ https://rustup.rs/ and then check out the official book, and the book rust by example, that you can find here https://www.rust-lang.org/learn https://www.rust-lang.org/learn - and maybe write some code on the Rust playground https://play.rust-lang.org/ https://play.rust-lang.org/ - then, you use pyo3 to build Python libraries in Rust, and then use maturin https://www.maturin.rs/ https://www.maturin.rs/ to build and publish them to Pypi. But if you still prefer to begin with Rust by writing Python libraries (it's a valid strategy if you are very comfortable with working with multiple stacks), the Maturin link has a tutorial that setups a program that is half written in python, half written in Rust, https://www.maturin.rs/tutorial.html https://www.maturin.rs/tutorial.html (well the pyo3 link I sent also has one too. You should refer to the documentation of both, because you will use the two together) After learning Rust and building some stuff with pyo3, the next step is looking for libraries that you could leverage to make Python programs ultra fast. Here https://github.com/rayon-rs/rayon https://github.com/rayon-rs/rayon is an obvious choice, see some examples from the Rust cookbook https://rust-lang-nursery.github.io/rust-cookbook/concurrency/parallel.html https://rust-lang-nursery.github.io/rust-cookbook/concurrenc... - when you create a parallel iterator, it will distribute the processing to many threads (by default, one per core). The rust cookbook, by the way, is a nice reference to see some of the most used crates (Rust libraries) in the Rust ecosystem. If you are doing async stuff in Python, you probably need to look up https://github.com/awestlake87/pyo3-asyncio https://github.com/awestlake87/pyo3-asyncio and https://tokio.rs/ https://tokio.rs/ - that's two libraries that Robyn uses (see Robyn dependencies here https://github.com/sansyrox/robyn/blob/main/Cargo.toml https://github.com/sansyrox/robyn/blob/main/Cargo.toml) Anyway there are some posts about pyo3 on the web, like this blog post https://boring-guy.sh/posts/river-rust/ https://boring-guy.sh/posts/river-rust/ (note: it uses an outdated version of pyo3, and doesn't seem to use maturin which is a newer tool). This post was written by the developers of https://github.com/online-ml/river https://github.com/online-ml/river - another Python library written in Rust

bayesian_horse 4y ago

When I did most of my work in Python, typehints weren't a thing. I still feel like static typechecking is less useful in Python since its datamodel is overally less of a hot mess compared to Javascript for example. And Python modules/frameworks can be a lot more "typed" than you'd expect. Django models, forms etc offer more runtime validation than typesystems can easily achieve. Yes, at runtime. But between not having a build process and in general being more readable and concise, I feel like Python lets you run into the runtime error long before you get the same thing done "error free" in C#. Not that C# doesn't have runtime errors...

phailhaus 4y ago
> its datamodel is overally less of a hot mess compared to Javascript for example. I don't understand this, what do you mean? TypeScript is such a success partly because everything is an object in Javascript. There's no need to distinguish between classes and "dicts", as you have to with Python. Another example: Python functions have args and kwargs, which make typing significantly more complex (as the article points out, Callable doesn't even support kwargs). Javascript doesn't, so typing functions is trivial. Part of the point of static typechecking is documentation that can be verified by tooling. If you have no types in your codebase, and you decide to change something from a dict to a class, good luck! You're going to have to manually trace all the data flows through your entire application to figure out what's affected.
- pdonis 4y ago
  > everything is an object in Javascript Everything is an object in Python too. > There's no need to distinguish between classes and "dicts", as you have to with Python. "dict" in Python (or at least Python 3, but Python 2 is EOL now so Python 3 is the only active Python there is) is a class. You can even subclass it, which if you want a customized dict for some reason in your particular application is often the best way to do it. Or you can subclass MutableMapping, which dict itself is a subclass of, and which is what is supposed to be used for isinstance checks now that Python has established abstract base classes for all of its standard "built-in type" interfaces.

wheelerof4te 4y ago

Either embrace dynamic typing and provide good error guards...or try to use type hints and still make good error guards. We had an entire history of Python 2 without type hints. Why use them now?

ok123456 4y ago
And most of that python2 code is completely rotted.
- markrages 4y ago
  By definition, since it won't run on a supported runtime.
9dev 4y ago
How come everyone realised the benefits of typing by now, except the Python community?
- pdonis 4y ago
  The benefits of typing for certain kinds of applications. Many people in the Python community simply aren't writing those kinds of applications. They're writing applications where typing is not a benefit, it's a hindrance, so they don't use it, and Python makes that easy.
  - zerodensity 4y ago
    Could you give an example of a type of application where typing is a hindrance?
    - randomluck040 4y ago
      Not a hindrance but I work with image data a lot. Usually what comes in is any kind of Numpy array and what comes out at the end is either statistics or a Numpy array. I try to type hint everywhere I can but I feel stupid doing so because all I do is work with arrays. Maybe I’m not proficient/knowledgeable enough though so take it with a grain of salt.

teaearlgraycold 4y ago

TypeScript is the language Python wants to be. If only JS came with a Go-level stdlib there’d be nothing to complain about.

deleted 4y ago
[deleted]
revskill 4y ago
Seems valid, most modern Python code is almost the same as Typescript without {}

KurtMueller 4y ago

Python the language doesn't give you super powers and I would say that its clunkier and less elegant than Ruby and obviously not as powerful as languages like F#. Python's ecosystem and contributors are its biggest benefits and they help you succeed in spite of the language itself.

jgeada 4y ago
The ecosystem exists because Python does make it possible. That was one of the main thrusts of the article!
- pdimitar 4y ago
  > The ecosystem exists because Python does make it possible. Only to a certain point, after which the network effects override everything else.
deleted 4y ago
[deleted]

ImprobableTruth 4y ago

Metaprogramming does not require dynamic types, but this post seems to equate them. As far as I can see all this could be done in e.g. Java (and probably is). IME this kind of "magic" very quickly loses its appeal when you have to debug it. The author's idea of libraries wrapping this stuff up so users don't have to care about it just doesn't pan out at all in my experience.

ajuc 4y ago
LISPs are the quintessential metaprogramming languages, and they learnt the lesson of "use macros sparingly and only as the last resort" very early on.
- taeric 4y ago
  I mean, you aren't wrong. But most of what they learned was "step debugging is hard in the presence of macros." And step debugging has largely been tossed out of the window in many modern setups. Just look at Java's "stream" apis. I swear they did what they could to replicate the LOOP macro.
  - vkou 4y ago
    I am very rarely concerned about debugging my usage of Java streams, and step debugging works fine in the context surrounding them.
    - taeric 4y ago
      That is also the case for macros in lisp, at large. Similarly, any heavy use of lambdas in java will make step debugging confusing. As will any annotations you may use. Which is largely why many people grow to hate annotations. Which is all a fancy way of saying we learn the same lessons again and again. Used smartly, all of these are great tools. Defining what is "smartly" is a place of dragons.
  - ajuc 4y ago
    My main problem with metaprogramming in Java is that it breaks automatic IDE refactors and static code analysis. At least the metaprogramming we did with java back in the day (using reflections). If you use common java libraries made with metaprogramming - IDEs usually have plugins to work with that.

revskill 4y ago

Let's stop calling "dynamic" and "static" versus "runtime" and "comptime" instead ? Also, "dynamic code generation" for "meta programming" ?

deleted 4y ago
[deleted]
ablob 4y ago
static and dynamic refers to where you find an object, not when it lives. Some people like to know which type of meta programming is being used at a given point. This is something akin to "all dynamic code generation is meta programming, but not all meta programming is dynamic code generation". One should be careful to not merge or eliminate words with subtle differences. They usually exist for a reason.

fasttriggerfish 4y ago

I think type hints have mostly changed Python for the better but I still get frustrated by the number of half baked features and inconsistencies in the language. You end up fighting quirks ( like isinstance not working properly with generics ) all the time and it can get pretty tedious.

packetlost 4y ago
... what? You're not supposed to use isinstance with generics??? It's a type-hinting only feature afaik

Spivak 4y ago

> But typing.Callable has zero support for them, meaning they can’t be typed in a higher-order context. https://peps.python.org/pep-0612/ https://peps.python.org/pep-0612/ from typing import Awaitable, Callable, ParamSpec, TypeVar P = ParamSpec("P") R = TypeVar("R") def add_logging(f: Callable[P, R]) -> Callable[P, Awaitable[R]]: async def inner(*args: P.args, **kwargs: P.kwargs) -> R: await log_to_database() return f(*args, **kwargs) return inner @add_logging def takes_int_str(x: int, y: str) -> int: return x + 7 await takes_int_str(1, "A") # Accepted await takes_int_str("B", 2) # Correctly rejected by the type checker I think the author and the people working on Python's type annotations actually agree here. The goal is to have type annotations powerful enough to statically describe all behavior generically without library specific extensions. There hasn't been much push for it but I expect a lot of the author's concerns would be alleviated by a type annotation for types themselves. Being able to say, "alright I'm gonna so some dynamic metaprogramming magic bullshit, but at the end will pop out a class of this shape" probably gets you 80% of the way where.

thereisnospork 4y ago

While this might be about my most unpopular opinion, it feels like it is time to start putting together a Python 4.0. I don't particularly have skin in the game but there is(?) enough meat on the bone around things like improving the GIL status quo, JIT compiling, static typing, and presumably etc. to be worth a breaking change.

JonChesterfield 4y ago
That's definitely an unpopular opinion. The 2 to 3 change caused a lot of pain. I can't see the python community attempting that again.
deleted 4y ago
[deleted]
codexb 4y ago
The GIL is already set to be optionally removable via a compile-time flag in an upcoming release. That's about as meaty a change as you can get, and it's still not going to require a 4.x release.
adamj9431 4y ago
Maybe string can be ebcdic this time around?

TeeWEE 4y ago

All these features are nice and stuff... But they are often runtime features... At compile time Python doesnt tell you wether a program is correct. Which is fine for small programs or small services. But any big system is written in python is really hard to maintain without LOTS of unit tests... Static typing, compile time checks just win in the long run. And with languages like kotlin you still have all the advantages, and you have nice tools like python has which execute at COMPILE time using DSL, which are type safe themselve.. Python has become mostly a toy language for me to write scripts in.

chatmasta 4y ago
What does "compile time" even mean for Python?
- RedTachyon 4y ago
  The moment you run the script, before it actually gets executed.
CoastalCoder 4y ago
> At compile time Python doesnt tell you wether a program is correct. This is a general limitation of type systems, IIUC. If you want a type system to guarantee 100% program correctness, I think it can't handle an arbitrary Turing-complete program. Disclaimer: I'm not sure I really understand this topic, so take this with a grain of salt.
ReflectedImage 4y ago
No programming language tells you if a program is correct at compile time. Type errors aren't a very common type of bug either.
- danpla 4y ago
  As I know, SPARK can do this (in particular, thanks to the strong type system). https://learn.adacore.com/courses/intro-to-spark/chapters/01_Overview.html https://learn.adacore.com/courses/intro-to-spark/chapters/01...

deleted 4y ago

[deleted]

benreesman 4y ago

I kinda hope we can just let Python be Python: it’s in a global-ish maximum for what it’s for, and I suspect any big move away from that will make it worse overall. Obviously things like UTF-8 support, or maybe getting a clean lexical scope option, or other “fixes” are good. Performance improvements are good. But there are lots of mainstream languages that have mature static type systems, strong metaprogramming facilities, higher performance, or all of the above. I’d way rather see effort go into getting the package management thing figured out, or the static analysis stuff improved further than any big movement in the language proper. Just my 2c.

nine_k 4y ago
Nit: Python has strong metaprogramming facilities since version 2.2, when "types" and "classes" were made equal, and it became trivial to create a class using the `type` function. It's so easy exactly because Python is so dynamic.
codexb 4y ago
I still think python has a ton of headroom to grow. It's maxing out the data science space, and it does reasonably well in the web service space, but they really have not pushed very far into the distributable GUI and CLI space. Dependency, packaging, and distribution have been improving in leaps and bounds still for python, and type-hinting makes large projects manageable.

analog31 4y ago

I'm a so called "scientific" programmer, not a software developer. My work is upstream from turning anything I make into a product by the real devs, who have their own languages. Maybe I'm in a minority, but I use dynamic typing. I can haul data around in flexible formats such as JSON, asdf, pickle, and so forth, and manipulate them as dict structures. Python builds the structure automatically. One of my first languages was Pascal, and so I understand the principle of letting your data structures and type checking do your work, and I still live that way when programming in C for embedded stuff. There may be better ways of doing what I'm doing, but there are also worse ways. From my own experience with getting software changed to suit my own development work, data structures and file formats are right up there in terms of why software is so inflexible.

spoils19 4y ago

I used to be a huge dynamic languages fan, with python being my favorite language over the majority of my career. Then I worked on a large python project of ~100k LOC with a team of ten. That's when I realized that writing code faster isn't the problem. Reading it is, making changes to code someone else wrote is, and refactoring across dozens of modules is a problem. Static languages help a lot with all three. I still love dynamic languages for small tasks, but I'd rather use a static language like Go which still keeps much of the dynamic feel, and helps catch my mistakes at compile time. I actually just set it to watch the project directory for changes and recompile automatically in a terminal on another screen (actually to run the unit tests, which includes compiling.) That's constant feedback and comparable to dynamic languages speed for edit-compile-test runs. But the best part is the unit tests. They run so much faster with Go that I don't get distracted waiting for them to finish, and that keeps me in the zone and much more productive.

taeric 4y ago
This is almost always framed in a way that shows large systems need the extra rigidity of some static systems. And that dynamic systems are only about rapid prototyping. I think that is a bit of a false framing. To wit, any system of 100k LOC will be hard to get into. Even harder to make changes in. There is no getting around that. None. Even worse if you have many entities flying about these 100k LOC. Each change to an entity will be dangerous. Static or dynamic. Especially if they are persisted anywhere, as at that point it is all too easy to get static guarantees that aren't reflecting actual data.
- reissbaker 4y ago
  Anecdotal but — I've worked on many-million-LOC codebases in Ruby with large teams (pre-Sorbet and other type checkers, at Airbnb), and many-million-LOC codebases in typechecker-checked Python with large teams (Instagram). The typechecker-checked Python was way easier to reason about and refactor.
- cageface 4y ago
  This is a false equivalence. You need all the help you can get maintaining large codebases and static typing is a huge help.

codexb 4y ago

His first example of python's Callable type-hinting not supporting keyword only arguments was true 5 years ago when type hinting was first catching on, but Protocols have existed since python 3.8 and they support keyword only arguments.

fithisux 4y ago

My personal experience is that static typing not only eliminates a huge number of unit tests but increases documentation expression in combinationn with good names. Dynamic typing is excellent for experimentation though. Python hits the sweet spot. But I can definitely recommend Groovy as an excellent alternative having all the strengths of the Java ecosystem. Scala and Kotlin are also quickly catching up.