12 ms·

Python is two languages now

mikece 4y ago
I get that the author is asserting that untyped Pythong is superior for infrastructure code but I don't understand why the author thinks this.
- sdfghswe 4y ago
  It's funny that you typoed Python as Pythong. I do that a lot - in my case I know it's faulty muscle memory, but I don't know where it comes from.
  - mikece 4y ago
    And to my chagrin I didn't realize it until AFTER the time window for editing a comment passed. Oh well.
- stu2b50 4y ago
  I’m not sure I understand the “infrastracture” label, but certainly for the cases the author listed as “infra” typing can be cumbersome. Famously just json deserialization can be rather painful in statically typed languages, especially ones without complex compile time features.
  - eropple 4y ago
    Serialization/deserialization can be really gross in something like Java, but TypeScript gives you some really good tools (zod, typebox, etc.) to validate and type-guard your inputs that I'm sure could be implemented in Python--if they're not already.
    - pbalau 4y ago
      Pydantic will do that for Python.
      - eropple 4y ago
        Yeah, I assumed as much. I've only looked at Pydantic in the context of FastAPI, as Python's not really my thing, but it looks really well-thought-out.
    - amarant 4y ago
      If serialisation is painful in Java, you're doing it wrong. Libraries like Jackson make it entirely painless. I would assume JSON parsing is a solved problem in any typed language.
      - eropple 4y ago
        I've written a lot of code using Jackson and I strongly disagree that it's a solved problem. If you're Java on both ends, it's probably fine, but then your serde style is an implementation detail. And that's great, but if you're not, you are in a bear pit. The lingua franca for JSON specification is JSON Schema, and there is significant impedance mismatch between the standard JSON tools folks use in Java and with JSON Schema as used by most other folks. (Jackson has a nice JSON Schema module for exporting it, but importing/codegen is pretty gross.)
    - sixbrx 4y ago
      I find Java pretty nice for handling JSON serialization/deserialization these days. Just define some records for the expected structure, and Jackson's object mapper handles the rest.
      - eropple 4y ago
        It's the "defining some records" part that's slow and gritty. JSON is usually an interchange format, and Java class construction is either slow, tedious, and manual, or involves massaging a code generator (which, at least the last time I checked a few months ago, weren't very good in Java).
        vips7L 4y ago
        I don’t see how declaring a data model class is anymore difficult in Java than another typed language. record Thing(String name, int size, LocalDate when) {} The only way I could see this being easier is to forgo type safety completely.
        eropple 4y ago
        You're assuming you only have to define it in one language. JSON is an interchange format. The canonical way to express it is JSON Schema, and the tools are unpleasant in Java. So is dealing with structural typing of polymorphic responses. In an ideal world we wouldn't have those, but we live in this one. These things are just better when you aren't in a nominatively typed language with some pretty crusty assumptions.
  - marcosdumay 4y ago
    Hum... Almost all of the problems of serialization/deserialization in statically typed languages turn into testing problems in dynamically typed languages. That's not a gain in any way.
    - stu2b50 4y ago
      In this case it’s more about shifting where the static typing occurs, since the author is still arguing for a typed business layer. Since I/O is usually untyped, you need to cram it into a typed schema somewhere. In strictly typed languages, that must occur upon ingestion. The author is saying you can delay it by having a layer on top of the I/O transform the data in untyped fashion before being typed later on.
      - Yoric 4y ago
        What's the benefit of delaying typing? By typing it at ingestion, you guarantee that invalid (with respect to structure) untrusted data will not break your code down the line. Doing otherwise means that you're manipulating untrusted data with a strong risk of forgetting that it is untrusted. I have seen large applications (think hundreds of millions of daily users) break because of such oopses.
  - Yoric 4y ago
    > Famously just json deserialization can be rather painful in statically typed languages, especially ones without complex compile time features. That's funny, because last time I had to write json-related code in Python, I really missed Rust's serde, which makes deserialization + validation much easier to write in Rust than in Python. YMMV, of course.
    - hewlett 4y ago
      Pydantic is that for Python
      - Yoric 4y ago
        Thanks, you remind me why I wanted to try Pydantic :)
- pjc50 4y ago
  I think from the perspective that "infrastructure" code being that which rummages around in objects, where the only distinction that matters being dict vs primitive.
- animuchan 4y ago
  Yup! For me it's precisely the opposite of that, the deeper in the systems code, the more types I write. Same with JS, the bulk of libraries IMO should be written in TypeScript.
- mandevil 4y ago
  Not the writer, but based on what the author wrote, I think it's based on the author's experience with Java. It can be a hideous nightmare of java.util.Date versus java.sql.Date versus java.time.LocalDate (just to pick one very well known fiasco in the language- Java defenders please don't come after me) and all of them have to exist to avoid breaking things but you have to have different ways to handle each of them, so dealing at the boundaries between business logic and "infrastructure" is always a mess of converting types- oh this library is old enough that it needs a utilDate, so I need to convert it back down, then convert it back up at every boundary. By contrast, duck-typing means that the intricacies of which exact type you are deserializing into is a lot less important in python.
  - jjoonathan 4y ago
    Duck Typing always struck me as utterly absurd wishful thinking. Two different types built for the same task will almost certainly not happen to be API compatible. Am I missing something?
    - hobs 4y ago
      No, in my experience you are trading off ease of coding for order of magnitude more production bugs.
      - mharig 4y ago
        So you do not write tests?
        hobs 4y ago
        I absolutely do, and one of those effective tests is a simple type annotation.
    - mandevil 4y ago
      I have bounced back and forth between typed and untyped languages in my two decades of employment history, and I prefer typed languages myself, even when I've been writing libraries nested deep in what I think the author would describe as the "infrastructure layer," but this is what I believe the original author of the linked piece is arguing.
- Spivak 4y ago
  I think it would have been better to say "internal library code for foundational projects." That code that takes advantage of all the freedom in Python to mess with types, generate classes on the fly, use type annotations at runtime, add and remove attributes dynamically. Code that is basically a notch away from `eval` (or sometimes literately for dataclasses). There's no way you'll ever be able to fully type this stuff outside of Any Any Any Any. But to be useful, the boundary between your library and it's user should ideally be fully typed so the type checker can help your users use the library correctly.
apetresc 4y ago
> Allow me to propose a model of thinking about code: there's infrastructure code and there's business logic code. > Infrastructure code is exciting, powerful code that exposes easy-to-use interfaces that solve difficult and tricky problems, like talking to browsers (Flask), talking to databases (the Django ORM, SQLAlchemy), dependency injection frameworks (incant), serialization (cattrs) or defining classes (attrs, dataclasses). > Business logic code is boring and unexciting code that enables you to solve problems and finish tickets and sprints at your day job. What a weird take, unironically asserting that the only "exciting" code is the glue, and all of the actual things software does is just an excuse to build more frameworks.
- galdor 4y ago
  If you only worked in companies producing web applications, writing CRUD routes all day is indeed incredibly boring. The glue at least gives you the opportunity to work with a larger range of technical subjects.
- eugenekolo 4y ago
  I found those definitions to be a weird line. But, I believe the overall premise is you use types for large projects that many people will be interfacing with and making small changes to. You go cowboy when it's some fun scripts.
  - kvdveer 4y ago
    This is not limited to type annotations. The same typically applies to comments, documentation, tests, etc. It's just one of the many code quality tools available to you (at a slight cost) , and one typically applies those tools where they have the most benefit: code that is often reused or reread.
- jnwatson 4y ago
  The hardest lesson I’ve learned in turning a programming hobby into a programming career is the following: The vast majority of truly impactful code is really boring at a technical level. So, yes, that’s sometimes why people write frameworks: to get some excitement in their code.
nindalf 4y ago
> I suppose you can look at Rust macros as a different, infrastructure language on top of Rust. That's a strange take, not one I've heard anywhere else. Rust macros are code generators - they're run once at build time and generate regular Rust code that's strongly typed. Maybe it's because the author has a different interpretation of "infrastructure" than the commonly accepted one.
- danpalmer 4y ago
  I can understand this take, Rust macros run at a different time, with different constraints, and are used to solve different problems. In many ways that may feel like a different language. However, a more technically correct interpretation would likely divide based on hygienic vs unhygienic macros. The C Preprocessor is most definitely a different language to C – it has the same differences as Rust/Rust Macros do, but additionally is implemented by a different binary, is de-coupled from the C language to some extent, and most importantly, has different syntax and semantics making it actually a different _language_ in the strict definition of the word.
- TechBro8615 4y ago
  I think the author really means "library code" (i.e. code where the "user" is another developer). He also alludes to such code using metaprogramming which makes internal type hints hard to implement. In this context, macros can be seen analogously to meta programming, or "infrastructure code" to use the terminology of OP.
- pg_1234 4y ago
  As with Rust and macros ... it would be more accurate to say Python is one language that can be used in two ways/modes ... as fits your needs. If anything this is just a sign of a more versatile language.
- ptsneves 4y ago
  Just checked PEP3107 and PEP484 and type hints are not taken into account at runtime[1]. > While these annotations are available at runtime through the usual __annotations__ attribute, no type checking happens at runtime. At first I thought it was disproving the commenter's "strange take" assertion but it actually proves him right. While it is true that python's type hints are a build time feature like in Rust, they do not generate code for the interpreter in the way Rust does. Actually they do not generate anything besides metadata that can be interpreted (at build time) in any way undefined way. The undefined is explicit. I personally liked the article and agree that eventually the undefined specification of the PEP will be removed and a new language a-la Typescript will emerge. [1] https://peps.python.org/pep-0484/ https://peps.python.org/pep-0484/
jmclnx 4y ago
Is that a good idea ? Just look at Perl on why I think this could be concerning. But YMMV, I know very little about Python so I can be persuaded this is no big deal.
- kvdveer 4y ago
  There's still usually only one way to do things in python, there's just the option to annotate it. Annotations are clearly deliniated in th code, and if you don't understand them you can safely ignore them. The author's take that annotated code and unannotated code are different languages is a bit silly. It's like saying code with comments is a different language than code without. While it's true that different camps favor annotations differently, but the same is true for code documentation, and we don't treat it like different languages, even if the comments are in a language you don't speak.
onychomys 4y ago
Just an hour ago I had to downgrade my npm install to build a project from four years ago because some random npm dependency used the print statement from 2 instead of 3. So annoying, especially since I'll need to switch back over to run a more modern app.
- eropple 4y ago
  I've had really good luck with asdf (https://asdf-vm.com https://asdf-vm.com) to manage per-project versions to avoid this sort of thing.
  - lanstin 4y ago
    Weird, there are two package management things called asdf. Here is the lisp one: https://asdf.common-lisp.dev/ https://asdf.common-lisp.dev/
  - jdxcode 4y ago
    also check out rtx (my project), which does everything asdf does but 20x-200x faster and with better UX https://rtx.pub https://rtx.pub
- jimmaswell 4y ago
  I'm still mad about this disaster of a language upgrade. I want my dumb byte strings and convenient print statement back.
  - nick238 4y ago
    Having print as a function is actually useful if you want to use it like any other function, e.g. `def whatever(x, y, z, , output=print):` then you could use the default or put in whatever that's like `print`. I also don't get what you mean by "dumb byte strings", it's just that the default* string type, i.e. when you have a literal `"foo"` in source, it's a sequence of codepoints vs. a sequence of octets. Assuming file systems use UTF-8 was broken in very early 3.x (I think before 3.3?) but it's something I've never struggled with (pure ASCII filenames for me). Forcing the programmer to know if they're dealing with characters & code points vs. bytes/structs/network buffers is a good thing, IMHO.
INTPenis 4y ago
I dunno about that. I've been sneaking type hints into my code bit by bit, so is my program polyglot then? :D The author means that in the sense of there being two distinct camps in the Python community, but using type hints is not exclusive in any way.
- hgs3 4y ago
  > I've been sneaking type hints into my code bit by bit, so is my program polyglot then? Gradually typed [1] is the recognized term. [1] https://en.wikipedia.org/wiki/Gradual_typing https://en.wikipedia.org/wiki/Gradual_typing
graderjs 4y ago
I think this is a great write up of the difference of two types of code, and I think it could apply to JS and TS.
deleted 4y ago
[deleted]
xhkkffbf 4y ago
Just two? I've gotten used to the fact that there are just enough breaking changes between the versions that each major release is different. Now if we add another axis that measures the number of type hints, that's twice as many languages. That's easily getting close to 20. Luckily there's plenty of overlap in the syntax.
bjt2n3904 4y ago
This is an interesting argument that Python can be both flexible and strict. I think the author made a good argument here that this is more of a benefit to the language than a risk. The problem I have currently with Python is that all these options are increasing the number of "pythonic" ways to do something, and making the language more complicated. Python was exciting because it was quick to write, and elegantly simple. It didn't make exciting and risky choices on syntactic sugar -- it took a moderate approach. But I get the impression that Python is trying to evolve into more and more use cases that it isn't well suited for. I think this runs the real risk of becoming too generalized -- doing many things, but not doing them well.
djmax 4y ago
My concern about this is that typed Python doesn't seem nearly as much of a feat of engineering as Typescript is. It seems like a "simplistic" typing system as opposed to a fully thought out compile-time programming language (Rust probably also is a fully thought out one, but Rust doesn't have the untyped equivalent).
kissgyorgy 4y ago
Good thing is that you don't have to do anything to combine them. If you use types, they will help you, if not, business as before. Same with library code, it's just nice to have annotations, so you don't need to look up documentation all the time.
zitterbewegung 4y ago
I think that adding an optional type system while allows you to write python in two different ways or "languages" it also allows you to incrementally make your application be eventually transitioned. You would first add types to things that are obvious as you implement features or fix bugs and leave the rest to Any and then add more complex types or even construct types for your application.
- hbrn 4y ago
  "Transitioning" an incredibly stupid goal, which unfortunately is becoming trendy among engineers who have nothing better to do. Typed Python is not better or worse than untyped Python, it has different tradeoffs. If you're setting a goal to "transition" your application, it means you don't understand what those tradeoffs are. You have a non-nuanced thinking is "types equals good" or "new equals good". > then add more complex types or even construct types for your application In other words, your intention is to add complexity to your application, while preserving it's functionality. Neither engineering nor business as a whole is going to benefit from additional complexity. The "edge case" where complexity can be beneficial is where the ratio of library maintainers to library users differs by orders of magnitude. E.g. if you have a library with 5 maintainers and 1,000,000 users, then types can be a net positive.
travisd 4y ago
The author just spends the final paragraphs… guessing? > TypeScript… JavaScript… Going to guess it's similar. > I haven't touched Java for almost a decade > I don't really know enough Rust to be able to comment with any confidence
- LadyCailin 4y ago
  If the only language you know is Python, your opinion on type systems is… not particularly interesting. No offense to the author.
  - john-radio 4y ago
    Well, the OP blog says the author used to know Java well, just hasn't worked with it for a long time.
danpalmer 4y ago
Perhaps a different way of looking at this: there exists a boundary between first party code and 3rd party code, in that most developers do not actively maintain the 3rd party code they use, and interact with it via a documented API boundary. Types at this boundary are valuable as another form of documentation with automation of enforcement.
patagonia 4y ago
ELI5: The difficulty and resources involved in adding explicit types to the language aside, besides a little more typing, what added burden is placed on the developer? It seems to me that the improvements to performance, code correctness, and code readability farrrr outweigh the cost of the added keystrokes.
- Bedon292 4y ago
  I have had times where I have had to spend a great deal of effort to make the type checker happy. Finding exactly where to import the correct type from the library you are interacting with can be quite annoying at points. And it can be painful making sure everything lines up perfectly so the code (which is already well tested and functionally correct) can make it through the type checker without having to just `# type: ignore` or use `Any`. I have had a few times where it took significantly longer to get the types working than the actual code.
- Myrmornis 4y ago
  I like typed Python, but it's not as simple as just adding types with a few keystrokes. It is a rabbit hole: if you're typing simple types, you will eventually want to type more complex types: functions, iterables, generators, use type variables, etc (Remember what the difference is between an iterable and an iterator? You'll need to.) And Python's type system is its own thing, with its own idiosyncracies. Furthermore, invoking the typechecker itself has complexities: are you doing it in an IDE, or as part of a pre-commit hook, or CI test? The default mypy options aren't great; there's one called `--check-untyped-defs` that IMO you should usually use. Anyway, I am not an expert and not currently using it professionally, but my point is it's much more complex than you're suggesting.
  - Hasnep 4y ago
    I've had lots of success using pyright [1] for Python projects, it has sensible defaults and can be configured with a pyproject.toml file so everyone's using the same settings. I use the Pylance VSCode extension to catch errors earlier, but I also put it in pre-commit and as a CI check, so all contributors are committing the same quality of typed code. With more complex types, I've found it isn't necessary to do anything more complex than specifying the element type of a list or dict most of the time. I have typehinted lambda function arguments before, but just using the plain Callable typehint is usually enough. When I forget if I need to use an Iterator or an Iterable (which is every time), then I just try one and run the type checker and change it to the other if i guessed incorrectly. Type checking Python can become complex if you expect to be able to express everything in the typehints, but Python's type system isn't powerful enough for that. I feel its strength is that it lets you add as much detail to the types as is convenient. [1] https://github.com/microsoft/pyright https://github.com/microsoft/pyright
woeirua 4y ago
The line of reasoning here seems completely backwards. Type hinting should be used in libraries and other pieces of code where type ambiguity can lead to unexpected results. In my experience, that tends to be in "infrastructure" like pieces of code. I think the author confuses the fact that many "infrastructure" libraries are relatively old, and thereby existed _before_ type hinting existed with some kind of conscious choice by the developers.
- Bedon292 4y ago
  And in my experience, for most of those cases there are additional type libraries that correspond with them and it just needs to be installed separately if you want to use the types.
- hbrn 4y ago
  This, 100%. Infrastructure by definition is supposed to be stable with well established interfaces, this is a perfect match for types. Business logic is constantly evolving, focusing too much on types will cause you to be disconnected from business value and you'll just waste time fighting typing system.
Bedon292 4y ago
Going in I was expecting a different take. For me type hints are great and just as important for the 'infrastructure' as they are for the 'business logic'. My default is using them everywhere possible now. So I don't really see a difference there. I see a potential separation as more: Application vs Data Science. Someone purely writing Python inside Jupyter notebooks for data science purposes tends to be writing a very different flavor of Python than I do while writing a web application. As least in my experience there is less concern for formatting guides, idiomatic Python, PEPs, type hinting, or things along those lines. And that is totally fine, its not something they need to really be concerned within that context. It does result in very different outputs though.
- ActorNightly 4y ago
  > It does result in very different outputs though. Not really. The thing with typing is that it gets hyped, but in practice, it matters very little. Both science code and web application code don't really need typing or really type hints. If you go back to your CS professors and ask them to explain what is the advantage to types and OOP, the overarching reason would be that they create contracts in your code that reduce the risk of errors. In practice, if you look at well set up deployment workflow and pipelines, you still have unit tests/integration test that aim to have 96%-99% code coverage with sufficient test coverage, and people don't realize that it makes all the typing system moot. You can write equivalent code in any language and make it fully correct solely by ensuring that unit tests and integration tests for your usecase pass. There is value however in extremely strong typing, where you can essentially do static proofs on the codebase to ensure correctness without running it.
  - Gunax 4y ago
    Unit tests might be exhaustive in 'lines of code covered' but never in 'possible inputs'. That is, did you remember to check for null? Did you remember to write a test for floating points in languages that just have generic number types? I don't see the value in relying on unit tests to enforce types. It might be possible, but it relies on the programmer to: 1. Remember all of the types (was this an int? Or a float? Or a int | null union? 2. Implement exhaustive tests for each type 3. Remember to update the tests when something changes. 4. When reading code, if the types emitted are not specified, having to examine the unit tests to determine the type limits. And the advantage of this is that it saves me a few keystrokes so i can write: X = 5 Instead of int X = 5
satisfice 4y ago
“Everyone doing Python is aware…” No. Neither me, nor anyone I’ve been helping with Python has been aware of type hints. You can do a lot of productive work with a language without knowing everything about it. It might be safe to say everyone working with Python is aware of indentation, though. Sometimes the disconnect between what people in the field are doing and what bloggers say is really striking.
- japhib 4y ago
  This is very surprising to me. Type hints have been a feature of Python since 2015. How are you, and everyone you know, unaware of such a large language feature? I'm guessing you (and the "people you've been helping") either don't use Python that much, or literally do not care one single bit about any of the new features that have happened in the past almost decade. I'm sure such people exist. I'm just surprised to see them on Hacker News.
  - Laaas 4y ago
    They might be using Python 2.
    - tomwojcik 4y ago
      Which would be even more surprising
      - DougMerritt 4y ago
        A surprising number of people have never been happy with Python 3 and even say it was a mistake. Whether you hear this attitude probably just depends on who you're around.
        what_ever 4y ago
        If someone is doing that as an informed decision, they would have heard about typing in Python.
dagw 4y ago
I kind of agree with the title, but for very different reasons. I think the python world as bifurcated into two camps, those programs that always start with something like import numpy import pandas ... and those that don't. Everything the follows from there might as well be two different languages. I write some 'python' code that starts with "import django" and some code that starts with "import numpy", and they're completely different in both structure and semantics. Being familiar in one style won't really help you much in understanding the other style.
kgbcia 4y ago
I was thinking more of numpy arrays versus python arrays.

I get that the author is asserting that untyped Pythong is superior for infrastructure code but I don't understand why the author thinks this.

sdfghswe 4y ago
It's funny that you typoed Python as Pythong. I do that a lot - in my case I know it's faulty muscle memory, but I don't know where it comes from.
- mikece 4y ago
  And to my chagrin I didn't realize it until AFTER the time window for editing a comment passed. Oh well.
stu2b50 4y ago
I’m not sure I understand the “infrastracture” label, but certainly for the cases the author listed as “infra” typing can be cumbersome. Famously just json deserialization can be rather painful in statically typed languages, especially ones without complex compile time features.
- eropple 4y ago
  Serialization/deserialization can be really gross in something like Java, but TypeScript gives you some really good tools (zod, typebox, etc.) to validate and type-guard your inputs that I'm sure could be implemented in Python--if they're not already.
  - pbalau 4y ago
    Pydantic will do that for Python.
    - eropple 4y ago
      Yeah, I assumed as much. I've only looked at Pydantic in the context of FastAPI, as Python's not really my thing, but it looks really well-thought-out.
  - amarant 4y ago
    If serialisation is painful in Java, you're doing it wrong. Libraries like Jackson make it entirely painless. I would assume JSON parsing is a solved problem in any typed language.
    - eropple 4y ago
      I've written a lot of code using Jackson and I strongly disagree that it's a solved problem. If you're Java on both ends, it's probably fine, but then your serde style is an implementation detail. And that's great, but if you're not, you are in a bear pit. The lingua franca for JSON specification is JSON Schema, and there is significant impedance mismatch between the standard JSON tools folks use in Java and with JSON Schema as used by most other folks. (Jackson has a nice JSON Schema module for exporting it, but importing/codegen is pretty gross.)
  - sixbrx 4y ago
    I find Java pretty nice for handling JSON serialization/deserialization these days. Just define some records for the expected structure, and Jackson's object mapper handles the rest.
    - eropple 4y ago
      It's the "defining some records" part that's slow and gritty. JSON is usually an interchange format, and Java class construction is either slow, tedious, and manual, or involves massaging a code generator (which, at least the last time I checked a few months ago, weren't very good in Java).
      - vips7L 4y ago
        I don’t see how declaring a data model class is anymore difficult in Java than another typed language. record Thing(String name, int size, LocalDate when) {} The only way I could see this being easier is to forgo type safety completely.
        eropple 4y ago
        You're assuming you only have to define it in one language. JSON is an interchange format. The canonical way to express it is JSON Schema, and the tools are unpleasant in Java. So is dealing with structural typing of polymorphic responses. In an ideal world we wouldn't have those, but we live in this one. These things are just better when you aren't in a nominatively typed language with some pretty crusty assumptions.
- marcosdumay 4y ago
  Hum... Almost all of the problems of serialization/deserialization in statically typed languages turn into testing problems in dynamically typed languages. That's not a gain in any way.
  - stu2b50 4y ago
    In this case it’s more about shifting where the static typing occurs, since the author is still arguing for a typed business layer. Since I/O is usually untyped, you need to cram it into a typed schema somewhere. In strictly typed languages, that must occur upon ingestion. The author is saying you can delay it by having a layer on top of the I/O transform the data in untyped fashion before being typed later on.
    - Yoric 4y ago
      What's the benefit of delaying typing? By typing it at ingestion, you guarantee that invalid (with respect to structure) untrusted data will not break your code down the line. Doing otherwise means that you're manipulating untrusted data with a strong risk of forgetting that it is untrusted. I have seen large applications (think hundreds of millions of daily users) break because of such oopses.
- Yoric 4y ago
  > Famously just json deserialization can be rather painful in statically typed languages, especially ones without complex compile time features. That's funny, because last time I had to write json-related code in Python, I really missed Rust's serde, which makes deserialization + validation much easier to write in Rust than in Python. YMMV, of course.
  - hewlett 4y ago
    Pydantic is that for Python
    - Yoric 4y ago
      Thanks, you remind me why I wanted to try Pydantic :)
pjc50 4y ago
I think from the perspective that "infrastructure" code being that which rummages around in objects, where the only distinction that matters being dict vs primitive.
animuchan 4y ago
Yup! For me it's precisely the opposite of that, the deeper in the systems code, the more types I write. Same with JS, the bulk of libraries IMO should be written in TypeScript.
mandevil 4y ago
Not the writer, but based on what the author wrote, I think it's based on the author's experience with Java. It can be a hideous nightmare of java.util.Date versus java.sql.Date versus java.time.LocalDate (just to pick one very well known fiasco in the language- Java defenders please don't come after me) and all of them have to exist to avoid breaking things but you have to have different ways to handle each of them, so dealing at the boundaries between business logic and "infrastructure" is always a mess of converting types- oh this library is old enough that it needs a utilDate, so I need to convert it back down, then convert it back up at every boundary. By contrast, duck-typing means that the intricacies of which exact type you are deserializing into is a lot less important in python.
- jjoonathan 4y ago
  Duck Typing always struck me as utterly absurd wishful thinking. Two different types built for the same task will almost certainly not happen to be API compatible. Am I missing something?
  - hobs 4y ago
    No, in my experience you are trading off ease of coding for order of magnitude more production bugs.
    - mharig 4y ago
      So you do not write tests?
      - hobs 4y ago
        I absolutely do, and one of those effective tests is a simple type annotation.
  - mandevil 4y ago
    I have bounced back and forth between typed and untyped languages in my two decades of employment history, and I prefer typed languages myself, even when I've been writing libraries nested deep in what I think the author would describe as the "infrastructure layer," but this is what I believe the original author of the linked piece is arguing.
Spivak 4y ago
I think it would have been better to say "internal library code for foundational projects." That code that takes advantage of all the freedom in Python to mess with types, generate classes on the fly, use type annotations at runtime, add and remove attributes dynamically. Code that is basically a notch away from `eval` (or sometimes literately for dataclasses). There's no way you'll ever be able to fully type this stuff outside of Any Any Any Any. But to be useful, the boundary between your library and it's user should ideally be fully typed so the type checker can help your users use the library correctly.

apetresc 4y ago

> Allow me to propose a model of thinking about code: there's infrastructure code and there's business logic code. > Infrastructure code is exciting, powerful code that exposes easy-to-use interfaces that solve difficult and tricky problems, like talking to browsers (Flask), talking to databases (the Django ORM, SQLAlchemy), dependency injection frameworks (incant), serialization (cattrs) or defining classes (attrs, dataclasses). > Business logic code is boring and unexciting code that enables you to solve problems and finish tickets and sprints at your day job. What a weird take, unironically asserting that the only "exciting" code is the glue, and all of the actual things software does is just an excuse to build more frameworks.

galdor 4y ago
If you only worked in companies producing web applications, writing CRUD routes all day is indeed incredibly boring. The glue at least gives you the opportunity to work with a larger range of technical subjects.
eugenekolo 4y ago
I found those definitions to be a weird line. But, I believe the overall premise is you use types for large projects that many people will be interfacing with and making small changes to. You go cowboy when it's some fun scripts.
- kvdveer 4y ago
  This is not limited to type annotations. The same typically applies to comments, documentation, tests, etc. It's just one of the many code quality tools available to you (at a slight cost) , and one typically applies those tools where they have the most benefit: code that is often reused or reread.
jnwatson 4y ago
The hardest lesson I’ve learned in turning a programming hobby into a programming career is the following: The vast majority of truly impactful code is really boring at a technical level. So, yes, that’s sometimes why people write frameworks: to get some excitement in their code.

nindalf 4y ago

> I suppose you can look at Rust macros as a different, infrastructure language on top of Rust. That's a strange take, not one I've heard anywhere else. Rust macros are code generators - they're run once at build time and generate regular Rust code that's strongly typed. Maybe it's because the author has a different interpretation of "infrastructure" than the commonly accepted one.

danpalmer 4y ago
I can understand this take, Rust macros run at a different time, with different constraints, and are used to solve different problems. In many ways that may feel like a different language. However, a more technically correct interpretation would likely divide based on hygienic vs unhygienic macros. The C Preprocessor is most definitely a different language to C – it has the same differences as Rust/Rust Macros do, but additionally is implemented by a different binary, is de-coupled from the C language to some extent, and most importantly, has different syntax and semantics making it actually a different _language_ in the strict definition of the word.
TechBro8615 4y ago
I think the author really means "library code" (i.e. code where the "user" is another developer). He also alludes to such code using metaprogramming which makes internal type hints hard to implement. In this context, macros can be seen analogously to meta programming, or "infrastructure code" to use the terminology of OP.
pg_1234 4y ago
As with Rust and macros ... it would be more accurate to say Python is one language that can be used in two ways/modes ... as fits your needs. If anything this is just a sign of a more versatile language.
ptsneves 4y ago
Just checked PEP3107 and PEP484 and type hints are not taken into account at runtime[1]. > While these annotations are available at runtime through the usual __annotations__ attribute, no type checking happens at runtime. At first I thought it was disproving the commenter's "strange take" assertion but it actually proves him right. While it is true that python's type hints are a build time feature like in Rust, they do not generate code for the interpreter in the way Rust does. Actually they do not generate anything besides metadata that can be interpreted (at build time) in any way undefined way. The undefined is explicit. I personally liked the article and agree that eventually the undefined specification of the PEP will be removed and a new language a-la Typescript will emerge. [1] https://peps.python.org/pep-0484/ https://peps.python.org/pep-0484/

jmclnx 4y ago

Is that a good idea ? Just look at Perl on why I think this could be concerning. But YMMV, I know very little about Python so I can be persuaded this is no big deal.

kvdveer 4y ago
There's still usually only one way to do things in python, there's just the option to annotate it. Annotations are clearly deliniated in th code, and if you don't understand them you can safely ignore them. The author's take that annotated code and unannotated code are different languages is a bit silly. It's like saying code with comments is a different language than code without. While it's true that different camps favor annotations differently, but the same is true for code documentation, and we don't treat it like different languages, even if the comments are in a language you don't speak.

onychomys 4y ago

Just an hour ago I had to downgrade my npm install to build a project from four years ago because some random npm dependency used the print statement from 2 instead of 3. So annoying, especially since I'll need to switch back over to run a more modern app.

eropple 4y ago
I've had really good luck with asdf (https://asdf-vm.com https://asdf-vm.com) to manage per-project versions to avoid this sort of thing.
- lanstin 4y ago
  Weird, there are two package management things called asdf. Here is the lisp one: https://asdf.common-lisp.dev/ https://asdf.common-lisp.dev/
- jdxcode 4y ago
  also check out rtx (my project), which does everything asdf does but 20x-200x faster and with better UX https://rtx.pub https://rtx.pub
jimmaswell 4y ago
I'm still mad about this disaster of a language upgrade. I want my dumb byte strings and convenient print statement back.
- nick238 4y ago
  Having print as a function is actually useful if you want to use it like any other function, e.g. `def whatever(x, y, z, , output=print):` then you could use the default or put in whatever that's like `print`. I also don't get what you mean by "dumb byte strings", it's just that the default* string type, i.e. when you have a literal `"foo"` in source, it's a sequence of codepoints vs. a sequence of octets. Assuming file systems use UTF-8 was broken in very early 3.x (I think before 3.3?) but it's something I've never struggled with (pure ASCII filenames for me). Forcing the programmer to know if they're dealing with characters & code points vs. bytes/structs/network buffers is a good thing, IMHO.

INTPenis 4y ago

I dunno about that. I've been sneaking type hints into my code bit by bit, so is my program polyglot then? :D The author means that in the sense of there being two distinct camps in the Python community, but using type hints is not exclusive in any way.

hgs3 4y ago
> I've been sneaking type hints into my code bit by bit, so is my program polyglot then? Gradually typed [1] is the recognized term. [1] https://en.wikipedia.org/wiki/Gradual_typing https://en.wikipedia.org/wiki/Gradual_typing

graderjs 4y ago

I think this is a great write up of the difference of two types of code, and I think it could apply to JS and TS.

deleted 4y ago

[deleted]

xhkkffbf 4y ago

Just two? I've gotten used to the fact that there are just enough breaking changes between the versions that each major release is different. Now if we add another axis that measures the number of type hints, that's twice as many languages. That's easily getting close to 20. Luckily there's plenty of overlap in the syntax.

bjt2n3904 4y ago

This is an interesting argument that Python can be both flexible and strict. I think the author made a good argument here that this is more of a benefit to the language than a risk. The problem I have currently with Python is that all these options are increasing the number of "pythonic" ways to do something, and making the language more complicated. Python was exciting because it was quick to write, and elegantly simple. It didn't make exciting and risky choices on syntactic sugar -- it took a moderate approach. But I get the impression that Python is trying to evolve into more and more use cases that it isn't well suited for. I think this runs the real risk of becoming too generalized -- doing many things, but not doing them well.

djmax 4y ago

My concern about this is that typed Python doesn't seem nearly as much of a feat of engineering as Typescript is. It seems like a "simplistic" typing system as opposed to a fully thought out compile-time programming language (Rust probably also is a fully thought out one, but Rust doesn't have the untyped equivalent).

kissgyorgy 4y ago

Good thing is that you don't have to do anything to combine them. If you use types, they will help you, if not, business as before. Same with library code, it's just nice to have annotations, so you don't need to look up documentation all the time.

zitterbewegung 4y ago

I think that adding an optional type system while allows you to write python in two different ways or "languages" it also allows you to incrementally make your application be eventually transitioned. You would first add types to things that are obvious as you implement features or fix bugs and leave the rest to Any and then add more complex types or even construct types for your application.

hbrn 4y ago
"Transitioning" an incredibly stupid goal, which unfortunately is becoming trendy among engineers who have nothing better to do. Typed Python is not better or worse than untyped Python, it has different tradeoffs. If you're setting a goal to "transition" your application, it means you don't understand what those tradeoffs are. You have a non-nuanced thinking is "types equals good" or "new equals good". > then add more complex types or even construct types for your application In other words, your intention is to add complexity to your application, while preserving it's functionality. Neither engineering nor business as a whole is going to benefit from additional complexity. The "edge case" where complexity can be beneficial is where the ratio of library maintainers to library users differs by orders of magnitude. E.g. if you have a library with 5 maintainers and 1,000,000 users, then types can be a net positive.

travisd 4y ago

The author just spends the final paragraphs… guessing? > TypeScript… JavaScript… Going to guess it's similar. > I haven't touched Java for almost a decade > I don't really know enough Rust to be able to comment with any confidence

LadyCailin 4y ago
If the only language you know is Python, your opinion on type systems is… not particularly interesting. No offense to the author.
- john-radio 4y ago
  Well, the OP blog says the author used to know Java well, just hasn't worked with it for a long time.

danpalmer 4y ago

Perhaps a different way of looking at this: there exists a boundary between first party code and 3rd party code, in that most developers do not actively maintain the 3rd party code they use, and interact with it via a documented API boundary. Types at this boundary are valuable as another form of documentation with automation of enforcement.

patagonia 4y ago

ELI5: The difficulty and resources involved in adding explicit types to the language aside, besides a little more typing, what added burden is placed on the developer? It seems to me that the improvements to performance, code correctness, and code readability farrrr outweigh the cost of the added keystrokes.

Bedon292 4y ago
I have had times where I have had to spend a great deal of effort to make the type checker happy. Finding exactly where to import the correct type from the library you are interacting with can be quite annoying at points. And it can be painful making sure everything lines up perfectly so the code (which is already well tested and functionally correct) can make it through the type checker without having to just `# type: ignore` or use `Any`. I have had a few times where it took significantly longer to get the types working than the actual code.
Myrmornis 4y ago
I like typed Python, but it's not as simple as just adding types with a few keystrokes. It is a rabbit hole: if you're typing simple types, you will eventually want to type more complex types: functions, iterables, generators, use type variables, etc (Remember what the difference is between an iterable and an iterator? You'll need to.) And Python's type system is its own thing, with its own idiosyncracies. Furthermore, invoking the typechecker itself has complexities: are you doing it in an IDE, or as part of a pre-commit hook, or CI test? The default mypy options aren't great; there's one called `--check-untyped-defs` that IMO you should usually use. Anyway, I am not an expert and not currently using it professionally, but my point is it's much more complex than you're suggesting.
- Hasnep 4y ago
  I've had lots of success using pyright [1] for Python projects, it has sensible defaults and can be configured with a pyproject.toml file so everyone's using the same settings. I use the Pylance VSCode extension to catch errors earlier, but I also put it in pre-commit and as a CI check, so all contributors are committing the same quality of typed code. With more complex types, I've found it isn't necessary to do anything more complex than specifying the element type of a list or dict most of the time. I have typehinted lambda function arguments before, but just using the plain Callable typehint is usually enough. When I forget if I need to use an Iterator or an Iterable (which is every time), then I just try one and run the type checker and change it to the other if i guessed incorrectly. Type checking Python can become complex if you expect to be able to express everything in the typehints, but Python's type system isn't powerful enough for that. I feel its strength is that it lets you add as much detail to the types as is convenient. [1] https://github.com/microsoft/pyright https://github.com/microsoft/pyright

woeirua 4y ago

The line of reasoning here seems completely backwards. Type hinting should be used in libraries and other pieces of code where type ambiguity can lead to unexpected results. In my experience, that tends to be in "infrastructure" like pieces of code. I think the author confuses the fact that many "infrastructure" libraries are relatively old, and thereby existed _before_ type hinting existed with some kind of conscious choice by the developers.

Bedon292 4y ago
And in my experience, for most of those cases there are additional type libraries that correspond with them and it just needs to be installed separately if you want to use the types.
hbrn 4y ago
This, 100%. Infrastructure by definition is supposed to be stable with well established interfaces, this is a perfect match for types. Business logic is constantly evolving, focusing too much on types will cause you to be disconnected from business value and you'll just waste time fighting typing system.

Bedon292 4y ago

Going in I was expecting a different take. For me type hints are great and just as important for the 'infrastructure' as they are for the 'business logic'. My default is using them everywhere possible now. So I don't really see a difference there. I see a potential separation as more: Application vs Data Science. Someone purely writing Python inside Jupyter notebooks for data science purposes tends to be writing a very different flavor of Python than I do while writing a web application. As least in my experience there is less concern for formatting guides, idiomatic Python, PEPs, type hinting, or things along those lines. And that is totally fine, its not something they need to really be concerned within that context. It does result in very different outputs though.

ActorNightly 4y ago
> It does result in very different outputs though. Not really. The thing with typing is that it gets hyped, but in practice, it matters very little. Both science code and web application code don't really need typing or really type hints. If you go back to your CS professors and ask them to explain what is the advantage to types and OOP, the overarching reason would be that they create contracts in your code that reduce the risk of errors. In practice, if you look at well set up deployment workflow and pipelines, you still have unit tests/integration test that aim to have 96%-99% code coverage with sufficient test coverage, and people don't realize that it makes all the typing system moot. You can write equivalent code in any language and make it fully correct solely by ensuring that unit tests and integration tests for your usecase pass. There is value however in extremely strong typing, where you can essentially do static proofs on the codebase to ensure correctness without running it.
- Gunax 4y ago
  Unit tests might be exhaustive in 'lines of code covered' but never in 'possible inputs'. That is, did you remember to check for null? Did you remember to write a test for floating points in languages that just have generic number types? I don't see the value in relying on unit tests to enforce types. It might be possible, but it relies on the programmer to: 1. Remember all of the types (was this an int? Or a float? Or a int | null union? 2. Implement exhaustive tests for each type 3. Remember to update the tests when something changes. 4. When reading code, if the types emitted are not specified, having to examine the unit tests to determine the type limits. And the advantage of this is that it saves me a few keystrokes so i can write: X = 5 Instead of int X = 5

satisfice 4y ago

“Everyone doing Python is aware…” No. Neither me, nor anyone I’ve been helping with Python has been aware of type hints. You can do a lot of productive work with a language without knowing everything about it. It might be safe to say everyone working with Python is aware of indentation, though. Sometimes the disconnect between what people in the field are doing and what bloggers say is really striking.

japhib 4y ago
This is very surprising to me. Type hints have been a feature of Python since 2015. How are you, and everyone you know, unaware of such a large language feature? I'm guessing you (and the "people you've been helping") either don't use Python that much, or literally do not care one single bit about any of the new features that have happened in the past almost decade. I'm sure such people exist. I'm just surprised to see them on Hacker News.
- Laaas 4y ago
  They might be using Python 2.
  - tomwojcik 4y ago
    Which would be even more surprising
    - DougMerritt 4y ago
      A surprising number of people have never been happy with Python 3 and even say it was a mistake. Whether you hear this attitude probably just depends on who you're around.
      - what_ever 4y ago
        If someone is doing that as an informed decision, they would have heard about typing in Python.

dagw 4y ago

I kind of agree with the title, but for very different reasons. I think the python world as bifurcated into two camps, those programs that always start with something like import numpy import pandas ... and those that don't. Everything the follows from there might as well be two different languages. I write some 'python' code that starts with "import django" and some code that starts with "import numpy", and they're completely different in both structure and semantics. Being familiar in one style won't really help you much in understanding the other style.

kgbcia 4y ago

I was thinking more of numpy arrays versus python arrays.