8 ms·

I am an OOP programmer going back to the late 80s (including the cfront days of C++), and a serious user of Python since 2007. In Python, I sometimes try data-

I am an OOP programmer going back to the late 80s (including the cfront days of C++), and a serious user of Python since 2007. In Python, I sometimes try data-oriented programming, using lists and dicts to structure data. And I find that it does not work well. Once I get two or more levels of nesting, I find it far too easy to get confused about which level I'm on, which is not helped by Python's lack of strong typing. In these situations, I often introduce objects that wrap the map or dict, and have methods that make sense for that level. In other words, the objects can be viewed as providing clear documentation for the whole nested structure, and how it can be navigated.

deleted 2y ago
[deleted]
goostavos 2y ago
>Once I get two or more levels of nesting, I find it far too easy to get confused about which level I'm on Author here, I agree with you. I have the working memory of a small pigeon. The flavor of data orientation we cover in the book leverages strongly typed representations of data (as opposed to using hash maps everywhere). So you'll always know what's shape it's in (and the compiler enforces it!). We spend a lot of time exploring the role that the type system can play in our programming and how we represent data.
- joshlemer 2y ago
  Given the strongly typed flavour of data oriented programming, I wonder if you have any thoughts on the "proliferation of types" problem. How to avoid, especially in a nominally typed language like Java, an explosion of aggregate types for every context where there may be a slight change in what fields are present, what their types are, and which ones are optional. Basically, Rich Hickey's Maybe Not talk. record Make(makeId, name) record Model(modelId, name) record Car(make, model, year) record Car(makeId, modelId, year) record Car(make, model) record Car(makeId, modelId) record Car(make, year) record Car(makeId, year) record Car(make, model, year, colour) record Car(makeId, modelId, year, colour) record Car(year, colour) ....
  - nicoty 2y ago
    I'm not familiar with Java. Does it have no notion of structural types at all? If it does, maybe you could wrap those fields in `Car` with `Maybe`/`Option` (I’m not sure what the equivalent is in Java) so you get something like `Car(Maybe Make, Maybe Model, Maybe Year, Maybe Colour)`?
    - spullara 2y ago
      yes and it is called Optional (rather than Maybe)
    - vips7L 2y ago
      Records are structural types. Null restricted types are in draft: https://openjdk.org/jeps/8303099 https://openjdk.org/jeps/8303099
      - htuyfjsdkfw 2y ago
        Records in Java are nominal. In fact, it is syntax sugar for a class.
  - goostavos 2y ago
    I have a long convoluted answer to this. I love that talk (and most of Rich's stuff). I consider myself a Clojure fanboy that got converted to the dark side of strong static typing. I think, to some degree, he actually answers that question as part of his talk (in between beating up nominal types). Optionality often pops up in place of understanding (or representing) that data has a context. If you model your program so that it has "15 maybe sheep," then... you'll have 15 "maybe sheep" you've got to deal with. The possible combinations of all data types that could be made is very different from the subset that actually express themselves in our programs. Meaning, the actual "explosion" is fairly constrained in practice because (most) businesses can't function under combinatorial pressures. There's some stuff that matters, and some stuff that doesn't. We only have to apply typing rigor to the stuff that matters. Where I do find type explosions tedious and annoying is not in expressing every possible combination, but in trying to express the slow accretion of information. (I think he talks about this in one of his talks, too). Invoice, then InvoiceWithCustomer, then InvoiceWithCustomerAndId, etc... the world that microservices have doomed us to representing. I don't know a good way to model that without intersection types or something like Rows in purescript. In Java, it's a pain point for sure.
FpUser 2y ago
>"In these situations, I often introduce objects that wrap the map or dict, and have methods that make sense for that level." I've been doing the same thing since the end of the 80s as well starting with Turbo/Borland Pascal, C++, and later any other language that supports OOP.
deleted 2y ago
[deleted]
sodapopcan 2y ago
Python's not really built for that AFAIK, though. In languages built for it, you can type your dicts/hashes/maps/whatever and its easier to see what they are/know where the functions that operate on them live. I'm most familiar with Elixir which has structs which are simply specialized map (analogous to dict in Python) where their "type" is the name of the module they belong to. There can only be one struct per module, In this sense is easy to know exactly where its functions live and is almost like a class with the very key difference that modules are not stateful.
- cbsmith 2y ago
  > In languages built for it, you can type your dicts/hashes/maps/whatever and its easier to see what they are/know where the functions that operate on them live. I think I must be misunderstanding what you mean by that, because I can very much do that in Python.
  - sodapopcan 2y ago
    I think I misunderstand OP's problem then.
    - cbsmith 2y ago
      Their problem stems from the scenario where you don't type them. You just leave them as generic lists & dicts.
      - sodapopcan 2y ago
        That's what I thought. I obviously don't know Python well enough and didn't know you can name dicts (like, beyond setting them to a variable). I guess you can export from a module so they are prefixed! Didn't think of that one earlier.
        cbsmith 2y ago
        Python classes are basically dictionaries that have a distinct type bound to them. Alternatively you can subclass from dictionary to give yourself a distinct type but still be a dictionary. Slotted classes are basically named tuples (and of course, Python has actual named tuples and dataclasses), so there's a lot of ways to "tag" a collection with a specific type in mind.
mejutoco 2y ago
I recommend you use pydantic for type annotations. Alternatively, dataclasses. Then you pair it with typeguards @typechecked annotation and the types will be checked at runtime for each method/function. You can use mypy to check it at "compile time". Having clear data types without oop is possible, even in python.
js2 2y ago
> Python's lack of strong typing I see people conflate strong/weak and static/dynamic quite often. Python is strong[1]/dynamic, with optional static typing through annotations and a type checker (mypy, pyright, etc). Perhaps the easiest way to add static types to data is with pydantic. Here's an example of using pydantic to type-check data provided via an external yaml configuration file: https://news.ycombinator.com/item?id=41508243 https://news.ycombinator.com/item?id=41508243 [1] strong/weak are not strictly defined, as compared to dynamic/static, but Python is absolutely on the strong end of the scale. You'll get a runtime TypeError if you try to add a number to a string, for example, compared to say JavaScript which will happily provide a typically meaningless "wat?"-style result.
- thelastparadise 2y ago
  You're being pydantic =)
- jstimpfle 2y ago
  In some significant ways, it's not strong at all. It's stronger than Javascript but it's difficult not to be. Python is a duck typing language for the most part.
  - htuyfjsdkfw 2y ago
    Duck typing doesn't exist. What you refer to as duck typing is the inherit nature of dynamic typing.
  - js2 2y ago
    Duck typing is an aspect of it being dynamically typed, not whether it is strong/weak. But strong/weak is not formally defined, so if duck typing disqualifies it for you, so be it. https://langdev.stackexchange.com/questions/3741/how-are-strong-and-weak-typing-defined https://langdev.stackexchange.com/questions/3741/how-are-str...
    - cbsmith 2y ago
      I always think of Python as having "fairly strong" typing, because you can override the type of objects by just assigning to __class__.
ederamen 2y ago
Use Data Classes
cbsmith 2y ago
Living this dream in Python right now (inherited a code base that used nasty nesting of lists & dicts). You don't strictly need to do OOP to solve the problem, but it really does help to have a data model. Using dataclasses to map out the data structures makes the code so much more readible, and the support for type hints in Python is good enough that you can even debug problems with the type system.
chipdart 2y ago
> (...) by Python's lack of strong typing. In these situations, I often (...) Python does support strong typing, albeit optional, with type annotations and tools like mypy. If a problem is caused by the lack of strong typing, why not use strong typing?
- baq 2y ago
  static. python was always strongly typed: you could not do 2 + '3', ever. nowadays mypy/pyright will tell you that before runtime, hence static.
  - chipdart 2y ago
    I stand corrected.
ninetyninenine 2y ago
Python now has type hints which can be used with an external type checker in the IDE. You'd probably type it with a class.
clepto 2y ago
I see a lot of people mentioning Pydantic here, but you should take a look into TypedDict. It provides a type structure ontop of a plain dictionary, and sounds like exactly what you’d want, and is a built-in that you don’t need a dependency for. Mypy for example can also see the types of the dictionary are supposed to be when you use it just like a normal dictionary.

deleted 2y ago

[deleted]

goostavos 2y ago

>Once I get two or more levels of nesting, I find it far too easy to get confused about which level I'm on Author here, I agree with you. I have the working memory of a small pigeon. The flavor of data orientation we cover in the book leverages strongly typed representations of data (as opposed to using hash maps everywhere). So you'll always know what's shape it's in (and the compiler enforces it!). We spend a lot of time exploring the role that the type system can play in our programming and how we represent data.

joshlemer 2y ago
Given the strongly typed flavour of data oriented programming, I wonder if you have any thoughts on the "proliferation of types" problem. How to avoid, especially in a nominally typed language like Java, an explosion of aggregate types for every context where there may be a slight change in what fields are present, what their types are, and which ones are optional. Basically, Rich Hickey's Maybe Not talk. record Make(makeId, name) record Model(modelId, name) record Car(make, model, year) record Car(makeId, modelId, year) record Car(make, model) record Car(makeId, modelId) record Car(make, year) record Car(makeId, year) record Car(make, model, year, colour) record Car(makeId, modelId, year, colour) record Car(year, colour) ....
- nicoty 2y ago
  I'm not familiar with Java. Does it have no notion of structural types at all? If it does, maybe you could wrap those fields in `Car` with `Maybe`/`Option` (I’m not sure what the equivalent is in Java) so you get something like `Car(Maybe Make, Maybe Model, Maybe Year, Maybe Colour)`?
  - spullara 2y ago
    yes and it is called Optional (rather than Maybe)
  - vips7L 2y ago
    Records are structural types. Null restricted types are in draft: https://openjdk.org/jeps/8303099 https://openjdk.org/jeps/8303099
    - htuyfjsdkfw 2y ago
      Records in Java are nominal. In fact, it is syntax sugar for a class.
- goostavos 2y ago
  I have a long convoluted answer to this. I love that talk (and most of Rich's stuff). I consider myself a Clojure fanboy that got converted to the dark side of strong static typing. I think, to some degree, he actually answers that question as part of his talk (in between beating up nominal types). Optionality often pops up in place of understanding (or representing) that data has a context. If you model your program so that it has "15 maybe sheep," then... you'll have 15 "maybe sheep" you've got to deal with. The possible combinations of all data types that could be made is very different from the subset that actually express themselves in our programs. Meaning, the actual "explosion" is fairly constrained in practice because (most) businesses can't function under combinatorial pressures. There's some stuff that matters, and some stuff that doesn't. We only have to apply typing rigor to the stuff that matters. Where I do find type explosions tedious and annoying is not in expressing every possible combination, but in trying to express the slow accretion of information. (I think he talks about this in one of his talks, too). Invoice, then InvoiceWithCustomer, then InvoiceWithCustomerAndId, etc... the world that microservices have doomed us to representing. I don't know a good way to model that without intersection types or something like Rows in purescript. In Java, it's a pain point for sure.

FpUser 2y ago

>"In these situations, I often introduce objects that wrap the map or dict, and have methods that make sense for that level." I've been doing the same thing since the end of the 80s as well starting with Turbo/Borland Pascal, C++, and later any other language that supports OOP.

deleted 2y ago

[deleted]

sodapopcan 2y ago

Python's not really built for that AFAIK, though. In languages built for it, you can type your dicts/hashes/maps/whatever and its easier to see what they are/know where the functions that operate on them live. I'm most familiar with Elixir which has structs which are simply specialized map (analogous to dict in Python) where their "type" is the name of the module they belong to. There can only be one struct per module, In this sense is easy to know exactly where its functions live and is almost like a class with the very key difference that modules are not stateful.

cbsmith 2y ago
> In languages built for it, you can type your dicts/hashes/maps/whatever and its easier to see what they are/know where the functions that operate on them live. I think I must be misunderstanding what you mean by that, because I can very much do that in Python.
- sodapopcan 2y ago
  I think I misunderstand OP's problem then.
  - cbsmith 2y ago
    Their problem stems from the scenario where you don't type them. You just leave them as generic lists & dicts.
    - sodapopcan 2y ago
      That's what I thought. I obviously don't know Python well enough and didn't know you can name dicts (like, beyond setting them to a variable). I guess you can export from a module so they are prefixed! Didn't think of that one earlier.
      - cbsmith 2y ago
        Python classes are basically dictionaries that have a distinct type bound to them. Alternatively you can subclass from dictionary to give yourself a distinct type but still be a dictionary. Slotted classes are basically named tuples (and of course, Python has actual named tuples and dataclasses), so there's a lot of ways to "tag" a collection with a specific type in mind.

mejutoco 2y ago

I recommend you use pydantic for type annotations. Alternatively, dataclasses. Then you pair it with typeguards @typechecked annotation and the types will be checked at runtime for each method/function. You can use mypy to check it at "compile time". Having clear data types without oop is possible, even in python.

js2 2y ago

> Python's lack of strong typing I see people conflate strong/weak and static/dynamic quite often. Python is strong[1]/dynamic, with optional static typing through annotations and a type checker (mypy, pyright, etc). Perhaps the easiest way to add static types to data is with pydantic. Here's an example of using pydantic to type-check data provided via an external yaml configuration file: https://news.ycombinator.com/item?id=41508243 https://news.ycombinator.com/item?id=41508243 [1] strong/weak are not strictly defined, as compared to dynamic/static, but Python is absolutely on the strong end of the scale. You'll get a runtime TypeError if you try to add a number to a string, for example, compared to say JavaScript which will happily provide a typically meaningless "wat?"-style result.

thelastparadise 2y ago
You're being pydantic =)
jstimpfle 2y ago
In some significant ways, it's not strong at all. It's stronger than Javascript but it's difficult not to be. Python is a duck typing language for the most part.
- htuyfjsdkfw 2y ago
  Duck typing doesn't exist. What you refer to as duck typing is the inherit nature of dynamic typing.
- js2 2y ago
  Duck typing is an aspect of it being dynamically typed, not whether it is strong/weak. But strong/weak is not formally defined, so if duck typing disqualifies it for you, so be it. https://langdev.stackexchange.com/questions/3741/how-are-strong-and-weak-typing-defined https://langdev.stackexchange.com/questions/3741/how-are-str...
  - cbsmith 2y ago
    I always think of Python as having "fairly strong" typing, because you can override the type of objects by just assigning to __class__.

ederamen 2y ago

Use Data Classes

cbsmith 2y ago

Living this dream in Python right now (inherited a code base that used nasty nesting of lists & dicts). You don't strictly need to do OOP to solve the problem, but it really does help to have a data model. Using dataclasses to map out the data structures makes the code so much more readible, and the support for type hints in Python is good enough that you can even debug problems with the type system.

chipdart 2y ago

> (...) by Python's lack of strong typing. In these situations, I often (...) Python does support strong typing, albeit optional, with type annotations and tools like mypy. If a problem is caused by the lack of strong typing, why not use strong typing?

baq 2y ago
static. python was always strongly typed: you could not do 2 + '3', ever. nowadays mypy/pyright will tell you that before runtime, hence static.
- chipdart 2y ago
  I stand corrected.

ninetyninenine 2y ago

Python now has type hints which can be used with an external type checker in the IDE. You'd probably type it with a class.

clepto 2y ago

I see a lot of people mentioning Pydantic here, but you should take a look into TypedDict. It provides a type structure ontop of a plain dictionary, and sounds like exactly what you’d want, and is a built-in that you don’t need a dependency for. Mypy for example can also see the types of the dictionary are supposed to be when you use it just like a normal dictionary.