3 ms·

Ack on migrations, tks! --- For completeness, you can do the same in Ecto: from t in MyTable, where: t.key >= 12, select: ["field1", "field2", "field3"]

Ack on migrations, tks! --- For completeness, you can do the same in Ecto: from t in MyTable, where: t.key >= 12, select: ["field1", "field2", "field3"] However, that will still allocate a "MyTable" struct in Ecto. And if that struct is large (say 50 fields), slots for it are allocated (but none of the actual data on the fields). I am not familiar with Python Object Model enough but, if your Python example still allocates an object with slots for all 47 other fields besides the three selected, then you may still find yourself under circumstances you would rather have a "slim" version of the model and reduce memory allocation. This may be needed when you need to load dozens of thousands of entries into memory for processing or similar. I am not saying this is a must have but I am just trying to clarify what my initial comment was about. :)

Xzya 4y ago
> I am not familiar with Python Object Model enough but, if your Python example still allocates an object with slots for all 47 other fields besides the three selected, then you may still find yourself under circumstances you would rather have a "slim" version of the model and reduce memory allocation. This may be needed when you need to load dozens of thousands of entries into memory for processing or similar. Alternatively you can also tell the ORM to return a dictionary rather than a model instance: https://docs.djangoproject.com/en/4.1/ref/models/querysets/#values https://docs.djangoproject.com/en/4.1/ref/models/querysets/#... For example, you may get a list of dictionaries with only "field1", "field2" and "field3" from a model like this: SomeModel.objects.filter(...).values("field1", "field2", "field3") [{"field1": "foo", "field2": "bar", "field3": "baz"}]
spapas82 4y ago
Please notice that nothing stops you from defining a non managed (ie Django migrations won't mess with it) model pointing to the same table and having declared only three of the fields of the original model.
- traverseda 4y ago
  That is more elegant than using a proxy model
  - spapas82 4y ago
    The Django docs also mention that https://docs.djangoproject.com/en/4.1/topics/db/models/#differences-between-proxy-inheritance-and-unmanaged-models https://docs.djangoproject.com/en/4.1/topics/db/models/#diff... > If you are mirroring an existing model or database table and don’t want all the original database table columns, use Meta.managed=False.
traverseda 4y ago
>I am not familiar with Python Object Model enough but, if your Python example still allocates an object with slots for all 47 other fields besides the three selected, then you may still find yourself under circumstances you would rather have a "slim" version of the model and reduce memory allocation. Honestly at that point probably don't be using python. ORM objects don't use slots, they use a hashmap to look up everything every time, and probably that hashmap points to a method that points to another hashmap that points to some kind of data structure. If you're worried about that than django probably is not for you. That being said it scales out very nicely, you probably are not loading dozens of thousands of entries in to memory at once but are instead using a celery distributed task queue. If you did need to load that much in to memory for some reason (presumably some kind of data science?) You can use the `in_bulk("field1","field2","etc")` query option to return raw data as dictionaries/lists instead of ORM objects. You don't get the benefit of ORM methods but you're not instantiating an ORM object for every row and probably for that kind of thing you're better off taking a more functional approach any way. Basically if you're worried about how much memory your data is going to take up you probably shouldn't be using django, hacking __slots__ or other memory efficient stuff into the django ORM is going to be tricky. In that case ignore the ORM and just construct your own objects directly or use something better suited for that.
- josevalim 4y ago
  That’s great, thanks for taking the time! Appreciated.

Xzya 4y ago

spapas82 4y ago

Please notice that nothing stops you from defining a non managed (ie Django migrations won't mess with it) model pointing to the same table and having declared only three of the fields of the original model.

traverseda 4y ago
That is more elegant than using a proxy model
- spapas82 4y ago
  The Django docs also mention that https://docs.djangoproject.com/en/4.1/topics/db/models/#differences-between-proxy-inheritance-and-unmanaged-models https://docs.djangoproject.com/en/4.1/topics/db/models/#diff... > If you are mirroring an existing model or database table and don’t want all the original database table columns, use Meta.managed=False.

traverseda 4y ago

>I am not familiar with Python Object Model enough but, if your Python example still allocates an object with slots for all 47 other fields besides the three selected, then you may still find yourself under circumstances you would rather have a "slim" version of the model and reduce memory allocation. Honestly at that point probably don't be using python. ORM objects don't use slots, they use a hashmap to look up everything every time, and probably that hashmap points to a method that points to another hashmap that points to some kind of data structure. If you're worried about that than django probably is not for you. That being said it scales out very nicely, you probably are not loading dozens of thousands of entries in to memory at once but are instead using a celery distributed task queue. If you did need to load that much in to memory for some reason (presumably some kind of data science?) You can use the `in_bulk("field1","field2","etc")` query option to return raw data as dictionaries/lists instead of ORM objects. You don't get the benefit of ORM methods but you're not instantiating an ORM object for every row and probably for that kind of thing you're better off taking a more functional approach any way. Basically if you're worried about how much memory your data is going to take up you probably shouldn't be using django, hacking __slots__ or other memory efficient stuff into the django ORM is going to be tricky. In that case ignore the ORM and just construct your own objects directly or use something better suited for that.

josevalim 4y ago
That’s great, thanks for taking the time! Appreciated.