6 ms·

I don't hear people complaining about say LINQ. Maybe it's just that your language and/or ORM or integration thereof suck?

Someone1234 9y ago
LINQ isn't a ORM. I assume you mean Entity Framework? EF definitely has the foreign key issue. We have around a thousand tables, we tried to generate the classes for all of them including foreign keys, problem is that when you create a context that references even only a single table, it will load everything that is foreign keyed including siblings of siblings of siblings, until you run out of memory. Only way around it is to not set up foreign keys which massively diminishes the value of using EF, so we wound up creating two copies of each table's classes, one with and one without foreign keys. That causes its own issues.
- bloaf 9y ago
  Wait, it ran out of memory because it couldn't hold the class structure?
  - Someone1234 9y ago
    Essentially yes. It created giant object map with millions of objects in it and hit a memory limit (rightfully so, it wouldn't have been usable anyway). It would start at the initial table, get the siblings, then the siblings of the siblings, and so on until it was trying to generate an object for every property of almost every table we had (even if we only referenced the original table).
- flukus 9y ago
  Sounds like you disabled lazy loading and that forced everything to be loaded at once. That's a user error, not a EF fault. This is a common theme I've seen with people blaming ORM's for being slow, it's the devs not using them appropriately more than the ORM's themselves. Not to say that they don't have their own issues.
  - Someone1234 9y ago
    With or without lazy loading enabled the result was the same. Generating the structure took seconds and went OOM with enough tables. LazyLoading impacts what data is retrieved from the database (or more to the point when), this is a structural issue before a query was even sent to the database. It would die while generating the query, not sending the query or populating the result. You likely should have asked for more information before concluding it was "user error."
    - useerup 9y ago
      I have been working with EF for years now. It has it quirks - but this is not something that I have ever experienced, nor have I heard of anything like it before today. What I have heard of is traversing the entire graph and causing cascading loading of navigational properties. Yes, I have done that. Something like AutoMapper will do that to you, if you are not careful. Been there and done that. How did you determine that it OOMed while generating the query?
      - Someone1234 9y ago
        > How did you determine that it OOMed while generating the query? By looking at the call stack when the exception was thrown (and the fact that nothing hit the database).
    - flukus 9y ago
      > LazyLoading impacts what data is retrieved from the database (or more to the point when) It impacts more than just that, it will impact the query generation as well. If you have a property that is not lazy loaded then it will attempt to join the relation or load it very another query in the same round trip. Turning it off tells the ORM that every single time you want A it needs to go and get B as well. If you have it off universally it will attempt to load the entire database, or as may be the case here, crashing while trying to generate a query to do so. > You likely should have asked for more information before concluding it was "user error." Perhaps, but you've got multiple conflicting accounts of what went wrong, some comments indicate that it returned data, others say it never touched the database.
- Merad 9y ago
  Were you doing serialization or something like that that recursively accessed all of the properties in the model? If so, and if lazy loading was turned on (the unfortunate default) then the serializer would indeed keep exploring the object graph until it either loaded your whole database or hit OOM. That's about the only scenario I can think of that would cause such an issue - EF doesn't eagerly load related entities unless you explicitly tell it to.
  - dragonwriter 9y ago
    Sounds like they're actually referencing some of the well-known query-generation issues in EF, though giant queries and long generation times are more common manifestations than OOM, which seems to be an extremely pathological case.
- useerup 9y ago
  You were doing it wrong then. EF does not eagerly load child collections. You cannot even configure it to do so. As someone else suggested, you probably had lazy loading enabled, and some of your code tried - e.g. through reflection - to get all properties. EF has some of it's own issues - but you can most certainly create composite primary keys, composite foreign keys and work with projections right from within the code. None of the issues the original author had with SQLAcademy and Hibernate are really a pain in EF. Some prefer to switch off lazy loading in EF and instead either explicit eagerly load specific child collections or explicitly load them right before use. In EF you can do var customers = Customers.Include(c => c.Orders).Single(c => c.CustomerNo == '1234') This will load Customer '1234' with the Orders collection eagerly loaded.
dragonwriter 9y ago
> I don't hear people complaining about say LINQ. LINQ isn’t ORM; ADO.NET and, on top of that, Entity Framework are the (core) ORMs that work with LINQ. ADO.NET and EF get plenty of complaints.

Someone1234 9y ago

LINQ isn't a ORM. I assume you mean Entity Framework? EF definitely has the foreign key issue. We have around a thousand tables, we tried to generate the classes for all of them including foreign keys, problem is that when you create a context that references even only a single table, it will load everything that is foreign keyed including siblings of siblings of siblings, until you run out of memory. Only way around it is to not set up foreign keys which massively diminishes the value of using EF, so we wound up creating two copies of each table's classes, one with and one without foreign keys. That causes its own issues.

bloaf 9y ago
Wait, it ran out of memory because it couldn't hold the class structure?
- Someone1234 9y ago
  Essentially yes. It created giant object map with millions of objects in it and hit a memory limit (rightfully so, it wouldn't have been usable anyway). It would start at the initial table, get the siblings, then the siblings of the siblings, and so on until it was trying to generate an object for every property of almost every table we had (even if we only referenced the original table).
flukus 9y ago
Sounds like you disabled lazy loading and that forced everything to be loaded at once. That's a user error, not a EF fault. This is a common theme I've seen with people blaming ORM's for being slow, it's the devs not using them appropriately more than the ORM's themselves. Not to say that they don't have their own issues.
- Someone1234 9y ago
  With or without lazy loading enabled the result was the same. Generating the structure took seconds and went OOM with enough tables. LazyLoading impacts what data is retrieved from the database (or more to the point when), this is a structural issue before a query was even sent to the database. It would die while generating the query, not sending the query or populating the result. You likely should have asked for more information before concluding it was "user error."
  - useerup 9y ago
    I have been working with EF for years now. It has it quirks - but this is not something that I have ever experienced, nor have I heard of anything like it before today. What I have heard of is traversing the entire graph and causing cascading loading of navigational properties. Yes, I have done that. Something like AutoMapper will do that to you, if you are not careful. Been there and done that. How did you determine that it OOMed while generating the query?
    - Someone1234 9y ago
      > How did you determine that it OOMed while generating the query? By looking at the call stack when the exception was thrown (and the fact that nothing hit the database).
  - flukus 9y ago
    > LazyLoading impacts what data is retrieved from the database (or more to the point when) It impacts more than just that, it will impact the query generation as well. If you have a property that is not lazy loaded then it will attempt to join the relation or load it very another query in the same round trip. Turning it off tells the ORM that every single time you want A it needs to go and get B as well. If you have it off universally it will attempt to load the entire database, or as may be the case here, crashing while trying to generate a query to do so. > You likely should have asked for more information before concluding it was "user error." Perhaps, but you've got multiple conflicting accounts of what went wrong, some comments indicate that it returned data, others say it never touched the database.
Merad 9y ago
Were you doing serialization or something like that that recursively accessed all of the properties in the model? If so, and if lazy loading was turned on (the unfortunate default) then the serializer would indeed keep exploring the object graph until it either loaded your whole database or hit OOM. That's about the only scenario I can think of that would cause such an issue - EF doesn't eagerly load related entities unless you explicitly tell it to.
- dragonwriter 9y ago
  Sounds like they're actually referencing some of the well-known query-generation issues in EF, though giant queries and long generation times are more common manifestations than OOM, which seems to be an extremely pathological case.
useerup 9y ago
You were doing it wrong then. EF does not eagerly load child collections. You cannot even configure it to do so. As someone else suggested, you probably had lazy loading enabled, and some of your code tried - e.g. through reflection - to get all properties. EF has some of it's own issues - but you can most certainly create composite primary keys, composite foreign keys and work with projections right from within the code. None of the issues the original author had with SQLAcademy and Hibernate are really a pain in EF. Some prefer to switch off lazy loading in EF and instead either explicit eagerly load specific child collections or explicitly load them right before use. In EF you can do var customers = Customers.Include(c => c.Orders).Single(c => c.CustomerNo == '1234') This will load Customer '1234' with the Orders collection eagerly loaded.

dragonwriter 9y ago

> I don't hear people complaining about say LINQ. LINQ isn’t ORM; ADO.NET and, on top of that, Entity Framework are the (core) ORMs that work with LINQ. ADO.NET and EF get plenty of complaints.