4 ms·

I think the bigger problem with ORM's or database driven application design in general is that you start to see the world through the eyes of the database and h

I think the bigger problem with ORM's or database driven application design in general is that you start to see the world through the eyes of the database and how you move things around inside it. Thus, your code resembles your database more than your database resembles your code. Ironically, developers would never think to do this with a storage mechanism like the filesystem. There is no great popularity in filesystem based ORM's. Somehow when we deal with the filesystem we treat it as it is - data storage and retrieval. When we deal with 3rd party api's we tend to keep them at arms length as well. Yet, when we deal with the database, we treat it as some other thing that seeks to influence the design of our code on a fundamental level. I wrote about this like a year ago: http://brianknapp.me/the-filesystem-test/ http://brianknapp.me/the-filesystem-test/ and I don't think much has changed since then.

lugg 13y ago
Sounds like you want to start using go. It basically assumes a database to be a black box much like file storage or an api, you keep away from it until you really need to put that data somewhere. This has the added benefit of being able to unit test almost all of your code. The only parts you can't effectively test are the storage layers like apis, fs, and db. Hmm to be a little more clear, I dont mean you avoid it, I mean you abstract it away into simple storage access interfaces the same way you would with fs or apis. The last thing you want is orm riddled logic.
- zurn 13y ago
  Can you provide a link? A quick web search only turns up the "sql" package - that seems to be just a low-level SQL interface that doesn't prescribe much.
  - grey-area 13y ago
    That's all golang has, it's pretty much like other languages :) github.com/lib/pq is a typical driver, which returns rows of values which you read to recreate your objects.
- grey-area 13y ago
  Having experimented with golang a bit I don't see any significant advantages over other languages in interfacing with sql dbs - the db drivers return rows which you convert back to your objects, and it's up to you to wire up the objects and relations from that, and there are even a few issues like calls to insert not returning the rowid inserted so you have to call LastInsertId for Mysql/sqlite (last time I used it). You still have to decide how rows returned are converted back to your golang objects, deal with nils, relations etc, all that logic has to go somewhere, and it's not in the db drivers. The problem with ORMs is not with just storing data, that's pretty simple and you don't even really need an ORM for that. The most difficult problem comes when you introduce complex relations, and need a way to retrieve those datasets from the database. AFAIK golang does not help you in any way with that, so you'll have to invent your own ORM and conventions for representing things like belongs_to, has_many and join relationships. If you have join tables you'll be doing the relations yourself which is not always simple or performant - this is what ORMs are useful for. Clearly ORMs break down in apps of a certain size and have their own issues (largely due to non-optimal SQL and/or a reluctance to break out of the ORM when necessary), but every app contains the sort of logic which is in an ORM, because it must map storage to in-memory and view representations. If you have logic to convert *sql.Rows to your objects, including relations, you have written a simple ORM.
porker 13y ago
Isn't this because when using a filesystem, we - in almost all cases - don't have relations between files we need to use? If I needed to look in a folder containing a million files, find a subset containing the word 'foo' and match to the 'joining' files in another 4 folders (all of which contain 500k+ files) - I think my code would resemble my filesystem more than my filesystem resembles my code.
lucian1900 13y ago
Except files are modelled. You have a bunch of functions for finding the right one, then a bunch for manipulating files at rest, then a few for giving you a file descriptor. The file descriptor represents the file in your language and is the equivalent of parf of what some ORMs do. It's an explicit interface between external data and your program, just like a good SQL library (like SQLAlchemy).
jordanthoms 13y ago
Thing is, for a lot of applications, the structure of the database is _much_ more important than the structure of the code. It's easier to refactor your code later than to deal with a database which is structured incorrectly, since when you refactor that you risk losing the data, have to run change scripts and have downtime, etc. You can view the database as merely being there to support your program, or you can view your program as being there to manage the database. If it's the latter, then it's not surprising that the database design impacts the design of your code. So we don't keep it at arm's length, because it's much more important and complex than a 3rd party API or even a filesystem.
- collyw 13y ago
  I think is analogous to Linus's quote about good programmers worrying more about the data structures than the code. Certainly when my team leader suggests another quick hack at the code level to make the database do what we want, I usually push back, and suggest a change at the database level to model the situation more accurately. His solution would be quicker, but mine will be more robust and maintainable (so less work in the long term).

lugg 13y ago

Sounds like you want to start using go. It basically assumes a database to be a black box much like file storage or an api, you keep away from it until you really need to put that data somewhere. This has the added benefit of being able to unit test almost all of your code. The only parts you can't effectively test are the storage layers like apis, fs, and db. Hmm to be a little more clear, I dont mean you avoid it, I mean you abstract it away into simple storage access interfaces the same way you would with fs or apis. The last thing you want is orm riddled logic.

zurn 13y ago
Can you provide a link? A quick web search only turns up the "sql" package - that seems to be just a low-level SQL interface that doesn't prescribe much.
- grey-area 13y ago
  That's all golang has, it's pretty much like other languages :) github.com/lib/pq is a typical driver, which returns rows of values which you read to recreate your objects.
grey-area 13y ago
Having experimented with golang a bit I don't see any significant advantages over other languages in interfacing with sql dbs - the db drivers return rows which you convert back to your objects, and it's up to you to wire up the objects and relations from that, and there are even a few issues like calls to insert not returning the rowid inserted so you have to call LastInsertId for Mysql/sqlite (last time I used it). You still have to decide how rows returned are converted back to your golang objects, deal with nils, relations etc, all that logic has to go somewhere, and it's not in the db drivers. The problem with ORMs is not with just storing data, that's pretty simple and you don't even really need an ORM for that. The most difficult problem comes when you introduce complex relations, and need a way to retrieve those datasets from the database. AFAIK golang does not help you in any way with that, so you'll have to invent your own ORM and conventions for representing things like belongs_to, has_many and join relationships. If you have join tables you'll be doing the relations yourself which is not always simple or performant - this is what ORMs are useful for. Clearly ORMs break down in apps of a certain size and have their own issues (largely due to non-optimal SQL and/or a reluctance to break out of the ORM when necessary), but every app contains the sort of logic which is in an ORM, because it must map storage to in-memory and view representations. If you have logic to convert *sql.Rows to your objects, including relations, you have written a simple ORM.

porker 13y ago

Isn't this because when using a filesystem, we - in almost all cases - don't have relations between files we need to use? If I needed to look in a folder containing a million files, find a subset containing the word 'foo' and match to the 'joining' files in another 4 folders (all of which contain 500k+ files) - I think my code would resemble my filesystem more than my filesystem resembles my code.

lucian1900 13y ago

Except files are modelled. You have a bunch of functions for finding the right one, then a bunch for manipulating files at rest, then a few for giving you a file descriptor. The file descriptor represents the file in your language and is the equivalent of parf of what some ORMs do. It's an explicit interface between external data and your program, just like a good SQL library (like SQLAlchemy).

jordanthoms 13y ago

Thing is, for a lot of applications, the structure of the database is _much_ more important than the structure of the code. It's easier to refactor your code later than to deal with a database which is structured incorrectly, since when you refactor that you risk losing the data, have to run change scripts and have downtime, etc. You can view the database as merely being there to support your program, or you can view your program as being there to manage the database. If it's the latter, then it's not surprising that the database design impacts the design of your code. So we don't keep it at arm's length, because it's much more important and complex than a 3rd party API or even a filesystem.

collyw 13y ago
I think is analogous to Linus's quote about good programmers worrying more about the data structures than the code. Certainly when my team leader suggests another quick hack at the code level to make the database do what we want, I usually push back, and suggest a change at the database level to model the situation more accurately. His solution would be quicker, but mine will be more robust and maintainable (so less work in the long term).