7 ms·

Toy Store: ORM for Riak, Redis, Cassandra, Memcached, darn near anything

51 points by netherland 16y ago

tjpick 16y ago
This NoSQL stuff has me confused. I didn't think those systems were considered relational. Why does one need an object-relational mapper?
- netherland 16y ago
  Good point. Probably should have just called it a mapper.
- koudelka 16y ago
  I believe the de-facto nomenclature for a NoSQL database's equivalent of an ORM is an "ODM", Object Document Mapper.
- antirez 16y ago
  NoSQL systems are not SQL, but this does not mean they are unable to model relations. For instance in Redis you often have something like: user:1000 That is an hash with a few attributes, and among this attribs you can have 'group_id", that is an integer, for instance 50. Then if you access group:50 You have another data structure representing the group the user is associated with.
netghost 16y ago
There's a better article on the rationale here: http://railstips.org/blog/archives/2011/01/27/data-modeling-in-performant-systems http://railstips.org/blog/archives/2011/01/27/data-modeling-...
deleted 16y ago
[deleted]
jrockway 16y ago
So basically, Elephant or KiokuDB, but for Ruby? I think people tend to call these "object databases" rather than ORMs.
mariusmg 16y ago
if you need a ORM for key/value storage, you're doing it (very) wrong
- rapind 16y ago
  Exactly. I'm not sure why this is necessary. If you want to store rich objects, just serialize them and create your indexes as necessary. Anyways, I'll definitely check it out. Maybe I'm missing something.
  - wfarr 16y ago
    The scale of the application affects how much you can really do just by adding indexes to tables and throwing more hardware at it. At a certain point, servers are only so fast and disks only so large and fast.
- cheald 16y ago
  If it's simple "stuff it in, pull it out" stuff, sure. Having things like validations and callbacks and association sugar and the like are really nice, though, in cases beyond the very simple. I think the point is to let you focus more on writing business logic, and less on mucking about with your underlying datastore. If it works as promised, then it could very certainly result in a productivity gain.
JonnieCache 16y ago
And it's ActiveModel-based! Hallelujah! For those not of a rails persuasion, ActiveModel is basically all the non-datastore stuff abstracted out from ActiveRecord, rails' ORM. So, this Toy Store lib will share all its validation, accessor etc. apis with ActiveRecord and so will already automatically be compatible with the huge range of rails libraries out there. A good example is simple_form, a rails gem for doing lovely forms without any markup. In theory you should be able to hook this into Toy Store and therefore all those key/value stores with zero effort. https://github.com/plataformatec/simple_form https://github.com/plataformatec/simple_form ActiveModel is on course to become Rack for data objects. When this idea is fully explored it's gonna make so many things so easy. Soon we're all going to be auto-generating javascript client-side validation code from our server-side activemodel validation definitions, without ever having to think for a second about how or where the data is actually stored.
- kotrin 16y ago
  abstraction abstraction abstraction
tomafro 16y ago
I'm not sure I quite get this. The code looks great. The mapping features seem strong (thanks in part to the power of ActiveModel). However, by restricting the adapter interface to a simple key/value store, don't you lose access to many of the features that make each backend distinct? I think Redis, Cassandra, MongoDB, etc. are great, but to me it't the differences amongst them which are interesting, not the similarities.
- iamwil 16y ago
  New does travel quick. I was just reading about this thing this morning. http://railstips.org/blog/archives/2011/01/27/data-modeling-in-performant-systems/ http://railstips.org/blog/archives/2011/01/27/data-modeling-... So John in the above link was talking about how when you scale, you end up looking at bottlenecks, which are usually slow data accesses, and moving them into some sort of key-value store--that usually are based on a key-value access pattern. So by the time you're well into scaling your app, you noticed that much of your data access has turned into limiting yourself to doing key lookups to attain the performance you need. Since (according to his experiences scaling) most of the performance bottlenecks seem to have this pattern, he asked, "what if you restricted your data access to just using key-value pairs from the very beginning?" That way you avoid some of the data access headaches later on. In answering that question, this is what they got.
  - Smerity 16y ago
    The same tactic was used by Google for Google App Engine - "if it doesn't scale then we're not including it as a feature". This is why so much of Google App Engine's documentation focuses on scaling [1]. Depending on who you ask this is genius or folly. By forcing yourself into this tactic you end up having to consider all the scaling complexity in the prototyping stage before your app has even proven itself potentially successful. If your app becomes wildly popular then the combination of your early work and Google's behind the scene scaling means there are far less problems for you to worry about. My major concern with this method is that people already focus far too much on premature optimizations before they even know where the real bottlenecks are in their application. If premature optimization leads to burn out or less features then it's a poison to our projects and should be avoided. [1] http://code.google.com/appengine/articles/scaling/overview.html http://code.google.com/appengine/articles/scaling/overview.h...
deleted 16y ago
[deleted]
- wildmXranat 16y ago
  'R'elevant. Good question.
j2d2j2d2 16y ago
I wrote something similar for Python. Not quite as robust, but on a similar path. DictShield: https://github.com/j2labs/dictshield https://github.com/j2labs/dictshield

51 points by netherland 16y ago

This NoSQL stuff has me confused. I didn't think those systems were considered relational. Why does one need an object-relational mapper?

netherland 16y ago
Good point. Probably should have just called it a mapper.
koudelka 16y ago
I believe the de-facto nomenclature for a NoSQL database's equivalent of an ORM is an "ODM", Object Document Mapper.
antirez 16y ago
NoSQL systems are not SQL, but this does not mean they are unable to model relations. For instance in Redis you often have something like: user:1000 That is an hash with a few attributes, and among this attribs you can have 'group_id", that is an integer, for instance 50. Then if you access group:50 You have another data structure representing the group the user is associated with.

netghost 16y ago

There's a better article on the rationale here: http://railstips.org/blog/archives/2011/01/27/data-modeling-in-performant-systems http://railstips.org/blog/archives/2011/01/27/data-modeling-...

deleted 16y ago

[deleted]

jrockway 16y ago

So basically, Elephant or KiokuDB, but for Ruby? I think people tend to call these "object databases" rather than ORMs.

mariusmg 16y ago

if you need a ORM for key/value storage, you're doing it (very) wrong

rapind 16y ago
Exactly. I'm not sure why this is necessary. If you want to store rich objects, just serialize them and create your indexes as necessary. Anyways, I'll definitely check it out. Maybe I'm missing something.
- wfarr 16y ago
  The scale of the application affects how much you can really do just by adding indexes to tables and throwing more hardware at it. At a certain point, servers are only so fast and disks only so large and fast.
cheald 16y ago
If it's simple "stuff it in, pull it out" stuff, sure. Having things like validations and callbacks and association sugar and the like are really nice, though, in cases beyond the very simple. I think the point is to let you focus more on writing business logic, and less on mucking about with your underlying datastore. If it works as promised, then it could very certainly result in a productivity gain.

JonnieCache 16y ago

And it's ActiveModel-based! Hallelujah! For those not of a rails persuasion, ActiveModel is basically all the non-datastore stuff abstracted out from ActiveRecord, rails' ORM. So, this Toy Store lib will share all its validation, accessor etc. apis with ActiveRecord and so will already automatically be compatible with the huge range of rails libraries out there. A good example is simple_form, a rails gem for doing lovely forms without any markup. In theory you should be able to hook this into Toy Store and therefore all those key/value stores with zero effort. https://github.com/plataformatec/simple_form https://github.com/plataformatec/simple_form ActiveModel is on course to become Rack for data objects. When this idea is fully explored it's gonna make so many things so easy. Soon we're all going to be auto-generating javascript client-side validation code from our server-side activemodel validation definitions, without ever having to think for a second about how or where the data is actually stored.

kotrin 16y ago
abstraction abstraction abstraction

tomafro 16y ago

I'm not sure I quite get this. The code looks great. The mapping features seem strong (thanks in part to the power of ActiveModel). However, by restricting the adapter interface to a simple key/value store, don't you lose access to many of the features that make each backend distinct? I think Redis, Cassandra, MongoDB, etc. are great, but to me it't the differences amongst them which are interesting, not the similarities.

iamwil 16y ago
New does travel quick. I was just reading about this thing this morning. http://railstips.org/blog/archives/2011/01/27/data-modeling-in-performant-systems/ http://railstips.org/blog/archives/2011/01/27/data-modeling-... So John in the above link was talking about how when you scale, you end up looking at bottlenecks, which are usually slow data accesses, and moving them into some sort of key-value store--that usually are based on a key-value access pattern. So by the time you're well into scaling your app, you noticed that much of your data access has turned into limiting yourself to doing key lookups to attain the performance you need. Since (according to his experiences scaling) most of the performance bottlenecks seem to have this pattern, he asked, "what if you restricted your data access to just using key-value pairs from the very beginning?" That way you avoid some of the data access headaches later on. In answering that question, this is what they got.
- Smerity 16y ago
  The same tactic was used by Google for Google App Engine - "if it doesn't scale then we're not including it as a feature". This is why so much of Google App Engine's documentation focuses on scaling [1]. Depending on who you ask this is genius or folly. By forcing yourself into this tactic you end up having to consider all the scaling complexity in the prototyping stage before your app has even proven itself potentially successful. If your app becomes wildly popular then the combination of your early work and Google's behind the scene scaling means there are far less problems for you to worry about. My major concern with this method is that people already focus far too much on premature optimizations before they even know where the real bottlenecks are in their application. If premature optimization leads to burn out or less features then it's a poison to our projects and should be avoided. [1] http://code.google.com/appengine/articles/scaling/overview.html http://code.google.com/appengine/articles/scaling/overview.h...

deleted 16y ago

[deleted]

wildmXranat 16y ago
'R'elevant. Good question.

j2d2j2d2 16y ago

I wrote something similar for Python. Not quite as robust, but on a similar path. DictShield: https://github.com/j2labs/dictshield https://github.com/j2labs/dictshield