4 ms·

As is to be expected from this author, this is definitely on the flame-bait side of things. I submit it because I believe there is an important point here: for

by jbyers 17y ago

As is to be expected from this author, this is definitely on the flame-bait side of things. I submit it because I believe there is an important point here: for the vast majority of startups, going with a relatively unproven "NoSQL" database is a premature optimization and an unneeded technical risk. I disagree with the author that these databases are a flash in the pan, but their over-application is.

psadauskas 17y ago
I dunno, I see the opposite: RDBMS's are a premature optimization. In my experience, it's /much/ easier to hack together a quick webapp in MongoDB, because you don't have to worry about relations, migrating schema, etc. Sure, it might be slower than Postgres on a billion-row table, but wait until you have a million rows before you shackle yourself to the relational constraints.
- bilbo0s 17y ago
  Absolutely right. He has it backwards. You use NoSQL to 'get shit done'. When you have a billion rows, then worry about schemas. By that point you will have a much better idea, a - what said schema should look like, b - what the architecture of the Postgres, or mysql, or Oracle should look like, and c - how much money you will have to solve the problem.
  - papachito 17y ago
    I don't really see the point of starting with NoSQL and then going for Oracle. NoSQL DBs are real tools you know, not toys you through after your app gets traction. It's actually the contrary that has happened so far.
    - bilbo0s 17y ago
      Maybe I should have been clear. You only go to SQL RDBMS if you need it. Which, in the vast majority of cases you will not. Further, you only go to SQL where you need it. For instance, right now someone is developing a street car racing game for Facebook. The XBox kind, not the FarmVille kind. At any rate, one of the features is obviously, playback. Now keeping all of those physics updates in an SQL is pointless. And figuring out a schema for that data would have only gotten in the way of them getting that out the door. Throw the physics messages in a queue and write them to Cassandra. If you have even 10000 MAUs, you will easily generate billions of rows. It's just not data that really needs to go into mysql. I don't think Cassandra is a toy. I think you should 'get it out the door' with everything in Cassandra, and then slowly, move the business stuff off. User names, what cars they bought for instance. Stuff that is not read often. But at first, get it out the door. Don't stop to figure out a perfectly normalized schema with balanced indexes.
      - lmz 17y ago
        Isn't Cassandra slightly more complicated than MySQL? Sure adding another column is trivial, but the access patterns & indexing need to be determined first.
        bilbo0s 17y ago
        Not if you use AKKA!
  - prodigal_erik 17y ago
    I once worked in a Notes shop. Notes has no schema for documents and nothing to enforce migrating data from older documents to the current format. After a few years of customers manipulating data with various versions of the code, they had documents in such bizzare combinations of states that it was no longer possible for anyone on our dev team to inspect them and say which behavior would be right for the workflow. Schemaless data should only be a summary of data properly maintained elsewhere, which you can regenerate at need. If your authoritative data has no schema, it will decay to garbage.
    - bilbo0s 17y ago
      That's because you waited years to address the problem. Not only that, you also rewrote the code, as I advise. But you did not take the opportunity to address the structural data issues you were having, contrary to my advice. My strategy is to rewrite the code, if needed, but with an eye towards addressing structural data issues. After a few months use of a web app you have a good idea of any surprising usage patterns that may appear. Readjust at that point when you are 'talking with data'. This advice is for small startups of the HN variety, where 'customers' are a lot more important than 'authoritative' data stores initially. NoSQL systems are useful tools for mitigating the danger of doing too much engineering upfront. Many tech entrepreneurs fall victim to doing too much upfront engineering in the hopes of their data store not 'decaying to garbage', only to find that no one wants to use their product. NoSQL makes it easy to go back and migrate off the data you want to store 'on the move'. When you have a better idea of how much of it there is, and how it is used.
      - prodigal_erik 17y ago
        If you are not doing data migration, after n revisions to the data management code, each record can be in any of 2^n states, depending on which code revisions did or did not modify it. How many revisions can you make before your code can no longer handle some of your older data? I'd say days' worth, not months, because you're trying to iterate a lot faster than we did. And the odds of a complete rewrite understanding all your old data are even worse. If you are doing data migration, you necessarily have an old and new schema in your head. At that point you're just refusing to write it down and let the tools tell you whether the code agrees with you.

by jbyers 17y ago

psadauskas 17y ago

I dunno, I see the opposite: RDBMS's are a premature optimization. In my experience, it's /much/ easier to hack together a quick webapp in MongoDB, because you don't have to worry about relations, migrating schema, etc. Sure, it might be slower than Postgres on a billion-row table, but wait until you have a million rows before you shackle yourself to the relational constraints.

bilbo0s 17y ago
Absolutely right. He has it backwards. You use NoSQL to 'get shit done'. When you have a billion rows, then worry about schemas. By that point you will have a much better idea, a - what said schema should look like, b - what the architecture of the Postgres, or mysql, or Oracle should look like, and c - how much money you will have to solve the problem.
- papachito 17y ago
  I don't really see the point of starting with NoSQL and then going for Oracle. NoSQL DBs are real tools you know, not toys you through after your app gets traction. It's actually the contrary that has happened so far.
  - bilbo0s 17y ago
    Maybe I should have been clear. You only go to SQL RDBMS if you need it. Which, in the vast majority of cases you will not. Further, you only go to SQL where you need it. For instance, right now someone is developing a street car racing game for Facebook. The XBox kind, not the FarmVille kind. At any rate, one of the features is obviously, playback. Now keeping all of those physics updates in an SQL is pointless. And figuring out a schema for that data would have only gotten in the way of them getting that out the door. Throw the physics messages in a queue and write them to Cassandra. If you have even 10000 MAUs, you will easily generate billions of rows. It's just not data that really needs to go into mysql. I don't think Cassandra is a toy. I think you should 'get it out the door' with everything in Cassandra, and then slowly, move the business stuff off. User names, what cars they bought for instance. Stuff that is not read often. But at first, get it out the door. Don't stop to figure out a perfectly normalized schema with balanced indexes.
    - lmz 17y ago
      Isn't Cassandra slightly more complicated than MySQL? Sure adding another column is trivial, but the access patterns & indexing need to be determined first.
      - bilbo0s 17y ago
        Not if you use AKKA!
- prodigal_erik 17y ago
  I once worked in a Notes shop. Notes has no schema for documents and nothing to enforce migrating data from older documents to the current format. After a few years of customers manipulating data with various versions of the code, they had documents in such bizzare combinations of states that it was no longer possible for anyone on our dev team to inspect them and say which behavior would be right for the workflow. Schemaless data should only be a summary of data properly maintained elsewhere, which you can regenerate at need. If your authoritative data has no schema, it will decay to garbage.
  - bilbo0s 17y ago
    That's because you waited years to address the problem. Not only that, you also rewrote the code, as I advise. But you did not take the opportunity to address the structural data issues you were having, contrary to my advice. My strategy is to rewrite the code, if needed, but with an eye towards addressing structural data issues. After a few months use of a web app you have a good idea of any surprising usage patterns that may appear. Readjust at that point when you are 'talking with data'. This advice is for small startups of the HN variety, where 'customers' are a lot more important than 'authoritative' data stores initially. NoSQL systems are useful tools for mitigating the danger of doing too much engineering upfront. Many tech entrepreneurs fall victim to doing too much upfront engineering in the hopes of their data store not 'decaying to garbage', only to find that no one wants to use their product. NoSQL makes it easy to go back and migrate off the data you want to store 'on the move'. When you have a better idea of how much of it there is, and how it is used.
    - prodigal_erik 17y ago
      If you are not doing data migration, after n revisions to the data management code, each record can be in any of 2^n states, depending on which code revisions did or did not modify it. How many revisions can you make before your code can no longer handle some of your older data? I'd say days' worth, not months, because you're trying to iterate a lot faster than we did. And the odds of a complete rewrite understanding all your old data are even worse. If you are doing data migration, you necessarily have an old and new schema in your head. At that point you're just refusing to write it down and let the tools tell you whether the code agrees with you.