6 ms·

Everyone seems to want to compare against MongoDB, but when I think NoSQL I think about Google Cloud Datastore and Amazon DynamoDB. Databases which are fully ho

• qaq 8y ago
  Outside of very niche use cases pushing enforcing consistency to the app layer adds enough complexity to dev process that a one/two programmer team will not generally handle it well. That's why even Google ended up building Spanner.
  • stickfigure 8y ago
    Having built several large, complicated, high-traffic apps on the Google Datastore, I disagree. Eventual consistency was at worst a minor nuisance. Rather, I found the major pain points to be 1) no joins and 2) no aggregations.
    • zzzcpan 8y ago
      Exactly. Eventual consistency was never hard, people should stop taking Google's research so seriously. They mix it with marketing too much.
    • qaq 8y ago
      so you have a distributed system that is handling eventual consistency at the app layer and it was a minor nuisance :)?
      • stickfigure 8y ago
        Yes. Genuinely, why is that hard to understand?
        • qaq 8y ago
          Because for avg. developer reasoning about eventual consistency and handling all the edge cases is fairly hard. If it's easy for you I guess you are far above avg. dev.
          • stickfigure 8y ago
            I hired a couple junior devs and they did fine. There really aren't that many edge cases. If you're changing data, you're in a transaction and that's always strongly consistent. If you're querying data, nobody really cares if it's 10s behind (and usually it was 1-2s). On the other hand, counting a few thousand things is a pretty major PITA. And doing bulk updates requires map/reduce. So there's a cost... but it's not what you seem to think it is.
            • qaq 8y ago
              Some of your projects looks like are in e-commerce space so say if I look at my cart and it is missing products I added say 5 sec ago that would be an issue.
              • lrem 8y ago
                That's what session affinity is for. I'm pretty ok with 5 seconds delay between adding something to the cart on my laptop and it appearing on my phone. Or, you may do user affinity and I'll have both the same second, but my latency will go bad the first day I travel overseas.
    • ecnahc515 8y ago
      Those major pain points are exactly what you're trading off off for when you pick eventual consistency.
• nojvek 8y ago
  I don’t understand the “call to get a quote” sales cycle. I bet memsql has a sales time the size of their engineering team trying to get those fat Enterprise deals but in the grand scheme of things why should I care? It doesn’t have a free tier, I can’t use it easily and just seems like a lot of hype without any 3rd party to back up their claims. Its the kind of things mongodb used to say to gain mindshare. Most popular databases like MySQL and Postgres gained its mindshare because they were open source and anyone could verify they did what they said. Only a couple closed source databases have won it big. Behind them were huge monoliths like Microsoft and Oracle with armies of sales teams. I’m not saying MemSQL will die. They are probably quite profitable. I work for an analytics company with a custom database. It’s just the average joe doesn’t get much from it. The cloud based dbs gets the average joe very far.
  • adrianpike 8y ago
    > I don’t understand the “call to get a quote” sales cycle. It's about maximizing revenue from enterprise clients - when the customer LTV is huge but the volume is low, it's smart to focus your resources on those dollar amounts. When the volume is high but the dollar amounts are low, it's smart to focus on optimizing the volume funnel.
    • bunderbunder 8y ago
      More specifically, it's about price discrimination. If you publish a price, you need to publish a price that's going to be palatable to your smallest clients. But the moment you do so, your most deep-pocketed clients will demand the same price. This could be disastrous, because those same clients also tend to be more demanding, so a profitable price for one client could be a disastrous money-loser for another. If you're the kind of person who needs to know the price up-front, it's probably safe to interpret this as a way of saying, "If you have to ask, this isn't for you." At the very least, it's probably the case that you have a limited appetite for products where success requires significant assistance from support engineers to help you deal with all the critical details that typically aren't in the documentation for these kinds of products.
• scarface74 8y ago
  I am an AWS true believer but I hate DynamoDB. It is far more limiting than Mongo DB. MongoDB works great with C#. Using a JSON based database with a statically typed object oriented language goes hand to hand. You don’t have the object relational mismatch and the languages type system helps enforce a “schema” on the database. With C# you work with a MongoCollection<T> And all of your inserts, Linq queries, etc are statically checked by the compiler.
  • 52-6F-62 8y ago
    It's been a pleasure working with MongoDB in TypeScript as well
    • scarface74 8y ago
      I could see that. But I can’t tell from briefly perusing the docs, does Typescript give you transpile time checking of objects that it will allow you to put in Mongo? Using a dynamically type language with a schemaless database isn’t something I would want to do for a large project.
      • 52-6F-62 8y ago
        I think it depends on your implementation. For example, I tend to use Mongoose as the interface to Mongo, with which you're required to set a Schema for each collection of documents. You can easily set every single field type to `Mixed` (analogous to `any`, `auto`, etc), but if you're the least bit disciplined you can set any fields to any variation of: * String * Number * Date * Buffer * Boolean * Mixed * ObjectId * Array * Decimal128 * Map (http://mongoosejs.com/docs/guide.html http://mongoosejs.com/docs/guide.html) And there are TypeScript typings supplied for Mongoose, so I typically recreate the models as TS interfaces for wherever I use them in other parts of my code. It's extremely intuitive and quick to develop. It's a tad redundant defining the Schema types, and then a TS interface, but I usually don't mind. I include them in the same file for coherence so that I'm always importing from the same module for anything relating to the model. For (a basic) example: /* MyCollection.ts */ import { Document, Model, model, Schema, Types } from 'mongoose'; export interface IMySchemaFields { _id?: Types.ObjectId; // † name: string; age?: number; fungi: boolean; ET?: "phone home" | "owwww"; } export type TMySchemaDocument = Document & IMySchemaFields; export type TMySchemaModel = Model<TMySchemaDocument>; export const MySchema: Schema = new Schema( { _id: Schema.Types.ObjectId // † NOTE name: String, age: Number, fungi: Boolean, ET: String, }); export const MyCollection: TMySchemaModel = model('myCollection', MySchema); † NOTE: there are a couple of quirks. Mongoose uses constructors to define types whereas TypeScript types are a different symbol understood only by the compiler— so in this case they use different aliases of the same object... So when I move to make any queries via the ODM, I import and use the associated types. The types won't allow you to reference fields that don't exist on the schema that way. You can force it by cheating the compiler and casting any arbitrary non-existent field to `<any>`, but I try to reserve that for hungover POC cases. It does suggest immutability of the object received from the query, but again— there are many ways to ignore that if you wanted to, or just didn't care. Example ODM query: /* MyController.ts */ import * as mongoose from 'mongoose'; import { MyCollection, IMySchemaFields } from '../models/MyCollection.ts' /* ... pretend there's relevant stuff happening here */ MyCollection.findOne({ _id: '436rgerr25gw4gfdhdsf', }, (error: Error, collection: IMyCollectionFields, }) : any => { // Do something with your `collection` object. // Just don't try to access, modify, or add fields // that don't exist on the referenced interface. }); /* More things probably happen here */ As for TypeScript for a large project... I think we get into different kinds of discussions here (eg, is Node or Deno the right tool, etc etc). The kind of applications I've used this design on have not had to face significant scaling or load problems. They're very applied circumstances that face small-to-medium/large traffic. I haven't benchmarked for anything that operated on a real significant scale. We will be testing for around the ~1k-2k concurrent user mark later this summer. But that doesn't involve consistent read/writes— it will involve regular reads, but seldom writes. I might try to write about it later, but it's kind of a fast-track thing so I'm a bit buried in it. Other projects following a similar design tend to be internal tooling for editorial and other teams in media and so don't see significant traffic or structures/changes beyond initial versions. I can understand many peoples' reservations and concerns. I definitely think it's a matter of what's right for the job, considering the circumstances and resources available.
• kodablah 8y ago
  > allow a one- or two-programmer team to achieve massive scale without hiring devops Know what isn't massive scale? Subjecting yourself to the off switch of a single company. So in that light, AWS-specific or Google-specific options are the worst scale you could ever have.
  • mylons 8y ago
    a meteor could also hit your datacenter, neutrinos can fly through your RAM, a person with malicious intent could detonate an EMP next to your servers, etc etc etc.
    • kodablah 8y ago
      Probability is not binary. The probability that you will want to move your app, cross-cloud distribute your app, or get it turned off is much higher than those things you mentioned. This is also true about the contents of the data instead of just the ifaces since using NoSQL databases often means distributed in lieu of backups. It's not that hard to run a few Cassandra machines, or if you can accept relational, RDS or other managed Postgres options are fine. It's just this vendor lock in stuff gets really annoying to refactor/migrate IME and in many cases if your time to market is that important, use a cloud-provided RDBMS that will let you be portable. I'd rather go from PG to Cassandra than Google data store to AWS data store to whatever other provider data store du jour.
      • stickfigure 8y ago
        This is terrible advice for startups. While you're busy building your platform-agnostic wondertrap with Cassandra nodes and Docker images and Chef scripts and monitoring and all the other bells and whistles, I'm sitting down at my keyboard and implementing customer-facing features. The probability that you will want to cross-cloud distribute your app is infinitesimally small compared to the probability that you will never get the right set of features to make your product take off. Platform portability is fake work that makes developers feel productive about doing something that their customers couldn't give a rat's ass about. There's a reason Pivotal doesn't let you point chores - your velocity is determined only by how fast you make the customer happy. Focus on developing features, and use whatever tools get you to product-market fit fastest.
• eikenberry 8y ago
  Just as an FYI DynamoDB has a free tier as well; per-month you get 25GB storage, 200 Million requests and 2.5 million stream requests. https://aws.amazon.com/dynamodb/pricing/ https://aws.amazon.com/dynamodb/pricing/