3 ms·
It's definitely true that putting the burden of consistency on developers (instead of on the DB) results in a lot more tricky work for developers. On my projec
by CydeWeys 8y ago
It's definitely true that putting the burden of consistency on developers (instead of on the DB) results in a lot more tricky work for developers. On my project, which started six years ago, we use Cloud Datastore, because Cloud Spanner hadn't come out yet. It results in complicated, painful code that would be completely unnecessary with stronger transactional guarantees. Some examples: https://github.com/google/nomulus/blob/master/java/google/registry/backup/CommitLogCheckpointStrategy.java https://github.com/google/nomulus/blob/master/java/google/re... https://github.com/google/nomulus/blob/master/java/google/registry/model/ofy/CommitLoggedWork.java https://github.com/google/nomulus/blob/master/java/google/re... https://github.com/google/nomulus/blob/master/java/google/registry/model/index/ForeignKeyIndex.java https://github.com/google/nomulus/blob/master/java/google/re...
It's no surprise that we're currently running screaming to something with stronger transactional guarantees.
- wikibob 8y agoI’m guessing that Nomulus is your project? I ran across it before and just wanted to say it’s really cool that this is open sourced.
- CydeWeys 8y agoI'm the tech lead of it. Glad to hear that you've heard of it before, and yeah, I think it's pretty cool it's open sourced too, which is why I made it happen! https://opensource.googleblog.com/2016/10/introducing-nomulus-open-source.html https://opensource.googleblog.com/2016/10/introducing-nomulu...
- guillon 8y agoAny idea if any company is planning to create an offer dedicates To .BRAND new gTLDs?
- erikb 8y agoWhy does anybody need to provide consistency? Often you don't have complete consistency anyways. There are bugs, there are time delays, there is parallel processing. Why even have the requirement?
- zzzcpan 8y agoWell, it's pretty hard to even find applications that require strong global consistency and are willing to sacrifice latency for that. Typically apps don't need much consistency at all and can sacrifice some data instead, like with most RDBMS setups in the wild. Beyond that SEC (strong eventual consistency) covers pretty much all consistency needs there are.
- erikb 8y ago> it's pretty hard to even find applications that require strong global consistency Exactly my point, even a little generalized. The world isn't consistent, why always put unrealistic constraints unto ourselves. If we change from "I really need to have all the datasets in all of their truthest form" and instead go with "well there is some data coming, better than nothing" the whole system might be even more reliable. Things in the middle won't die just because the world is imperfect.
- CydeWeys 8y agoLet me expand on an example issue that comes up. Nomulus is software that runs a domain name registry, including most notably the .app TLD. There are three fundamental objects at play here; the domains themselves, contacts (ownership information that goes into WHOIS), and hosts (nameserver information that feeds into DNS). There's a many-to-many relationship here, in that contacts and hosts can be reused over an arbitrarily large number of domains. The problem is that you can't perform transactions over an arbitrarily large number of different objects in Cloud Datastore; you're limited to enlisting a maximum of 25 entity groups. This means that you can't perform operations that are strongly consistent when contacts or hosts are reused too often. This situation comes up a lot; registrars tend to reuse the same nameservers across many domains, as well as the contacts used for privacy/proxy services. These problems don't arise in a relational SQL database, because you can simply JOIN the relevant tables together (provided you have the correct indexes set up) and then perform your operations in a strongly consistent manner. That trades off scalability for consistency though, whereas in Spanner you give up neither.
- 8y ago
- davidgay 8y agoIt's worth noting that Cloud Datastore's follow-on, Cloud Firestore, does provide strong consistency, and includes a "Datastore mode" that supports the Datastore API. Firestore is currently in beta, but once it's GA we will be migrating all Datastore users to Firestore: https://cloud.google.com/datastore/docs/upgrade-to-firestore https://cloud.google.com/datastore/docs/upgrade-to-firestore Disclaimer: I work on Cloud Datastore/Firestore.
- stingraycharles 8y agoI always got the impression that Firestore is more aimed towards mobile apps rather than backend applications, and as such more or less a different kind of product. Is this not the case?
- davidgay 8y agoDatastore was initially designed for use from App Engine, i.e., an easy start, no management, automatic scaling environment ("serverless" to use the current in-vogue term). I would view the Firestore API as a further extension (the "Datastore Mode" functionality was always an element of the design) of that paradigm, extending to the case where you have no trusted piece of code to mediate requests to the database, thus allowing direct use from, e.g., mobile apps (at which point other issues such as disconnected operation surface). So not so much a "different kind of product" and more a product that supports a strict superset of use cases.
- thesandlord 8y agoWhile Firestore has a Android/iOS/Web SDK, it also has great backend support (Python, Java, Node, Go, Ruby, C#, PHP) as well. The "realtime" features of Firestore are better suited for mobile IMO, but using Firestore as a scalable, consistent, document/nosql database for your backend is definetly a good use for it. I actually think most of the server SDKs don't even expose many of the realtime APIs. Maybe they will in the future, but it shows that you can use Firestore like a normal database just fine. (I work for GCP)