25 ms·

Cloud Firestore: A New Document Database for Apps

377 points by crb 9y ago

jamest 9y ago
[Firebase founder] This new database has been in the works for 2.5 years, since shortly after we joined Google. It was developed in close collaboration with the Cloud Datastore[1] team, and uses Google’s core database infrastructure. We built it because we know it can be challenging to build complex apps with our original database -- Firebase Realtime Database -- where we optimized for ease-of-use & real-time sync over querying functionality. For more info, see this comparison between the two[2]. This is a open beta launch, so the product has limitations[3] you should be aware of. We’ll be working to remove/raise these before General Availability. A note on naming: Firebase launched in 2012[4] with a single product, the original database. As we added products (Firebase Auth / Firebase Hosting / etc), we began calling the original product ‘Firebase Realtime Database’, or RTDB for short. If you haven’t followed us lately, we’ve grown to become Google’s app platform, and now have 16 products to help you build/grow apps. We’re grateful for the support HN has given us over the years and we hope you enjoy Cloud Firestore! [1] https://cloud.google.com/datastore/ https://cloud.google.com/datastore/ [2] https://firebase.google.com/docs/firestore/rtdb-vs-firestore https://firebase.google.com/docs/firestore/rtdb-vs-firestore [3] https://firebase.google.com/docs/firestore/quotas https://firebase.google.com/docs/firestore/quotas [4] https://news.ycombinator.com/item?id=3832877 https://news.ycombinator.com/item?id=3832877
- aphextron 9y ago
  Very exciting. I found this to be the one "missing piece" keeping me from being able to build a whole app on Firebase when checking it out earlier this year. One question though, will this support BLOBs like GridFS?
  - habosa 9y ago
    You can store Blob data, but the maximum size of a single document is still 1MB. If you need to store larger blobs check out Cloud Storage for Firebase.
    - e12e 9y ago
      Could you share some of the reasoning behind arriving at a 1mb limit (why not 10mb or 512kb for example)?
      - itcmcgrath 9y ago
        Hard to explain externally, but it relates to RAM size allocation + throughput in the underlying distributed database system.
- TY 9y ago
  Just read through your link #2 and was very impressed. Firestore addresses every single concern I had with RDTB and more and I'm looking forward to using it in my side projects. Thank you!
  - mayop100 9y ago
    That's great to hear! Our design for Cloud Firestore was based on years of developer feedback. Glad we hit the mark for you : )
- judah 9y ago
  What's your story on ACID, particularly on transactions? Transactions: If I want to insert 2 documents, but zero if either fails, what does that look like with Firestore? And if I store a document and run a query milliseconds later, will the query include document I just stored? Or are queries eventually consistent?
  - habosa 9y ago
    Cloud Firestore has support for transactions: https://firebase.google.com/docs/firestore/manage-data/update-data#update_data_with_transactions https://firebase.google.com/docs/firestore/manage-data/updat... So yes, if any of the operations in the transaction fails (and cannot be retried) the whole transaction will fail. It's atomic as you'd expect. If you've ever used transactions in Firebase Realtime Database you'll be happy to know that the transactions in Cloud Firestore are much easier to use since they don't require selecting a common data parent. Queries are always strongly consistent. If you do a write-read sequence you'll see the value of the latest write.
    - ibdknox 9y ago
      How do you deal with consistency in the offline case? E.g. If I make a bunch of edits while disconnected, and others have made edits while connected, how are the conflicts resolved?
      - mikelehen 9y ago
        This works similar to the Realtime Database in that it's last-write wins (where in the offline case, "last" is the last person to come back online and send their write to the backend). This model is very easy for developers to understand and directly solves many use cases, especially since we allow very granular writes which reduces the risk of conflicts. But for more complex use cases, you can get clever and implement things like OT conflict resolution as a layer on top of last-write wins, e.g. similar to how we implemented collaborative editing with www.firepad.io on the realtime database. PS: Hi Chris! :-)
- cfilipov 9y ago
  Regarding offline mode. How is sync done? Is this OT based? How are conflicts resolved?
  - mikelehen 9y ago
    Good question, and to answer this well we should probably do a blog post or something. In the meantime you could dig into the code since the clients are all open source. :-) But basically, sync is split into two halves: writes and listens. Clients store pending writes locally until they're flushed to the backend (which could be a long time if the app is running offline). While online, listen results are streamed from the backend and persisted in a local client cache so that the results will also be visible while offline (and any pending writes are merged into this offline view). When a client comes back online, it flushes its pending writes to the backend which are executed in a last-write-wins manner (see my answer above to ibdknox for more details on this). To resume listens, the client can use a "resume token" which allows the backend to quickly get the client back up-to-date without needing to re-send already retrieved results (there are some nuances here depending on how old the resume token is, etc.).
    - cfilipov 9y ago
      Thank you!
- wahnfrieden 9y ago
  Do you have a rough timeline on beta => General Availability?
  - itcmcgrath 9y ago
    Nothing we can share at this time. Really looking forward to the feedback we get now we're in beta as this will feed into our GA plans.
- sandGorgon 9y ago
  Do you see this as an equivalent to Dynamodb ? Very timely ;)
  - jaxondu 9y ago
    DynamoDB does not come with offline sync.
- gfosco 9y ago
  Awesome to see this. I couldn't get behind the RTDB structure, but Cloud Firestore I will definitely use!
  - inlined 9y ago
    Woohoo! Is this an official parse endorsement? (Hi Fosco )
    - gfosco 9y ago
      Hey Thomas :) I speak only for myself, Firebase just got way better.
      - inlined 9y ago
        I'm glad. Hoping you like cloud functions too!
- insomniacity 9y ago
  How goes the work to improve customer service and support? Thinking about https://news.ycombinator.com/item?id=14359801 https://news.ycombinator.com/item?id=14359801, what improvements have you made in the last 139 days that would make us want to build on this?
  - steego 9y ago
    For their own sake, I hope the team gives this question attention and respect. What sets Amazon apart from many other companies is their reputation for relentless customer service. Customers pay close attention to the quality of answer or lack of answers to questions like this one. Every cloud company that asks us to invest our time in their proprietary API is asking us to trust them. If you violate one vocal customer's trust, we're going to notice. If you don't make things right, we're going to notice. Even before I read the article, I first read the comments to gauge the reception because I care more about the perception of trust before I even consider using something like this. Customers don't always stop trusting companies for fair or valid reasons, but that doesn't matter. Trust is much more of a visceral thing than a cerebral thing. So here's my question to other HN readers. What would Google need to do to improve its reputation for customer service? What would they need to do to make that commitment resonate with you on a visceral level?
    - techdragon 9y ago
      For me it's pretty easy. I want to start hearing and seeing positive anecdotes about real time communication with Google Cloud support. That is to say, I want to hear people start saying things like "I rang them, and everything was sorted out" ideally with a platitude attached like "it was actually a pretty good experience". I hear this about AWS all the time, and I've even experienced it myself. One client some time ago had an AWS novice who was confident they could handle setting up the auto-scale groups. They made a small mistake, which lead to the scaling trigger being permanently on, and it auto scaling to 1024 ec2 nodes within the first half hour. Immediately after deploying a fix for the scaling trigger, I phone AWS to see if there was anything that could be done about the cost of this accident, and I had a small monthly credit to compensate, and a respectable discount on the next appropriate AWS training course so that the novice could learn more and hopefully avoid future mistakes. THAT is the kind of story I want to start hearing about Google Cloud. "I used it and did not have problems", and "It works and was cheaper than AWS" is just not enough.
- novaleaf 9y ago
  glad you mentioned Cloud Datastore. It seems that Firestore supersedes Datastore and Firebase. Is that correct? Any comparisons of Firestore vs Datastore?
  - habosa 9y ago
    It does not "supersede" Datastore or Firebase Realtime Database, neither of those products are going anywhere. This is a totally new product that builds on what we learned from those two products, sharing some of the best features of each and bringing in some totally new things as well. All three are (in my opinion) great choices for building a new app today so make sure you evaluate all options.
    - revmoo 9y ago
      You posted this after two controversial but fair questions has been asked. Please answer them.
- mjw1007 9y ago
  Can you provide anything to reassure potential users that this will still exist in five or ten years time?
  - mbleigh 9y ago
    The Firebase Realtime Database has been around for ~5 years (and is still fully supported after this launch). Cloud Datastore has been around in some form (started as App Engine Datastore) since 2008. Cloud Firestore was a massive multi-year joint effort between Firebase and Google Cloud. Google is investing heavily in both, and this is a big deal for our teams.
    - mjw1007 9y ago
      Perhaps you could consider persuading the powers that be to make a loud public commitment. After all, if they are truly confident, there should be little cost in doing so.
      - mbleigh 9y ago
        We announced the product, that's a loud public commitment. Once it reaches general availability, it will be covered by the Cloud deprecation policy requiring a minimum of one year notice for deprecation. I'm not sure what other guarantees would even make sense to offer. If anything, I'd look at this announcement as ongoing proof in the magnitude of investment Google is making in Firebase and Cloud.
        Cthulhu_ 9y ago
        I'd guess a forward commitment, like a LTS version? That would probably have an adverse effect though, since you'd be the only provider that goes (e.g.) "We will support this product for at least three years from now", implying (to people making Decisions) you'd pull the plug after three years. Maybe publish a long term (5+ year) plan / roadmap? idk.
        9y ago
- jonasb 9y ago
  The enhanced query capabilities are very much appreciated. I hope full-text search is in the works, since adding a 3rd party FTS engine adds quite a bit of complexity.
- paulsharratt 9y ago
  Good work guys, sounds like good progress! I'm struggling to see what this is giving me that I didn't have with RTDB though The queries seem to be doing what orderByChild equalTo startAt endAt limitToFirst and limitToLast were already allowing. Is there mostly a performance gain, or am I missing something that I can do now that I couldn't do before? Cheers
  - paulsharratt 9y ago
    I'll answer myself here for anyone else trying to figure out what we can do with queries now: --- We can now apply a filter and sort in the same command We can chain filters, though they can only apply to one field if they specify a range We can make a query that will only return documents, not the subfield data (i.e. what a lot of us were doing with the deprecated REST interface shallow=1) Queries in large data sets will be faster I guess? Things it'd still be nice to see - References that allow you to pull down related documents with a single query OR queries --- It's been a bit of rollercoaster seeing this announcement.. "Oh wow we can do proper queries now! Oh wait no, the queries don't seem to allow more than what we had before.. oh wait there are some improvements, it's a bit better now.. oh the shallow querying will be super handy! and they do geopoints.. but don't appear to have any way of searching for radius.. but they say they will soon" Anyway, good work all the same :)
- ec109685 9y ago
  While I know this is a beta and you’re solving a difficult technical problem, some of the limitations are concernig. With a system like dynamo, you can in most cases pay your way out of bottlenecks. It would be nice to develop a way in Firestore for users to be able to pay for the number of shards assigned to their workload to reduce relatively artificial limits such as write limits within collections and index update rate. That way, developers wouldn’t have to plan to leave the platform if their app is successful.
tannhaeuser 9y ago
> Document Database for Apps From linked page: > designed to easily store and sync app data Maybe I'm misunderstanding, but that's not what I understand a "document" to be. For me a document - is a file that can be stored on a file system - can be send via mail - is using a standard document representation so can be used by different applications, such as markup (XML, HTML, SGML), or maybe PDF
- habosa 9y ago
  I can see how that might be confusing. We are using "Document" as it's used in database circles: https://en.wikipedia.org/wiki/Document-oriented_database https://en.wikipedia.org/wiki/Document-oriented_database
- sametmax 9y ago
  Why the downvote ? The person is polite and genuinly doesn't know something. Just answer. Besides, a "document database", like many term in computing, can be very confusing. Come on, we all had to be explained what the difference is between a software server vs hardware. This is not different.
  - tannhaeuser 9y ago
    > Why the downvote? Smells like ring voting: 4 downvotes in < 0.5 min? @dang?
    - staunch 9y ago
      Any post having to do with YC, Google, Microsoft, Facebook, Apple, etc are voting ringed to hell by their thousands of employees. As the site has grown, HN has become a bit of a mouthpiece for large organizations through these de facto voting rings. Best idea I have is for HN to add a profile field like: "Organizations: [google]" which would prevent voting on any Google-related submissions. It could also add a disclaimer in each comment of these submissions, so users wouldn't have to remember to do that.
      - 9y ago
jgmmo 9y ago
How is this different than using datastore?
- deleted 9y ago
  [deleted]
- itcmcgrath 9y ago
  Great question! I'm the PM for both Cloud Datastore and Cloud Firestore. The biggest difference is the integration with Firebase, so you have access to Mobile (iOS/Android) and Web SDKs along with a native offline mode. This comes along with the real-time synchronization feature that makes serverless app development a breeze. Cloud Datastore is great for large scale server-side development where you manage your own connection to your app, such as running your own website on App Engine or via Compute/Container Engine.
willchen 9y ago
Looks very exciting! I've used firebase for some small weekend projects and it's been very productive for me. I'm curious, what's the react native support like for Firestore / firebase in general?
- deleted 9y ago
  [deleted]
- adufetel 9y ago
  There's a pretty complete React Native library you can use to build with Firebase services, including Cloud Firestore: https://github.com/invertase/react-native-firebase https://github.com/invertase/react-native-firebase The Cloud Firestore integration is fresh out of the box, but I'd certainly recommend trying it out!
niftich 9y ago
Grats! Can you talk a bit about your consistency model?
- itcmcgrath 9y ago
  Sure, it's strongly consistent on the server-side part (strict serializability), whereas for the mobile/web clients we make sure we move you through consistency snapshots with the real-time sync functionality.
  - irfansharif 9y ago
    Congratulations on the release! > multi-region replicated database [..] once data is committed, it's durable [...] > strongly consistent on the server-side Do you mind elaborating a bit more here? Around perhaps what happens underneath the hood when failing over, etc. Do you have a single "co-ordinator" of sorts ensuring strict serializability, if so what do you do when failing this over? Or is it a quorum based approach like Paxos/Raft?
    - itcmcgrath 9y ago
      Quorum based (Paxos)
nkw 9y ago
This is great. I had a feeling something like this was coming given most firebase people were pretty open about what the shortcomings of the Realtime DB were and AngularFire received some much needed attention to its database API last week. That said, I really hope there are plans for some full text search ability beyond the current suggestions[1]. I would very much like to ditch Elasticsearch in favor of db engine provided search. Even a small subset of the Elasticsearch/Solr feature set (similar to the full text search capability now available for Postgres[2]) would be a very welcome addition. [1] https://firebase.google.com/docs/firestore/solutions/search https://firebase.google.com/docs/firestore/solutions/search [2] https://www.postgresql.org/docs/9.5/static/textsearch.html https://www.postgresql.org/docs/9.5/static/textsearch.html
- itcmcgrath 9y ago
  Thanks for the feedback, this request is definitely been a frequent one. :)
- sampl 9y ago
  I struggled with this too, but recently started using a small backend cloud function to send updates to Algolia (and query docs straight from the front end with the Algolia js script).
hmexx 9y ago
I’m still looking for a product that provides firebase-levels of ease of getting up and running (no API to design, rule-based authentication, etc) , but runs on your own infrastructure, off of a traditional RDBMS. Someone tell me they’ve found the holy grail?!
- joking 9y ago
  You can use Apache usergrid or the open source version of parse for example.
  - hmexx 9y ago
    I believe usergrid runs off of Cassandra and Parse runs off another NoSQL DB. Maybe Mongo?
    - jamesjyu 9y ago
      Yep, Parse runs on Mongo.
- jkarneges 9y ago
  If such a project were based on top of an RDBMS, would you still expect a document/KV-oriented API, or one more SQL-ish?
  - hmexx 9y ago
    Honestly I don't mind. A document API with a little manipulation on the front-end would be fine, as long as the back-end has some way of running through the relations. A SQL-ish API might even be better, although I can see some pitfalls. Maybe base it on GraphQL as a compromise? Either way, I'd love to have something like this.
- bryanlarsen 9y ago
  horizon.io seemed to be a promising entry, but unlike RethinkDB the database, this project appears to have died with Rethink the company.
dotmanish 9y ago
What happens to "Backups" with Firestore? (Firestore console doesn't have a Backups tab)
- adufetel 9y ago
  Thanks for the feedback. We don't have backups for Cloud Firestore yet, but this will be available down the road.
  - afro88 9y ago
    What are our options until this is available? Will built in backups be available once you're out of beta, or further down the road?
adamonkey 9y ago
How is the pricing? Firebase Realtime Database is fairly expensive ($5/GB of storage per month), so it makes economic sense for developers to migrate to their own backend once they hit a certain scale. This is why third-party mobile backends are nowhere near the popularity of non-mobile backends with Google Cloud / AWS. Do you think $0.18 per 100000 writes solves this problem?
- habosa 9y ago
  As you've noticed, the pricing model is totally different for Cloud Firestore. Link for others reading: https://firebase.google.com/docs/firestore/pricing https://firebase.google.com/docs/firestore/pricing If you're concerned about price per GB stored, Cloud Firestore will be much cheaper than Realtime Database (0.18/GiB/month). We evaluated many use cases when deciding on prices and we believe developers will be happy with the new model. However since Cloud Firestore charges by operation, it's important to evaluate your use case when thinking about pricing. For example if you're running a fleet of IoT devices checking in a few times per second with very small payloads, you'd be doing a lot of write operations with very little storage and Cloud Firestore could be more expensive in that case.
  - jopsen 9y ago
    So: firebase <-- presence, real-time editing, chat, in-memory things firestore <-- things that have a save/submit button, transactional database quick question: So the real-time features in firestore are not ideal for real-time text editors and chats. But saving documents from firebase after to user has left is perhaps a good middle ground. Would firestore charge read for each reader that's listening in real-time when a document is updated?
    - habosa 9y ago
      I wouldn't say the difference is exactly that clear cut. Presence is a clear example of a situation where you want to use Realtime Database over Cloud Firestore. And you're right, for something that you would otherwise do "in memory" or use memcached/redis/etc Cloud Firestore may not be the right fit. In my experience both databases are totally appropriate for a chat app. Even though in Cloud Firestore you will pay a document write for each new chat message, that's only $1.80 for a million chat messages and you get all the rich querying from the Cloud Firestore API. Regarding your pricing question: https://firebase.google.com/docs/firestore/pricing#operations https://firebase.google.com/docs/firestore/pricing#operation... > When you listen to the results of a query, you are charged for a read each time a document in the result set is added or updated. You are also charged for a read when a document is removed from the result set because the document has changed. (In constrast, when a document is deleted, you are not charged for a read.)
plexicle 9y ago
Very excited to see a Go SDK for it, too! Can I use it for real-time as well?
- habosa 9y ago
  The Go SDK does not yet have realtime capabilities but that's on our roadmap! Right now you can use realtime with Android, iOS, Web, and Node.
polskibus 9y ago
A couple of questions: What's the underlying synchronisation mechanism? Is it based on CRDTs? How does this product differ from Gun.js and realm.io?
- code-is-code 9y ago
  It would also be great to see a comparison to the couchDB-sync which I currently use via pouchdb/rxdb.
- archagon 9y ago
  If this is similar to what powers GDocs, it's probably OT-based.
- itcmcgrath 9y ago
  It does not use CRDTs as they cannot be used to handle all the update requirements of this system. It uses Paxos as the consensus algorithm, along with internal systems like the TrueTime API to enable us synchronously replicate across multiple data centers.
deepsun 9y ago
Sounds interesting, but is there any independent description of what it cannot do, or what would be hard to do? I just noticed it's much easier to understand a new datastore by reading its limitations (usually carefully omitted from pr articles or documentation). For example, I suspect that Firestore must be built on top of Spanner infrastructure, as it's the only way to get usable cross-datacenter many-row transactions. And Spanner's limitation is it's high price. And if it's not Spanner, but more like Cloud Datastore or the old Megastore, then there should be limitations on transactions. Sounds amazing anyway, more datastorage choices is always better for the world.
- habosa 9y ago
  We try to be honest about what the product can't do natively and provide best practices, take a look at the "solutions" section of the documentation: https://firebase.google.com/docs/firestore/solutions/ https://firebase.google.com/docs/firestore/solutions/ For example Cloud Firestore can't do native full text search, but we made sure it's easy to integrate with a search provider like Algolia. As for your guess about Spanner, you're right that Cloud Firestore uses the same technology as Cloud Spanner to ensure consistency at scale. Would love to know what you think about the product once you've tried it!
- itcmcgrath 9y ago
  We've also tried to be transparent and document our current limits here: https://firebase.google.com/docs/firestore/quotas https://firebase.google.com/docs/firestore/quotas
bikamonki 9y ago
What's the difference with Firebase RTDB?
- gfosco 9y ago
  If you had read the post, you would know this, it even links to a more detailed post specifically about the differences towards the bottom.
- UnfalseDesign 9y ago
  Here's a document they wrote comparing the two: https://firebase.google.com/docs/firestore/rtdb-vs-firestore https://firebase.google.com/docs/firestore/rtdb-vs-firestore
projectpublius 9y ago
Will GeoFire remain solely on RTDB?
- habosa 9y ago
  Cloud Firestore has native support for the "GeoPoint" data type and will eventually have support for native geo queries without any external library. For this reason we are not going to invest in making a GeoFire library for Cloud Firestore and spend that effort getting the native functionality ready.
HammadB 9y ago
This is amazing! I just spent yesterday writing a separate app engine service to use cloud datastore even though the rest of our stuff is on Firebase (RTDB wasn't a fit for us). Great to see this is out
pritambarhate 9y ago
Does any of you feel cheated by Google? Rather than improving the existing product and providing backward compatibility, Google has chosen to build a new product with its own proprietary API. I have a client who has invested significant amount time and money in Firebase Realtime Database. Now with this move, I am not sure if Google will support Firebase Realtime Database for next 5 years. So a full rewrite might be needed. Once more it seems that going Cloud Native on one of Google's proprietary tools is very risky. I know many of the readers will say that Firebase Real-time Database is still supported. But the main question is: will it stay supported for years to come? Google please please make an announcement and make a commitment to keep Firebase Realtime Database alive for "X" years to come. Otherwise, you are just making us developers lose faith in you.
- stefano 9y ago
  This is a very reasonable comment, why is it downvoted so heavily?
- jamest 9y ago
  [Firebase founder] To offer better querying, improve the data model, and increase scalability we had to build an entirely new database. The original technical choices we made as a startup weren't able to support the featureset Cloud Firestore has. Even if we tried to improve upon the existing Realtime Database product, we would have had to make breaking changes that would have required you to rewrite your code (and likely shipped a worse product). Regarding deprecation: you can be comfortable continuing to build on the Realtime Database. We don't intend to deprecate either database, since both are useful in different situations, depending on what you're building. We recommend using the Realtime Database for a number of usecases[1] We're not posting a "Realtime Database will be supported for X years" statement because many may interpret this as "the Realtime Database is deprecating in X years", which isn't the case. [1] https://firebase.googleblog.com/2017/10/cloud-firestore-for-rtdb-developers.html https://firebase.googleblog.com/2017/10/cloud-firestore-for-...
  - pritambarhate 9y ago
    Thanks for the clarification. I certainly feel better with the confirmation. I hope it will give some certainty to thousands of developers out there who decided to make Firebase Realtime Database a critical component of their product architecture.
fictionfuture 9y ago
Looks great, however, I still have hopes that something similar to a GraphQL interface for a SQL based database is coming. NoSQL is cool, but ultimately GraphQL/Apollo serves many of the same issues but has the capability of a much richer, standardized and potentially lower cost backend.
- ruslan_talpa 9y ago
  It's coming https://subzero.cloud/ https://subzero.cloud/ :)
  - fictionfuture 9y ago
    looks cool, interested to see the backend you're using and if it requires lockin. did Kreatank do that logo?
    - ruslan_talpa 9y ago
      the stack is like this graphql/rest -> openresty ( + custom code ) -> postgrest (custom) -> postgresql yes he did :)
troika 9y ago
Great update, congratulations. It sounds similar to https://deepstream.io https://deepstream.io records - small JSON documents that are stored and synced in realtime across clients and can be arranged in lists and reference each other.
stefano 9y ago
I don't like this trend of creating new closed source database systems that only exist on a single cloud provider. Your data is your most valuable asset, and by using this you're locking it inside Google servers. If they decide five years from now to discontinue it, or to raise the pricing 10x, you're screwed. Are most developers only working on short term projects? Why would you put yourself in such a situation instead of using open source technologies that can be deployed anywhere?
- cobookman 9y ago
  > Your data is your most valuable asset, and by using this you're locking it inside Google servers. If they decide five years from now to discontinue it, or to raise the pricing 10x, you're screwed. You'd be able to move your data off of firestore. And there's legal business contracts around pricing. Google can't just raise pricing 10x overnight.
  - stefano 9y ago
    Sure, you can move it, but how much code will you need to rewrite? How much effort to convert the data to a new database format? And how much more development time to migrate your data without taking your service down during the migration? It's a very expensive move that I don't think people consider when choosing this kind of solution.
    - kerryritter 9y ago
      Maybe I'm naive but I think a lot of people consider this, but the benefits of development speed outweigh the risk for a number of cases.
    - anderspitman 9y ago
      Have you used Firebase before? In my (rather limited) experience, the development speed increases from not needing to manage your own Mongo store, not to mention implementing all of their real-time features on top of that, are quite valuable. At least for an MVP I wouldn't hesitate to do the first version with Firebase to test my assumption even if I knew I wasn't going to use it at scale. Plus you can always wrap Firebase in your own layer from the beginning.
wildchild 9y ago
Yet another useless nosql snake oil.
sagadotworld 9y ago
We're one of the startups shown in the announcement (Crypto Portfolio Tracker [1] - bottom right logo). Firestore enabled us to build the first version of our app in under a week. We've been using the product for over two months now, and it's given us a competitive edge in being able to develop features rapidly and not having to spend time on operations. One of the things we liked about Firestore is that it takes the best practices of Realtime Database and makes them more explicit. Before, your RD database structure would look like `collection/{id}` and `collection_sub_collection/{id}/{sub_id}` in order to avoid loading sub-collections in top-level queries. With Firestore, this collections pattern is now part of the API itself, and sub-collections aren't fetched when the parent is fetched. Another feature we liked is that transactions are no longer limited to a subtree of your database. Before, you would have to structure all of your transactional data under a single path. This would sometimes lead to having to pile-in unrelated data into a single object, such as adding payment data under a users object instead of a separate collection, so that you could atomically modify both user and payment data. With Firestore, transactions are global, so this isn't a concern anymore - we are free to structure our data in any way that makes sense for our app. Overall, we had a great experience with Firestore during the alpha, and we'll definitely be keeping it as part of our technology stack. Congrats on the launch! (disclaimer: this post isn't sponsored by Firebase) [1] https://cryptoportfoliotracker.com https://cryptoportfoliotracker.com
- Scarbutt 9y ago
  Is your use of Firebase limited to Firestore? If not, Does Crypto Portfolio runs entirely in Firebase?
  - sagadotworld 9y ago
    We run entirely on Firebase with Auth, Storage, Hosting, Firestore, and Cloud Functions. As a result, we actually don't have any backend to manage.
    - ultrafez 9y ago
      Do you think that the time-to-market advantage that using Firebase offers you justifies the (presumably) increased cost when compared to the cost of managing your own infrastructure?
drumttocs8 9y ago
Should I really be so excited for built-in query support in 2017? For a mature product supported by a dominant tech company? Because I am.
jcz 9y ago
The firestore security rules dsl [1] looks like it could become cumbersome as the number of edge cases increases. I've briefly used firebase realtime database json rules [2] in a side project, and while basic acl scenarios can be covered cleanly, rules quickly tangle without a full programming language. It's hard to check syntax, lint, test, and collaboratively edit. I believe this custom auth layer is best handled by allowing code with restricted api access and time constraints. The technical approach cloudflare used to implement its edge workers by embedding v8/js [3] would be perfect here. [1] https://firebase.google.com/docs/firestore/security/get-started https://firebase.google.com/docs/firestore/security/get-star... example: service cloud.firestore { match /databases/{database}/documents { match /{document=**} { allow read, write: if false; } } } [2] https://firebase.google.com/docs/database/security/ https://firebase.google.com/docs/database/security/ [3] https://blog.cloudflare.com/introducing-cloudflare-workers/ https://blog.cloudflare.com/introducing-cloudflare-workers/
- habosa 9y ago
  The new rules language (which was already used by Cloud Storage) is a full programming language. It's not turing complete, but that's by design. Still, you can declare functions and do other complex things that were never possible with the Firebase Realtime Database JSON rules. We're working on opening up the tools to work with the rules language so you can test and iterate more easily.
jinjin2 9y ago
The offline database experience still looks really primitive compared to Realm, and all the documentation seems to purposefully avoid any details with regards to conflict resolution. Are there any plans to address this?
vira28 9y ago
Can this be used with the old firebase sdk? Upgrading looks like pain
tnolet 9y ago
I'm sorry if I overlooked, but no aggregations? Like calculating sums or averages?
- habosa 9y ago
  That's correct, no native aggregations right now. See this doc for some more info on how to achieve what you want: https://firebase.google.com/docs/firestore/solutions/aggregation https://firebase.google.com/docs/firestore/solutions/aggrega...
  - tnolet 9y ago
    Ok,thanks for the helpful link. I'm hoping native aggregations come soon!
p2t2p 9y ago
Another cool Google tech which will SUDDENLY go extinct when you need it most and which will ignore all of you support requests if it doesn't work properly and which will block you from all of you stuff in Google's system if you break smallest shady statement line in the TOS. Do you really want to base your business on top of Google's stuff? Come on, even MS is more trustworthy than modern Google.
riston 9y ago
I do like the new solution more than the Realtime Database, but I see only this solution to fit smaller scale. Applications with heavy write could get really expensive. Also, the standard plan has only "Maximum write rate to a document 1 per second". The write is currently three times expensive operation as read hopefully, this would be lowered in future.
alexnewman 9y ago
Why anyone still uses firebase when there's trivial to get running OSS ways of getting this done I'll never understand. People think stuff like this makes time to development smaller, I say it increases it deceptively
- Kiro 9y ago
  I'm a one-man shop and the less time I need to spend with servers the better. The Firebase + Google Cloud Functions approach has served me well.
  - alexnewman 9y ago
    Yea if I could give you a script to run which made setting up your own version of those would you?
ralusek 9y ago
Couple of questions. For the querying: https://firebase.google.com/docs/firestore/query-data/queries https://firebase.google.com/docs/firestore/query-data/querie... It seems as though there is just the capability for AND logic between the WHERE queries, is that correct. Is the any plans to add AND/OR and perhaps a concept for representing grouping like parentheses? And for the data structures: https://firebase.google.com/docs/firestore/manage-data/structure-data https://firebase.google.com/docs/firestore/manage-data/struc... Is there any capability to perform joins or have the data populated at query time? And for the last thing: It is mentioned multiple times in the documentation that nested collections will not be deleted if the parent is deleted. I'm just curious why there isn't the capability to insert a document with options that would allow something like a cascade delete. Since you allow indexes on collections, there is obviously some sort of metadata maintained, why not just add an additional flag that could be set so that whenever a record is deleted it can optionally have its subcollections removed as well?
jadbox 9y ago
Hmm, is there react-native support?
- jamest 9y ago
  Yes: https://github.com/invertase/react-native-firebase https://github.com/invertase/react-native-firebase
kkotak 9y ago
I just hope your Firestore Status Dashboard is going to look very differerent than your Realtime DB one. I mean - much less yellow and red dots.
ben75011 9y ago
Having a consistent relational db background, I always feel a bit lost about the patterns provided to associate models in no-sql land. Firebase db was 'weird' when it came to associations, don't see that much changes here. The feature is available, but... Fire store provide 3 ways to do so, but why ? Going through the docs the results/benefits for each patterns remains unclear, the way to query each of those is not really understandable reading the docs. is this state will remains or is it a field that is potentially to be improved in the next releases ?
- ben75011 9y ago
  It might just be me misinterpreting your example, but having created the required records to run the following command ('var messageRef = db.collection('rooms').doc('roomA').collection('messages').doc('message1');'), i don't see no results. Or people is expected to understand that 'message1' stands for an object ?
fiokoden 9y ago
Just never use Google for building applications. They have invested heavily over the lifetime of the company in avoiding providing support. That reputation can never be revived. Google does not have direct customer support in its DNA, it has the opposite, whatever that is. They have not demonstrated relentless commitment to being available to resolve issues, and nothing matters more than this if you've bet your company on their platform. If something goes really wrong. Amazon has your back and you'll find someone who'll listen who has the power to resolve it. Google, you're stuffed. If you've built you're business around that thing that went wrong, well time for regret.
- andygeers 9y ago
  Personally, this has not been my experience of the Firebase team actually - for me they have always gone out of their way to help find solutions and give all kinds of advice.
us0r 9y ago
So read through the docs, decided to give the free tier a go and of course they are punishing me for being an existing customer. "The Blaze billing plan (pay-as-you-go) for Firebase is required for Google projects with billing enabled. To use the free tier, you must first turn off billing in your Google project." I don't want 15 "projects". I want to add this to my existing one. Why on earth is this not possible?
unitboolean 9y ago
Cosmos DB from Microsoft Azure seems to be a much better choose, because it provides tons of useful features, but it also compatible with MongoDB, so if they decide to increase prices or shut it down some day, you will be able to use regular MongoDB instead. With Firestore it's not the case and you are locked with a proprietary solution.

377 points by crb 9y ago

[Firebase founder] This new database has been in the works for 2.5 years, since shortly after we joined Google. It was developed in close collaboration with the Cloud Datastore[1] team, and uses Google’s core database infrastructure. We built it because we know it can be challenging to build complex apps with our original database -- Firebase Realtime Database -- where we optimized for ease-of-use & real-time sync over querying functionality. For more info, see this comparison between the two[2]. This is a open beta launch, so the product has limitations[3] you should be aware of. We’ll be working to remove/raise these before General Availability. A note on naming: Firebase launched in 2012[4] with a single product, the original database. As we added products (Firebase Auth / Firebase Hosting / etc), we began calling the original product ‘Firebase Realtime Database’, or RTDB for short. If you haven’t followed us lately, we’ve grown to become Google’s app platform, and now have 16 products to help you build/grow apps. We’re grateful for the support HN has given us over the years and we hope you enjoy Cloud Firestore! [1] https://cloud.google.com/datastore/ https://cloud.google.com/datastore/ [2] https://firebase.google.com/docs/firestore/rtdb-vs-firestore https://firebase.google.com/docs/firestore/rtdb-vs-firestore [3] https://firebase.google.com/docs/firestore/quotas https://firebase.google.com/docs/firestore/quotas [4] https://news.ycombinator.com/item?id=3832877 https://news.ycombinator.com/item?id=3832877

aphextron 9y ago
Very exciting. I found this to be the one "missing piece" keeping me from being able to build a whole app on Firebase when checking it out earlier this year. One question though, will this support BLOBs like GridFS?
- habosa 9y ago
  You can store Blob data, but the maximum size of a single document is still 1MB. If you need to store larger blobs check out Cloud Storage for Firebase.
  - e12e 9y ago
    Could you share some of the reasoning behind arriving at a 1mb limit (why not 10mb or 512kb for example)?
    - itcmcgrath 9y ago
      Hard to explain externally, but it relates to RAM size allocation + throughput in the underlying distributed database system.
TY 9y ago
Just read through your link #2 and was very impressed. Firestore addresses every single concern I had with RDTB and more and I'm looking forward to using it in my side projects. Thank you!
- mayop100 9y ago
  That's great to hear! Our design for Cloud Firestore was based on years of developer feedback. Glad we hit the mark for you : )
judah 9y ago
What's your story on ACID, particularly on transactions? Transactions: If I want to insert 2 documents, but zero if either fails, what does that look like with Firestore? And if I store a document and run a query milliseconds later, will the query include document I just stored? Or are queries eventually consistent?
- habosa 9y ago
  Cloud Firestore has support for transactions: https://firebase.google.com/docs/firestore/manage-data/update-data#update_data_with_transactions https://firebase.google.com/docs/firestore/manage-data/updat... So yes, if any of the operations in the transaction fails (and cannot be retried) the whole transaction will fail. It's atomic as you'd expect. If you've ever used transactions in Firebase Realtime Database you'll be happy to know that the transactions in Cloud Firestore are much easier to use since they don't require selecting a common data parent. Queries are always strongly consistent. If you do a write-read sequence you'll see the value of the latest write.
  - ibdknox 9y ago
    How do you deal with consistency in the offline case? E.g. If I make a bunch of edits while disconnected, and others have made edits while connected, how are the conflicts resolved?
    - mikelehen 9y ago
      This works similar to the Realtime Database in that it's last-write wins (where in the offline case, "last" is the last person to come back online and send their write to the backend). This model is very easy for developers to understand and directly solves many use cases, especially since we allow very granular writes which reduces the risk of conflicts. But for more complex use cases, you can get clever and implement things like OT conflict resolution as a layer on top of last-write wins, e.g. similar to how we implemented collaborative editing with www.firepad.io on the realtime database. PS: Hi Chris! :-)
cfilipov 9y ago
Regarding offline mode. How is sync done? Is this OT based? How are conflicts resolved?
- mikelehen 9y ago
  Good question, and to answer this well we should probably do a blog post or something. In the meantime you could dig into the code since the clients are all open source. :-) But basically, sync is split into two halves: writes and listens. Clients store pending writes locally until they're flushed to the backend (which could be a long time if the app is running offline). While online, listen results are streamed from the backend and persisted in a local client cache so that the results will also be visible while offline (and any pending writes are merged into this offline view). When a client comes back online, it flushes its pending writes to the backend which are executed in a last-write-wins manner (see my answer above to ibdknox for more details on this). To resume listens, the client can use a "resume token" which allows the backend to quickly get the client back up-to-date without needing to re-send already retrieved results (there are some nuances here depending on how old the resume token is, etc.).
  - cfilipov 9y ago
    Thank you!
wahnfrieden 9y ago
Do you have a rough timeline on beta => General Availability?
- itcmcgrath 9y ago
  Nothing we can share at this time. Really looking forward to the feedback we get now we're in beta as this will feed into our GA plans.
sandGorgon 9y ago
Do you see this as an equivalent to Dynamodb ? Very timely ;)
- jaxondu 9y ago
  DynamoDB does not come with offline sync.
gfosco 9y ago
Awesome to see this. I couldn't get behind the RTDB structure, but Cloud Firestore I will definitely use!
- inlined 9y ago
  Woohoo! Is this an official parse endorsement? (Hi Fosco )
  - gfosco 9y ago
    Hey Thomas :) I speak only for myself, Firebase just got way better.
    - inlined 9y ago
      I'm glad. Hoping you like cloud functions too!
insomniacity 9y ago
How goes the work to improve customer service and support? Thinking about https://news.ycombinator.com/item?id=14359801 https://news.ycombinator.com/item?id=14359801, what improvements have you made in the last 139 days that would make us want to build on this?
- steego 9y ago
  For their own sake, I hope the team gives this question attention and respect. What sets Amazon apart from many other companies is their reputation for relentless customer service. Customers pay close attention to the quality of answer or lack of answers to questions like this one. Every cloud company that asks us to invest our time in their proprietary API is asking us to trust them. If you violate one vocal customer's trust, we're going to notice. If you don't make things right, we're going to notice. Even before I read the article, I first read the comments to gauge the reception because I care more about the perception of trust before I even consider using something like this. Customers don't always stop trusting companies for fair or valid reasons, but that doesn't matter. Trust is much more of a visceral thing than a cerebral thing. So here's my question to other HN readers. What would Google need to do to improve its reputation for customer service? What would they need to do to make that commitment resonate with you on a visceral level?
  - techdragon 9y ago
    For me it's pretty easy. I want to start hearing and seeing positive anecdotes about real time communication with Google Cloud support. That is to say, I want to hear people start saying things like "I rang them, and everything was sorted out" ideally with a platitude attached like "it was actually a pretty good experience". I hear this about AWS all the time, and I've even experienced it myself. One client some time ago had an AWS novice who was confident they could handle setting up the auto-scale groups. They made a small mistake, which lead to the scaling trigger being permanently on, and it auto scaling to 1024 ec2 nodes within the first half hour. Immediately after deploying a fix for the scaling trigger, I phone AWS to see if there was anything that could be done about the cost of this accident, and I had a small monthly credit to compensate, and a respectable discount on the next appropriate AWS training course so that the novice could learn more and hopefully avoid future mistakes. THAT is the kind of story I want to start hearing about Google Cloud. "I used it and did not have problems", and "It works and was cheaper than AWS" is just not enough.
novaleaf 9y ago
glad you mentioned Cloud Datastore. It seems that Firestore supersedes Datastore and Firebase. Is that correct? Any comparisons of Firestore vs Datastore?
- habosa 9y ago
  It does not "supersede" Datastore or Firebase Realtime Database, neither of those products are going anywhere. This is a totally new product that builds on what we learned from those two products, sharing some of the best features of each and bringing in some totally new things as well. All three are (in my opinion) great choices for building a new app today so make sure you evaluate all options.
  - revmoo 9y ago
    You posted this after two controversial but fair questions has been asked. Please answer them.
mjw1007 9y ago
Can you provide anything to reassure potential users that this will still exist in five or ten years time?
- mbleigh 9y ago
  The Firebase Realtime Database has been around for ~5 years (and is still fully supported after this launch). Cloud Datastore has been around in some form (started as App Engine Datastore) since 2008. Cloud Firestore was a massive multi-year joint effort between Firebase and Google Cloud. Google is investing heavily in both, and this is a big deal for our teams.
  - mjw1007 9y ago
    Perhaps you could consider persuading the powers that be to make a loud public commitment. After all, if they are truly confident, there should be little cost in doing so.
    - mbleigh 9y ago
      We announced the product, that's a loud public commitment. Once it reaches general availability, it will be covered by the Cloud deprecation policy requiring a minimum of one year notice for deprecation. I'm not sure what other guarantees would even make sense to offer. If anything, I'd look at this announcement as ongoing proof in the magnitude of investment Google is making in Firebase and Cloud.
      - Cthulhu_ 9y ago
        I'd guess a forward commitment, like a LTS version? That would probably have an adverse effect though, since you'd be the only provider that goes (e.g.) "We will support this product for at least three years from now", implying (to people making Decisions) you'd pull the plug after three years. Maybe publish a long term (5+ year) plan / roadmap? idk.
        9y ago
jonasb 9y ago
The enhanced query capabilities are very much appreciated. I hope full-text search is in the works, since adding a 3rd party FTS engine adds quite a bit of complexity.
paulsharratt 9y ago
Good work guys, sounds like good progress! I'm struggling to see what this is giving me that I didn't have with RTDB though The queries seem to be doing what orderByChild equalTo startAt endAt limitToFirst and limitToLast were already allowing. Is there mostly a performance gain, or am I missing something that I can do now that I couldn't do before? Cheers
- paulsharratt 9y ago
  I'll answer myself here for anyone else trying to figure out what we can do with queries now: --- We can now apply a filter and sort in the same command We can chain filters, though they can only apply to one field if they specify a range We can make a query that will only return documents, not the subfield data (i.e. what a lot of us were doing with the deprecated REST interface shallow=1) Queries in large data sets will be faster I guess? Things it'd still be nice to see - References that allow you to pull down related documents with a single query OR queries --- It's been a bit of rollercoaster seeing this announcement.. "Oh wow we can do proper queries now! Oh wait no, the queries don't seem to allow more than what we had before.. oh wait there are some improvements, it's a bit better now.. oh the shallow querying will be super handy! and they do geopoints.. but don't appear to have any way of searching for radius.. but they say they will soon" Anyway, good work all the same :)
ec109685 9y ago
While I know this is a beta and you’re solving a difficult technical problem, some of the limitations are concernig. With a system like dynamo, you can in most cases pay your way out of bottlenecks. It would be nice to develop a way in Firestore for users to be able to pay for the number of shards assigned to their workload to reduce relatively artificial limits such as write limits within collections and index update rate. That way, developers wouldn’t have to plan to leave the platform if their app is successful.

tannhaeuser 9y ago

> Document Database for Apps From linked page: > designed to easily store and sync app data Maybe I'm misunderstanding, but that's not what I understand a "document" to be. For me a document - is a file that can be stored on a file system - can be send via mail - is using a standard document representation so can be used by different applications, such as markup (XML, HTML, SGML), or maybe PDF

habosa 9y ago
I can see how that might be confusing. We are using "Document" as it's used in database circles: https://en.wikipedia.org/wiki/Document-oriented_database https://en.wikipedia.org/wiki/Document-oriented_database
sametmax 9y ago
Why the downvote ? The person is polite and genuinly doesn't know something. Just answer. Besides, a "document database", like many term in computing, can be very confusing. Come on, we all had to be explained what the difference is between a software server vs hardware. This is not different.
- tannhaeuser 9y ago
  > Why the downvote? Smells like ring voting: 4 downvotes in < 0.5 min? @dang?
  - staunch 9y ago
    Any post having to do with YC, Google, Microsoft, Facebook, Apple, etc are voting ringed to hell by their thousands of employees. As the site has grown, HN has become a bit of a mouthpiece for large organizations through these de facto voting rings. Best idea I have is for HN to add a profile field like: "Organizations: [google]" which would prevent voting on any Google-related submissions. It could also add a disclaimer in each comment of these submissions, so users wouldn't have to remember to do that.
    - 9y ago

jgmmo 9y ago

How is this different than using datastore?

deleted 9y ago
[deleted]
itcmcgrath 9y ago
Great question! I'm the PM for both Cloud Datastore and Cloud Firestore. The biggest difference is the integration with Firebase, so you have access to Mobile (iOS/Android) and Web SDKs along with a native offline mode. This comes along with the real-time synchronization feature that makes serverless app development a breeze. Cloud Datastore is great for large scale server-side development where you manage your own connection to your app, such as running your own website on App Engine or via Compute/Container Engine.

willchen 9y ago

Looks very exciting! I've used firebase for some small weekend projects and it's been very productive for me. I'm curious, what's the react native support like for Firestore / firebase in general?

deleted 9y ago
[deleted]
adufetel 9y ago
There's a pretty complete React Native library you can use to build with Firebase services, including Cloud Firestore: https://github.com/invertase/react-native-firebase https://github.com/invertase/react-native-firebase The Cloud Firestore integration is fresh out of the box, but I'd certainly recommend trying it out!

niftich 9y ago

Grats! Can you talk a bit about your consistency model?

itcmcgrath 9y ago
Sure, it's strongly consistent on the server-side part (strict serializability), whereas for the mobile/web clients we make sure we move you through consistency snapshots with the real-time sync functionality.
- irfansharif 9y ago
  Congratulations on the release! > multi-region replicated database [..] once data is committed, it's durable [...] > strongly consistent on the server-side Do you mind elaborating a bit more here? Around perhaps what happens underneath the hood when failing over, etc. Do you have a single "co-ordinator" of sorts ensuring strict serializability, if so what do you do when failing this over? Or is it a quorum based approach like Paxos/Raft?
  - itcmcgrath 9y ago
    Quorum based (Paxos)

nkw 9y ago

This is great. I had a feeling something like this was coming given most firebase people were pretty open about what the shortcomings of the Realtime DB were and AngularFire received some much needed attention to its database API last week. That said, I really hope there are plans for some full text search ability beyond the current suggestions[1]. I would very much like to ditch Elasticsearch in favor of db engine provided search. Even a small subset of the Elasticsearch/Solr feature set (similar to the full text search capability now available for Postgres[2]) would be a very welcome addition. [1] https://firebase.google.com/docs/firestore/solutions/search https://firebase.google.com/docs/firestore/solutions/search [2] https://www.postgresql.org/docs/9.5/static/textsearch.html https://www.postgresql.org/docs/9.5/static/textsearch.html

itcmcgrath 9y ago
Thanks for the feedback, this request is definitely been a frequent one. :)
sampl 9y ago
I struggled with this too, but recently started using a small backend cloud function to send updates to Algolia (and query docs straight from the front end with the Algolia js script).

hmexx 9y ago

I’m still looking for a product that provides firebase-levels of ease of getting up and running (no API to design, rule-based authentication, etc) , but runs on your own infrastructure, off of a traditional RDBMS. Someone tell me they’ve found the holy grail?!

joking 9y ago
You can use Apache usergrid or the open source version of parse for example.
- hmexx 9y ago
  I believe usergrid runs off of Cassandra and Parse runs off another NoSQL DB. Maybe Mongo?
  - jamesjyu 9y ago
    Yep, Parse runs on Mongo.
jkarneges 9y ago
If such a project were based on top of an RDBMS, would you still expect a document/KV-oriented API, or one more SQL-ish?
- hmexx 9y ago
  Honestly I don't mind. A document API with a little manipulation on the front-end would be fine, as long as the back-end has some way of running through the relations. A SQL-ish API might even be better, although I can see some pitfalls. Maybe base it on GraphQL as a compromise? Either way, I'd love to have something like this.
bryanlarsen 9y ago
horizon.io seemed to be a promising entry, but unlike RethinkDB the database, this project appears to have died with Rethink the company.

dotmanish 9y ago

What happens to "Backups" with Firestore? (Firestore console doesn't have a Backups tab)

adufetel 9y ago
Thanks for the feedback. We don't have backups for Cloud Firestore yet, but this will be available down the road.
- afro88 9y ago
  What are our options until this is available? Will built in backups be available once you're out of beta, or further down the road?

adamonkey 9y ago

How is the pricing? Firebase Realtime Database is fairly expensive ($5/GB of storage per month), so it makes economic sense for developers to migrate to their own backend once they hit a certain scale. This is why third-party mobile backends are nowhere near the popularity of non-mobile backends with Google Cloud / AWS. Do you think $0.18 per 100000 writes solves this problem?

habosa 9y ago
As you've noticed, the pricing model is totally different for Cloud Firestore. Link for others reading: https://firebase.google.com/docs/firestore/pricing https://firebase.google.com/docs/firestore/pricing If you're concerned about price per GB stored, Cloud Firestore will be much cheaper than Realtime Database (0.18/GiB/month). We evaluated many use cases when deciding on prices and we believe developers will be happy with the new model. However since Cloud Firestore charges by operation, it's important to evaluate your use case when thinking about pricing. For example if you're running a fleet of IoT devices checking in a few times per second with very small payloads, you'd be doing a lot of write operations with very little storage and Cloud Firestore could be more expensive in that case.
- jopsen 9y ago
  So: firebase <-- presence, real-time editing, chat, in-memory things firestore <-- things that have a save/submit button, transactional database quick question: So the real-time features in firestore are not ideal for real-time text editors and chats. But saving documents from firebase after to user has left is perhaps a good middle ground. Would firestore charge read for each reader that's listening in real-time when a document is updated?
  - habosa 9y ago
    I wouldn't say the difference is exactly that clear cut. Presence is a clear example of a situation where you want to use Realtime Database over Cloud Firestore. And you're right, for something that you would otherwise do "in memory" or use memcached/redis/etc Cloud Firestore may not be the right fit. In my experience both databases are totally appropriate for a chat app. Even though in Cloud Firestore you will pay a document write for each new chat message, that's only $1.80 for a million chat messages and you get all the rich querying from the Cloud Firestore API. Regarding your pricing question: https://firebase.google.com/docs/firestore/pricing#operations https://firebase.google.com/docs/firestore/pricing#operation... > When you listen to the results of a query, you are charged for a read each time a document in the result set is added or updated. You are also charged for a read when a document is removed from the result set because the document has changed. (In constrast, when a document is deleted, you are not charged for a read.)

plexicle 9y ago

Very excited to see a Go SDK for it, too! Can I use it for real-time as well?

habosa 9y ago
The Go SDK does not yet have realtime capabilities but that's on our roadmap! Right now you can use realtime with Android, iOS, Web, and Node.

polskibus 9y ago

A couple of questions: What's the underlying synchronisation mechanism? Is it based on CRDTs? How does this product differ from Gun.js and realm.io?

code-is-code 9y ago
It would also be great to see a comparison to the couchDB-sync which I currently use via pouchdb/rxdb.
archagon 9y ago
If this is similar to what powers GDocs, it's probably OT-based.
itcmcgrath 9y ago
It does not use CRDTs as they cannot be used to handle all the update requirements of this system. It uses Paxos as the consensus algorithm, along with internal systems like the TrueTime API to enable us synchronously replicate across multiple data centers.

deepsun 9y ago

Sounds interesting, but is there any independent description of what it cannot do, or what would be hard to do? I just noticed it's much easier to understand a new datastore by reading its limitations (usually carefully omitted from pr articles or documentation). For example, I suspect that Firestore must be built on top of Spanner infrastructure, as it's the only way to get usable cross-datacenter many-row transactions. And Spanner's limitation is it's high price. And if it's not Spanner, but more like Cloud Datastore or the old Megastore, then there should be limitations on transactions. Sounds amazing anyway, more datastorage choices is always better for the world.

habosa 9y ago
We try to be honest about what the product can't do natively and provide best practices, take a look at the "solutions" section of the documentation: https://firebase.google.com/docs/firestore/solutions/ https://firebase.google.com/docs/firestore/solutions/ For example Cloud Firestore can't do native full text search, but we made sure it's easy to integrate with a search provider like Algolia. As for your guess about Spanner, you're right that Cloud Firestore uses the same technology as Cloud Spanner to ensure consistency at scale. Would love to know what you think about the product once you've tried it!
itcmcgrath 9y ago
We've also tried to be transparent and document our current limits here: https://firebase.google.com/docs/firestore/quotas https://firebase.google.com/docs/firestore/quotas

bikamonki 9y ago

What's the difference with Firebase RTDB?

gfosco 9y ago
If you had read the post, you would know this, it even links to a more detailed post specifically about the differences towards the bottom.
UnfalseDesign 9y ago
Here's a document they wrote comparing the two: https://firebase.google.com/docs/firestore/rtdb-vs-firestore https://firebase.google.com/docs/firestore/rtdb-vs-firestore

projectpublius 9y ago

Will GeoFire remain solely on RTDB?

habosa 9y ago
Cloud Firestore has native support for the "GeoPoint" data type and will eventually have support for native geo queries without any external library. For this reason we are not going to invest in making a GeoFire library for Cloud Firestore and spend that effort getting the native functionality ready.

HammadB 9y ago

This is amazing! I just spent yesterday writing a separate app engine service to use cloud datastore even though the rest of our stuff is on Firebase (RTDB wasn't a fit for us). Great to see this is out

pritambarhate 9y ago

Does any of you feel cheated by Google? Rather than improving the existing product and providing backward compatibility, Google has chosen to build a new product with its own proprietary API. I have a client who has invested significant amount time and money in Firebase Realtime Database. Now with this move, I am not sure if Google will support Firebase Realtime Database for next 5 years. So a full rewrite might be needed. Once more it seems that going Cloud Native on one of Google's proprietary tools is very risky. I know many of the readers will say that Firebase Real-time Database is still supported. But the main question is: will it stay supported for years to come? Google please please make an announcement and make a commitment to keep Firebase Realtime Database alive for "X" years to come. Otherwise, you are just making us developers lose faith in you.

stefano 9y ago
This is a very reasonable comment, why is it downvoted so heavily?
jamest 9y ago
[Firebase founder] To offer better querying, improve the data model, and increase scalability we had to build an entirely new database. The original technical choices we made as a startup weren't able to support the featureset Cloud Firestore has. Even if we tried to improve upon the existing Realtime Database product, we would have had to make breaking changes that would have required you to rewrite your code (and likely shipped a worse product). Regarding deprecation: you can be comfortable continuing to build on the Realtime Database. We don't intend to deprecate either database, since both are useful in different situations, depending on what you're building. We recommend using the Realtime Database for a number of usecases[1] We're not posting a "Realtime Database will be supported for X years" statement because many may interpret this as "the Realtime Database is deprecating in X years", which isn't the case. [1] https://firebase.googleblog.com/2017/10/cloud-firestore-for-rtdb-developers.html https://firebase.googleblog.com/2017/10/cloud-firestore-for-...
- pritambarhate 9y ago
  Thanks for the clarification. I certainly feel better with the confirmation. I hope it will give some certainty to thousands of developers out there who decided to make Firebase Realtime Database a critical component of their product architecture.

fictionfuture 9y ago

Looks great, however, I still have hopes that something similar to a GraphQL interface for a SQL based database is coming. NoSQL is cool, but ultimately GraphQL/Apollo serves many of the same issues but has the capability of a much richer, standardized and potentially lower cost backend.

ruslan_talpa 9y ago
It's coming https://subzero.cloud/ https://subzero.cloud/ :)
- fictionfuture 9y ago
  looks cool, interested to see the backend you're using and if it requires lockin. did Kreatank do that logo?
  - ruslan_talpa 9y ago
    the stack is like this graphql/rest -> openresty ( + custom code ) -> postgrest (custom) -> postgresql yes he did :)

troika 9y ago

Great update, congratulations. It sounds similar to https://deepstream.io https://deepstream.io records - small JSON documents that are stored and synced in realtime across clients and can be arranged in lists and reference each other.

stefano 9y ago

I don't like this trend of creating new closed source database systems that only exist on a single cloud provider. Your data is your most valuable asset, and by using this you're locking it inside Google servers. If they decide five years from now to discontinue it, or to raise the pricing 10x, you're screwed. Are most developers only working on short term projects? Why would you put yourself in such a situation instead of using open source technologies that can be deployed anywhere?

cobookman 9y ago
> Your data is your most valuable asset, and by using this you're locking it inside Google servers. If they decide five years from now to discontinue it, or to raise the pricing 10x, you're screwed. You'd be able to move your data off of firestore. And there's legal business contracts around pricing. Google can't just raise pricing 10x overnight.
- stefano 9y ago
  Sure, you can move it, but how much code will you need to rewrite? How much effort to convert the data to a new database format? And how much more development time to migrate your data without taking your service down during the migration? It's a very expensive move that I don't think people consider when choosing this kind of solution.
  - kerryritter 9y ago
    Maybe I'm naive but I think a lot of people consider this, but the benefits of development speed outweigh the risk for a number of cases.
  - anderspitman 9y ago
    Have you used Firebase before? In my (rather limited) experience, the development speed increases from not needing to manage your own Mongo store, not to mention implementing all of their real-time features on top of that, are quite valuable. At least for an MVP I wouldn't hesitate to do the first version with Firebase to test my assumption even if I knew I wasn't going to use it at scale. Plus you can always wrap Firebase in your own layer from the beginning.

wildchild 9y ago

Yet another useless nosql snake oil.

sagadotworld 9y ago

We're one of the startups shown in the announcement (Crypto Portfolio Tracker [1] - bottom right logo). Firestore enabled us to build the first version of our app in under a week. We've been using the product for over two months now, and it's given us a competitive edge in being able to develop features rapidly and not having to spend time on operations. One of the things we liked about Firestore is that it takes the best practices of Realtime Database and makes them more explicit. Before, your RD database structure would look like `collection/{id}` and `collection_sub_collection/{id}/{sub_id}` in order to avoid loading sub-collections in top-level queries. With Firestore, this collections pattern is now part of the API itself, and sub-collections aren't fetched when the parent is fetched. Another feature we liked is that transactions are no longer limited to a subtree of your database. Before, you would have to structure all of your transactional data under a single path. This would sometimes lead to having to pile-in unrelated data into a single object, such as adding payment data under a users object instead of a separate collection, so that you could atomically modify both user and payment data. With Firestore, transactions are global, so this isn't a concern anymore - we are free to structure our data in any way that makes sense for our app. Overall, we had a great experience with Firestore during the alpha, and we'll definitely be keeping it as part of our technology stack. Congrats on the launch! (disclaimer: this post isn't sponsored by Firebase) [1] https://cryptoportfoliotracker.com https://cryptoportfoliotracker.com

Scarbutt 9y ago
Is your use of Firebase limited to Firestore? If not, Does Crypto Portfolio runs entirely in Firebase?
- sagadotworld 9y ago
  We run entirely on Firebase with Auth, Storage, Hosting, Firestore, and Cloud Functions. As a result, we actually don't have any backend to manage.
  - ultrafez 9y ago
    Do you think that the time-to-market advantage that using Firebase offers you justifies the (presumably) increased cost when compared to the cost of managing your own infrastructure?

drumttocs8 9y ago

Should I really be so excited for built-in query support in 2017? For a mature product supported by a dominant tech company? Because I am.

jcz 9y ago

The firestore security rules dsl [1] looks like it could become cumbersome as the number of edge cases increases. I've briefly used firebase realtime database json rules [2] in a side project, and while basic acl scenarios can be covered cleanly, rules quickly tangle without a full programming language. It's hard to check syntax, lint, test, and collaboratively edit. I believe this custom auth layer is best handled by allowing code with restricted api access and time constraints. The technical approach cloudflare used to implement its edge workers by embedding v8/js [3] would be perfect here. [1] https://firebase.google.com/docs/firestore/security/get-started https://firebase.google.com/docs/firestore/security/get-star... example: service cloud.firestore { match /databases/{database}/documents { match /{document=**} { allow read, write: if false; } } } [2] https://firebase.google.com/docs/database/security/ https://firebase.google.com/docs/database/security/ [3] https://blog.cloudflare.com/introducing-cloudflare-workers/ https://blog.cloudflare.com/introducing-cloudflare-workers/

habosa 9y ago
The new rules language (which was already used by Cloud Storage) is a full programming language. It's not turing complete, but that's by design. Still, you can declare functions and do other complex things that were never possible with the Firebase Realtime Database JSON rules. We're working on opening up the tools to work with the rules language so you can test and iterate more easily.

jinjin2 9y ago

The offline database experience still looks really primitive compared to Realm, and all the documentation seems to purposefully avoid any details with regards to conflict resolution. Are there any plans to address this?

vira28 9y ago

Can this be used with the old firebase sdk? Upgrading looks like pain

tnolet 9y ago

I'm sorry if I overlooked, but no aggregations? Like calculating sums or averages?

habosa 9y ago
That's correct, no native aggregations right now. See this doc for some more info on how to achieve what you want: https://firebase.google.com/docs/firestore/solutions/aggregation https://firebase.google.com/docs/firestore/solutions/aggrega...
- tnolet 9y ago
  Ok,thanks for the helpful link. I'm hoping native aggregations come soon!

p2t2p 9y ago

Another cool Google tech which will SUDDENLY go extinct when you need it most and which will ignore all of you support requests if it doesn't work properly and which will block you from all of you stuff in Google's system if you break smallest shady statement line in the TOS. Do you really want to base your business on top of Google's stuff? Come on, even MS is more trustworthy than modern Google.

riston 9y ago

I do like the new solution more than the Realtime Database, but I see only this solution to fit smaller scale. Applications with heavy write could get really expensive. Also, the standard plan has only "Maximum write rate to a document 1 per second". The write is currently three times expensive operation as read hopefully, this would be lowered in future.

alexnewman 9y ago

Why anyone still uses firebase when there's trivial to get running OSS ways of getting this done I'll never understand. People think stuff like this makes time to development smaller, I say it increases it deceptively

Kiro 9y ago
I'm a one-man shop and the less time I need to spend with servers the better. The Firebase + Google Cloud Functions approach has served me well.
- alexnewman 9y ago
  Yea if I could give you a script to run which made setting up your own version of those would you?

ralusek 9y ago

Couple of questions. For the querying: https://firebase.google.com/docs/firestore/query-data/queries https://firebase.google.com/docs/firestore/query-data/querie... It seems as though there is just the capability for AND logic between the WHERE queries, is that correct. Is the any plans to add AND/OR and perhaps a concept for representing grouping like parentheses? And for the data structures: https://firebase.google.com/docs/firestore/manage-data/structure-data https://firebase.google.com/docs/firestore/manage-data/struc... Is there any capability to perform joins or have the data populated at query time? And for the last thing: It is mentioned multiple times in the documentation that nested collections will not be deleted if the parent is deleted. I'm just curious why there isn't the capability to insert a document with options that would allow something like a cascade delete. Since you allow indexes on collections, there is obviously some sort of metadata maintained, why not just add an additional flag that could be set so that whenever a record is deleted it can optionally have its subcollections removed as well?

jadbox 9y ago

Hmm, is there react-native support?

jamest 9y ago
Yes: https://github.com/invertase/react-native-firebase https://github.com/invertase/react-native-firebase

kkotak 9y ago

I just hope your Firestore Status Dashboard is going to look very differerent than your Realtime DB one. I mean - much less yellow and red dots.

ben75011 9y ago

Having a consistent relational db background, I always feel a bit lost about the patterns provided to associate models in no-sql land. Firebase db was 'weird' when it came to associations, don't see that much changes here. The feature is available, but... Fire store provide 3 ways to do so, but why ? Going through the docs the results/benefits for each patterns remains unclear, the way to query each of those is not really understandable reading the docs. is this state will remains or is it a field that is potentially to be improved in the next releases ?

ben75011 9y ago
It might just be me misinterpreting your example, but having created the required records to run the following command ('var messageRef = db.collection('rooms').doc('roomA').collection('messages').doc('message1');'), i don't see no results. Or people is expected to understand that 'message1' stands for an object ?

fiokoden 9y ago

Just never use Google for building applications. They have invested heavily over the lifetime of the company in avoiding providing support. That reputation can never be revived. Google does not have direct customer support in its DNA, it has the opposite, whatever that is. They have not demonstrated relentless commitment to being available to resolve issues, and nothing matters more than this if you've bet your company on their platform. If something goes really wrong. Amazon has your back and you'll find someone who'll listen who has the power to resolve it. Google, you're stuffed. If you've built you're business around that thing that went wrong, well time for regret.

andygeers 9y ago
Personally, this has not been my experience of the Firebase team actually - for me they have always gone out of their way to help find solutions and give all kinds of advice.

us0r 9y ago

So read through the docs, decided to give the free tier a go and of course they are punishing me for being an existing customer. "The Blaze billing plan (pay-as-you-go) for Firebase is required for Google projects with billing enabled. To use the free tier, you must first turn off billing in your Google project." I don't want 15 "projects". I want to add this to my existing one. Why on earth is this not possible?

unitboolean 9y ago

Cosmos DB from Microsoft Azure seems to be a much better choose, because it provides tons of useful features, but it also compatible with MongoDB, so if they decide to increase prices or shut it down some day, you will be able to use regular MongoDB instead. With Firestore it's not the case and you are locked with a proprietary solution.