4 ms·

What open source components does Cloud Firestore use? Can you point towards papers that describe the algorithms and data structures that Firestore uses? What i

What open source components does Cloud Firestore use? Can you point towards papers that describe the algorithms and data structures that Firestore uses? What infrastructure does it rely on to perform reliably?

roneythomas6 8y ago
.
- itcmcgrath 8y ago
  This isn't true, but Cloud Spanner is awesome too!
  - wilhuff 8y ago
    It's probably better said that Cloud Firestore and Cloud Spanner share common infrastructure. For example, Cloud Firestore uses TrueTime to provide similarly strong consistency guarantees to Cloud Spanner (see https://cloud.google.com/spanner/docs/true-time-external-consistency https://cloud.google.com/spanner/docs/true-time-external-con...) I'm not aware of any whitepapers describing the listen feature that provides real-time updates.
wilhuff 8y ago
The backend is almost entirely custom, though many common low-level open-source components are used pervasively. For example, for components written in C++ we use Abseil, Google Test, etc. The SDKs are all open source and make use of other open source components: * Android: https://github.com/firebase/firebase-android-sdk/tree/master/firebase-firestore * iOS: https://github.com/firebase/firebase-ios-sdk/tree/master/Firestore * Web: https://github.com/firebase/firebase-js-sdk/tree/master/packages/firestore For example, the iOS SDK uses LevelDB for local key-value storage and gRPC for communication with the backend.
- chupapuma 8y ago
  You can find language specific libraries for many popular languages under the GoogleAPIs org on GitHub. To install, go to the defacto package manager for the specific language. * .NET: https://github.com/googleapis/google-cloud-dotnet/tree/master/apis/Google.Cloud.Firestore https://github.com/googleapis/google-cloud-dotnet/tree/maste... * Go: https://github.com/googleapis/google-cloud-go/tree/master/firestore https://github.com/googleapis/google-cloud-go/tree/master/fi... * Java: https://github.com/googleapis/google-cloud-java/tree/master/google-cloud-clients/google-cloud-firestore https://github.com/googleapis/google-cloud-java/tree/master/... * Node.js: https://github.com/googleapis/nodejs-firestore https://github.com/googleapis/nodejs-firestore * PHP: https://github.com/googleapis/google-cloud-php-firestore https://github.com/googleapis/google-cloud-php-firestore * Python: https://github.com/googleapis/google-cloud-python/tree/master/firestore https://github.com/googleapis/google-cloud-python/tree/maste... * Ruby: https://github.com/googleapis/google-cloud-ruby/tree/master/google-cloud-firestore https://github.com/googleapis/google-cloud-ruby/tree/master/...
  - polskibus 8y ago
    I'm interested in learning about Firestore architecture and how it solves difficult problems to understand and trust it. I'm not interested in client libraries.
    - chupapuma 8y ago
      Sorry I misunderstood. I was simply sharing additional open source components that are part of the Firestore ecosystem. Best of luck in your search.

roneythomas6 8y ago

itcmcgrath 8y ago
This isn't true, but Cloud Spanner is awesome too!
- wilhuff 8y ago
  It's probably better said that Cloud Firestore and Cloud Spanner share common infrastructure. For example, Cloud Firestore uses TrueTime to provide similarly strong consistency guarantees to Cloud Spanner (see https://cloud.google.com/spanner/docs/true-time-external-consistency https://cloud.google.com/spanner/docs/true-time-external-con...) I'm not aware of any whitepapers describing the listen feature that provides real-time updates.

wilhuff 8y ago

The backend is almost entirely custom, though many common low-level open-source components are used pervasively. For example, for components written in C++ we use Abseil, Google Test, etc. The SDKs are all open source and make use of other open source components: * Android: https://github.com/firebase/firebase-android-sdk/tree/master/firebase-firestore * iOS: https://github.com/firebase/firebase-ios-sdk/tree/master/Firestore * Web: https://github.com/firebase/firebase-js-sdk/tree/master/packages/firestore For example, the iOS SDK uses LevelDB for local key-value storage and gRPC for communication with the backend.

chupapuma 8y ago
You can find language specific libraries for many popular languages under the GoogleAPIs org on GitHub. To install, go to the defacto package manager for the specific language. * .NET: https://github.com/googleapis/google-cloud-dotnet/tree/master/apis/Google.Cloud.Firestore https://github.com/googleapis/google-cloud-dotnet/tree/maste... * Go: https://github.com/googleapis/google-cloud-go/tree/master/firestore https://github.com/googleapis/google-cloud-go/tree/master/fi... * Java: https://github.com/googleapis/google-cloud-java/tree/master/google-cloud-clients/google-cloud-firestore https://github.com/googleapis/google-cloud-java/tree/master/... * Node.js: https://github.com/googleapis/nodejs-firestore https://github.com/googleapis/nodejs-firestore * PHP: https://github.com/googleapis/google-cloud-php-firestore https://github.com/googleapis/google-cloud-php-firestore * Python: https://github.com/googleapis/google-cloud-python/tree/master/firestore https://github.com/googleapis/google-cloud-python/tree/maste... * Ruby: https://github.com/googleapis/google-cloud-ruby/tree/master/google-cloud-firestore https://github.com/googleapis/google-cloud-ruby/tree/master/...
- polskibus 8y ago
  I'm interested in learning about Firestore architecture and how it solves difficult problems to understand and trust it. I'm not interested in client libraries.
  - chupapuma 8y ago
    Sorry I misunderstood. I was simply sharing additional open source components that are part of the Firestore ecosystem. Best of luck in your search.