8 ms·

SapphireDb – Open-Source Alternative to Firebase

225 points by morrisjdev 7y ago

ivankolev 7y ago
based on .net core, for what it's worth
- ComputerGuru 7y ago
  Which is a bit weird since .NET Core has standardized on the open source SignalR, which is really good. It would make more sense to build on top of that.
  - onesmalldrop 7y ago
    depends on your audience
  - manishjhawar 7y ago
    > SapphireDb should serve as a self hosted alternative to firebase and also gives you an alternative to SignalR.
  - morrisjdev 7y ago
    I took SignlaR into consideration when building that. SignalR does not have some client implementations. I decided to create a small protocol that is easily adoptable in many technologies. SignalR also has some limitations that made it impracticable for the project.
arpit 7y ago
Firebase is a collection of services. This seems to be an alternative to the Firebase Realtime Database specifically
- dudus 7y ago
  Which is sort of a legacy database they keep probably more for compatibility reasons. Firestore is the actual database used by most new apps.
- dsun180 7y ago
  This is not even true. Firebase Realtime Database supports offline-first while SapphireDB is just streaming results to the client, like RethinkDB or Meteor.
  - morrisjdev 7y ago
    Offline-first is already a planned feature and only missing because of the lack of development capacity.
rkrzr 7y ago
If you are looking for an open-source replacement of the Firebase Realtime Database, you might also be interested in Icepeak (https://github.com/channable/icepeak https://github.com/channable/icepeak) (full disclosure, I'm one of the authors). Icepeak was written because we were unsatisfied with the reliability and with the (occasionally) high latency of Firebase, both for pushing updates via HTTP (>1s for some requests) and for receiving updates over websockets. This was about 3 years ago though, so I hope that Google improved the quality of the service in the mean time.
- bumblebee4 7y ago
  That sounds interesting. Have you compared your service to pouchdb and gun? Is it possible to use icepeak in offline mode and automatically sync when getting online again? [1] https://pouchdb.com/ https://pouchdb.com/ [2] https://gun.js.org/ https://gun.js.org/
  - xrd 7y ago
    PouchDB looks like the client side library. Do you still need CouchDB on the backend? I see from the mailing list it is still a very active project, which surprised me.
    - bumblebee4 7y ago
      PouchDB is CouchDB-Protocol implemented in javascript so you can run it wherever there is javascript, not just as a client. If you want to have a backend, you can use any DB that supports that protocol (there are several), which includes using a PouchDB instance run by node.js.
      - lhuser123 7y ago
        Is PouchDB + CouchDB considered an alternative to Firebase?
        bumblebee4 7y ago
galonk 7y ago
"SapphireDb also supports running in an NLB with multiple instances and scales very good." In English, when speaking of actions you must say "well" instead of "good", ie. "scales very well". While there are a million weird things about English, this particular mistake is used as a cultural marker of "childish" or "low-intelligence" speech, so you probably want to fix it ASAP. Also what is an NLB???
- deleted 7y ago
  [deleted]
- cprecioso 7y ago
  Network Load Balancer
- morrisjdev 7y ago
  Thanks for the feedback. I'll fix it soon.
deleted 7y ago
[deleted]
marknadal 7y ago
I'm a competitor, and I want to applaud this work! - SapphireDB is truly Open Source, MIT! - There demo worked like a charm! We could really use more projects like this out there. Here is a criticism though: - It seems currently to only support Angular (?), which is gonna harm its adoption. My recommendation would be to commit to Svelte, it seems like it will be the "next big thing" in the JS world, and could help drive adoption. Obviously, this is something we're trying to do as well (https://gun.eco/docs/Svelte https://gun.eco/docs/Svelte). If you're looking for a Firebase alternative, GUN is used in production with 11M+ downloads a month, but is NodeJS based so it won't work for the .NET audience that Sapphire is targeting. What I think would be cool to add: (from my bias) - Support graph data. - Work offline / local-first. Either way, thanks for building & posting about Sapphire, it is nice to see there is still big demand for Open Source Firebase alternatives!
- tluyben2 7y ago
  Offtopic: > to commit to Svelte, it seems like it will be the "next big thing" in the JS world How long does it usually take? In the JS world? Serious question as I am not in the JS world but reading HN/Reddit I would believe that things change quite fast. Someone told me over 1 year ago Svelte would be the next big thing and I see ‘no-one’ using it besides some hardcore enthusiasts; it is even hardly ever mentioned here or on Reddit; if it was a the next big thing, I think it would be? Sorry for being uninformed, I am just curious.
  - marknadal 7y ago
    React was like this for a long while too, compared to Angular. Then once some big "used in production by NYT/x/y/z" hits, a premier conference talk, then things explode with everyone virtue-signaling how they use the modern/best stack, floods of article/blogs are posted about how they switched over before you did, and the FOMO cargo cult bandwagon rocketships. Svelte does look legit, but it is better to not buy into the JS virtue culture.
  - dangoor 7y ago
    I don't think this is clear cut. React has been around for more than 6 years. According to the latest State of JS survey[1], 72% of respondents have used React before and would use it again. That number is 7% for Svelte. Svelte has a lot of interest, but it also has a very long way to go against a popular, 6 year old project. [1]: https://2019.stateofjs.com/front-end-frameworks/ https://2019.stateofjs.com/front-end-frameworks/
niblettc 7y ago
In the Stackblitz example the Angular client is connecting directly to the SapphireDB database with credentials stored in the app.module.ts. Could you build a scalable real world, production application this way? With no in the middle NodeJS web server handing business logic? Handle all the business logic in the Angular client and just CRUD the objects back to SapphireDB? Essentially just Client--> SapphireDB Or am I missing something here? I see there's a asp.net configuration, which would be Client--> Server--> SapphireDB Is there a package or support for connecting a NodeJS server to SapphireDB database? Also, is this production ready?
- morrisjdev 7y ago
  The credentials you found in the app.module.ts are just credentials to identify the client (not the user). It is just to give an extra layer of security and only allow specific applications access to the server. User authentication is handled through tokens for example. The architecture looks more like this: Client -> Asp.Net Core server with SapphireDb -> any database server SapphireDb is not a database. Its more like the connector between database and clients. Because of that there is no package for a NodeJS because it would just be proxy. Yes I think it is production ready. I'll add a section in the docs describing the criteria this opinion is based on.
brynb 7y ago
Creator of Redwood here, which is similar in many ways to this and GunDB (hi Mark!). Agreed that we need more projects like this. Redwood might be better described as a "p2p application platform" than a realtime database, but there's plenty of overlap. It's written in Go, supports HTTPS, Libp2p, and WebRTC transports (and soon, a Cap'n Proto RPC interface). It can also: - serve as a Git remote - stand in as an application server (you can upload assets directly to it, and they're served over regular HTTPS) - execute Lambda-like functions written in Go, Lua, and Javascript that can be installed at any point in a given state tree Would love some early feedback, and also to discuss similarities and differences with Sapphire's model. I've set up some demos in the "scenarios" folder -- a chat app, a realtime document editor, and a Git integration. Take a look if you're interested: https://github.com/brynbellomy/redwood https://github.com/brynbellomy/redwood
- anderspitman 7y ago
  > and soon, a Cap'n Proto RPC interface This is interesting, and good to hear. Are there still a lot of projects adding capnproto support? It's a great protocol but I've worried it might be dying.
  - brynb 7y ago
    Dying? Hm, not to my knowledge. In fact, I saw a highly-commented thread on HN just a couple of weeks ago about Cap'n Proto. What gives you the sense that it's on its way out?
    - anderspitman 7y ago
      Sandstorm Oasis going away and the last time I checked (maybe a year ago?) work seemed to have stopped on getting the protocol implemented in the browser.
      - kentonv 7y ago
        Hi, I'm the author of Cap'n Proto, and co-founder of Sandstorm.io. When Sandstorm failed, I went to Cloudflare where I'm leading the Cloudflare Workers project. We're using Cap'n Proto (including RPC) pretty heavily there! That said, it's true that Cap'n Proto relies on its users to contribute improvements to the implementation. We don't currently have a team who can dedicate their time to building out capnp implementations in every language. At present, within Cloudflare, the heaviest use is C++, with some stuff in Go, Rust, Java, and Lua. Hence, if you look at the C++ implementation you'll see it's been pretty active. At the moment we don't have a JavaScript use case so haven't put effort into that implementation. But given that Workers itself is a JavaScript platform, it seems reasonably likely that we'll end up putting some work into the JavaScript/TypeScript implementation at some point down the road.
tracker1 7y ago
Need to followup, but angular-only examples... would be nice to see a straight/simple client that can be used with, for example, angular listed... It is probably out there already.
tracker1 7y ago
Needs a base npm library, that is not tethered to Angular. https://github.com/SapphireDb/SapphireDb/issues/3 https://github.com/SapphireDb/SapphireDb/issues/3
- morrisjdev 7y ago
  This is planned and will come soon.
pspeter3 7y ago
Is there a self hosted server for auto merge?
- folex 7y ago
  Hi! What features are you interested in for integrating with auto-merge? If it's OK to ask, what's your use case? Thanks :)
virtuallyvivek 7y ago
Cool - I'm glad to see more tech similar to Firebase pop up that's self hosted. The big request would be mobile. I mostly use FB for the web but knowing that I could create a mobile version using the same platform would be great.
kiwicopple 7y ago
For anyone looking to replace Firebase with plain Postgres, I am went through the same exercise last year an I’m opensourcing the work I did - https://supabase.io https://supabase.io The hardest part was the real-time functionality. I ended up writing an elixir server that listens to Postgres’ built-in replication stream, converts it into JSON, then sends it over websockets.
- danr4 7y ago
  That looks preeeetty cool. Keepin an eye on it.
- kaishin 7y ago
  Is it supposed to be self-hosted? Wasn’t clear to me after taking a quick look.
  - kiwicopple 7y ago
    Yes, it can be. I realise it's not too obvious so I'll add the documentation for self hosting over the next 1 week
    - kaishin 7y ago
      Thanks! Will keep an eye on it.
- _frkl 7y ago
  Neat. Did you have a look at Hasura before you implemented this, or does your solution have other features you needed? Edit: also, Debezium might be a good backend for this, that way you would be able to support other databases than Postgres.
  - kiwicopple 7y ago
    I took a lot of inspiration from Hasura and Debezium. At the time they used triggers to send events via NOTIFY (not sure if they still do) The issue with NOTIFY is that it has an 8000 byte limit. Some of our updates are large (JSONB columns) so we need more than 8000 bytes, or we would have to notify just the row identifier, then fetch the data (inefficient). Supabase hooks in to Postgres' replication stream so there is no issue. It converts the streaming bytes to JSON. One other bonus - you don't have to set a trigger on every table. Just run `CREATE PUBLICATION supabase_realtime ON ALL TABLES`, then you're good to go.
quest88 7y ago
Was building an opensource version cheaper for you than just using firebase? I'm generally curious. I commonly see OS alternatives to paid services but I've never seen why it was made. Is it cheaper to run and maintain your own servers? Are longer outages and downtimes ok for your company? Security vulnerabilities? Maybe these are things you don't care about and that's fine. It would be nice to see "Use this if.." section.
hardwaresofton 7y ago
Another open source "realtime" database, which supports joins, has a nice admin interface, sharding, and realtime queries: https://rethinkdb.com https://rethinkdb.com It's what mongo should have been from the beginning -- they won the correctness battlew hile mongo won the marketing war. I should mention that I've had some colleagues run into problems with scaling it so it's not all roses but a database that scales painlessly in every case is AFAIK a fantasy.

225 points by morrisjdev 7y ago

ivankolev 7y ago

based on .net core, for what it's worth

ComputerGuru 7y ago
Which is a bit weird since .NET Core has standardized on the open source SignalR, which is really good. It would make more sense to build on top of that.
- onesmalldrop 7y ago
  depends on your audience
- manishjhawar 7y ago
  > SapphireDb should serve as a self hosted alternative to firebase and also gives you an alternative to SignalR.
- morrisjdev 7y ago
  I took SignlaR into consideration when building that. SignalR does not have some client implementations. I decided to create a small protocol that is easily adoptable in many technologies. SignalR also has some limitations that made it impracticable for the project.

arpit 7y ago

Firebase is a collection of services. This seems to be an alternative to the Firebase Realtime Database specifically

dudus 7y ago
Which is sort of a legacy database they keep probably more for compatibility reasons. Firestore is the actual database used by most new apps.
dsun180 7y ago
This is not even true. Firebase Realtime Database supports offline-first while SapphireDB is just streaming results to the client, like RethinkDB or Meteor.
- morrisjdev 7y ago
  Offline-first is already a planned feature and only missing because of the lack of development capacity.

rkrzr 7y ago

If you are looking for an open-source replacement of the Firebase Realtime Database, you might also be interested in Icepeak (https://github.com/channable/icepeak https://github.com/channable/icepeak) (full disclosure, I'm one of the authors). Icepeak was written because we were unsatisfied with the reliability and with the (occasionally) high latency of Firebase, both for pushing updates via HTTP (>1s for some requests) and for receiving updates over websockets. This was about 3 years ago though, so I hope that Google improved the quality of the service in the mean time.

bumblebee4 7y ago
That sounds interesting. Have you compared your service to pouchdb and gun? Is it possible to use icepeak in offline mode and automatically sync when getting online again? [1] https://pouchdb.com/ https://pouchdb.com/ [2] https://gun.js.org/ https://gun.js.org/
- xrd 7y ago
  PouchDB looks like the client side library. Do you still need CouchDB on the backend? I see from the mailing list it is still a very active project, which surprised me.
  - bumblebee4 7y ago
    PouchDB is CouchDB-Protocol implemented in javascript so you can run it wherever there is javascript, not just as a client. If you want to have a backend, you can use any DB that supports that protocol (there are several), which includes using a PouchDB instance run by node.js.
    - lhuser123 7y ago
      Is PouchDB + CouchDB considered an alternative to Firebase?
      - bumblebee4 7y ago

galonk 7y ago

"SapphireDb also supports running in an NLB with multiple instances and scales very good." In English, when speaking of actions you must say "well" instead of "good", ie. "scales very well". While there are a million weird things about English, this particular mistake is used as a cultural marker of "childish" or "low-intelligence" speech, so you probably want to fix it ASAP. Also what is an NLB???

deleted 7y ago
[deleted]
cprecioso 7y ago
Network Load Balancer
morrisjdev 7y ago
Thanks for the feedback. I'll fix it soon.

deleted 7y ago

[deleted]

marknadal 7y ago

I'm a competitor, and I want to applaud this work! - SapphireDB is truly Open Source, MIT! - There demo worked like a charm! We could really use more projects like this out there. Here is a criticism though: - It seems currently to only support Angular (?), which is gonna harm its adoption. My recommendation would be to commit to Svelte, it seems like it will be the "next big thing" in the JS world, and could help drive adoption. Obviously, this is something we're trying to do as well (https://gun.eco/docs/Svelte https://gun.eco/docs/Svelte). If you're looking for a Firebase alternative, GUN is used in production with 11M+ downloads a month, but is NodeJS based so it won't work for the .NET audience that Sapphire is targeting. What I think would be cool to add: (from my bias) - Support graph data. - Work offline / local-first. Either way, thanks for building & posting about Sapphire, it is nice to see there is still big demand for Open Source Firebase alternatives!

tluyben2 7y ago
Offtopic: > to commit to Svelte, it seems like it will be the "next big thing" in the JS world How long does it usually take? In the JS world? Serious question as I am not in the JS world but reading HN/Reddit I would believe that things change quite fast. Someone told me over 1 year ago Svelte would be the next big thing and I see ‘no-one’ using it besides some hardcore enthusiasts; it is even hardly ever mentioned here or on Reddit; if it was a the next big thing, I think it would be? Sorry for being uninformed, I am just curious.
- marknadal 7y ago
  React was like this for a long while too, compared to Angular. Then once some big "used in production by NYT/x/y/z" hits, a premier conference talk, then things explode with everyone virtue-signaling how they use the modern/best stack, floods of article/blogs are posted about how they switched over before you did, and the FOMO cargo cult bandwagon rocketships. Svelte does look legit, but it is better to not buy into the JS virtue culture.
- dangoor 7y ago
  I don't think this is clear cut. React has been around for more than 6 years. According to the latest State of JS survey[1], 72% of respondents have used React before and would use it again. That number is 7% for Svelte. Svelte has a lot of interest, but it also has a very long way to go against a popular, 6 year old project. [1]: https://2019.stateofjs.com/front-end-frameworks/ https://2019.stateofjs.com/front-end-frameworks/

niblettc 7y ago

In the Stackblitz example the Angular client is connecting directly to the SapphireDB database with credentials stored in the app.module.ts. Could you build a scalable real world, production application this way? With no in the middle NodeJS web server handing business logic? Handle all the business logic in the Angular client and just CRUD the objects back to SapphireDB? Essentially just Client--> SapphireDB Or am I missing something here? I see there's a asp.net configuration, which would be Client--> Server--> SapphireDB Is there a package or support for connecting a NodeJS server to SapphireDB database? Also, is this production ready?

morrisjdev 7y ago
The credentials you found in the app.module.ts are just credentials to identify the client (not the user). It is just to give an extra layer of security and only allow specific applications access to the server. User authentication is handled through tokens for example. The architecture looks more like this: Client -> Asp.Net Core server with SapphireDb -> any database server SapphireDb is not a database. Its more like the connector between database and clients. Because of that there is no package for a NodeJS because it would just be proxy. Yes I think it is production ready. I'll add a section in the docs describing the criteria this opinion is based on.

brynb 7y ago

Creator of Redwood here, which is similar in many ways to this and GunDB (hi Mark!). Agreed that we need more projects like this. Redwood might be better described as a "p2p application platform" than a realtime database, but there's plenty of overlap. It's written in Go, supports HTTPS, Libp2p, and WebRTC transports (and soon, a Cap'n Proto RPC interface). It can also: - serve as a Git remote - stand in as an application server (you can upload assets directly to it, and they're served over regular HTTPS) - execute Lambda-like functions written in Go, Lua, and Javascript that can be installed at any point in a given state tree Would love some early feedback, and also to discuss similarities and differences with Sapphire's model. I've set up some demos in the "scenarios" folder -- a chat app, a realtime document editor, and a Git integration. Take a look if you're interested: https://github.com/brynbellomy/redwood https://github.com/brynbellomy/redwood

anderspitman 7y ago
> and soon, a Cap'n Proto RPC interface This is interesting, and good to hear. Are there still a lot of projects adding capnproto support? It's a great protocol but I've worried it might be dying.
- brynb 7y ago
  Dying? Hm, not to my knowledge. In fact, I saw a highly-commented thread on HN just a couple of weeks ago about Cap'n Proto. What gives you the sense that it's on its way out?
  - anderspitman 7y ago
    Sandstorm Oasis going away and the last time I checked (maybe a year ago?) work seemed to have stopped on getting the protocol implemented in the browser.
    - kentonv 7y ago
      Hi, I'm the author of Cap'n Proto, and co-founder of Sandstorm.io. When Sandstorm failed, I went to Cloudflare where I'm leading the Cloudflare Workers project. We're using Cap'n Proto (including RPC) pretty heavily there! That said, it's true that Cap'n Proto relies on its users to contribute improvements to the implementation. We don't currently have a team who can dedicate their time to building out capnp implementations in every language. At present, within Cloudflare, the heaviest use is C++, with some stuff in Go, Rust, Java, and Lua. Hence, if you look at the C++ implementation you'll see it's been pretty active. At the moment we don't have a JavaScript use case so haven't put effort into that implementation. But given that Workers itself is a JavaScript platform, it seems reasonably likely that we'll end up putting some work into the JavaScript/TypeScript implementation at some point down the road.

tracker1 7y ago

Need to followup, but angular-only examples... would be nice to see a straight/simple client that can be used with, for example, angular listed... It is probably out there already.

tracker1 7y ago

Needs a base npm library, that is not tethered to Angular. https://github.com/SapphireDb/SapphireDb/issues/3 https://github.com/SapphireDb/SapphireDb/issues/3

morrisjdev 7y ago
This is planned and will come soon.

pspeter3 7y ago

Is there a self hosted server for auto merge?

folex 7y ago
Hi! What features are you interested in for integrating with auto-merge? If it's OK to ask, what's your use case? Thanks :)

virtuallyvivek 7y ago

Cool - I'm glad to see more tech similar to Firebase pop up that's self hosted. The big request would be mobile. I mostly use FB for the web but knowing that I could create a mobile version using the same platform would be great.

kiwicopple 7y ago

For anyone looking to replace Firebase with plain Postgres, I am went through the same exercise last year an I’m opensourcing the work I did - https://supabase.io https://supabase.io The hardest part was the real-time functionality. I ended up writing an elixir server that listens to Postgres’ built-in replication stream, converts it into JSON, then sends it over websockets.

danr4 7y ago
That looks preeeetty cool. Keepin an eye on it.
kaishin 7y ago
Is it supposed to be self-hosted? Wasn’t clear to me after taking a quick look.
- kiwicopple 7y ago
  Yes, it can be. I realise it's not too obvious so I'll add the documentation for self hosting over the next 1 week
  - kaishin 7y ago
    Thanks! Will keep an eye on it.
_frkl 7y ago
Neat. Did you have a look at Hasura before you implemented this, or does your solution have other features you needed? Edit: also, Debezium might be a good backend for this, that way you would be able to support other databases than Postgres.
- kiwicopple 7y ago
  I took a lot of inspiration from Hasura and Debezium. At the time they used triggers to send events via NOTIFY (not sure if they still do) The issue with NOTIFY is that it has an 8000 byte limit. Some of our updates are large (JSONB columns) so we need more than 8000 bytes, or we would have to notify just the row identifier, then fetch the data (inefficient). Supabase hooks in to Postgres' replication stream so there is no issue. It converts the streaming bytes to JSON. One other bonus - you don't have to set a trigger on every table. Just run `CREATE PUBLICATION supabase_realtime ON ALL TABLES`, then you're good to go.

quest88 7y ago

Was building an opensource version cheaper for you than just using firebase? I'm generally curious. I commonly see OS alternatives to paid services but I've never seen why it was made. Is it cheaper to run and maintain your own servers? Are longer outages and downtimes ok for your company? Security vulnerabilities? Maybe these are things you don't care about and that's fine. It would be nice to see "Use this if.." section.

hardwaresofton 7y ago

Another open source "realtime" database, which supports joins, has a nice admin interface, sharding, and realtime queries: https://rethinkdb.com https://rethinkdb.com It's what mongo should have been from the beginning -- they won the correctness battlew hile mongo won the marketing war. I should mention that I've had some colleagues run into problems with scaling it so it's not all roses but a database that scales painlessly in every case is AFAIK a fantasy.