4 ms·

Poor Man's Firebase: LevelDB, REST, and WebSockets

89 points by jonpaul 13y ago

octernion 13y ago
Pretty cool project! I remember the pleasure I got out of (partially) re-implementing the Pusher API. I wonder why real-time features aren't more popular? Real-time order status, live-updating maps, the list goes on!
- rgbrgb 13y ago
  I think they're not that popular because the potential revenue lift for implementing the features you listed does not seem significant.
  - octernion 13y ago
    Absolutely true! The geek in me sees a sort of halo effect of having those king of features, though, in how much I appreciate a project; I'm sure that's not as easily quantifiable.
deleted 13y ago
[deleted]
nateabele 13y ago
> (As an aside, it seems that it would work most of the time in most of environments that I'd try, but for some reason, it rarely worked in one). So I needed to find a new solution. 'Instead of debugging the one environment, I'll just reimplement the entire service!' Obviously. :-)
- saraid216 13y ago
  It seems to be a personal project. Since there aren't any meaningful consequences for writing a bug-ridden, unsupported app, it's way more valuable to reinvent the wheel than to become an expert in someone else's work.
  - nateabele 13y ago
    I know, I'm not saying he's a bad person, I'm just poking a little fun at the mentality that leads to wheel-reinvention as the first and only conclusion. I still think it's a cool project.
    - vezzy-fnord 13y ago
      I'm not sure if this is wheel reinvention, so much as wheel subset reimplementation. The latter of which is a very legitimate practice, quite frankly.
      - nateabele 13y ago
        You're right. I forgot to put my pedantry hat on before typing that comment.
- javajosh 13y ago
  Well, it might also have something to do with money or control, as Firebase is SaaS, and this solution is open source software. https://www.firebase.com/pricing.html https://www.firebase.com/pricing.html
  - jaytaylor 13y ago
    As an aside, what happens with FireBase when you need more than 30GB of storage in a single app?
    - tekacs 13y ago
      You switch to the Inferno plan (scroll down! :P) and then on to Enterprise plans (or just pay overages, though I'm not sure if they apply to disk?)
      - jaytaylor 13y ago
        Ah, thanks. The fold looked like a complete page to me! Regardless, I still find the potential lock-in costs and issues incurred with a proprietary datastore like FireBase to be a total and complete deal breaker. For truly production-grade systems, relinquishing control over your data stores does not seem like a viable path to better reliability or control for any business with significantly large swaths of data. If there were a compatible open-source equivalent with easy import functionality for a FireBase dump, I'd be singing a different tune. Am I missing or overlooking something here? I get the use case for a small project or app, but for mission-critical production, it doesn't seem like a good idea.
yeleti 13y ago
This is quite nice. Before we (https://www.tesla.im https://www.tesla.im) switched to Firebase we had (still have) our own realtime system built with NodeJS, Websockets, Amazon SQS (as backup to ensure messages don't get lost) and a RIAK cluster.
- yeleti 13y ago
  Just to clarify things a bit; our setup worked fine. The only reason we switched to Firebase was, being a small team, we didnt want to maintain all those servers.
  - ing33k 13y ago
    Sounds cool, have you thought about open sourcing what you have buid?
    - deleted 13y ago
      [deleted]
    - yeleti 13y ago
      Yes we can open source it. It works pretty well. You will have to maintain these 'minimum' servers to get > 99% availability. 2 NodeJS app servers 1 Redis Master + 1 Slave 3 RIAK cluster (recommended 5) You might want to checkout https://www.meteor.com/ https://www.meteor.com/ as well.
nicklovescode 13y ago
Would love if someone produced a decent open source firebase-clone with turn-key angularfire integration. I really want to use it, but some of my clients are not OK with having data stored on someone else's server(rationally or not)
- yeleti 13y ago
  You might like https://www.meteor.com/ https://www.meteor.com/
greencarpet 13y ago
I built an npm module with a very similar goal - an open source alternative to FireBase. It's open source at https://npmjs.org/package/waferdb https://npmjs.org/package/waferdb
rakoo 13y ago
I don't want to sound negative towards you project, but why not use CouchDB [0], along with TouchDB [1] for mobiles and PouchDB [2] for the browser ? [0] https://couchdb.apache.org/ https://couchdb.apache.org/ [1] http://touchdb.org/ http://touchdb.org/ [2] http://pouchdb.com/ http://pouchdb.com/
- jonpaul 13y ago
  There is no project... just experimentation with LevelDB. All of those solutions that you provided require an installation of CouchDB on the server side. Therefore, a bit out of scope of building a quick hacky Firebase clone using LevelDB modules.
  - Lazare 13y ago
    Actually, that's not true: PouchDB runs on node (via LevelDB) and doesn't need an actual couch server. :)
    - rakoo 13y ago
      I'm not sure about that. The doc [0] says that there are some static files that should be distributed by whatever HTTP server you want (they suggest python's simple http server) After that, it's all js in the browser. Edit: I see what you mean: Pouchdb-server [1] is a couchdb-compatible server that speaks the same sync protocol. If you don't want to install it, you can work with any hosted couchdb-speaking cloud instance, such as iriscouch [2] or cloudant [3] [0] http://pouchdb.com/getting-started.html http://pouchdb.com/getting-started.html [1] https://github.com/nick-thompson/pouchdb-server https://github.com/nick-thompson/pouchdb-server [2] http://www.iriscouch.com/ http://www.iriscouch.com/ [3] https://cloudant.com/ https://cloudant.com/
kayoone 13y ago
i always thought that firebase should be easy to replicate with some node + express REST + websockets + client code if you plan on doing a bigger longterm project with sensitive data. For smaller projects (experiments, campaign related stuff) firebase should be fine.

89 points by jonpaul 13y ago

Pretty cool project! I remember the pleasure I got out of (partially) re-implementing the Pusher API. I wonder why real-time features aren't more popular? Real-time order status, live-updating maps, the list goes on!

rgbrgb 13y ago
I think they're not that popular because the potential revenue lift for implementing the features you listed does not seem significant.
- octernion 13y ago
  Absolutely true! The geek in me sees a sort of halo effect of having those king of features, though, in how much I appreciate a project; I'm sure that's not as easily quantifiable.

deleted 13y ago

[deleted]

nateabele 13y ago

> (As an aside, it seems that it would work most of the time in most of environments that I'd try, but for some reason, it rarely worked in one). So I needed to find a new solution. 'Instead of debugging the one environment, I'll just reimplement the entire service!' Obviously. :-)

saraid216 13y ago
It seems to be a personal project. Since there aren't any meaningful consequences for writing a bug-ridden, unsupported app, it's way more valuable to reinvent the wheel than to become an expert in someone else's work.
- nateabele 13y ago
  I know, I'm not saying he's a bad person, I'm just poking a little fun at the mentality that leads to wheel-reinvention as the first and only conclusion. I still think it's a cool project.
  - vezzy-fnord 13y ago
    I'm not sure if this is wheel reinvention, so much as wheel subset reimplementation. The latter of which is a very legitimate practice, quite frankly.
    - nateabele 13y ago
      You're right. I forgot to put my pedantry hat on before typing that comment.
javajosh 13y ago
Well, it might also have something to do with money or control, as Firebase is SaaS, and this solution is open source software. https://www.firebase.com/pricing.html https://www.firebase.com/pricing.html
- jaytaylor 13y ago
  As an aside, what happens with FireBase when you need more than 30GB of storage in a single app?
  - tekacs 13y ago
    You switch to the Inferno plan (scroll down! :P) and then on to Enterprise plans (or just pay overages, though I'm not sure if they apply to disk?)
    - jaytaylor 13y ago
      Ah, thanks. The fold looked like a complete page to me! Regardless, I still find the potential lock-in costs and issues incurred with a proprietary datastore like FireBase to be a total and complete deal breaker. For truly production-grade systems, relinquishing control over your data stores does not seem like a viable path to better reliability or control for any business with significantly large swaths of data. If there were a compatible open-source equivalent with easy import functionality for a FireBase dump, I'd be singing a different tune. Am I missing or overlooking something here? I get the use case for a small project or app, but for mission-critical production, it doesn't seem like a good idea.

yeleti 13y ago

This is quite nice. Before we (https://www.tesla.im https://www.tesla.im) switched to Firebase we had (still have) our own realtime system built with NodeJS, Websockets, Amazon SQS (as backup to ensure messages don't get lost) and a RIAK cluster.

yeleti 13y ago
Just to clarify things a bit; our setup worked fine. The only reason we switched to Firebase was, being a small team, we didnt want to maintain all those servers.
- ing33k 13y ago
  Sounds cool, have you thought about open sourcing what you have buid?
  - deleted 13y ago
    [deleted]
  - yeleti 13y ago
    Yes we can open source it. It works pretty well. You will have to maintain these 'minimum' servers to get > 99% availability. 2 NodeJS app servers 1 Redis Master + 1 Slave 3 RIAK cluster (recommended 5) You might want to checkout https://www.meteor.com/ https://www.meteor.com/ as well.

nicklovescode 13y ago

Would love if someone produced a decent open source firebase-clone with turn-key angularfire integration. I really want to use it, but some of my clients are not OK with having data stored on someone else's server(rationally or not)

yeleti 13y ago
You might like https://www.meteor.com/ https://www.meteor.com/

greencarpet 13y ago

I built an npm module with a very similar goal - an open source alternative to FireBase. It's open source at https://npmjs.org/package/waferdb https://npmjs.org/package/waferdb

rakoo 13y ago

I don't want to sound negative towards you project, but why not use CouchDB [0], along with TouchDB [1] for mobiles and PouchDB [2] for the browser ? [0] https://couchdb.apache.org/ https://couchdb.apache.org/ [1] http://touchdb.org/ http://touchdb.org/ [2] http://pouchdb.com/ http://pouchdb.com/

jonpaul 13y ago
There is no project... just experimentation with LevelDB. All of those solutions that you provided require an installation of CouchDB on the server side. Therefore, a bit out of scope of building a quick hacky Firebase clone using LevelDB modules.
- Lazare 13y ago
  Actually, that's not true: PouchDB runs on node (via LevelDB) and doesn't need an actual couch server. :)
  - rakoo 13y ago
    I'm not sure about that. The doc [0] says that there are some static files that should be distributed by whatever HTTP server you want (they suggest python's simple http server) After that, it's all js in the browser. Edit: I see what you mean: Pouchdb-server [1] is a couchdb-compatible server that speaks the same sync protocol. If you don't want to install it, you can work with any hosted couchdb-speaking cloud instance, such as iriscouch [2] or cloudant [3] [0] http://pouchdb.com/getting-started.html http://pouchdb.com/getting-started.html [1] https://github.com/nick-thompson/pouchdb-server https://github.com/nick-thompson/pouchdb-server [2] http://www.iriscouch.com/ http://www.iriscouch.com/ [3] https://cloudant.com/ https://cloudant.com/

kayoone 13y ago

i always thought that firebase should be easy to replicate with some node + express REST + websockets + client code if you plan on doing a bigger longterm project with sensitive data. For smaller projects (experiments, campaign related stuff) firebase should be fine.