4 ms·

Show HN: GUN Starter App Boilerplate – An Easy Open Source Firebase Replacement

89 points by marknadal 10y ago

j4pe 10y ago
Wow, this looks really promising. I went with Firebase when it became too obnoxious to manage websocket concurrency. I didn't even know about gun. I appreciate that the starter app doesn't try to make decisions for me about what I'm using clientside. Can you think of any reasons it wouldn't play nice with react-native?
- marknadal 10y ago
  Thanks! It works with react-native but there is currently a bug in a react-native module loader (according to: https://github.com/amark/gun/issues/236#issuecomment-255555300 https://github.com/amark/gun/issues/236#issuecomment-2555553... ) that is causing some problems. But I'm sure this will get sorted out pretty soon.
mholt 10y ago
Been keeping an eye on gun since the early demos more than a year ago. Nice work, Mark. :)
- arxpoetica 10y ago
  As have I. Keep it coming. :)
liamcurry 10y ago
First time hearing of Gun. Sounds very interesting! I'm a little confused about a few things though. Is Gun meant to run in the browser? If so how does it compare to PouchDB? And does syncing with peers just mean peer server nodes, or is there some WebRTC data channel stuff going on?
- marknadal 10y ago
  Sorry that the docs aren't clear enough! We'll definitely try and get that fixed with this next version that is coming out. It can run in both the browser and/or the server. Because WebRTC unfortunately requires a STUN/ICE or signaling server, you still have to run or use a community server. A gun server is nothing fancy though, it is just another peer that helps replicate/synchronize data between browser peers and other server peers. Full WebRTC support is planned as an extension but isn't finished yet: https://github.com/PsychoLlama/gun-rtc https://github.com/PsychoLlama/gun-rtc . See my other comment elsewhere in this thread about Pouch. Happy to answer any other questions you have! And please let us know what we can do to further clarify things in docs.
TheAceOfHearts 10y ago
I'd never heard of gun before, but the project looks interesting. I see there's a few videos in the main repo's readme, so I'll be checking em out. Any comments on how it compares to something like PouchDB [0]? What are examples of good use-cases where you'd want to reach for something like gun? Although I've looked into stuff like CouchDB and NoSQL databases, I can never intuitively wrap my head around it. For most problems that cross my mind, my first response is usually "Postgres sounds perfect for this". Does anyone have suggestions for resources that discuss this in a way that's approachable? While looking through their docs and wiki, one of my questions was: how do you protect yourself from invalid data and all the related "baddies"? They have a wiki entry on the subject of authn and authz [1], but unfortunately it doesn't look like they included a complete example. In another wiki entry [2] they mention an authorization filter, but don't go into more detail. [0] https://pouchdb.com/ https://pouchdb.com/ [1] https://github.com/amark/gun/wiki/Security%2C-Authentication%2C-Authorization https://github.com/amark/gun/wiki/Security%2C-Authentication... [2] https://github.com/amark/gun/wiki/Mesh-Network-Messaging-Algorithm https://github.com/amark/gun/wiki/Mesh-Network-Messaging-Alg...
- marknadal 10y ago
  Thanks for the comment! I'll start with what you should not use gun for: It is a strong eventually consistent system (AP if you are familiar with the CAP Theorem) which means it is a bad choice for banking/accounting. You should use a strongly consistent system like RethinkDB or Postgres instead. Some things that people are using gun for are: - Distributed Machine Learning ( http://myrighttocode.org/blog/artificial%20intelligence/particle%20swarm/genetic%20algorithm/collective%20knowledge/machine%20learning/gun-db-artificial-knowledge-sharing http://myrighttocode.org/blog/artificial%20intelligence/part... ) - Multiplayer VR games in the browser with WebGL. - Realtime IoT data viz with webcomponents. ( https://github.com/Stefdv/gun-ui-lcd https://github.com/Stefdv/gun-ui-lcd ) - Peer-to-Peer electron/react-native fully encrypted social networks. How it compares to PouchDB? PouchDB is great! They pioneered offline-first before people had even termed the phrase, very much were forward thinkers. The biggest difference is that Pouch/Couch is a document database while gun is a graph database which can support key/value, relational, document, circular references, or graph data structures. I haven't used Couch since 2011 though, so I can't speak to any of their more recent features, but their latest release looked really solid. NoSQL was definitely a fad, so I don't blame you for not being able to wrap your head around it. We plan on introducing a SQL query extension for gun within the next 8 months. SQL is great, relations are great, and the NoSQL/document-only hype hurt the industry although it was based on some legitimate needs. From my perspective there is no question that Postgres should be considered the best solution for Master-Slave query based databases. I do predict, obviously biased though, that the industry will slowly migrate towards Master-Master push based graph databases though. And my experience with gun, the VC funding I've raised, the pain points I've had and heard from others, and the traction we're getting is confirmation of that (or confirmation bias?). Preventing invalid data from being saved is as simple as adding a schema ( https://github.com/RangerMauve/gun-schema https://github.com/RangerMauve/gun-schema ). Preventing hackers is as not-simple as using real end-to-end cryptography, but we're making an explainer video series to try and help consolidate these ideas for people ( http://gun.js.org/explainers/data/security.html http://gun.js.org/explainers/data/security.html ). Or you can opt to use gun in a traditional centralized server setup where all reads/writes are proxied by your app's routes, like you would normally do with any other database. Anything else I can answer? Thanks for asking these questions!
fiatjaf 10y ago
Is it an actual graph? Can I get all objects pointing to some other and vice-versa? Is that efficient? Does that mean making thousands of queries?
- marknadal 10y ago
  If you don't index that query it won't be efficient, but you absolutely can do that! In fact there are awesome tagging extensions that people in the community have built ( https://github.com/Stefdv/guntagger https://github.com/Stefdv/guntagger ) that easily lets you do stuff like that. These are efficient because they are (if you are familiar with Big O Notation) O(1) queries not dynamic (because they are indexed). So no, it would not take thousands of queries. I feel like that doesn't really answer your question though? Want to give a more concrete example that I can expand on? Thanks for the question!
  - fiatjaf 10y ago
    Actually, it has answered it. Thank you! I want to build something with some simple graph things. I couldn't manage to do with Cayley and Gremlin I'll try now with Gun. I'll ask on Gitter when I have something more concrete to work with.
    - marknadal 10y ago
      That is sad! Graphs are actually super easy. Here is a sweet&short guide that introduces graphs to relational/NoSQL people: https://github.com/amark/gun/wiki/graphs https://github.com/amark/gun/wiki/graphs . It even has a runnable JSbin example at the bottom so you can try it out in the browser without installing anything!
felipeccastro 10y ago
This seems very interesting, powerful and yet simple, congratulations! I've noticed in one of the samples how the data is stored in a single json file on the server, so I'm curious about how would this scale for very large databases: if I don't want to send the entire dataset to all clients, and send just parts of it for each user, while keeping everything else stored in a traditional SQL db, would that be possible? A real world application of this would be something like Trello, for example.
- marknadal 10y ago
  Thanks for the compliments, that is very encouraging! And for the constructive questions. You point out something very important - the json dump definitely is not scalable! That is why we warn people it should only be used for local development and testing, and never to run it in production (these warnings are littered through out the documentation). Your instinct is correct! By default only a subset of the data is sent to clients - specifically, only the pieces of data that they are subscribed to. This is already built in so you don't have to worry about managing it yourself. Using the Level storage adapter, you could plug into a SQL database. I don't recommend doing this (because you'll get consistency issues if you query SQL separately from gun) but it is possible - the one exception is using SQLite as a storage layer. There is actually somebody building a P2P Trello app, I referenced his security/authorization gun framework in one of the other comments, if you are interested. Hit me up with anything whenever. Thanks for being so constructive!

89 points by marknadal 10y ago

j4pe 10y ago

Wow, this looks really promising. I went with Firebase when it became too obnoxious to manage websocket concurrency. I didn't even know about gun. I appreciate that the starter app doesn't try to make decisions for me about what I'm using clientside. Can you think of any reasons it wouldn't play nice with react-native?

marknadal 10y ago
Thanks! It works with react-native but there is currently a bug in a react-native module loader (according to: https://github.com/amark/gun/issues/236#issuecomment-255555300 https://github.com/amark/gun/issues/236#issuecomment-2555553... ) that is causing some problems. But I'm sure this will get sorted out pretty soon.

mholt 10y ago

Been keeping an eye on gun since the early demos more than a year ago. Nice work, Mark. :)

arxpoetica 10y ago
As have I. Keep it coming. :)

liamcurry 10y ago

First time hearing of Gun. Sounds very interesting! I'm a little confused about a few things though. Is Gun meant to run in the browser? If so how does it compare to PouchDB? And does syncing with peers just mean peer server nodes, or is there some WebRTC data channel stuff going on?

marknadal 10y ago
Sorry that the docs aren't clear enough! We'll definitely try and get that fixed with this next version that is coming out. It can run in both the browser and/or the server. Because WebRTC unfortunately requires a STUN/ICE or signaling server, you still have to run or use a community server. A gun server is nothing fancy though, it is just another peer that helps replicate/synchronize data between browser peers and other server peers. Full WebRTC support is planned as an extension but isn't finished yet: https://github.com/PsychoLlama/gun-rtc https://github.com/PsychoLlama/gun-rtc . See my other comment elsewhere in this thread about Pouch. Happy to answer any other questions you have! And please let us know what we can do to further clarify things in docs.

TheAceOfHearts 10y ago

I'd never heard of gun before, but the project looks interesting. I see there's a few videos in the main repo's readme, so I'll be checking em out. Any comments on how it compares to something like PouchDB [0]? What are examples of good use-cases where you'd want to reach for something like gun? Although I've looked into stuff like CouchDB and NoSQL databases, I can never intuitively wrap my head around it. For most problems that cross my mind, my first response is usually "Postgres sounds perfect for this". Does anyone have suggestions for resources that discuss this in a way that's approachable? While looking through their docs and wiki, one of my questions was: how do you protect yourself from invalid data and all the related "baddies"? They have a wiki entry on the subject of authn and authz [1], but unfortunately it doesn't look like they included a complete example. In another wiki entry [2] they mention an authorization filter, but don't go into more detail. [0] https://pouchdb.com/ https://pouchdb.com/ [1] https://github.com/amark/gun/wiki/Security%2C-Authentication%2C-Authorization https://github.com/amark/gun/wiki/Security%2C-Authentication... [2] https://github.com/amark/gun/wiki/Mesh-Network-Messaging-Algorithm https://github.com/amark/gun/wiki/Mesh-Network-Messaging-Alg...

marknadal 10y ago
Thanks for the comment! I'll start with what you should not use gun for: It is a strong eventually consistent system (AP if you are familiar with the CAP Theorem) which means it is a bad choice for banking/accounting. You should use a strongly consistent system like RethinkDB or Postgres instead. Some things that people are using gun for are: - Distributed Machine Learning ( http://myrighttocode.org/blog/artificial%20intelligence/particle%20swarm/genetic%20algorithm/collective%20knowledge/machine%20learning/gun-db-artificial-knowledge-sharing http://myrighttocode.org/blog/artificial%20intelligence/part... ) - Multiplayer VR games in the browser with WebGL. - Realtime IoT data viz with webcomponents. ( https://github.com/Stefdv/gun-ui-lcd https://github.com/Stefdv/gun-ui-lcd ) - Peer-to-Peer electron/react-native fully encrypted social networks. How it compares to PouchDB? PouchDB is great! They pioneered offline-first before people had even termed the phrase, very much were forward thinkers. The biggest difference is that Pouch/Couch is a document database while gun is a graph database which can support key/value, relational, document, circular references, or graph data structures. I haven't used Couch since 2011 though, so I can't speak to any of their more recent features, but their latest release looked really solid. NoSQL was definitely a fad, so I don't blame you for not being able to wrap your head around it. We plan on introducing a SQL query extension for gun within the next 8 months. SQL is great, relations are great, and the NoSQL/document-only hype hurt the industry although it was based on some legitimate needs. From my perspective there is no question that Postgres should be considered the best solution for Master-Slave query based databases. I do predict, obviously biased though, that the industry will slowly migrate towards Master-Master push based graph databases though. And my experience with gun, the VC funding I've raised, the pain points I've had and heard from others, and the traction we're getting is confirmation of that (or confirmation bias?). Preventing invalid data from being saved is as simple as adding a schema ( https://github.com/RangerMauve/gun-schema https://github.com/RangerMauve/gun-schema ). Preventing hackers is as not-simple as using real end-to-end cryptography, but we're making an explainer video series to try and help consolidate these ideas for people ( http://gun.js.org/explainers/data/security.html http://gun.js.org/explainers/data/security.html ). Or you can opt to use gun in a traditional centralized server setup where all reads/writes are proxied by your app's routes, like you would normally do with any other database. Anything else I can answer? Thanks for asking these questions!

fiatjaf 10y ago

Is it an actual graph? Can I get all objects pointing to some other and vice-versa? Is that efficient? Does that mean making thousands of queries?

marknadal 10y ago
If you don't index that query it won't be efficient, but you absolutely can do that! In fact there are awesome tagging extensions that people in the community have built ( https://github.com/Stefdv/guntagger https://github.com/Stefdv/guntagger ) that easily lets you do stuff like that. These are efficient because they are (if you are familiar with Big O Notation) O(1) queries not dynamic (because they are indexed). So no, it would not take thousands of queries. I feel like that doesn't really answer your question though? Want to give a more concrete example that I can expand on? Thanks for the question!
- fiatjaf 10y ago
  Actually, it has answered it. Thank you! I want to build something with some simple graph things. I couldn't manage to do with Cayley and Gremlin I'll try now with Gun. I'll ask on Gitter when I have something more concrete to work with.
  - marknadal 10y ago
    That is sad! Graphs are actually super easy. Here is a sweet&short guide that introduces graphs to relational/NoSQL people: https://github.com/amark/gun/wiki/graphs https://github.com/amark/gun/wiki/graphs . It even has a runnable JSbin example at the bottom so you can try it out in the browser without installing anything!

felipeccastro 10y ago

This seems very interesting, powerful and yet simple, congratulations! I've noticed in one of the samples how the data is stored in a single json file on the server, so I'm curious about how would this scale for very large databases: if I don't want to send the entire dataset to all clients, and send just parts of it for each user, while keeping everything else stored in a traditional SQL db, would that be possible? A real world application of this would be something like Trello, for example.

marknadal 10y ago
Thanks for the compliments, that is very encouraging! And for the constructive questions. You point out something very important - the json dump definitely is not scalable! That is why we warn people it should only be used for local development and testing, and never to run it in production (these warnings are littered through out the documentation). Your instinct is correct! By default only a subset of the data is sent to clients - specifically, only the pieces of data that they are subscribed to. This is already built in so you don't have to worry about managing it yourself. Using the Level storage adapter, you could plug into a SQL database. I don't recommend doing this (because you'll get consistency issues if you query SQL separately from gun) but it is possible - the one exception is using SQLite as a storage layer. There is actually somebody building a P2P Trello app, I referenced his security/authorization gun framework in one of the other comments, if you are interested. Hit me up with anything whenever. Thanks for being so constructive!