9 ms·

Scaling to 1M active GraphQL subscriptions on Postgres

353 points by tango12 7y ago

rj5 7y ago
I can’t wait to try this out!
dmitryminkovsky 7y ago
Hasura is such a promising application platform. I tried it a few months ago and was really pleased. In a way it kind of felt like RethinkDB, except GraphQL and Postgres. Was very easy to turn on with Docker Compose and play around. Looking forward to 1.0!
- z3t4 7y ago
  I guessed it would use the pub/sub capabilities of postgres. But it seems from the readme that it use polling, although using the same query which allows cache optimizations. The naive implementation can often be surprisingly effective.
  - tango12 7y ago
    Yep. We experimented around a lot with pub/sub (listen/notify, wal-lr). But it was getting hard to scale and heavy/spiky write loads needed careful thought. This seemed like the right way to get started esp. given that Hasura can get the entire sql query including authz, before moving to tackling that problem more effectively[1]. For folks that want to play around with wal and listen/notify: https://github.com/hasura/pgdeltastream https://github.com/hasura/pgdeltastream https://github.com/hasura/skor https://github.com/hasura/skor [1] https://github.com/hasura/graphql-engine/blob/master/architecture/live-queries.md#when-do-we-refetch https://github.com/hasura/graphql-engine/blob/master/archite...
Thaxll 7y ago
I was curious about the code so I went to check it, it's written in Haskell, never seen Haskell before, it's really hard to follow for someone not used to it: https://github.com/hasura/graphql-engine/blob/master/server/src-lib/Hasura/Server/App.hs https://github.com/hasura/graphql-engine/blob/master/server/... Not sure how people enjoy FP.
- mrkurt 7y ago
  It's worth writing something in an ML or Haskell sometime. Functional programming is great, and it makes my code in things like JavaScript much more robust.
  - Cyph0n 7y ago
    Scala is probably as far as I’ll get when it comes to FP.
    - estsauver 7y ago
      Even within scala, you can dabble in quite a bit of functional programming concepts. Have you used typelevel's cats or scalaz? ~Earl
- lpage 7y ago
  Haskell and ML languages look like symbol soup when you're used to mainstream/C style languages. Once you understand some of the basic syntax they become quite readable and easy to reason about as a series of transformations. Step one, which applies to most functional languages, is knowing how to read a Hindley-Milner type signature [1]. Armed with that you can often gain a surface level understanding of what some function does - and what the program does in aggregate - just by reading type signatures and seeing what calls what. Hoogle [2] is a good resource for figuring out Haskell syntax, like what's up with all of those $ signs. Converting OCaml to ReasonML syntax with sketch or reason-tools [3] makes OCaml less foreign. With basic knowledge of Haskell and ML syntax you'll find many FP languages readable. [1] https://drboolean.gitbooks.io/mostly-adequate-guide-old/content/ch7.html#tales-from-the-cryptic https://drboolean.gitbooks.io/mostly-adequate-guide-old/cont... [2] https://hoogle.haskell.org/ https://hoogle.haskell.org/ [3] https://reasonml.github.io/docs/en/extra-goodies https://reasonml.github.io/docs/en/extra-goodies
  - jakear 7y ago
    “Quite readable” is a stretch... I know of no other language that needs a hoogle equivalent. That being said, my experience with Haskell is limited to working on some meta-meta-programming research in college, where every file used at minimum 3 different GHC extensions. So I may be biased.
thegoleffect 7y ago
When I first saw Hasura, it was AGPL licensed which makes it unusable in a lot of use cases so we passed on it. Looks like they converted it to Apache license two months ago. I suppose it is worth another look, I'm glad they made the change.
StreamBright 7y ago
Nice scaling numbers. Can you use Hasura without Kubernetes or Docker?
- tango12 7y ago
  You can compile the binary yourself, but otherwise docker is the way to go. What’re you looking at?
agrippanux 7y ago
Hasura is fantastic. I am using it for several projects atm. Coming from Prisma it's refreshingly easy to use and well thought out. The security/ACL model is esp. nice and easy to grok.
pier25 7y ago
We evaluated Hasura and the only problem we found is the integration with our own authorization system. Otherwise Hasura is quite frankly amazing. Our auth system has many roles per user, then each role has its permissions, pretty common setup. The problem is that Hasura expects a single default role per request to then evaluate against its permissions. The Hasura team has been looking into accepting an array of roles and merging permissions and whatnot, but AFAIK this hasn't been solved. If you start with Hasura from scratch this is not really a problem, it happened to us because we had to figure out how to integrate Hasura with our current permissions.
- contrasti 7y ago
  > the only problem we found is the integration with our own authorization system But integration with external authorization systems is an extremely basic requirement for every piece of software, certainly on the top 5 on the requirements list for every serious solution - I can not understand how did you forget about this?
  - sk5t 7y ago
    I haven't found this ("integration with external authorization systems is an extremely basic requirement") to be the case at all; most solutions ship with tightly-coupled access control mechanisms, and it is an exercise for the user to figure out how to wield them effectively. In Hasura's case, it looks like the flow of data to the authorization bits shows a simple/naive approach, although it also appears this is something Hasura are working to correct. Even as it stands, moderately complex authz logic could be implemented via x-hasura headers or in-database associations.
jazoom 7y ago
I know there's an issue about this but it doesn't really indicate Hasura's intentions regarding it: Is supporting CockroachDB something Hasura is actively working on?
- tango12 7y ago
  It’s on the cards! Don’t want to spread ourselves too thin too early. Also working with the yugabyte folks closely to have Hasura supported with their 2.0 release. https://twitter.com/karthikr/status/1128003465370693636?s=21 https://twitter.com/karthikr/status/1128003465370693636?s=21
  - jazoom 7y ago
    > Don’t want to spread ourselves too thin too early. I certainly understand that, but I wonder since CockroachDB is essentially Postgres, if it wouldn't be easier to ensure compatibility from the start. It seems like it'd be easier to just avoid using special Postgres features that aren't in CockroachDB than to go back later and re-write all your code that relies on those features.
    - tango12 7y ago
      Definitely. That’s why cockroach & yugabyte that speak Postgres but are black magic underneath are a great target. However, it’s not fully compatible quite yet https://github.com/cockroachdb/cockroach/issues?q=is%3Aopen+is%3Aissue+label%3AA-sql-pgcompat https://github.com/cockroachdb/cockroach/issues?q=is%3Aopen+... Citus / timescale / pipelinedb are much easier and we already support timescale and pipelinedb well. Citus with native support for their distribution columns is going to happen soon too! These are packaged as Postgres extensions which is definitely first priority for us before moving to databases that speak Postgres and eventually to other databases as well. :)
      - jazoom 7y ago
        > it’s not fully compatible quite yet https://github.com/cockroachdb/cockroach/issues?q=is%3Aopen+.. https://github.com/cockroachdb/cockroach/issues?q=is%3Aopen+.... So you're waiting for CockroachDB to implement features before you suppoort it?
londons_explore 7y ago
> we’ve currently fallen back to interval based polling to refetch queries. This fancy "subscribe to events when the results of this query change" system boils down to just polling the database...
- aidos 7y ago
  Sometimes the simple option is the best. Having said that, it’s not that simple, for them. I believe it relies on being able to fetch and dissect different results for different clients in the same query (all while applying their own form of RLS).
- GorgeRonde 7y ago
  Future work: Reduce load on Postgres by: - Mapping events to active live queries - Incremental computation of result set Fake it till you make it. Meanwhile: https://opensource.janestreet.com/incremental/ https://opensource.janestreet.com/incremental/
  - aidos 7y ago
    I’m not entirely sure how the library you posted is related. Because it does incremental computation? It doesn’t seem to have anything to do with Postgres. As far as I can tell it looks more like mobx.
    - GorgeRonde 7y ago
      They built other libraries on top of this [1] [2], and when compared to mobx [3] it indeed looks very similar, however I think the target ideal of having a small core on top of which one would build more complex computation is essential. Since Clojure is pervasively built on laziness, there has been some discussions in the community about doing the same but for incremental computation. And doing it pervasively most importantly guarantees it will work at any granularity without any manual fiddling with observables and reactions. A recent contribution to the solution field in Clojure is micro_adapton [4][5] which looks like it is to incremental computing what miniKanren is to logic programming, i.e. a tiny core for a a challenging problem. Also, considering the literature around it it seems to be pretty tricky to do properly, especially when it comes to databases [6]. [1] https://github.com/janestreet/incr_map https://github.com/janestreet/incr_map [2] https://github.com/janestreet/incr_select https://github.com/janestreet/incr_select [3] https://github.com/mobxjs/mobx https://github.com/mobxjs/mobx [4] https://github.com/aibrahim/micro_adapton https://github.com/aibrahim/micro_adapton [5] https://arxiv.org/abs/1609.05337 https://arxiv.org/abs/1609.05337 [6] https://scholar.google.fr/scholar?hl=fr&as_sdt=0%2C5&q=incremental+computation+database&btnG= https://scholar.google.fr/scholar?hl=fr&as_sdt=0%2C5&q=incre...
aidos 7y ago
Just another little shout out for Hasura here. It’s a really lovely system to use and the team behind it are awesome to deal with. Super friendly and helpful. We evaluated a lot of stuff recently before settling on Hasura. Prisma, aws appsync, postgraphile, roll our own socketio backend etc. I had a few hesitations about Hasura. In particular I was hoping to lean on Postgres RLS for the authorisation, and to be able to customise the graphql endpoints a bit more. After hearing the rational from the guys about how their system is better than RLS and seeing the patterns they have for implementing the logic I’m really happy we made the leap. We’re still in early days, but it’s been really solid and a real pleasure to use. (No affiliation, but happily paying for their support licence)
- ndarilek 7y ago
  Out of curiosity, what is the rationalle for abandoning RLS? In a previous project, I used Postgraphile and enjoyed it. Sure, RLS and PLPGSQL were a bit awkward, but I knew they were well-tested and had lots of eyes on them, so I felt comfortable that if an RLS policy blocked access to a row, then that particular role wasn't getting access. I also enjoyed working with a text-based format that more clearly fit within my other workflows of, well, working with other text-based formats. :) Another part of it is that I'm blind, and Hasura seems to make doing things outside of its web interface somewhat painful. Sure I can write a YAML migration by hand, but the YAML migration format seems more machine-friendly than user-friendly. Yesterday I needed to create a function, and the SQL web interface wasn't showing me the line number on which my function had an error. Ultimately I dropped to the psql command line, cut-and-pasted the function in manually, and right away got back the line number and fixed the issue. Please don't get me wrong. I'm not trying to hate on Hasura. But the fact that I can't just drop to a non-YAML text-based format and throw down a few checks to secure my tables has been an endless source of frustration for me. So if there is a non-NIH reason for abandoning RLS in favor of a separate security system, then maybe knowing that might help me be a bit less annoyed. :)
  - aidos 7y ago
    To be totally honest, I haven’t implemented anything using RLS, it just seemed the way to go since it’s native Postgres. The Hasura team gave me the same explanation as they give in this article - when you use RLS, you have to rerun every query for every client to get your results. By doing it in the application layer, you can run a single query instead. In terms of the setup and migrations, I also have the same reservation about it being harder to do with text. So far I’ve left it to one of the other guys to implement so it hasn’t been painful for me, yet! Edit to add: I believe they built their version of RLS before Postgres had it. So it was more a case of Not Invented Yet
lubesGordi 7y ago
I'm curious about the underlying push tech. Is Hasura using firebase to handle that? Then firebase relies on different push services depending on the client, like Apple's push service for iOS and http/2's push for web (which is supported by what percentage of browsers)?
- pier25 7y ago
  Firebase? No, not at all. It's all Hasura and Postgres.
  - lubesGordi 7y ago
    What I'm getting at is what is the underlying tech handling push to clients? Reading further I see that the graphql libraries handle subscriptions via websockets. But elsewhere in this github I was seeing firebase. So it's a little confusing. Websockets are pretty ubiquitous now but but from what I understand maintain a tcp connection to the server. That has scalability implications.
    - tango12 7y ago
      I think you were probably looking at docs/sample-code for a firebase auth integration. Hasura GraphQL subscriptions are over websockets. This post is a benchmark that should address those scalability concerns. It’s about getting to 1M concurrent active websocket connections.
machbio 7y ago
Write Ahead Logs would have been amazing - hate that AWS RDS does not support WAL - could have saved so much time for some of the applications we build.
- gunnarmorling 7y ago
  WDYM by "RDS does not support WAL"? I wouldn't know how Postgres works without them, and they are accessible using logical decoding. wal2json is supported on RDS and used by tools such as Debezium for streaming data changes into Apache Kafka (Disclaimer: I work on Debezium).
  - machbio 7y ago
    Thank you, had missed this announcement.. I did check Debezium a while back, found it appealing
themccallister 7y ago
I’ve been using Hasura for around 3 months now. It has changed the velocity and way I approach our projects. I have a small team that works as a startup inside of a larger organization and this was crucial to delivering some key projects... plus very little code. The event trigger really help reduce our costs as well, using the Serverless.com framework with Lambda Go has been a huge success in our organization! Thank you to the Hasura team for all you do and in turn allow me and my team to do!
rishav_sharan 7y ago
As an Indian engineer, Hasura is one the few indian startups that i really am proud of. They are innovative, distinct, product focused and super young. The only other company which made me feel like this was Notion Ink, but those guys seem to have faded away. I wish Hasura the best and would be cheering for them from the sidelines
- contrasti 7y ago
  The concept of nationalism has brought more pain than joy to millions of humans on this planet. Seems to be better for global peace to forget it and train your mind to not think in these political categories of "nations".
gunnarmorling 7y ago
I'm impressed by the numbers, but I'm having some doubts about the general idea of exposing your database via GraphQL. It seems to expose your internal data structures, and any change to those will immediatly impact that public GraphQL API. Also it seems that this approach would only work well for applications without any business logic, or with business logic solely implemented in the database. At least the applications I've been working on wouldn't fall into this category. But in any case, impressive work and demos!
- aleksei 7y ago
  You aren't limited to a single database though, so you could stream your main database to a public facing one with a more stable structure. But you'd have to handle changes through another API, or at least route them differently, and then worry about consistency so probably not ideal for all business cases.
  - tango12 7y ago
    True! We’ve added event triggers to Hasura to kind of support this pattern via Hasura itself. So you can create “action” tables that basically have a log of request data (a mutation inserts an action). Hasura will then call an event handler which can run with the action data, user session information, related data in case there are any relationships etc. This handler can then go update the tables that can be queried from. Ofcourse, if the eventing system is in-order / exactly once quite a few use-cases become feasible ;)
he0001 7y ago
How does it handle transactions and locking? It seem to support multiple databases, does it handle distributed transactions?
james4532 7y ago
NEED A HACKED ATM CARD WITH PIN? We Sell Physical Loaded ATM Cards . It Is A Cloned Card That Can Be Used To Withdraw Cash At Any ATM Machine. This Cards Comes In Visa/MasterCard. Therefore It Works At Any ATM Machine That Accept Visa/MasterCard Worldwide. CAN I USE THIS READY MADE ATM CARD TO BUY STUFF IN STORES? OR ONLINE SHOPPING? PAY BILLS? Yes, With This Physical ATM Card, You Can Use It To Pay Stuff At Stores Through POS. With This ATM Card Information, You Can Use It Online To Pay Bills Or Do Online Shopping. When You Order For This Card, Full Information About The Card Will Be Given To You. We Also Reload Your Card When Funds Exhausted. DO I NEED TO ORDER NEW CARD EVERY TIME I SPENT THE FUNDS FINISH? No, If You Have Already Ordered Our Card, There Is No Need To Keep Ordering New Cards, Just Contact Us For A Reload. We Shall Easily Reload The ATM Card Already In Your Possession HOW LONG DOES IT TAKE TO RECEIVE ATM CARD IN MY COUNTRY? If You Are In ASIA, You Will Receive Your Card In 2-3 DAYS With Guaranteed. If You Are Outside Asia Your Card Will Arrive To You Between 3 – 5 Business Days Guaranteed. HOW SAFE IS THIS CARD? It Is 100% Safe To Use This Card. Because It Will Be Shipped To You As A Gift Card. DO YOU ALSO RELOAD ANY OTHER CARD NOT FROM THIS CLONED CARDS? Yes, We Can Reload Any Active And Valid Cards, Any Type Of Card Just Contact Us For A Reload (Prepaid Cards, Credit/Debit Cards). HOW DO I ORDER FOR THE ATM CARD? EMAIL: braeckmansj@outlook.com HOW DO WE MAKE THIS CARD? We Use A Machine MSR To Clone This Cards . You Can Also Buy This Machine From Us Also. You Can Order For The ATM Card Either The Designed Card Or The Blank Card But Still Same Information On Them. EMAIL: (braeckmansj@outlook.com) whatsapp number..+393512840729

353 points by tango12 7y ago

rj5 7y ago

I can’t wait to try this out!

Hasura is such a promising application platform. I tried it a few months ago and was really pleased. In a way it kind of felt like RethinkDB, except GraphQL and Postgres. Was very easy to turn on with Docker Compose and play around. Looking forward to 1.0!

z3t4 7y ago
I guessed it would use the pub/sub capabilities of postgres. But it seems from the readme that it use polling, although using the same query which allows cache optimizations. The naive implementation can often be surprisingly effective.
- tango12 7y ago
  Yep. We experimented around a lot with pub/sub (listen/notify, wal-lr). But it was getting hard to scale and heavy/spiky write loads needed careful thought. This seemed like the right way to get started esp. given that Hasura can get the entire sql query including authz, before moving to tackling that problem more effectively[1]. For folks that want to play around with wal and listen/notify: https://github.com/hasura/pgdeltastream https://github.com/hasura/pgdeltastream https://github.com/hasura/skor https://github.com/hasura/skor [1] https://github.com/hasura/graphql-engine/blob/master/architecture/live-queries.md#when-do-we-refetch https://github.com/hasura/graphql-engine/blob/master/archite...

Thaxll 7y ago

I was curious about the code so I went to check it, it's written in Haskell, never seen Haskell before, it's really hard to follow for someone not used to it: https://github.com/hasura/graphql-engine/blob/master/server/src-lib/Hasura/Server/App.hs https://github.com/hasura/graphql-engine/blob/master/server/... Not sure how people enjoy FP.

mrkurt 7y ago
It's worth writing something in an ML or Haskell sometime. Functional programming is great, and it makes my code in things like JavaScript much more robust.
- Cyph0n 7y ago
  Scala is probably as far as I’ll get when it comes to FP.
  - estsauver 7y ago
    Even within scala, you can dabble in quite a bit of functional programming concepts. Have you used typelevel's cats or scalaz? ~Earl
lpage 7y ago
Haskell and ML languages look like symbol soup when you're used to mainstream/C style languages. Once you understand some of the basic syntax they become quite readable and easy to reason about as a series of transformations. Step one, which applies to most functional languages, is knowing how to read a Hindley-Milner type signature [1]. Armed with that you can often gain a surface level understanding of what some function does - and what the program does in aggregate - just by reading type signatures and seeing what calls what. Hoogle [2] is a good resource for figuring out Haskell syntax, like what's up with all of those $ signs. Converting OCaml to ReasonML syntax with sketch or reason-tools [3] makes OCaml less foreign. With basic knowledge of Haskell and ML syntax you'll find many FP languages readable. [1] https://drboolean.gitbooks.io/mostly-adequate-guide-old/content/ch7.html#tales-from-the-cryptic https://drboolean.gitbooks.io/mostly-adequate-guide-old/cont... [2] https://hoogle.haskell.org/ https://hoogle.haskell.org/ [3] https://reasonml.github.io/docs/en/extra-goodies https://reasonml.github.io/docs/en/extra-goodies
- jakear 7y ago
  “Quite readable” is a stretch... I know of no other language that needs a hoogle equivalent. That being said, my experience with Haskell is limited to working on some meta-meta-programming research in college, where every file used at minimum 3 different GHC extensions. So I may be biased.

thegoleffect 7y ago

When I first saw Hasura, it was AGPL licensed which makes it unusable in a lot of use cases so we passed on it. Looks like they converted it to Apache license two months ago. I suppose it is worth another look, I'm glad they made the change.

StreamBright 7y ago

Nice scaling numbers. Can you use Hasura without Kubernetes or Docker?

tango12 7y ago
You can compile the binary yourself, but otherwise docker is the way to go. What’re you looking at?

agrippanux 7y ago

Hasura is fantastic. I am using it for several projects atm. Coming from Prisma it's refreshingly easy to use and well thought out. The security/ACL model is esp. nice and easy to grok.

pier25 7y ago

We evaluated Hasura and the only problem we found is the integration with our own authorization system. Otherwise Hasura is quite frankly amazing. Our auth system has many roles per user, then each role has its permissions, pretty common setup. The problem is that Hasura expects a single default role per request to then evaluate against its permissions. The Hasura team has been looking into accepting an array of roles and merging permissions and whatnot, but AFAIK this hasn't been solved. If you start with Hasura from scratch this is not really a problem, it happened to us because we had to figure out how to integrate Hasura with our current permissions.

contrasti 7y ago
> the only problem we found is the integration with our own authorization system But integration with external authorization systems is an extremely basic requirement for every piece of software, certainly on the top 5 on the requirements list for every serious solution - I can not understand how did you forget about this?
- sk5t 7y ago
  I haven't found this ("integration with external authorization systems is an extremely basic requirement") to be the case at all; most solutions ship with tightly-coupled access control mechanisms, and it is an exercise for the user to figure out how to wield them effectively. In Hasura's case, it looks like the flow of data to the authorization bits shows a simple/naive approach, although it also appears this is something Hasura are working to correct. Even as it stands, moderately complex authz logic could be implemented via x-hasura headers or in-database associations.

jazoom 7y ago

I know there's an issue about this but it doesn't really indicate Hasura's intentions regarding it: Is supporting CockroachDB something Hasura is actively working on?

tango12 7y ago
It’s on the cards! Don’t want to spread ourselves too thin too early. Also working with the yugabyte folks closely to have Hasura supported with their 2.0 release. https://twitter.com/karthikr/status/1128003465370693636?s=21 https://twitter.com/karthikr/status/1128003465370693636?s=21
- jazoom 7y ago
  > Don’t want to spread ourselves too thin too early. I certainly understand that, but I wonder since CockroachDB is essentially Postgres, if it wouldn't be easier to ensure compatibility from the start. It seems like it'd be easier to just avoid using special Postgres features that aren't in CockroachDB than to go back later and re-write all your code that relies on those features.
  - tango12 7y ago
    Definitely. That’s why cockroach & yugabyte that speak Postgres but are black magic underneath are a great target. However, it’s not fully compatible quite yet https://github.com/cockroachdb/cockroach/issues?q=is%3Aopen+is%3Aissue+label%3AA-sql-pgcompat https://github.com/cockroachdb/cockroach/issues?q=is%3Aopen+... Citus / timescale / pipelinedb are much easier and we already support timescale and pipelinedb well. Citus with native support for their distribution columns is going to happen soon too! These are packaged as Postgres extensions which is definitely first priority for us before moving to databases that speak Postgres and eventually to other databases as well. :)
    - jazoom 7y ago
      > it’s not fully compatible quite yet https://github.com/cockroachdb/cockroach/issues?q=is%3Aopen+.. https://github.com/cockroachdb/cockroach/issues?q=is%3Aopen+.... So you're waiting for CockroachDB to implement features before you suppoort it?

londons_explore 7y ago

> we’ve currently fallen back to interval based polling to refetch queries. This fancy "subscribe to events when the results of this query change" system boils down to just polling the database...

aidos 7y ago
Sometimes the simple option is the best. Having said that, it’s not that simple, for them. I believe it relies on being able to fetch and dissect different results for different clients in the same query (all while applying their own form of RLS).
GorgeRonde 7y ago
Future work: Reduce load on Postgres by: - Mapping events to active live queries - Incremental computation of result set Fake it till you make it. Meanwhile: https://opensource.janestreet.com/incremental/ https://opensource.janestreet.com/incremental/
- aidos 7y ago
  I’m not entirely sure how the library you posted is related. Because it does incremental computation? It doesn’t seem to have anything to do with Postgres. As far as I can tell it looks more like mobx.
  - GorgeRonde 7y ago
    They built other libraries on top of this [1] [2], and when compared to mobx [3] it indeed looks very similar, however I think the target ideal of having a small core on top of which one would build more complex computation is essential. Since Clojure is pervasively built on laziness, there has been some discussions in the community about doing the same but for incremental computation. And doing it pervasively most importantly guarantees it will work at any granularity without any manual fiddling with observables and reactions. A recent contribution to the solution field in Clojure is micro_adapton [4][5] which looks like it is to incremental computing what miniKanren is to logic programming, i.e. a tiny core for a a challenging problem. Also, considering the literature around it it seems to be pretty tricky to do properly, especially when it comes to databases [6]. [1] https://github.com/janestreet/incr_map https://github.com/janestreet/incr_map [2] https://github.com/janestreet/incr_select https://github.com/janestreet/incr_select [3] https://github.com/mobxjs/mobx https://github.com/mobxjs/mobx [4] https://github.com/aibrahim/micro_adapton https://github.com/aibrahim/micro_adapton [5] https://arxiv.org/abs/1609.05337 https://arxiv.org/abs/1609.05337 [6] https://scholar.google.fr/scholar?hl=fr&as_sdt=0%2C5&q=incremental+computation+database&btnG= https://scholar.google.fr/scholar?hl=fr&as_sdt=0%2C5&q=incre...

aidos 7y ago

Just another little shout out for Hasura here. It’s a really lovely system to use and the team behind it are awesome to deal with. Super friendly and helpful. We evaluated a lot of stuff recently before settling on Hasura. Prisma, aws appsync, postgraphile, roll our own socketio backend etc. I had a few hesitations about Hasura. In particular I was hoping to lean on Postgres RLS for the authorisation, and to be able to customise the graphql endpoints a bit more. After hearing the rational from the guys about how their system is better than RLS and seeing the patterns they have for implementing the logic I’m really happy we made the leap. We’re still in early days, but it’s been really solid and a real pleasure to use. (No affiliation, but happily paying for their support licence)

ndarilek 7y ago
Out of curiosity, what is the rationalle for abandoning RLS? In a previous project, I used Postgraphile and enjoyed it. Sure, RLS and PLPGSQL were a bit awkward, but I knew they were well-tested and had lots of eyes on them, so I felt comfortable that if an RLS policy blocked access to a row, then that particular role wasn't getting access. I also enjoyed working with a text-based format that more clearly fit within my other workflows of, well, working with other text-based formats. :) Another part of it is that I'm blind, and Hasura seems to make doing things outside of its web interface somewhat painful. Sure I can write a YAML migration by hand, but the YAML migration format seems more machine-friendly than user-friendly. Yesterday I needed to create a function, and the SQL web interface wasn't showing me the line number on which my function had an error. Ultimately I dropped to the psql command line, cut-and-pasted the function in manually, and right away got back the line number and fixed the issue. Please don't get me wrong. I'm not trying to hate on Hasura. But the fact that I can't just drop to a non-YAML text-based format and throw down a few checks to secure my tables has been an endless source of frustration for me. So if there is a non-NIH reason for abandoning RLS in favor of a separate security system, then maybe knowing that might help me be a bit less annoyed. :)
- aidos 7y ago
  To be totally honest, I haven’t implemented anything using RLS, it just seemed the way to go since it’s native Postgres. The Hasura team gave me the same explanation as they give in this article - when you use RLS, you have to rerun every query for every client to get your results. By doing it in the application layer, you can run a single query instead. In terms of the setup and migrations, I also have the same reservation about it being harder to do with text. So far I’ve left it to one of the other guys to implement so it hasn’t been painful for me, yet! Edit to add: I believe they built their version of RLS before Postgres had it. So it was more a case of Not Invented Yet

lubesGordi 7y ago

I'm curious about the underlying push tech. Is Hasura using firebase to handle that? Then firebase relies on different push services depending on the client, like Apple's push service for iOS and http/2's push for web (which is supported by what percentage of browsers)?

pier25 7y ago
Firebase? No, not at all. It's all Hasura and Postgres.
- lubesGordi 7y ago
  What I'm getting at is what is the underlying tech handling push to clients? Reading further I see that the graphql libraries handle subscriptions via websockets. But elsewhere in this github I was seeing firebase. So it's a little confusing. Websockets are pretty ubiquitous now but but from what I understand maintain a tcp connection to the server. That has scalability implications.
  - tango12 7y ago
    I think you were probably looking at docs/sample-code for a firebase auth integration. Hasura GraphQL subscriptions are over websockets. This post is a benchmark that should address those scalability concerns. It’s about getting to 1M concurrent active websocket connections.

machbio 7y ago

Write Ahead Logs would have been amazing - hate that AWS RDS does not support WAL - could have saved so much time for some of the applications we build.

gunnarmorling 7y ago
WDYM by "RDS does not support WAL"? I wouldn't know how Postgres works without them, and they are accessible using logical decoding. wal2json is supported on RDS and used by tools such as Debezium for streaming data changes into Apache Kafka (Disclaimer: I work on Debezium).
- machbio 7y ago
  Thank you, had missed this announcement.. I did check Debezium a while back, found it appealing

themccallister 7y ago

I’ve been using Hasura for around 3 months now. It has changed the velocity and way I approach our projects. I have a small team that works as a startup inside of a larger organization and this was crucial to delivering some key projects... plus very little code. The event trigger really help reduce our costs as well, using the Serverless.com framework with Lambda Go has been a huge success in our organization! Thank you to the Hasura team for all you do and in turn allow me and my team to do!

rishav_sharan 7y ago

As an Indian engineer, Hasura is one the few indian startups that i really am proud of. They are innovative, distinct, product focused and super young. The only other company which made me feel like this was Notion Ink, but those guys seem to have faded away. I wish Hasura the best and would be cheering for them from the sidelines

contrasti 7y ago
The concept of nationalism has brought more pain than joy to millions of humans on this planet. Seems to be better for global peace to forget it and train your mind to not think in these political categories of "nations".

gunnarmorling 7y ago

I'm impressed by the numbers, but I'm having some doubts about the general idea of exposing your database via GraphQL. It seems to expose your internal data structures, and any change to those will immediatly impact that public GraphQL API. Also it seems that this approach would only work well for applications without any business logic, or with business logic solely implemented in the database. At least the applications I've been working on wouldn't fall into this category. But in any case, impressive work and demos!

aleksei 7y ago
You aren't limited to a single database though, so you could stream your main database to a public facing one with a more stable structure. But you'd have to handle changes through another API, or at least route them differently, and then worry about consistency so probably not ideal for all business cases.
- tango12 7y ago
  True! We’ve added event triggers to Hasura to kind of support this pattern via Hasura itself. So you can create “action” tables that basically have a log of request data (a mutation inserts an action). Hasura will then call an event handler which can run with the action data, user session information, related data in case there are any relationships etc. This handler can then go update the tables that can be queried from. Ofcourse, if the eventing system is in-order / exactly once quite a few use-cases become feasible ;)

he0001 7y ago

How does it handle transactions and locking? It seem to support multiple databases, does it handle distributed transactions?

james4532 7y ago

NEED A HACKED ATM CARD WITH PIN? We Sell Physical Loaded ATM Cards . It Is A Cloned Card That Can Be Used To Withdraw Cash At Any ATM Machine. This Cards Comes In Visa/MasterCard. Therefore It Works At Any ATM Machine That Accept Visa/MasterCard Worldwide. CAN I USE THIS READY MADE ATM CARD TO BUY STUFF IN STORES? OR ONLINE SHOPPING? PAY BILLS? Yes, With This Physical ATM Card, You Can Use It To Pay Stuff At Stores Through POS. With This ATM Card Information, You Can Use It Online To Pay Bills Or Do Online Shopping. When You Order For This Card, Full Information About The Card Will Be Given To You. We Also Reload Your Card When Funds Exhausted. DO I NEED TO ORDER NEW CARD EVERY TIME I SPENT THE FUNDS FINISH? No, If You Have Already Ordered Our Card, There Is No Need To Keep Ordering New Cards, Just Contact Us For A Reload. We Shall Easily Reload The ATM Card Already In Your Possession HOW LONG DOES IT TAKE TO RECEIVE ATM CARD IN MY COUNTRY? If You Are In ASIA, You Will Receive Your Card In 2-3 DAYS With Guaranteed. If You Are Outside Asia Your Card Will Arrive To You Between 3 – 5 Business Days Guaranteed. HOW SAFE IS THIS CARD? It Is 100% Safe To Use This Card. Because It Will Be Shipped To You As A Gift Card. DO YOU ALSO RELOAD ANY OTHER CARD NOT FROM THIS CLONED CARDS? Yes, We Can Reload Any Active And Valid Cards, Any Type Of Card Just Contact Us For A Reload (Prepaid Cards, Credit/Debit Cards). HOW DO I ORDER FOR THE ATM CARD? EMAIL: braeckmansj@outlook.com HOW DO WE MAKE THIS CARD? We Use A Machine MSR To Clone This Cards . You Can Also Buy This Machine From Us Also. You Can Order For The ATM Card Either The Designed Card Or The Blank Card But Still Same Information On Them. EMAIL: (braeckmansj@outlook.com) whatsapp number..+393512840729