4 ms·

Looks kinda like Dokku + PostgREST + a fancy UI, is that mostly right? Not trying to disparage it, just trying to relate figure out what Hasura is, there's a lo

by Arcsech 10y ago

Looks kinda like Dokku + PostgREST + a fancy UI, is that mostly right? Not trying to disparage it, just trying to relate figure out what Hasura is, there's a lot going on. One thing that concerns me is the "Don't make changes to your Postgres schema outside of the UI, it'll mess things up" warning. If I already have a tool to manage database migrations, it sounds like using it with this would be a bad idea? Edit: Also, what accounts for the speed increase over a bare VM? I'm assuming something Hasura does for you, but it would be nice to know what that is.

tango12 10y ago
Hasura engg. here. No offence taken! :) Hasura is indeed kind of what you described. The warning on the UI is mostly for non power-users who probably don't have a good db migration kind of practice. Underneath, the Hasura data API stores its minimal metadata within Postgres itself. That means that you can always use Postgres directly, and your migration tool works exactly the way it should. Whenever your schema migrations affect a data API, you need to ensure a metadata update for the Data API as well. We're going to be publishing examples of how DB migrations look using just a raw SQL setup, and an alembic setup! The speed increase for the Data APIs is because of a few reasons: 1) Efficient serialization/deserialization of the JSON queries and responses (https://hackage.haskell.org/package/aeson https://hackage.haskell.org/package/aeson) 2) Prepared statements 3) JSON aggregations done within Postgres (not generating the JSON) 4) Every client API call needs to be authenticated. The authentication is done by a efficient middleware written inside nginx with redis as the session store. The data API layer, then just does the authorization (this user-id can access this row).
ruslan_talpa 10y ago
Actually Hasura is something that might be the only thing out there that is anything close to touching postgrest in this space (respect :)). The only thing that worries me a bit is that the client can generate any type of query, meaning it's easy to generate an unoptimized join that will kill the db (did i get it wrong? how do you protect against that?). The other thing i do not understand is why are you not relying on "reading" the schema and require to go through a gui to define the schema. Why did you not copy the queries from postgrest that introspect the database schema and the relations? If you guys want i can send you a single query that you run and get the structure of the database and all the relations and use that as your "metadata" :) (I'm from postgrest core team, probably competing with hasura in the future here https://graphqlapi.com https://graphqlapi.com)
- tango12 10y ago
  Hey there! Thanks for your comment. Well actually we do infer directly from the Postgres schema. Only for users not familiar with Postgres directly, we recommend going through the UI so that not only is the table created, but the Data API is also asked to track it and provide an API for it. There is other metadata that we do need the user to specify, via the UI or the API, and that's for stuff like permissions and relationships. Regarding, the unrestricted querying, we address it in one of the following ways: 1. Timing out queries running for longer than a specified interval. 2. Cursors and streaming data if the dataset requested is large. 3. Query templates: We allow the developer to create a RESTful API endpoint that has a limited pre-defined surface area (eg: GET data.mysite.com/v1/template/article?limit=10) which internally maps to a pre-defined query. Our docs are still lagging behind on these unfortunately! What do you guys do to restrict access? Any suggestions?
  - ruslan_talpa 10y ago
    about metadata: in Postgrest - you tell it which schema to expose and that means that anything inside it needs "tracking" and exposing, want it exposed, put it in that schema. Relations between entities ... that can be queried from the database itself, no need to ask the user for it. timing out - it works up to a point, if you for example set a hard limit to 1s then to an attacker it just means he needs to do a few parallel requests and he still kills your db. what postgrest does for protection: a - no fancy joins, joins are only done by a thing we call "embedding" and those are joins between tables that FKs defined between them so they are fast, if you need fancy joins then the user has to define the view doing the join and expose it to the api. b - no functions applied to the return columns, like you cant generate something like "select md5(bigtextcolumn) ..." c - the only way to "kill" the database under postgrest is to use a filter on a column that has no index ( name like "hello"), out of the box postgrest does not have protection build in for that, but it's easy to put a proxy in front and whenever there is a url parameter called "name" you just reply with "you are not allowed to filter by this column"
    - ruslan_talpa 10y ago

by Arcsech 10y ago

tango12 10y ago

Hasura engg. here. No offence taken! :) Hasura is indeed kind of what you described. The warning on the UI is mostly for non power-users who probably don't have a good db migration kind of practice. Underneath, the Hasura data API stores its minimal metadata within Postgres itself. That means that you can always use Postgres directly, and your migration tool works exactly the way it should. Whenever your schema migrations affect a data API, you need to ensure a metadata update for the Data API as well. We're going to be publishing examples of how DB migrations look using just a raw SQL setup, and an alembic setup! The speed increase for the Data APIs is because of a few reasons: 1) Efficient serialization/deserialization of the JSON queries and responses (https://hackage.haskell.org/package/aeson https://hackage.haskell.org/package/aeson) 2) Prepared statements 3) JSON aggregations done within Postgres (not generating the JSON) 4) Every client API call needs to be authenticated. The authentication is done by a efficient middleware written inside nginx with redis as the session store. The data API layer, then just does the authorization (this user-id can access this row).

ruslan_talpa 10y ago

Actually Hasura is something that might be the only thing out there that is anything close to touching postgrest in this space (respect :)). The only thing that worries me a bit is that the client can generate any type of query, meaning it's easy to generate an unoptimized join that will kill the db (did i get it wrong? how do you protect against that?). The other thing i do not understand is why are you not relying on "reading" the schema and require to go through a gui to define the schema. Why did you not copy the queries from postgrest that introspect the database schema and the relations? If you guys want i can send you a single query that you run and get the structure of the database and all the relations and use that as your "metadata" :) (I'm from postgrest core team, probably competing with hasura in the future here https://graphqlapi.com https://graphqlapi.com)

tango12 10y ago
Hey there! Thanks for your comment. Well actually we do infer directly from the Postgres schema. Only for users not familiar with Postgres directly, we recommend going through the UI so that not only is the table created, but the Data API is also asked to track it and provide an API for it. There is other metadata that we do need the user to specify, via the UI or the API, and that's for stuff like permissions and relationships. Regarding, the unrestricted querying, we address it in one of the following ways: 1. Timing out queries running for longer than a specified interval. 2. Cursors and streaming data if the dataset requested is large. 3. Query templates: We allow the developer to create a RESTful API endpoint that has a limited pre-defined surface area (eg: GET data.mysite.com/v1/template/article?limit=10) which internally maps to a pre-defined query. Our docs are still lagging behind on these unfortunately! What do you guys do to restrict access? Any suggestions?
- ruslan_talpa 10y ago
  about metadata: in Postgrest - you tell it which schema to expose and that means that anything inside it needs "tracking" and exposing, want it exposed, put it in that schema. Relations between entities ... that can be queried from the database itself, no need to ask the user for it. timing out - it works up to a point, if you for example set a hard limit to 1s then to an attacker it just means he needs to do a few parallel requests and he still kills your db. what postgrest does for protection: a - no fancy joins, joins are only done by a thing we call "embedding" and those are joins between tables that FKs defined between them so they are fast, if you need fancy joins then the user has to define the view doing the join and expose it to the api. b - no functions applied to the return columns, like you cant generate something like "select md5(bigtextcolumn) ..." c - the only way to "kill" the database under postgrest is to use a filter on a column that has no index ( name like "hello"), out of the box postgrest does not have protection build in for that, but it's easy to put a proxy in front and whenever there is a url parameter called "name" you just reply with "you are not allowed to filter by this column"
  - ruslan_talpa 10y ago