6 ms·

I've had my eye on Taxi for a while, and it's neat! I agree that the problems it aims to solve are real and painful in real life. In my experience, I'm not sur

I've had my eye on Taxi for a while, and it's neat! I agree that the problems it aims to solve are real and painful in real life. In my experience, I'm not sure people care about schemas or schema languages — they are just implementation details best left under the hood. This is why in my own work, I started on the query end of the spectrum instead. This is why I built Trustfall, a query engine able to query any data source: DBs, APIs, files like PDF, CSV, or JSON, or any combination of those. GitHub: https://github.com/obi1kenobi/trustfall https://github.com/obi1kenobi/trustfall Playground showing the query "which GitHub/Twitter users comment on HackerNews stories about OpenAI?": https://play.predr.ag/hackernews#?f=1&q=IyBDcm9zcyBBUEkgcXVlcnkgKEFsZ29saWEgKyBGaXJlYmFzZSk6CiMgRmluZCBjb21tZW50cyBvbiBzdG9yaWVzIGFib3V0ICJvcGVuYWkuY29tIiB3aGVyZQojIHRoZSBjb21tZW50ZXIncyBiaW8gaGFzIGF0IGxlYXN0IG9uZSBHaXRIdWIgb3IgVHdpdHRlciBsaW5rCnF1ZXJ5IHsKICAjIFRoaXMgaGl0cyB0aGUgQWxnb2xpYSBzZWFyY2ggQVBJIGZvciBIYWNrZXJOZXdzLgogICMgVGhlIHN0b3JpZXMvY29tbWVudHMvdXNlcnMgZGF0YSBpcyBmcm9tIHRoZSBGaXJlYmFzZSBITiBBUEkuCiAgIyBUaGUgdHJhbnNpdGlvbiBpcyBzZWFtbGVzcyAtLSBpdCBpc24ndCB2aXNpYmxlIGZyb20gdGhlIHF1ZXJ5LgogIFNlYXJjaEJ5RGF0ZShxdWVyeTogIm9wZW5haS5jb20iKSB7CiAgICAuLi4gb24gU3RvcnkgewogICAgICAjIEFsbCBkYXRhIGZyb20gaGVyZSBvbndhcmQgaXMgZnJvbSB0aGUgRmlyZWJhc2UgQVBJLgogICAgICBzdG9yeVRpdGxlOiB0aXRsZSBAb3V0cHV0CiAgICAgIHN0b3J5TGluazogdXJsIEBvdXRwdXQKICAgICAgc3Rvcnk6IHN1Ym1pdHRlZFVybCBAb3V0cHV0CiAgICAgICAgICAgICAgICAgICAgICAgICAgQGZpbHRlcihvcDogInJlZ2V4IiwgdmFsdWU6IFsiJHNpdGVQYXR0ZXJuIl0pCgogICAgICBjb21tZW50IHsKICAgICAgICByZXBseSBAcmVjdXJzZShkZXB0aDogNSkgewogICAgICAgICAgY29tbWVudDogdGV4dFBsYWluIEBvdXRwdXQKCiAgICAgICAgICBieVVzZXIgewogICAgICAgICAgICBjb21tZW50ZXI6IGlkIEBvdXRwdXQKICAgICAgICAgICAgY29tbWVudGVyQmlvOiBhYm91dFBsYWluIEBvdXRwdXQKCiAgICAgICAgICAgICMgVGhlIHByb2ZpbGUgbXVzdCBoYXZlIGF0IGxlYXN0IG9uZQogICAgICAgICAgICAjIGxpbmsgdGhhdCBwb2ludHMgdG8gZWl0aGVyIEdpdEh1YiBvciBUd2l0dGVyLgogICAgICAgICAgICBsaW5rCiAgICAgICAgICAgICAgQGZvbGQKICAgICAgICAgICAgICBAdHJhbnNmb3JtKG9wOiAiY291bnQiKQogICAgICAgICAgICAgIEBmaWx0ZXIob3A6ICI%2BPSIsIHZhbHVlOiBbIiRtaW5Qcm9maWxlcyJdKQogICAgICAgICAgICB7CiAgICAgICAgICAgICAgY29tbWVudGVySURzOiB1cmwgQGZpbHRlcihvcDogInJlZ2V4IiwgdmFsdWU6IFsiJHNvY2lhbFBhdHRlcm4iXSkKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBAb3V0cHV0CiAgICAgICAgICAgIH0KICAgICAgICAgIH0KICAgICAgICB9CiAgICAgIH0KICAgIH0KICB9Cn0%3D&v=ewogICJzaXRlUGF0dGVybiI6ICJodHRwW3NdOi8vKFteLl0qXFwuKSpvcGVuYWkuY29tLy4qIiwKICAibWluUHJvZmlsZXMiOiAxLAogICJzb2NpYWxQYXR0ZXJuIjogIihnaXRodWJ8dHdpdHRlcilcXC5jb20vIgp9 https://play.predr.ag/hackernews#?f=1&q=IyBDcm9zcyBBUEkgcXVl...

martypitt 3y ago
I agree that querying is where the real value is at, and Trustfall looks like an elegant approach. While Taxi is all about documenting & augmenting specs, Orbital is the query engine (which is a bit similar to Trustfall) that consumes those specs. Orbital's goal is to allow consumers to be able to query for data, without having to be aware of the specifics of the data sets / APIs / DBs, etc they're composing together. IMO GraphQL does a nice job on the query side of keeping consumers away from the wiring, but shifts the obligation to middleware resolvers. How data is wired together is ultimately an implementation detail, and one that changes. If consumers need to know about how stuff is connected, they're subject to breakages when those details change.
- gavinray 3y ago
  > GraphQL does a nice job on the query side of keeping consumers away from the wiring, but shifts the obligation to middleware resolvers. I'm not sure that I understand how Orbital is different from tools that autogenerate GraphQL API's on top of data sources like databases and API's. From the site, it reads: > "Orbital uses your existing API specs and Database schemas - enriched with Taxi metadata to describe links. Orbital turns this into rich API and data catalog, letting you explore all your data, and how it connects." It sounds like the same concept, except using a new language instead of GraphQL? The schema language looks very similar to something AWS has called Smithy by the way: - https://aws.amazon.com/blogs/developer/introducing-smithy-idl-2-0/ https://aws.amazon.com/blogs/developer/introducing-smithy-id... - https://smithy.io/2.0/index.html https://smithy.io/2.0/index.html
  - martypitt 3y ago
    Autogen tools do a nice job of creating GraphQL schemas from a single source. We're looking more at how to compose / stitch multiple sources into a single queryable API. (ie., Stitch data from a DB, some REST APIs, and a Kafka topic). In GraphQL, you need to write / maintain resolvers that handle the stitching for you - which means as upstream data sources change, resolvers need to be maintained too. Smithy is nice, and we'll probably add first-class support in Orbital for it at some point. Taxi was around before Smithy, so we're the OG :)

martypitt 3y ago

I agree that querying is where the real value is at, and Trustfall looks like an elegant approach. While Taxi is all about documenting & augmenting specs, Orbital is the query engine (which is a bit similar to Trustfall) that consumes those specs. Orbital's goal is to allow consumers to be able to query for data, without having to be aware of the specifics of the data sets / APIs / DBs, etc they're composing together. IMO GraphQL does a nice job on the query side of keeping consumers away from the wiring, but shifts the obligation to middleware resolvers. How data is wired together is ultimately an implementation detail, and one that changes. If consumers need to know about how stuff is connected, they're subject to breakages when those details change.

gavinray 3y ago
> GraphQL does a nice job on the query side of keeping consumers away from the wiring, but shifts the obligation to middleware resolvers. I'm not sure that I understand how Orbital is different from tools that autogenerate GraphQL API's on top of data sources like databases and API's. From the site, it reads: > "Orbital uses your existing API specs and Database schemas - enriched with Taxi metadata to describe links. Orbital turns this into rich API and data catalog, letting you explore all your data, and how it connects." It sounds like the same concept, except using a new language instead of GraphQL? The schema language looks very similar to something AWS has called Smithy by the way: - https://aws.amazon.com/blogs/developer/introducing-smithy-idl-2-0/ https://aws.amazon.com/blogs/developer/introducing-smithy-id... - https://smithy.io/2.0/index.html https://smithy.io/2.0/index.html
- martypitt 3y ago
  Autogen tools do a nice job of creating GraphQL schemas from a single source. We're looking more at how to compose / stitch multiple sources into a single queryable API. (ie., Stitch data from a DB, some REST APIs, and a Kafka topic). In GraphQL, you need to write / maintain resolvers that handle the stitching for you - which means as upstream data sources change, resolvers need to be maintained too. Smithy is nice, and we'll probably add first-class support in Orbital for it at some point. Taxi was around before Smithy, so we're the OG :)