4 ms·

Show HN: Tigris – open-source developer data platform for your next app

58 points by ovaistariq 4y ago

Tigris is the first truly open source developer data platform with a simple yet powerful, unified API that spans search, event streaming, and transactional document store. It enables you to focus on building your applications and stop worrying about the data infrastructure.

anigbrowl 4y ago
Build out the overview with some case studies. If I am a first time cloud builder at the 'what should I use...' stage - which seems to be a target market for you - then I don't know what I don't know. Right now your marketing speaks to people who have already implemented something across multiple different platforms - but having learned all that they're probably more willing to accept the marginal cost of maintenance than to abandon the time and effort already invested in setting it all up. Building from scratch is a much easier and more pleasant decision than migrating. So you'll gain more users from people who want an accessible infrastructure experience that can grow with their (presently unknown) needs than with grizzled veterans - who might prefer just demanding a larger budget or more hires for infrastructure on a future project than learning a new technology that will render their previous knowledge obsolete. Your short intro video kind of addresses this, but shows your example developer deploying her app and then 5 different infrastructure platforms with zero effort - she doesn't care about the problem until it comes to the boring maintenance stage. A lot of people would take away that the market leading technologies are super-easy to deploy and that they can just hire someone to do the maintenance while they focus on product or the business side. (Also, the music doesn't match the visual story at all and feels cringey - again, not the emotional impression you want to create.) In general, people will go with whatever the default tool is because it's socially proven and there will be a big community of prior users who can help them through the teething problems. And there's a sense that the right tool for the right job is usually better than a do-everything tool. So you need to show where the pain points are early in the process of cloud deployment - when your target user is figuring out what infrastructure they need, and why.
- VyseofArcadia 4y ago
  > Build out the overview with some case studies. If I am a first time cloud builder at the 'what should I use...' stage - which seems to be a target market for you - then I don't know what I don't know. >Right now your marketing speaks to people who have already implemented something across multiple different platforms This seems to be a pretty big gap in marketing across the industry. Everything is either utterly meaningless market-speak[0] or targeted at experts. [0] The sheer number of products marketing themselves as one-size-fits-all "solutions" without ever bothering to define the problem...
  - ovaistariq 4y ago
    We have prepared this use-case video to depict how Tigris can be used: https://youtu.be/G489hasvgio https://youtu.be/G489hasvgio
- Volundr 4y ago
  > I don't know what I don't know. This probably describes me. Reading over the linked documentation it's not clear to me how this is different from couchdb or any number of other document databases that work over http.
  - ovaistariq 4y ago
    Does this explain it in a better way: https://docs.tigrisdata.com/overview/features https://docs.tigrisdata.com/overview/features
- ovaistariq 4y ago
  Great feedback. We do want to speak to folks building new applications as Tigris takes away the infrastructure burden completely from the start. We want to bring the same API experience to "data" as you get when using Stripe API for financial transactions. Migration is often times necessary. In my experience I have seen countless examples of migrations once an application becomes popular and the backend technology choices at that point either become brittle (hard to keep adding new app features), or do not scale. So I wouldn't categorize it as marginal maintenance cost. The video, I agree could be improved. Because the pain is not just seen at the maintenance stage, it is felt at every step that you add a new technology to you stack. Regarding your other comment about "people will go with whatever the default tool". It is actually the opposite, if you give people the choice they will stick to far fewer technologies :) This is why when you see someone building a top Postgres, as an example, initially they use it for the OLTP workloads, then they use it for the full-text search use cases (knowing fully well there is a better alternative because PG is not a search engine), or use it for analytics (even though there are other special-purpose analytical products), or use it for modeling event stream (even though again there is a special-purpose product available for event streaming). This is how application architectures organically grow, and the reason this happens is exactly because it is much easier (development time-wise and maintenance-wise) for me to continue to use X until it is unsustainable. And this is where we are coming from. We want to be that general purpose solution that you can continue to use to build up your application and it will grow/evolve with you. We did prepare this use-case video, I am curious to hear your thoughts: https://youtu.be/G489hasvgio https://youtu.be/G489hasvgio
masukomi 4y ago
after reading the front page i honestly have no clue what this is, but i'm reasonably certain that part of it is a document db. the quickstart seem to imply that i throw JSON at it via APIs and that that ends up in the implied db. The quickstart is also all about "hey DB stuff" so, now i think it's... a MongoDB competitor? maybe?
- ovaistariq 4y ago
  Thanks for the feedback on the doc. At the core of it, it is a transactional document database that uses FoundationDB under the hood for data persistence. It automatically provides search capability on the data stored (similar to Elasticsearch) without you having to pipe data to Elasticsearch. Then there is event streaming capability that is also provided out-of-the box, so you can use simple publish/subscribe APIs to build event-driven applications without having to introduce Kafka. A better comparison would be against MongoDB Atlas. Here are some of the ways it is differentiated from Atlas: - Open source that you can deploy anywhere, the only requirement being Kubernetes. Whether it is your cloud environment or on-prem, it doesn't matter. - Much simpler API interface as compared to MongoDB with native support for HTTP enabling it to be used from anywhere where HTTP is accessible - edge, mobile, etc. - Built with strong transactional consistency guarantee (strict serializability) - this is the benefit of using FoundationDB for data persistence. - MongoDB Atlas is based on 10 years old architecture and the industry has evolved quite a bit thanks to the cloud. Tigris has a micro-service based architecture with separately deployed components (compute, storage, indexing) that can each be scaled independently. - True serverless experience, which is of course possible through a modern architecture. - Automatic indexing, so you don't spend time optimizing queries and secondary indexes.
throwaway2016a 4y ago
So where does the data actually live? I read through the page and I'm still not clear. Is Tigris itself a database? Specifically it seems, a multi-paradigm database? And if it is, what is it that makes it a platform and not simply a database? Not criticizing, these are just questions I have initially trying to grok it at a quick glance.
- ovaistariq 4y ago
  We use FoundationDB’s transactional key-value interface as the underlying storage engine. This provides us with a low level building block that can scale to peta bytes of data. The reason it is more than a database is because it provides more than the structured data storage that is provided by databases. At the core of Tigris is a transactional document database built on FoundationDB. Then Tigris automatically indexes the data so you can run search queries (think Elasticsearch) without having to pipe the data to Elasticsearch. Tigris also provides an integrated event streaming system so that you can build event-driven applications (think Kafka, Google PubSub, etc). The beauty of the event streaming system is that you can then also run search and read queries on the events data automatically. We have explained the high level architecture here: https://docs.tigrisdata.com/overview/architecture https://docs.tigrisdata.com/overview/architecture
  - throwaway2016a 4y ago
    Thank you for the link. That is helpful.
asdfasdfsdf1231 4y ago
it's a REST+json payload frontend for foundationDB. ...by people who doesn't feel that the underlying tech is important to mention. So, not for this crowd.
- ovaistariq 4y ago
  The REST+json part is one part of it. FoundationDB is an awesome tech and proven at scale. But it only provides a simple key-value interface. But we have built multiple features on top of it to enable it to be used as complete platform. Query processing layer that can do complex read queries similar to FoundationDB. Ability to do full-text searches similar to Elastisearch. Ability to use it for event streaming through simple and easy to use publish/subscribe APIs without the complexity of Kafka. All of it is done in a cohesive manner, so for example, when you use it as your event streaming system, you get out of the box capability to query and search data without having to pipe the data somewhere else.
dang 4y ago
This looks like a great project but it's not a valid Show HN if people can't try it out. Please see https://news.ycombinator.com/showhn.html https://news.ycombinator.com/showhn.html. Note the bit about signup pages. (I've taken Show HN out of the title now.)
- ovaistariq 4y ago
  People can try it out in local development environment: https://docs.tigrisdata.com/quickstart https://docs.tigrisdata.com/quickstart Besides that, it is Apache2 license which means folks can deploy it to their k8s and run it.
  - dang 4y ago
    Ok! I'll put Show HN back in the title. Sorry for misunderstanding.
UncleEntity 4y ago
Is it just a server process sitting in front of the db? All this “docker this” and “docker that” I can’t tell what things do anymore. I assume the docker image does a whole lot of magic to get things to work that one would have to reproduce to run this on a bare VM they have sitting around. I’m not trying to be negative here, this looks like something I’ve had rattling around in my head for a while but haven’t had the time to figure out how to make. Well, my imaginary version is based on graphQL (and not written in JavaScript like every other graphQL thing) simply because I want to try out graphQL and no other reason. Oh, and a python client library would be nice but I can knock something out on my own to play around with this when I get some free time.
- ovaistariq 4y ago
  Take a look at the high level architecture here: https://docs.tigrisdata.com/overview/architecture https://docs.tigrisdata.com/overview/architecture. I will be happy to go into more details. `docker run -d -p 8081:8081 tigrisdata/tigris-local:latest` essentially gives you a local development version of Tigris to play around on your local machine as well as to allow you to build and test your application locally. We have a managed version of this in the cloud that essentially runs the entire platform on Kubernetes. I am glad to hear that this is similar to what you have thought about building as well. As an application developer in my past life, I have seen so much pain building apps because eventually I end up spending more time on infrastructure and maintenance and less time on my application code. We have built Tigris to solve this by providing you with simple APIs to interface with data without dealing with a separate database system, a separate event storage system, a separate search system, etc. We didn't go with graphQL because we want a more broad adoption and it seems like HTTP APIs are still the most widely used. At some point though I do envision us having graphQL support. We are open source so would love your contribution regarding python client library.
j_kao 4y ago
Great job shipping! When it comes to open-source infrastructure projects like this, the first thing I generally look for is an architecture diagram on how to operate something like this. Having a great developer-friendly API is important, but operating the system(s) is just as important and often a differentiation point of using the system (e.g. scale up some nodes, and you're good vs a mish-mash of an OLTP database, distributed WAL, a denormalized search cluster, etc). I don't get a good sense of how this works right now with the docs.
- ovaistariq 4y ago
  I agree details are important as well. Do you think this is providing enough information: https://docs.tigrisdata.com/overview/architecture https://docs.tigrisdata.com/overview/architecture ? Or would you suggest adding more technical details? There are essentially 3 major components: - FoundationDB (providing transactional data storage) - Sharded version of Typesense (providing in-memory indexes) - Query engine return in Go (stateless compute) Each of these components can be scaled independently.

58 points by ovaistariq 4y ago

anigbrowl 4y ago

Build out the overview with some case studies. If I am a first time cloud builder at the 'what should I use...' stage - which seems to be a target market for you - then I don't know what I don't know. Right now your marketing speaks to people who have already implemented something across multiple different platforms - but having learned all that they're probably more willing to accept the marginal cost of maintenance than to abandon the time and effort already invested in setting it all up. Building from scratch is a much easier and more pleasant decision than migrating. So you'll gain more users from people who want an accessible infrastructure experience that can grow with their (presently unknown) needs than with grizzled veterans - who might prefer just demanding a larger budget or more hires for infrastructure on a future project than learning a new technology that will render their previous knowledge obsolete. Your short intro video kind of addresses this, but shows your example developer deploying her app and then 5 different infrastructure platforms with zero effort - she doesn't care about the problem until it comes to the boring maintenance stage. A lot of people would take away that the market leading technologies are super-easy to deploy and that they can just hire someone to do the maintenance while they focus on product or the business side. (Also, the music doesn't match the visual story at all and feels cringey - again, not the emotional impression you want to create.) In general, people will go with whatever the default tool is because it's socially proven and there will be a big community of prior users who can help them through the teething problems. And there's a sense that the right tool for the right job is usually better than a do-everything tool. So you need to show where the pain points are early in the process of cloud deployment - when your target user is figuring out what infrastructure they need, and why.

VyseofArcadia 4y ago
> Build out the overview with some case studies. If I am a first time cloud builder at the 'what should I use...' stage - which seems to be a target market for you - then I don't know what I don't know. >Right now your marketing speaks to people who have already implemented something across multiple different platforms This seems to be a pretty big gap in marketing across the industry. Everything is either utterly meaningless market-speak[0] or targeted at experts. [0] The sheer number of products marketing themselves as one-size-fits-all "solutions" without ever bothering to define the problem...
- ovaistariq 4y ago
  We have prepared this use-case video to depict how Tigris can be used: https://youtu.be/G489hasvgio https://youtu.be/G489hasvgio
Volundr 4y ago
> I don't know what I don't know. This probably describes me. Reading over the linked documentation it's not clear to me how this is different from couchdb or any number of other document databases that work over http.
- ovaistariq 4y ago
  Does this explain it in a better way: https://docs.tigrisdata.com/overview/features https://docs.tigrisdata.com/overview/features
ovaistariq 4y ago
Great feedback. We do want to speak to folks building new applications as Tigris takes away the infrastructure burden completely from the start. We want to bring the same API experience to "data" as you get when using Stripe API for financial transactions. Migration is often times necessary. In my experience I have seen countless examples of migrations once an application becomes popular and the backend technology choices at that point either become brittle (hard to keep adding new app features), or do not scale. So I wouldn't categorize it as marginal maintenance cost. The video, I agree could be improved. Because the pain is not just seen at the maintenance stage, it is felt at every step that you add a new technology to you stack. Regarding your other comment about "people will go with whatever the default tool". It is actually the opposite, if you give people the choice they will stick to far fewer technologies :) This is why when you see someone building a top Postgres, as an example, initially they use it for the OLTP workloads, then they use it for the full-text search use cases (knowing fully well there is a better alternative because PG is not a search engine), or use it for analytics (even though there are other special-purpose analytical products), or use it for modeling event stream (even though again there is a special-purpose product available for event streaming). This is how application architectures organically grow, and the reason this happens is exactly because it is much easier (development time-wise and maintenance-wise) for me to continue to use X until it is unsustainable. And this is where we are coming from. We want to be that general purpose solution that you can continue to use to build up your application and it will grow/evolve with you. We did prepare this use-case video, I am curious to hear your thoughts: https://youtu.be/G489hasvgio https://youtu.be/G489hasvgio

masukomi 4y ago

after reading the front page i honestly have no clue what this is, but i'm reasonably certain that part of it is a document db. the quickstart seem to imply that i throw JSON at it via APIs and that that ends up in the implied db. The quickstart is also all about "hey DB stuff" so, now i think it's... a MongoDB competitor? maybe?

ovaistariq 4y ago
Thanks for the feedback on the doc. At the core of it, it is a transactional document database that uses FoundationDB under the hood for data persistence. It automatically provides search capability on the data stored (similar to Elasticsearch) without you having to pipe data to Elasticsearch. Then there is event streaming capability that is also provided out-of-the box, so you can use simple publish/subscribe APIs to build event-driven applications without having to introduce Kafka. A better comparison would be against MongoDB Atlas. Here are some of the ways it is differentiated from Atlas: - Open source that you can deploy anywhere, the only requirement being Kubernetes. Whether it is your cloud environment or on-prem, it doesn't matter. - Much simpler API interface as compared to MongoDB with native support for HTTP enabling it to be used from anywhere where HTTP is accessible - edge, mobile, etc. - Built with strong transactional consistency guarantee (strict serializability) - this is the benefit of using FoundationDB for data persistence. - MongoDB Atlas is based on 10 years old architecture and the industry has evolved quite a bit thanks to the cloud. Tigris has a micro-service based architecture with separately deployed components (compute, storage, indexing) that can each be scaled independently. - True serverless experience, which is of course possible through a modern architecture. - Automatic indexing, so you don't spend time optimizing queries and secondary indexes.

throwaway2016a 4y ago

So where does the data actually live? I read through the page and I'm still not clear. Is Tigris itself a database? Specifically it seems, a multi-paradigm database? And if it is, what is it that makes it a platform and not simply a database? Not criticizing, these are just questions I have initially trying to grok it at a quick glance.

ovaistariq 4y ago
We use FoundationDB’s transactional key-value interface as the underlying storage engine. This provides us with a low level building block that can scale to peta bytes of data. The reason it is more than a database is because it provides more than the structured data storage that is provided by databases. At the core of Tigris is a transactional document database built on FoundationDB. Then Tigris automatically indexes the data so you can run search queries (think Elasticsearch) without having to pipe the data to Elasticsearch. Tigris also provides an integrated event streaming system so that you can build event-driven applications (think Kafka, Google PubSub, etc). The beauty of the event streaming system is that you can then also run search and read queries on the events data automatically. We have explained the high level architecture here: https://docs.tigrisdata.com/overview/architecture https://docs.tigrisdata.com/overview/architecture
- throwaway2016a 4y ago
  Thank you for the link. That is helpful.

asdfasdfsdf1231 4y ago

it's a REST+json payload frontend for foundationDB. ...by people who doesn't feel that the underlying tech is important to mention. So, not for this crowd.

ovaistariq 4y ago
The REST+json part is one part of it. FoundationDB is an awesome tech and proven at scale. But it only provides a simple key-value interface. But we have built multiple features on top of it to enable it to be used as complete platform. Query processing layer that can do complex read queries similar to FoundationDB. Ability to do full-text searches similar to Elastisearch. Ability to use it for event streaming through simple and easy to use publish/subscribe APIs without the complexity of Kafka. All of it is done in a cohesive manner, so for example, when you use it as your event streaming system, you get out of the box capability to query and search data without having to pipe the data somewhere else.

dang 4y ago

This looks like a great project but it's not a valid Show HN if people can't try it out. Please see https://news.ycombinator.com/showhn.html https://news.ycombinator.com/showhn.html. Note the bit about signup pages. (I've taken Show HN out of the title now.)

ovaistariq 4y ago
People can try it out in local development environment: https://docs.tigrisdata.com/quickstart https://docs.tigrisdata.com/quickstart Besides that, it is Apache2 license which means folks can deploy it to their k8s and run it.
- dang 4y ago
  Ok! I'll put Show HN back in the title. Sorry for misunderstanding.

UncleEntity 4y ago

Is it just a server process sitting in front of the db? All this “docker this” and “docker that” I can’t tell what things do anymore. I assume the docker image does a whole lot of magic to get things to work that one would have to reproduce to run this on a bare VM they have sitting around. I’m not trying to be negative here, this looks like something I’ve had rattling around in my head for a while but haven’t had the time to figure out how to make. Well, my imaginary version is based on graphQL (and not written in JavaScript like every other graphQL thing) simply because I want to try out graphQL and no other reason. Oh, and a python client library would be nice but I can knock something out on my own to play around with this when I get some free time.

ovaistariq 4y ago
Take a look at the high level architecture here: https://docs.tigrisdata.com/overview/architecture https://docs.tigrisdata.com/overview/architecture. I will be happy to go into more details. `docker run -d -p 8081:8081 tigrisdata/tigris-local:latest` essentially gives you a local development version of Tigris to play around on your local machine as well as to allow you to build and test your application locally. We have a managed version of this in the cloud that essentially runs the entire platform on Kubernetes. I am glad to hear that this is similar to what you have thought about building as well. As an application developer in my past life, I have seen so much pain building apps because eventually I end up spending more time on infrastructure and maintenance and less time on my application code. We have built Tigris to solve this by providing you with simple APIs to interface with data without dealing with a separate database system, a separate event storage system, a separate search system, etc. We didn't go with graphQL because we want a more broad adoption and it seems like HTTP APIs are still the most widely used. At some point though I do envision us having graphQL support. We are open source so would love your contribution regarding python client library.

j_kao 4y ago

Great job shipping! When it comes to open-source infrastructure projects like this, the first thing I generally look for is an architecture diagram on how to operate something like this. Having a great developer-friendly API is important, but operating the system(s) is just as important and often a differentiation point of using the system (e.g. scale up some nodes, and you're good vs a mish-mash of an OLTP database, distributed WAL, a denormalized search cluster, etc). I don't get a good sense of how this works right now with the docs.

ovaistariq 4y ago
I agree details are important as well. Do you think this is providing enough information: https://docs.tigrisdata.com/overview/architecture https://docs.tigrisdata.com/overview/architecture ? Or would you suggest adding more technical details? There are essentially 3 major components: - FoundationDB (providing transactional data storage) - Sharded version of Typesense (providing in-memory indexes) - Query engine return in Go (stateless compute) Each of these components can be scaled independently.