7 ms·

Tidis - Redis protocol on top of TiKV

65 points by jinqueeny 9y ago

jazoom 9y ago
>This repo is WIP now and has lots of work to do, and for test only. Cool project though.
justicezyx 9y ago
I am actually surprised that given today's wide availability of cloud service, new low-level storage softwares are still getting invented. Questions to fellow HNers: - If you need to deploy such software for your business, was it because optimizing GCP/AWS is not cost-effective or the same level of performance is not possible on public cloud?
- tyingq 9y ago
  Sometimes, for various reasons, on-prem is the only choice. Politics, bandwidth limitations, expediency, egress charges, heavy-handed IT security, etc. Similar for EC2 clones.
  - toomuchtodo 9y ago
    There are compliance and business continuity requirements as well. We don’t allow business apps on cloud providers unless the business unit can prove they can pull it back on prem within a certain number of days.
- gregwebs 9y ago
  Your view point of just using something already hosted by GCP/AWS is probably the default today for good reason. I would break down the reason to not do that to three reasons: cost, performance, and customization. Lets ignore cost (which is complex) and focus on the TiDB and AWS example. TiDB was build to operate at a scale for OLTP (scale writes horizontally just by adding more nodes) that doesn't exist on Amazon's MySQL. You would have to you DynamoDB and downgrade from SQL transactions to eventual consistency. In terms of customization, TiDB is offering something AWS does not: the ability to store ever more of your data in one distributed data store (TiKV) and to leverage that in different ways (query it with TiSpark, and perhaps more projects like this in the future).
  - elvinyung 9y ago
    Correction: AWS does have this kind of strongly consistent, scalable OLTP database in Aurora.
    - ddorian43 9y ago
      Correction: No. Aurora is single node. No horizontal scaling.
      - iampims 9y ago
        Not exactly. You can add as many replicas as you need to scale reads.
      - wgjordan 9y ago
        Aurora currently supports automatic horizontal scaling for reads [1], and horizontal scaling for writes is currently in preview [2]. [1]: https://aws.amazon.com/about-aws/whats-new/2017/11/amazon-aurora-now-supports-auto-scaling-for-aurora-replicas/ https://aws.amazon.com/about-aws/whats-new/2017/11/amazon-au... [2]: https://aws.amazon.com/about-aws/whats-new/2017/11/sign-up-for-the-preview-of-amazon-aurora-multi-master/ https://aws.amazon.com/about-aws/whats-new/2017/11/sign-up-f...
        antoncohen 9y ago
        > horizontal scaling for writes is currently in preview I don't expect multi-master to increase write performance much. It will scale out network connections and maybe query processing, but not actual writing of data. It will replicate all data to all nodes, and the replication will be synchronous to any node it can automatically fail over to. So a client will be able to connect to any master, but that master will have to synchronously write to at minimum all other masters, if not all other nodes, before the server can acknowledge the write back to the client. Every node will need the write performance to write every write. That isn't scaling out writes.
        9y ago
- alexnewman 9y ago
  I’ve used all of the amazon datastores and none of them write as quick as a data store designed like this or maybe something like Hbase or Cassandra depending on what you are doing. Something like this is even better. I think amazon does Hbase as a service btw.Those use cases are rare so Aurora is usually the right answer. Alternatively A hundred s3 buckets can handle a large amount of writes as well for large or medium sized writes but it’s annoying to manage.
- cortesoft 9y ago
  Well, my business is working for a cloud services provider, so I can’t exactly rely on cloud services.
- manigandham 9y ago
  Complex services are hard to manage, even at scale with billions of investment. We've had plenty of instances where managed services had issues but VMs (being relatively simple tech) were just fine and we would've been better off running it ourselves. Also there's the cost ratio where high performance becomes exponential more expensive for many services. Perhaps the biggest reason though is that software is easier than ever to operate now with the rise of containers and Kubernetes. In fact, you can use Kubernetes as the single managed service to get that taken care of by the cloud, and then do the rest yourself for a great balance of cost, performance, flexibility and reliability.
maxpert 9y ago
Benchmarks please?
imauld 9y ago
Are there docs or examples of how to use this somewhere? I love Redis and I love using Go so I would be interested in trying this out for hobby projects and it would be helpful to have some documentation/examples.
- ceohockey60 9y ago
  Not directly related to tidis, but I did read about another Redis-on-top-of-Tikv use case by Ele.me, which is apparently this big food delivery platform like Doordash that's super popular in China. Here's the post: https://zhuanlan.zhihu.com/p/35226447 https://zhuanlan.zhihu.com/p/35226447 (too bad it's in Chinese right now. Hopefully someone translates it into English soon...)
  - c4pt0r 9y ago
    I think the team is working on it right now.
etaioinshrdlu 9y ago
Redis has been called a "data structure server". Isn't that what a database is? Is there anything fundamentally better about Redis than say, a single connection to Postgres? Except with Postgres you would be allowed many concurrent connections and greater throughput, while having the same external behavior thanks to transaction serializability. It's as if the operations got applied by a single thread in a defined order, but in reality can be parallelized. Redis cannot do that intelligently. I use Redis occasionally because it's a very simple system and it's easy, and has been micro-optimized very well. I don't think it necessarily contains any good computer science ideas. Am I wrong here? Tidis is more evidence that Redis's single threaded design leaves a ton on the table.
- meowface 9y ago
  Many in-memory databases don't have native data types for sets, sorted sets, hash tables, bitmaps, HyperLogLogs. Postgres and Redis have different use cases.
  - etaioinshrdlu 9y ago
    A unique index in Postgres is an efficient set data structure. A sorted set is a index in Postgres, stored as a B-Tree. Postgres has hash indexes, basically the same as a hash table. And you can run Postgres in memory if you want via tmpfs. Bitmaps and hyperloglogs are basically math functions running in the database, and Postgres supports user-defined math functions. It seems disingenuous to point out individual math functions when both databases support arbitrary user defined math functions.
    - matthewmacleod 9y ago
      You are conflating structures used internally to implement a relational database, and structures exposed as a first-class building block for your own systems. Think of Redis a a shared, in-memory, well-tested implementation of basic data structure primitives that takes care of a lot of the fiddly nonsense around them without you having to do the work. Postgres is awesome, but it's targeted at a different use case and has different characteristics; it's overkill for something like a shared key-value store, for example.
deleted 9y ago
[deleted]
littlestymaar 9y ago
I think that's a good example of the different use-cases where Go and Rust excel at: The front-end (Tidis), which offers a lot of feature but little complexity, is written in Go, and the back-end(TiKV), where complex stuff is done and performance must be immaculate, is in Rust.
- ddorian43 9y ago
  Pretty soon you'll have to write the whole thing in rust/c++ depending on how much efficiency you really need. At least move many things in coprocessors in tikv layer.

65 points by jinqueeny 9y ago

>This repo is WIP now and has lots of work to do, and for test only. Cool project though.

I am actually surprised that given today's wide availability of cloud service, new low-level storage softwares are still getting invented. Questions to fellow HNers: - If you need to deploy such software for your business, was it because optimizing GCP/AWS is not cost-effective or the same level of performance is not possible on public cloud?

tyingq 9y ago
Sometimes, for various reasons, on-prem is the only choice. Politics, bandwidth limitations, expediency, egress charges, heavy-handed IT security, etc. Similar for EC2 clones.
- toomuchtodo 9y ago
  There are compliance and business continuity requirements as well. We don’t allow business apps on cloud providers unless the business unit can prove they can pull it back on prem within a certain number of days.
gregwebs 9y ago
Your view point of just using something already hosted by GCP/AWS is probably the default today for good reason. I would break down the reason to not do that to three reasons: cost, performance, and customization. Lets ignore cost (which is complex) and focus on the TiDB and AWS example. TiDB was build to operate at a scale for OLTP (scale writes horizontally just by adding more nodes) that doesn't exist on Amazon's MySQL. You would have to you DynamoDB and downgrade from SQL transactions to eventual consistency. In terms of customization, TiDB is offering something AWS does not: the ability to store ever more of your data in one distributed data store (TiKV) and to leverage that in different ways (query it with TiSpark, and perhaps more projects like this in the future).
- elvinyung 9y ago
  Correction: AWS does have this kind of strongly consistent, scalable OLTP database in Aurora.
  - ddorian43 9y ago
    Correction: No. Aurora is single node. No horizontal scaling.
    - iampims 9y ago
      Not exactly. You can add as many replicas as you need to scale reads.
    - wgjordan 9y ago
      Aurora currently supports automatic horizontal scaling for reads [1], and horizontal scaling for writes is currently in preview [2]. [1]: https://aws.amazon.com/about-aws/whats-new/2017/11/amazon-aurora-now-supports-auto-scaling-for-aurora-replicas/ https://aws.amazon.com/about-aws/whats-new/2017/11/amazon-au... [2]: https://aws.amazon.com/about-aws/whats-new/2017/11/sign-up-for-the-preview-of-amazon-aurora-multi-master/ https://aws.amazon.com/about-aws/whats-new/2017/11/sign-up-f...
      - antoncohen 9y ago
        > horizontal scaling for writes is currently in preview I don't expect multi-master to increase write performance much. It will scale out network connections and maybe query processing, but not actual writing of data. It will replicate all data to all nodes, and the replication will be synchronous to any node it can automatically fail over to. So a client will be able to connect to any master, but that master will have to synchronously write to at minimum all other masters, if not all other nodes, before the server can acknowledge the write back to the client. Every node will need the write performance to write every write. That isn't scaling out writes.
        9y ago
alexnewman 9y ago
I’ve used all of the amazon datastores and none of them write as quick as a data store designed like this or maybe something like Hbase or Cassandra depending on what you are doing. Something like this is even better. I think amazon does Hbase as a service btw.Those use cases are rare so Aurora is usually the right answer. Alternatively A hundred s3 buckets can handle a large amount of writes as well for large or medium sized writes but it’s annoying to manage.
cortesoft 9y ago
Well, my business is working for a cloud services provider, so I can’t exactly rely on cloud services.
manigandham 9y ago
Complex services are hard to manage, even at scale with billions of investment. We've had plenty of instances where managed services had issues but VMs (being relatively simple tech) were just fine and we would've been better off running it ourselves. Also there's the cost ratio where high performance becomes exponential more expensive for many services. Perhaps the biggest reason though is that software is easier than ever to operate now with the rise of containers and Kubernetes. In fact, you can use Kubernetes as the single managed service to get that taken care of by the cloud, and then do the rest yourself for a great balance of cost, performance, flexibility and reliability.

maxpert 9y ago

Benchmarks please?

imauld 9y ago

Are there docs or examples of how to use this somewhere? I love Redis and I love using Go so I would be interested in trying this out for hobby projects and it would be helpful to have some documentation/examples.

ceohockey60 9y ago
Not directly related to tidis, but I did read about another Redis-on-top-of-Tikv use case by Ele.me, which is apparently this big food delivery platform like Doordash that's super popular in China. Here's the post: https://zhuanlan.zhihu.com/p/35226447 https://zhuanlan.zhihu.com/p/35226447 (too bad it's in Chinese right now. Hopefully someone translates it into English soon...)
- c4pt0r 9y ago
  I think the team is working on it right now.

etaioinshrdlu 9y ago

Redis has been called a "data structure server". Isn't that what a database is? Is there anything fundamentally better about Redis than say, a single connection to Postgres? Except with Postgres you would be allowed many concurrent connections and greater throughput, while having the same external behavior thanks to transaction serializability. It's as if the operations got applied by a single thread in a defined order, but in reality can be parallelized. Redis cannot do that intelligently. I use Redis occasionally because it's a very simple system and it's easy, and has been micro-optimized very well. I don't think it necessarily contains any good computer science ideas. Am I wrong here? Tidis is more evidence that Redis's single threaded design leaves a ton on the table.

meowface 9y ago
Many in-memory databases don't have native data types for sets, sorted sets, hash tables, bitmaps, HyperLogLogs. Postgres and Redis have different use cases.
- etaioinshrdlu 9y ago
  A unique index in Postgres is an efficient set data structure. A sorted set is a index in Postgres, stored as a B-Tree. Postgres has hash indexes, basically the same as a hash table. And you can run Postgres in memory if you want via tmpfs. Bitmaps and hyperloglogs are basically math functions running in the database, and Postgres supports user-defined math functions. It seems disingenuous to point out individual math functions when both databases support arbitrary user defined math functions.
  - matthewmacleod 9y ago
    You are conflating structures used internally to implement a relational database, and structures exposed as a first-class building block for your own systems. Think of Redis a a shared, in-memory, well-tested implementation of basic data structure primitives that takes care of a lot of the fiddly nonsense around them without you having to do the work. Postgres is awesome, but it's targeted at a different use case and has different characteristics; it's overkill for something like a shared key-value store, for example.

deleted 9y ago

[deleted]

littlestymaar 9y ago

I think that's a good example of the different use-cases where Go and Rust excel at: The front-end (Tidis), which offers a lot of feature but little complexity, is written in Go, and the back-end(TiKV), where complex stuff is done and performance must be immaculate, is in Rust.

ddorian43 9y ago
Pretty soon you'll have to write the whole thing in rust/c++ depending on how much efficiency you really need. At least move many things in coprocessors in tikv layer.