7 ms·

Mixer – A MySQL Proxy powered by Go

42 points by siddontang 12y ago

swah 12y ago
I think you have to improve the description: "aims to supply a simple solution for using MySQL." doesn't really make sense, because I can just use MySQL directly. Is this some kind of connection pool / load balancer?
- fideloper 12y ago
  It sounds like it might help make otherwise complicated things (sharding) easier by pushing the logic of how to shard into the config. If I'm reading that correctly... the description could use clarification on the benefits. I'd use HAProxy I only wanted to proxy/balance read queries.
- siddontang 12y ago
  Sorry for my bad expression, Mix aims to solve some simple MySQL sharding problems. I should improve the document first, maybe:)
withjive 12y ago
I wonder how this compares to Vitess, which youtube has been using to scale their MySql databases for years. Which is also written in Go. Perhaps this is a direct competitor to Vitess, but I agree the description is in dire need of some specifics. Edit: adding link https://github.com/youtube/vitess https://github.com/youtube/vitess
- siddontang 12y ago
  Vitess is too huge for me, I can't fully understand it. I don't think mixer can be the direct competitor to Vitess.I develop it just for fun and only want to solve some simple MySQL sharding problems. Btw, Vitess is awesome and I use some codes from it.
davidroetzel 12y ago
This sounds a lot like mysql proxy (http://dev.mysql.com/doc/refman/5.7/en/mysql-proxy.html http://dev.mysql.com/doc/refman/5.7/en/mysql-proxy.html), which afaik is not recommended for production use. I would very much like to know what the differences are (apart from the choice of programming language) and how stable this is.
- siddontang 12y ago
  Mixer is more like Atlas(https://github.com/Qihoo360/Atlas https://github.com/Qihoo360/Atlas) or Cobar(https://github.com/alibaba/cobar https://github.com/alibaba/cobar). You can define some rules to route sql to different MySQL servers, maybe it is the main difference. Mixer can not be used in production now, I have been improving it and try to use it in my company first.
  - litewulf 12y ago
    These seem pretty cool, do you know if there is any English documentation for them?
    - siddontang 12y ago
      Sorry, only Chinese document. The authors may only focus Chinese MySQL users. Btw, writing english document is very hard for many Chinese programmers, the same to me:)
    - flike 12y ago
      I'm the developer of Atlas(https://github.com/Qihoo360/Atlas https://github.com/Qihoo360/Atlas).In current Atlas has used in 30+ companies,If you are interested in Atlas,you can contact me with email(flikecn@126.com),any question are welcom.:)
morgo 12y ago
There's lots of development in this space all of the sudden. From Oracle there is also MySQL Fabric. Fabric also provides sharding + failover, but does so by using modified MySQL connectors rather than a proxy.
- siddontang 12y ago
  There is no best solution for MySQL sharding at all, so many people develop the proxy for their real scenario. Maybe MySQL Fabric is a silver bullet, but I always think that sometimes proxy is enough.
webmonkeyuk 12y ago
Looks like it has potential and is exciting until you read the "Limitations" section
- siddontang 12y ago
  Well, I put many limitations now only to get proxy working right first. I need to improve it step by step. I am very glad that if anyone can help me. Many things to do but I have not enough time.
- cwyers 12y ago
  Yeah, wow. "Join not supported Subselects not supported 'group by' not supported" At that point you might as well be using MongoDB or what have you.
  - siddontang 12y ago
    Well, support cross sharding `join`, `subslect` and `group by` is not easy, I see Vitess does not fully support them now. I am the only guy to develop Mixer just for fun with little time, and have to improve it step by step. Any help is welcome. Btw, I have not used MongoDB, I will try to see it later.
fooyc 12y ago
What I would really love to see in a solution like this is the ability to automatically rebalance shards (like moving data between shards when adding new instances). Sharding can be achieved quite easily on the application level, or by using one of the few available proxies. On the other hand, re-balancing is not easily done yet.
- siddontang 12y ago
  Now I only see Vitess can support resharding with some restrictions, maybe I am ignorant. I don't think handing sharding on the application level is easy, programmers may care too much about MySQL sharding and can not write sql easily, at least in my company. I have only used Atlas (https://github.com/Qihoo360/Atlas https://github.com/Qihoo360/Atlas) and cobar (https://github.com/alibaba/cobar https://github.com/alibaba/cobar), but they are not suitable for my company, so I want to develop a simple one using Go. Auto resharding is very awesome but I don't have the power to do that now. Any help is very welcome.

42 points by siddontang 12y ago

swah 12y ago

I think you have to improve the description: "aims to supply a simple solution for using MySQL." doesn't really make sense, because I can just use MySQL directly. Is this some kind of connection pool / load balancer?

fideloper 12y ago
It sounds like it might help make otherwise complicated things (sharding) easier by pushing the logic of how to shard into the config. If I'm reading that correctly... the description could use clarification on the benefits. I'd use HAProxy I only wanted to proxy/balance read queries.
siddontang 12y ago
Sorry for my bad expression, Mix aims to solve some simple MySQL sharding problems. I should improve the document first, maybe:)

withjive 12y ago

I wonder how this compares to Vitess, which youtube has been using to scale their MySql databases for years. Which is also written in Go. Perhaps this is a direct competitor to Vitess, but I agree the description is in dire need of some specifics. Edit: adding link https://github.com/youtube/vitess https://github.com/youtube/vitess

siddontang 12y ago
Vitess is too huge for me, I can't fully understand it. I don't think mixer can be the direct competitor to Vitess.I develop it just for fun and only want to solve some simple MySQL sharding problems. Btw, Vitess is awesome and I use some codes from it.

davidroetzel 12y ago

This sounds a lot like mysql proxy (http://dev.mysql.com/doc/refman/5.7/en/mysql-proxy.html http://dev.mysql.com/doc/refman/5.7/en/mysql-proxy.html), which afaik is not recommended for production use. I would very much like to know what the differences are (apart from the choice of programming language) and how stable this is.

siddontang 12y ago
Mixer is more like Atlas(https://github.com/Qihoo360/Atlas https://github.com/Qihoo360/Atlas) or Cobar(https://github.com/alibaba/cobar https://github.com/alibaba/cobar). You can define some rules to route sql to different MySQL servers, maybe it is the main difference. Mixer can not be used in production now, I have been improving it and try to use it in my company first.
- litewulf 12y ago
  These seem pretty cool, do you know if there is any English documentation for them?
  - siddontang 12y ago
    Sorry, only Chinese document. The authors may only focus Chinese MySQL users. Btw, writing english document is very hard for many Chinese programmers, the same to me:)
  - flike 12y ago
    I'm the developer of Atlas(https://github.com/Qihoo360/Atlas https://github.com/Qihoo360/Atlas).In current Atlas has used in 30+ companies,If you are interested in Atlas,you can contact me with email(flikecn@126.com),any question are welcom.:)

morgo 12y ago

There's lots of development in this space all of the sudden. From Oracle there is also MySQL Fabric. Fabric also provides sharding + failover, but does so by using modified MySQL connectors rather than a proxy.

siddontang 12y ago
There is no best solution for MySQL sharding at all, so many people develop the proxy for their real scenario. Maybe MySQL Fabric is a silver bullet, but I always think that sometimes proxy is enough.

webmonkeyuk 12y ago

Looks like it has potential and is exciting until you read the "Limitations" section

siddontang 12y ago
Well, I put many limitations now only to get proxy working right first. I need to improve it step by step. I am very glad that if anyone can help me. Many things to do but I have not enough time.
cwyers 12y ago
Yeah, wow. "Join not supported Subselects not supported 'group by' not supported" At that point you might as well be using MongoDB or what have you.
- siddontang 12y ago
  Well, support cross sharding `join`, `subslect` and `group by` is not easy, I see Vitess does not fully support them now. I am the only guy to develop Mixer just for fun with little time, and have to improve it step by step. Any help is welcome. Btw, I have not used MongoDB, I will try to see it later.

fooyc 12y ago

What I would really love to see in a solution like this is the ability to automatically rebalance shards (like moving data between shards when adding new instances). Sharding can be achieved quite easily on the application level, or by using one of the few available proxies. On the other hand, re-balancing is not easily done yet.

siddontang 12y ago
Now I only see Vitess can support resharding with some restrictions, maybe I am ignorant. I don't think handing sharding on the application level is easy, programmers may care too much about MySQL sharding and can not write sql easily, at least in my company. I have only used Atlas (https://github.com/Qihoo360/Atlas https://github.com/Qihoo360/Atlas) and cobar (https://github.com/alibaba/cobar https://github.com/alibaba/cobar), but they are not suitable for my company, so I want to develop a simple one using Go. Auto resharding is very awesome but I don't have the power to do that now. Any help is very welcome.