4 ms·
Writing a MySQL storage engine from scratch (2016)
- mappu 9y agoGreat article, makes it very clear what parts of the database are- and are not- provided by the storage engine. I wrote a C wrapper for BoltDB recently, i wonder how MySQL would perform linked to that? I have some memories of looking at VB6 code on codeproject.com over a decade ago, interesting to see that the site is still around and useful.
- anton_gogolev 9y agoOh, and it becomes painfully obvious why MySQL cannot use more than one index while executing the query.
- sfilargi 9y agoone index per table (Sorry for being pedantic)
- hoschicz 9y agoCan PostgreSQL use more than one index per table while executing a query?
- anton_gogolev 9y agoAbsolutely: https://www.postgresql.org/docs/8.3/static/indexes-bitmap-scans.html https://www.postgresql.org/docs/8.3/static/indexes-bitmap-sc...
- deleted 9y ago[deleted]
- morgo 9y agoIt has been able to use more than one index per table since MySQL 5.0 (2005): https://dev.mysql.com/doc/refman/5.7/en/index-merge-optimization.html https://dev.mysql.com/doc/refman/5.7/en/index-merge-optimiza...
- willvarfar 9y agoSomewhat related, I recently started trying to reverse-engineer the MySQL wire protocol. My idea is that if this is better documented then it will be easier for people to make databases that interoperate with MySQL clients but aren't MySQL that much easier (personally I want a custom Shard-Query-ish thing). This is obstinately documented (http://dev.mysql.com/doc/dev/mysql-server/latest/PAGE_PROTOCOL.html http://dev.mysql.com/doc/dev/mysql-server/latest/PAGE_PROTOC...), but I can't find a nice cheat-sheet saying "to send a query, you send this packet and you receive these packets" etc. I've taken to looking at e.g. the pure-Python PyMySQL package source-code https://github.com/PyMySQL/PyMySQL/blob/master/pymysql/connections.py#L226 https://github.com/PyMySQL/PyMySQL/blob/master/pymysql/conne... Here's my progress so far: https://gist.github.com/williame/ebf003dd85d5949f551a72331aefec11 https://gist.github.com/williame/ebf003dd85d5949f551a72331ae... Anyone know what commands you can send, and what the expected responses are?
- ccommsxx 9y agoTake a closer look at the MySQL documentation. You dont have to reverse engineer anything - it's all there and documented (source: wrote a client impl based on the docs) http://imysql.com/mysql-internal-manual/client-server-protocol.html http://imysql.com/mysql-internal-manual/client-server-protoc... http://imysql.com/mysql-internal-manual/connection-phase.html http://imysql.com/mysql-internal-manual/connection-phase.htm... http://imysql.com/mysql-internal-manual/com-query.html http://imysql.com/mysql-internal-manual/com-query.html etc
- willvarfar 9y agoCrikey, thx. There's a lot of dead links, e.g. http://imysql.com/mysql-internal-manual/com-query.html http://imysql.com/mysql-internal-manual/com-query.html doesn't describe the response? Got any idea what a query response consists of?
- ccommsxx 9y agoTurns out that all of this stuff has apparently been deleted from the official docs some time ago, so that's probably why you didn't find it :) (see sibling comment). When I did a quick google to pull up the docs that I knew I had read when I implemented the mysql client I found & linked you to a non-official, old copy of the mysql docs from back then. You can still find the docs for COM_QUERY result on the net though: https://github.com/cwarden/mysql-proxy/blob/master/doc/protocol.rst#com_query-response https://github.com/cwarden/mysql-proxy/blob/master/doc/proto...
- michaelmior 9y agoAm I the only one who likes the "C with classes" approach over C++?
- Chris2048 9y agoNever got into C++, so in a way - yes :-)
- pmelendez 9y agoDepends on what you mean by C++. C++11, C++14, C++17 have features that are must have to me.
- paulddraper 9y agoWell thank goodness we live in 2017.
- Toenex 9y agoI've heard this called imperative programming with classes.
- deleted 9y ago[deleted]
- tyingq 9y agoSqlite has a fairly straightforward way to implement your own storage engine as well, see the VFS documentation: https://sqlite.org/vfs.html https://sqlite.org/vfs.html Virtual tables are also interesting, but higher level, somewhat like a fuse filesystem. https://sqlite.org/vtab.html https://sqlite.org/vtab.html
- nullnilvoid 9y agoInteresting article. Not many people work on low-level storage engines nowadays.