5 ms·
"We are moving more and more of the core database in C/C++, while still using many of the concurrency and reliability design principles we've proven with the Er
by plinkplonk 15y ago
"We are moving more and more of the core database in C/C++, while still using many of the concurrency and reliability design principles we've proven with the Erlang codebase. And Erlang is still going to be part of the product as well, particularly with cluster management, but most of the performance sensitive portions will be moving to over C code. Erlang is still a great language, but when you need top performance and low level control, C is hard to beat."
This is interesting. If I remember correctly, CouchDB was first written in C++ and then moved to Erlang. Now the project has come full circle (which is fine of course).
- Joakal 15y agoOof, seems like a lot of inefficiency in retraining, redocumentation, converting, etc. Not to mention the debugging issues between two different technologies. Why would it be fine?
- reitzensteinm 15y agoI think plinkplonk was saying that it's fine (not necessarily bad) to come full circle in general, not that it will work out in this particular case.
- plinkplonk 15y ago"it's fine (not necessarily bad) to come full circle in general, not that it will work out in this particular case." exactly. I was not saying this step is specifically a bad (or good!) one. I didn't want the thread to devolve into a "which is better Erlang or C++?" flame war. I am sure the devs on the project have good reasons for the change. I just thought it was interesting that the project came full circle.
- damienkatz 15y agoYes. Erlang is great for proving ideas and making something work reliably. I have mad respect for Erlang, it's the kind of language where one guy can code a whole a distributed database. But when you need high performance and low level control, and you have the resources and team to make it happen, you really can't beat C. I'll write more about that soon.
- plinkplonk 15y agoErlang for abstraction and C for performance sounds cool. I'd be interested in how much (or whether) you are using C++ (vs C). I hope you'll address that in your write up.
- dlsspy 15y agoep-engine (the core of membase and what handles all of the data moving around between networks, memory and disks) is almost entirely C++.
- sriram_sun 15y agoAre you using any C++ libraries for the networking stuff or plain OS specific socket APIs?
- dlsspy 15y agoThe networking stuff is the memcached engine branch. It's pure C and loads ep-engine as a module for doing all the backend junk.
- mjs 15y agoI wonder if Riak will ever get to this point.
- seancribbs 15y agoTo what point? Already large portions of Riak (the performance-critical ones) are in C, and have been for a long time.
- mjs 15y agoOh, I didn't realise much of Riak was in C, and the website now mentions Erlang a whole lot less than I remember. Still, in the technology stack http://basho.com/technology/technology-stack/ http://basho.com/technology/technology-stack/ Riak Core seems to be all Erlang. Bitcask is possibly C--is that what you're referring to?
- MatthewPhillips 15y agoYou usually get things correct the second time you write them. You are missing critical knowledge on the first-write.
- TheRevoltingX 15y agoI don't think it's coming full circle. They're just properly separating the project code, so that the time critical code is in C/C++ while the scale/fault-tolerant code will be in Erlang.
- nivertech 15y agoErlang or C++ is not that important. The real drug on performance is using javascript for queries & M/R and HTTP-based APIs vs. binary protocols. Moving to UnQL and Memcached protocol may solve some of the performance problems. There is a reason antirez choose Lua and not JS for a scripting language.
- dlsspy 15y agoI agree with much of what you've said there. You clearly understand enough that I don't need to go into detail describing why I agree. :) But this isn't quite right: > There is a reason antirez choose Lua and not JS for a scripting language. To be fair, I embedded lua in ep-engine well over a year ago, but haven't ever released it. As it turns out, v8 is performing very well lately (certainly faster than plain lua) and people get it more easily. I've done some fun things with lua and it's really fun and easy to embed, but whatever gets shipped has to be supported and lua has some pretty dark areas.