3 ms·

ZomboDB developer here. I don't frequent HN so please forgive any missteps... Just wanted to say that while the code has proven itself production worthy, I'm

by zombodb 11y ago

ZomboDB developer here. I don't frequent HN so please forgive any missteps... Just wanted to say that while the code has proven itself production worthy, I'm still re-writing documentation and the test suite and that stuff is going to take awhile. What ZDB does, how it performs, and the ways it can be (creatively) used aren't reflected in the current docs. Neither are the caveats, edge cases, and known-bugs. If this thread shows any interest I'll do my best to field questions and generally discuss how it's actually fun to work in a world with MVCC-correct Elasticsearch indexes.

dordoka 11y ago
Thanks a lot for open sourcing this!
- zombodb 11y ago
  You're welcome. It's hard not to when you've spent so much of your career using quality products like Postgres.
rattray 11y ago
I love that you publicly acknowledge what is and isn't done. The documentation topics you line out in this comment are super-important; it was great to see that you already include a list of "non-features". Would also be great to see the "production credentials" - where has it been used, under what load/conditions, how much data, what kinds of queries, etc.
- zombodb 11y ago
  I'm just some guy on the Internet that is fortunate enough to be able bring a thing he's spent the past two years working on behind closed doors to the public. I haven't participated in open-source since 1999 (no, really) so I'm out of the loop with how things work in 2015. But if I'm going to continue to shepherd this beast forward, there's no point in hiding its flaws. Besides, how will I know to fix them if I don't document them? Along those lines, I'm sure it's clear the docs are still a WIP, and the more I've written the more I realize needs to be written. I suspect the actual code won't change one bit (ha!) over the next few weeks. Regarding "production credentials", the README mentions the company where this started, and I've alluded to some large (to me) round numbers. Those will have to do for now. :) This sort of scenario (a PG index based on ES) isn't what I'd want to use for the scale of something like Netflix (for example) where you've got billions of rows and tens-of-thousands of queries a second. But at the same time, that level of scale only happens at the top. And it's lonely up there. There's a lot more room down here on the ground. ---- While I'm sitting here at 4:30am waiting for HN to stop telling me I'm posting too quickly (wth, I'd like to get some sleep eventually!), here's another thought... Data is hard. I've spent my entire professional career dealing with data. Trying to bridge the gap between two distinct databases has proven really challenging (and fun and rewarding) but there's quite a bit of work left to do, and there's thousands of programmers out there that are waay smarter than I (starting with the entire crew of postgresql-hackers), so I feel like if I can at least list the things I know I don't know, someone else may come along and have an answer.
rattray 11y ago
You mentioned elsewhere this is going to be your fulltime occupation. May I ask what the business model is, and the progress to-date?

by zombodb 11y ago

dordoka 11y ago

Thanks a lot for open sourcing this!

zombodb 11y ago
You're welcome. It's hard not to when you've spent so much of your career using quality products like Postgres.

rattray 11y ago

I love that you publicly acknowledge what is and isn't done. The documentation topics you line out in this comment are super-important; it was great to see that you already include a list of "non-features". Would also be great to see the "production credentials" - where has it been used, under what load/conditions, how much data, what kinds of queries, etc.

zombodb 11y ago
I'm just some guy on the Internet that is fortunate enough to be able bring a thing he's spent the past two years working on behind closed doors to the public. I haven't participated in open-source since 1999 (no, really) so I'm out of the loop with how things work in 2015. But if I'm going to continue to shepherd this beast forward, there's no point in hiding its flaws. Besides, how will I know to fix them if I don't document them? Along those lines, I'm sure it's clear the docs are still a WIP, and the more I've written the more I realize needs to be written. I suspect the actual code won't change one bit (ha!) over the next few weeks. Regarding "production credentials", the README mentions the company where this started, and I've alluded to some large (to me) round numbers. Those will have to do for now. :) This sort of scenario (a PG index based on ES) isn't what I'd want to use for the scale of something like Netflix (for example) where you've got billions of rows and tens-of-thousands of queries a second. But at the same time, that level of scale only happens at the top. And it's lonely up there. There's a lot more room down here on the ground. ---- While I'm sitting here at 4:30am waiting for HN to stop telling me I'm posting too quickly (wth, I'd like to get some sleep eventually!), here's another thought... Data is hard. I've spent my entire professional career dealing with data. Trying to bridge the gap between two distinct databases has proven really challenging (and fun and rewarding) but there's quite a bit of work left to do, and there's thousands of programmers out there that are waay smarter than I (starting with the entire crew of postgresql-hackers), so I feel like if I can at least list the things I know I don't know, someone else may come along and have an answer.

rattray 11y ago

You mentioned elsewhere this is going to be your fulltime occupation. May I ask what the business model is, and the progress to-date?