3 ms·
+1, thanks for this and your previous comment. Have you written up your approach in any more detail? I'd love to reference it.
by chrisweekly 6y ago
+1, thanks for this and your previous comment. Have you written up your approach in any more detail? I'd love to reference it.
- derekp7 6y agoThe code is on github.com/derekp7/snebu if you want to take a look. The next task on my list is to write up developer / contributor documentation on how the whole thing is put together along with various tricks (and simplified demonstration code). But specifically, look in "snebu-main.c" that is where the opendb function is (so you can see the pragma statements), and there is a busy_retry function that gets referenced (all it does is sleep for .1 seconds). I believe that you don't need the busy-retry function, if you use the built-in busy handler, but I'm not really sure and don't want to take a chance and break working code. For the temp tables, look in snebu-submitfiles.c -- the function at the top handles the DB operations, one towards the bottom handles the non-DB tar file consumption operations, and there is a circular queue in the middle to handle buffering so the data ingestion can keep going while the data is getting flushed (these three run as separate processes). I should learn threads, as there may be more flexibility in that, but not comfortable enough with thread programming yet.
- chrisweekly 6y agoThanks so much! The phrase "unreasonably helpful" comes to mind. Also, kudos for the extraordinarily high-quality documentation in snebu!