4 ms·
Heh, yeah.ZODB seems to be pretty good for the purpose here (it's performance is usually not the bottleneck). But when indexing large texts, it becomes a probl
by michwill 11y ago
Heh, yeah.ZODB seems to be pretty good for the purpose here (it's performance is usually not the bottleneck).
But when indexing large texts, it becomes a problem. One way to solve it is instead of using generic serializing for data structures, take real structure into account. Many isinstance() calls cause the CPU bottleneck [http://www.upfrontsystems.co.za/Members/roche/where-im-calling-from/zodb-benchmarks http://www.upfrontsystems.co.za/Members/roche/where-im-calli...]. Though, it turns out to be not as bad [http://www.upfrontsystems.co.za/Members/roche/where-im-calling-from/zodb-benchmarks-revisited/ http://www.upfrontsystems.co.za/Members/roche/where-im-calli...]
- michwill 11y agoIn fact, soon we want to switch away from Python pickles (which also could have security problems) into serializing to msgpack. And that makes everything much better in terms of cross-language compatibility. So, things people dreamed about inside ZODB, pretty much :-)