3 ms·
You can use a custom memory allocator in python? I wonder if this is somehow pluggable, or if they had to modify the interpreter.
by captainmuon 13y ago
You can use a custom memory allocator in python? I wonder if this is somehow pluggable, or if they had to modify the interpreter.
- sesqu 13y agoAccording to the article, they do use a modified interpreter.
- zaphar 13y agoThe dropbox guys gave a talk about this at pycon one year but I'm having trouble finding it now. I remember thinking it involved less work than I thought it would.
- plodit 13y agoI think this is what you're looking for: http://blip.tv/pycon-us-videos-2009-2010-2011/pycon-2011-how-dropbox-did-it-and-how-python-helped-4896698 http://blip.tv/pycon-us-videos-2009-2010-2011/pycon-2011-how...
- Scaevolus 13y agoAll you need to do is set the appropriate type's tp_alloc/tp_dealloc function pointers [1] (type-specific malloc/free functions). Dropbox was having fragmentation issues from the large amount of garbage generated while scanning the filesystem, and making memory allocation use type-specific memory pools fixed it. [1] http://docs.python.org/2/c-api/typeobj.html#PyTypeObject.tp_alloc