10 ms·
Same here. I have however seen a few out of memory cases in the past when given large input files.
by pletnes 8mo ago
Same here. I have however seen a few out of memory cases in the past when given large input files.
- maxldn 8mo agoReally? How large? I’ve only managed to crash it with hundreds/thousands of files so far, but haven’t so many huge files to deal with.
- skeeter2020 8mo agoit's not the focus or very performant but you can have it spill to disk if you run out of memory. I wouldn't suggest building a solution based on this approach though; the sweet-spot is memory-constrained.
- jastr 8mo agoBy default, it tries to take 80% of your memory. I've found that you need to set it to something much smaller in ~/.duckdbrc `set max_memory='1GB';`