3 ms·
If you have numeric data in e.g. NumPy arrays and you care about performance, numpy.savetxt() followed by text parsing in C++ will never match the speed of just
by jzwinck 12y ago
If you have numeric data in e.g. NumPy arrays and you care about performance, numpy.savetxt() followed by text parsing in C++ will never match the speed of just storing the binary data. For example, use numpy.save() then load the NumPy format in C++ (it's a pretty simple format), or use a more cross-language format like HDF5.
Even if you have received data from an outside source in text format, perhaps the first thing you should do is convert it to a binary format on disk; this will save you a lot more overhead than any amount of scanf() optimization will.
- cnvogel 12y agoCertainly it depends. One one side, if your input data is read only once or a few times, introducing any intermediate binary format will often not be worth it, and optimizing reading the ASCII is the way to go. On the other hand, if you have to crunch through huge amounts of locally stored data, try to make most of your binary file-format resemble the in-memory layout as closely as possible. That way you can, with some luck, mmap() your file and just use pointers into the bulk data. Saves at least one level of copying, from the filesystem cache to application heap. Be careful with validation of lengths and offsets, though!