5 ms·

The code quality seems questionable. A few examples: Repository claims modern code, but uses Python 2 (end of life January 1, 2020) https://github.com/Oneliner

The code quality seems questionable. A few examples: Repository claims modern code, but uses Python 2 (end of life January 1, 2020) https://github.com/Onelinerhub/onelinerhub/blob/main/python/display_all_the_users.md https://github.com/Onelinerhub/onelinerhub/blob/main/python/... Copied from https://wiki.python.org/moin/Powerful%20Python%20One-Liners https://wiki.python.org/moin/Powerful%20Python%20One-Liners Incorrect indentation causing unnecessarily large memory usage https://github.com/Onelinerhub/onelinerhub/blob/main/python/merge_txt_files.md https://github.com/Onelinerhub/onelinerhub/blob/main/python/... Undefined behavior for i = 31 https://github.com/Onelinerhub/onelinerhub/blob/main/c/bits_read_change.md https://github.com/Onelinerhub/onelinerhub/blob/main/c/bits_... Printing of uninitialized memory https://github.com/Onelinerhub/onelinerhub/blob/main/c/set_value_to_array.md https://github.com/Onelinerhub/onelinerhub/blob/main/c/set_v... Undefined behavior https://github.com/Onelinerhub/onelinerhub/blob/main/c/swap_two_variables_values.md https://github.com/Onelinerhub/onelinerhub/blob/main/c/swap_...

Beltiras 5y ago
Fixed some of those. Simple case of just pushing the edit button and sending a PR.
- johndough 5y ago
  The suggested solution https://github.com/Onelinerhub/onelinerhub/blob/34467e427cc679705b19ee0e2774502f3d789060/python/merge_txt_files.md https://github.com/Onelinerhub/onelinerhub/blob/34467e427cc6... still runs out of memory with many files. I believe the original intention behind using mode="a" was to append to the output file while reading the input files at the same time. This way, there is no need for an ever-growing string array. But there are still many other issues like using default platform string encoding instead of detecting it properly or at least using utf-8, checking for ".txt" anywhere in the path instead of at the end, and not closing the input files with a context manager like the output file, which suggests that this code is just pierced together from various sources. A robust solution would require many more lines.
  - mynameismon 5y ago
    Another memory issue (although unlikely) could also be the file itself being larger than the RAM size, which can be avoided by chunking the textfiles.
  - hermitdev 5y ago
    For bonus points: how about properly handling UTF-8 byte order marks (BOM), too? The way this code is written if the files contain a BOM, you're going to pass the BOM right into the destination file. Fine for the first file, not so fine for any other file. Dropping a BOM mark in the middle of a file isn't going to result in valid UTF-8...

Beltiras 5y ago

Fixed some of those. Simple case of just pushing the edit button and sending a PR.

johndough 5y ago
The suggested solution https://github.com/Onelinerhub/onelinerhub/blob/34467e427cc679705b19ee0e2774502f3d789060/python/merge_txt_files.md https://github.com/Onelinerhub/onelinerhub/blob/34467e427cc6... still runs out of memory with many files. I believe the original intention behind using mode="a" was to append to the output file while reading the input files at the same time. This way, there is no need for an ever-growing string array. But there are still many other issues like using default platform string encoding instead of detecting it properly or at least using utf-8, checking for ".txt" anywhere in the path instead of at the end, and not closing the input files with a context manager like the output file, which suggests that this code is just pierced together from various sources. A robust solution would require many more lines.
- mynameismon 5y ago
  Another memory issue (although unlikely) could also be the file itself being larger than the RAM size, which can be avoided by chunking the textfiles.
- hermitdev 5y ago
  For bonus points: how about properly handling UTF-8 byte order marks (BOM), too? The way this code is written if the files contain a BOM, you're going to pass the BOM right into the destination file. Fine for the first file, not so fine for any other file. Dropping a BOM mark in the middle of a file isn't going to result in valid UTF-8...