3 ms·
It’s still magic, just implemented with a shared crystal ball. If I learned about fsspec at 2AM during a prod outage via reading tracebacks instead of reading c
by kstrauser 3y ago
It’s still magic, just implemented with a shared crystal ball. If I learned about fsspec at 2AM during a prod outage via reading tracebacks instead of reading code, I’d be a font of creative profanity.
The idea is nice! I’m not saying fsspec is bad. It’s not. It’s neat! I just strongly abhor the idea that pip installing a package changes runtime behavior whether that packages is ever even imported.
- philomath_mn 3y agoYeah, I'd prefer the pandas call look like ``` from csvbase import CSVBASE_FS pd.read_csv('//calpaterson/onion-vox-pops', fsspec=CSVBASE_FS) ```
- kstrauser 3y agoI vastly, hugely, enormously prefer that pattern. It’s not as pretty as the blog post, but it’s so explicit and easy to understand at a glance. If I’d never heard of csvbase or fsspec before seeing that code, I’d still know what to look for and how to debug it. At 2PM when I’m well rested, caffeinated, and alert, the original code is clever and lovely. At 2AM, I want this kind of easy to understand explicitness.
- philomath_mn 3y agoLooks like explicit fsspec usage is possible: ``` fs = fsspec.implementations.local.LocalFileSystem() with fs.open('test.csv', mode='r') as fp: print(pd.read_csv(StringIO(fp.read()))) ```