5 ms·
> I like to have my lexers operate on `FILE*`, rather than string-views. [...] it does mean I can operate on streams. While I understand the desire to support
by teo_zero 1y ago
> I like to have my lexers operate on `FILE*`, rather than string-views. [...] it does mean I can operate on streams.
While I understand the desire to support one input interface for composability, reuse, etc. I can't help wondering why 'FILE*'. Isn't reading from a string more "universal"?
> If the user has a c-string, the string can be easily wrapped by `funopen()` or `fopencookie()` to provide a `FILE*` adapter layer.
And if the user has a file, it's easy to read it into memory in advance.
What's the benefit of FILE* over a string?
- trealira 1y agoPerhaps it's that you never have to read the whole file into memory at once if it's with a `FILE *` rather than a string. I'm not that person, this is just my assumption.
- tlb 1y agoThere was a time when a file of source code might not fit in memory, or would take up a significant fraction of it. But it hasn't been the case on any developer machine in 20+ years. And the overhead of FILE * accessors like fgetc is substantial. Strings in memory are always going to be faster.
- viega 1y agoWell, the overhead of the stream API is in the noise. If the lexer / parser do not support incremental parsing, it doesn't really matter. But incremental parsing can be important in some situations. For instance, if you're parsing a 1GB json blob keeping the whole thing in memory at once can easily be an issue. Plus, if you stall waiting for the entire input string, you end up adding to latency, if that matters.
- cyber_kinetist 1y agoYou can just use virtual memory (mmap / VirtualAlloc) to map an address region with a file and get the same effect while just using char* pointers.