3 ms·

So, the case is that the formatting string fits into memory, the data being formatted fits in memory but the output formatted text does not? This seems like an

So, the case is that the formatting string fits into memory, the data being formatted fits in memory but the output formatted text does not? This seems like an extremely rare case, which is itself a subset of the very rare case that you are formatting something that can be arbitrarily large. Oh, and the programmer must somehow have overlook this and not written it formatted output to output in reasonably-sized chunks. Meanwhile, having back-and-forth between the formatter and I/O means the formatter must correctly handles all the I/O problems and corner cases: disk full, disconnect, transient failures, ... not even going into that for some of them, you might want the caller to decide the outcome, which is easy when you split formatting and I/O, but hard when the formatting performs the I/O. I think this is a case that it may sounds superior to mix both, but it is actually buying next to nothing but both complicates matters and probably hides or ignores real issues.

ddevault 4y ago
> So, the case is that the formatting string fits into memory, the data being formatted fits in memory but the output formatted text does not? It takes a minimum of twice as much memory to store the original data and its formatted stream. Some systems don't have much memory, and some systems work with large amounts of data. Some systems work with more data than fits in RAM, or more data than even fits on disk, or even infinite streams of data, from sensors or generated data. All of these use-cases must be accommodated by Hare. That said, you can easily format into a dynamically allocated string in Hare if you so desire: let s = fmt::asprintf("Hello, {}!", user); defer free(s); This has the obvious downside of requiring an additional memory allocation, which can also fail just as easily as I/O can, and must be freed after use. However, this function does not require I/O, so all of the I/O failure cases are ruled out. There are trade-offs between these two approaches. Hare's design is meant to let the user evaluate these trade-offs for their use-case and make an explicit, planned decision regarding their needs and potential failure modes.
- kragen 4y ago
  It might be worth putting a short note on this into the library documentation so people understand the library's design objectives. Among other things it will help them find other things in the library. Incidentally this is one of the things that the limited form of laziness found in Python generators could help with, if Hare had it; you can think of an input stream as a lazy string. Unlike full laziness, generators have very predictable memory usage.

ddevault 4y ago

> So, the case is that the formatting string fits into memory, the data being formatted fits in memory but the output formatted text does not? It takes a minimum of twice as much memory to store the original data and its formatted stream. Some systems don't have much memory, and some systems work with large amounts of data. Some systems work with more data than fits in RAM, or more data than even fits on disk, or even infinite streams of data, from sensors or generated data. All of these use-cases must be accommodated by Hare. That said, you can easily format into a dynamically allocated string in Hare if you so desire: let s = fmt::asprintf("Hello, {}!", user); defer free(s); This has the obvious downside of requiring an additional memory allocation, which can also fail just as easily as I/O can, and must be freed after use. However, this function does not require I/O, so all of the I/O failure cases are ruled out. There are trade-offs between these two approaches. Hare's design is meant to let the user evaluate these trade-offs for their use-case and make an explicit, planned decision regarding their needs and potential failure modes.

kragen 4y ago
It might be worth putting a short note on this into the library documentation so people understand the library's design objectives. Among other things it will help them find other things in the library. Incidentally this is one of the things that the limited form of laziness found in Python generators could help with, if Hare had it; you can think of an input stream as a lazy string. Unlike full laziness, generators have very predictable memory usage.