11 ms·
I/O library 6x faster than fmt, 10x faster than stdio.h and iostream
- apankrat 6y agoJust keep in mind that stdio is really slow, even its fast paths. For example, here's a trivial sscanf() rewrite for a log parsing case that achieves 300x speed-up - https://gist.github.com/apankrat/20776d68d1d97bca12576a6e204b2f74 https://gist.github.com/apankrat/20776d68d1d97bca12576a6e204... - and that's with a completely unoptimized code.
- AceJohnny2 6y agoso the follow-up question (not necessarily to you but to the crowd) is: why is stdio so slow?
- why_only_15 6y agoWonder why... Maybe because of compatibility concerns?
- kstenerud 6y agoBecause it's an abstraction on top of an abstraction on top of an abstraction. You need to leak parts of the lower layers to higher levels in order to get the speed back (and allow zero-copy). There are probably some non-leaky optimizations that can be done as well, although that's been going on for decades. Another biggie is floating point. Converting binary floating point to decimal floating point and vice versa is VERY VERY VERY SLOW, difficult to get right, and difficult to predict results. Unfortunately, ieee754 decimal float just isn't getting any adoption, so we're stuck doing these costly conversions every time we deal with text formats or big float implementations. Actually, scanf and printf are just in general slow because of their general nature. They need to handle a TON of options and get really bloated and complicated as a result.
- 77pt77 6y ago> Another biggie is floating point. Converting binary floating point to decimal floating point and vice versa is VERY VERY VERY SLOW, difficult to get right Never forget the cluster bug in PHP because the developers decided they needed to reinvent the wheel[1]. [1] https://www.exploringbinary.com/php-hangs-on-numeric-value-2-2250738585072011e-308/ https://www.exploringbinary.com/php-hangs-on-numeric-value-2...
- ars 6y agoCan you explain what you mean? Because I read the bug report and it seems to be some intricacy about loading variables into a register solved by using volatile. Meaning it's a complicated implementation detail of the CPU, and does not seem to be because they reinvented the wheel. Plus is there some standardized library they should have been using?
- hyperman1 6y agoJava had a variant of that bug too: https://www.exploringbinary.com/java-hangs-when-converting-2-2250738585072012e-308/ https://www.exploringbinary.com/java-hangs-when-converting-2... It sits in a loop flip-flopping between 2 representations, each time deciding the other one is better. This turned into a DOS, as some web servers used floating point when decoding time zones from the http request header. So if your server has e.g. 20 http acceptor threads, someone could send 20 requests with an insane time zone, each sending 1 thread in an infinite loop.
- deleted 6y ago[deleted]
- thethirdone 6y ago> Another biggie is floating point. Converting binary floating point to decimal floating point and vice versa is VERY VERY VERY SLOW, difficult to get right, and difficult to predict results. I haven't read deeply into IEEE 754-2008 for decimal floating points, but it seems like it should be pretty fast (relative to system calls) to convert binary to decimal because 10 has a factor of 2. > Unfortunately, ieee754 decimal float just isn't getting any adoption, so we're stuck doing these costly conversions every time we deal with text formats or big float implementations. Is there any reason big float implementations should use decimal rather than binary? It seems like it is very straightforward to make a binary big float, and do operations on it. In fact, IEEE 754-2008 specifies interchange formats for all binary floats of bit lengths >= 128 where length is a multiple of 32.
- saagarjha 6y agoThey’re exceptionally complicated. No, really, take a look at the source to glibc’s vfprintf-internal.c (http://sourceware.org/git/?p=glibc.git;a=blob;f=stdio-common/vfprintf-internal.c http://sourceware.org/git/?p=glibc.git;a=blob;f=stdio-common...) or vfcanf-internal.c (http://sourceware.org/git/?p=glibc.git;a=blob;f=stdio-common/vfscanf-internal.c http://sourceware.org/git/?p=glibc.git;a=blob;f=stdio-common...) sometime. all the * (scanf|printf) functions are funneled through that, which means right off the bat you’re converting your string to a FILE * abstraction and marshaling all your arguments through a va_args. Then you get to the actual function itself, which needs to deal with locales and buffering and locking and positional arguments and multiple bases and memory allocation and rounding floating point and a huge amount of complexity that you might not even be aware of. And then there’s the somewhat awkward signature itself, which means it is all but required to do extra work to conform to the standard.
- geocar 6y agoBecause it's doing something else. Saying stdio is slow isn't the same thing as saying "they should just do what these other guys are doing" -- they can't do that without breaking everyone who uses stdio, and a lot of people use stdio because being slow isn't a problem they need to solve. Meanwhile, when being slow is a problem you need to solve, just because there are undoubtedly some quick wins in just about every stdio implementation doesn't mean there would be much value chasing them. Let's say for every 20% improvement in stdio you can find there, you could get a 20x improvement in your application some other way. And there are a lot of other ways: More formatting and IO libraries than you can shake a stick at, so it's easy to think about the things you're paying for by using stdio that you don't need and simply find something else that doesn't have those things. Some examples: - Compatibility: stdio offers a lot of options for no better reason than it's always offered them. This creates a certain amount of bloat and slowdown that can't be removed without breaking some other application that expects this behaviour. - Edge cases: More on that point, stdio is expected to handle a lot of situations to handle the application might never get into, but they have to test for anyway. Other libraries don't. - Locking: FILE* needs to be locked for concurrent access to get things like the file position or to update the buffer. Many implementations have an _unlocked() implementation of stdio operations that skip the lock, but the only way to know if they're safe is to know there's no part of the application (or a library) that could be using that stream in another thread. It also nearly doubled the number of API calls stdio offers. - Tuneables with bad defaults: Most people don't ever use setvbuf because choosing a good value is hard. Pick a value too big, and you'll see massive delays with data coming out, and too small and you waste time on syscalls. If people do use this, they choose something like line buffering (usually for logfiles) which means you're doing a syscall per line. - Impedance mismatch: fwrite() might look* like write() but it doesn't look anything like writev() or sendmmsg(); Similarly fread() versus recvmmsg() or readv(). stdio would need a bunch of new API calls (like the _unlocked() calls) just to keep up. - Abstraction in implementation: stdio probably uses malloc which is also slow, but taking an allocator as an argument would require a whole new API. Meanwhile, it is trivial to stack-allocate a buffer for writing your log-line and flush it out with a single write(). There are others. I like to say that writing fast software is 99% doing only what you need to, and being good at writing fast software requires being brutally narrow about that "need".
- kyllo 6y agoIs this generally the case across all major C standard library implementations or are some of them much faster than others?
- apankrat 6y agoOh, I'm sure it's implementation-specific.
- asveikau 6y ago> For example, here's a trivial sscanf() rewrite "sscanf rewrite" implies you wrote a general purpose replacement. This just parses a fixed format. That is not an sscanf rewrite. The way you phrased it sounds like they could drop it into an existing libc and do better.
- apankrat 6y agoDon't get carried away with misquoting or you end up with a strawman. It's "sscanf() rewrite for a log parsing case", not "sscanf rewrite".
- zerr 6y agoI also understood it as ((sscanf() rewrite) for a log parsing case) :)
- rat9988 6y agoI understood it the same way until I saw his limited implementation, and then revised my understanding. It seems pretty obvious to me that he isn't writing a sscanf clone.
- zoomablemind 6y agoThere are other word choices than 'rewrite' that describe that implementation without confusing the context. For example, "log parsing without using sscanf()", or "sscanf()-like parsing using fixed formats". Saves the need to lookup the actual code in disbelief.
- ChrisLomont 6y agoIt's not misquoting. It's poorly chosen words that were misleading. I too thought you meant you either replaced sscanf with a faster equivalent or used sscanf to beat something else. Instead write "I replaced sscanf with single use code", but then the speed improvement is not that interesting.
- 6y ago
- nchase 6y agothis adds some helpful perspective. thanks!
- andi999 6y agoSo is IO faster or 'conversion'?
- vitaut 6y agoThe formatting part but unfortunately it's just a clickbait: https://news.ycombinator.com/item?id=23311726 https://news.ycombinator.com/item?id=23311726
- expnkx 6y agoSorry dude. You are horribly wrong. https://github.com/expnkx/fast_io/commit/9b2b084e680b6cd28096a566b103f08b9ad3a714 https://github.com/expnkx/fast_io/commit/9b2b084e680b6cd2809... Benchmark: https://github.com/expnkx/fast_io/tree/master/benchmarks/0015.fast_io_vs_fmt/benchmark https://github.com/expnkx/fast_io/tree/master/benchmarks/001... Binary Size: https://bitbucket.org/ejsvifq_mabmip/fast_io/src/master/benchmarks/0015.fast_io_vs_fmt/binary_size/ https://bitbucket.org/ejsvifq_mabmip/fast_io/src/master/benc...
- deleted 6y ago[deleted]
- londons_explore 6y agoYou know what's even faster... write() I was writing part of a codebase where I couldn't use the heap, so all standard libraries were not on the cards, and I ended up just using write(2, "hello\n", 6) to send the data direct to the kernel, and found the experience remarkably simple and unpainful.
- smabie 6y agoNo it's not, write is wayyyy slower. I just did a test, I write "hello\n" to a file 10,000,000 times. Using fprintf() and stdio.h, it takes 0.04s on my machine. Using write(), it takes 7.6s, over 2 orders of magnitude slower. Syscalls are extremely expensive and dominant the execution time unless you are writing large chunks of data all at once. Which is precisely why stdio.h exists, because it only occasionally flushes the buffer with write(). This can be observed when you get a segfault, often it seems like your print statements did not execute. One can use fflush() to force stdio.h to flush the buffer. write() on the otherhand, always writes the data, well, at least to the file buffer cache. fsync() is required to ensure that the data is on disk, though I've seen systems lie about that too.
- ilammy 6y agoIt depends on what you want to do. If that's sending data into the kernel right now – like procfs or a socket – then write() is what you need. fprintf() will provide more throughput by buffering the data and sending a batched write() later, but sometimes you need to send stuff right now and be sure that it's in the kernel when you get control back.
- MaxBarraclough 6y agoI imagine your compiler optimises the fprintf call into a call to fputs or puts.
- smabie 6y agofwrite, but yeah.
- 6y ago
- CoolGuySteve 6y agoFormatting is a natural demarcation point for deferring to another thread as it can be surprisingly expensive. You can create a workqueue where each item is a copy of the argument list and a const char* of the format string. Then your program will only take a microsecond or two to print (depending on the argument length) on the critical path.
- klodolph 6y agoFormatting is generally fast enough in practice that synchronizing with another thread would slow things down. So it is actually not a good place to offload work to other threads, since you spend a similar amount of work just sending the message. Unless you are formatting a really large message.
- CoolGuySteve 6y agoYou should time it and come back. You're wrong.
- nkurz 6y agoYou might be right, but your response comes across as rather rude. I think you should either soften your message, or provide more details about the timing --- or both! My instinct is that for standard IO involving moderate amounts of string interpolation and number formatting in C running on Linux on modern Intel processors, klodolph is probably closer to right. But maybe you are right for some other setup? Or maybe klodolph and I are both wrong, in which case some more details would be helpful.
- CoolGuySteve 6y agoI gave a time in my original post: 1-2usec. Printing a handful of doubles with a cold instruction cache using the standard stdio headers takes longer than that due to all the special casing. klodolph made some handwavey claim about synchronization overhead that are clearly false. It doesn't even make sense to me since the device being printed to will need some sort of synchronization and syscall. Both you and klodlph are also making handwavey claims about formatting speed in Linux. You both need to measure before you argue. I find it extremely rude to argue from a position of ignorance as both of you are doing. "Well gee, I dunno but you're probably wrong". It's so asinine.
- nabla9 6y agoIn many applications I can get 10x faster I/O using stdio.h just by radically increasing the buffer size. It turns out that less you talk to the kernel, faster you are. /* typically 16 to 32 times stat.st_blksize or BUFSIZ is good */ int new_size = 16 * 4096; setvbuf(fp, NULL, _IOFBF, new_size);
- michaelcampbell 6y agoI can't remember how many times I've suggested that people use the optional pre-sizing parameter to Java's StringBuffer class constructor in code reviews. Many times the size of the final string is known absolutely, or at least close enough to pre-allocate. IIRC, the default initial size is 16 bytes which is absurdly low.
- underdeserver 6y agoIt's a shame there's no way to globally set the default StringBuffer size. I bet if you optimize it just by dumb binary search, on your average project you could save on both runtime and memory usage.
- grandinj 6y agoThe default size is in fact optimally sized for "normal" Java applications, which frequently do very little with string buffers, or use them for small tokens or numbers. Source: mailing list discussion with one of the Sun devs.
- jrpelkonen 6y agoI completely agree on the buffer size statement, but at the same time, find it interesting that you would use StringBuffer enough to make that comment. I personally haven’t found any compelling use cases for it since Java 1.5 introduced StringBuilder. Would you like to share some ways you find StringBuffer useful?
- cogman10 6y agoYou shouldn't use StringBuffer. The reason it is slow is because it is "thread safe". 99% of the time, you don't really want that (especially in cases where you know how big the string will be at the end of everything). StringBuilder should be preferred for pretty much everything.
- deleted 6y ago[deleted]
- quietbritishjim 6y agoI find the lack of clarity about this library quite suspicious. * What is the core implementation idea behind this library? Why is it supposedly so much faster than stdio and fmt? There doesn't seem to be much explanation in the readme. The only thing I can see is "Locale support optional" and "Zero copy IO" but these are mixing up formatted and unformatted interfaces - it is supposed to be faster at both? Does the "zero copy IO" mean that it's unbuffered (as another comment here mentioned, that usually makes things slower not faster)? * What does the API look like? There's no documentation whatsoever - the "documentation" heading in the readme just refers to "./doxygen/html/index.html" which doesn't exist in the repo; I can't even see a Doxyfile. Just a brief example in the readme showing reading and writing would be nice! * What exactly is it faster at? Reading or writing? Formatted or unformatted IO? If formatted, is it just the formatting that's faster (e.g. is there also a comparison against fputs)? The benchmarks section has no detail of what code was compared except a mention of the examples/ directory, but that contains dozens of subdirectories, most of which each have multiple files in them. I find it quite implausible that it's 6x faster at formatting than fmt, and benchmarks are notoriously hard, so combined with the extreme lack of clarity I find it hard to take these claims seriously.
- dingo_bat 6y agoDocumentation for C/C++ libraries is usually just the header file: https://github.com/expnkx/fast_io/blob/master/include/fast_io.h https://github.com/expnkx/fast_io/blob/master/include/fast_i...
- colejohnson66 6y agoI was expecting there to be Doxygen comments in there, but there wasn’t. So, no, this header is not documentation. I don’t want to read all the code to understand how to use a library; I want examples.
- dingo_bat 6y agoIt's not hard to find examples in the repo. https://github.com/expnkx/fast_io/blob/master/examples/0001.helloworld/hello_world_win32_box.cc https://github.com/expnkx/fast_io/blob/master/examples/0001.... The code is self documenting. I don't think doxygen comments are always needed. This is a simple library, with a simple interface.
- vitaut 6y agoUnfortunately performance claims appear to be bogus. 1. ospan, performance claims seem to be based on, doesn't do any bound checks, so you can easily get buffer overflow. 2. fast_io generates a whopping 50kB of static data just to format an integer. So if these benchmark results are correct (I was not able to verify because the author hasn't provided the benchmark source): > format_int 7867424 ns 7866027 ns 89 items_per_second=127.129M/s > fast_io_ospan_res 6871917 ns 6870708 ns 102 items_per_second=145.545M/s fast_io gives 15% perf improvement by replacing a safe format_int API from https://github.com/fmtlib/fmt https://github.com/fmtlib/fmt with a similar but unsafe one + 50kB of extra data. Adding safety will likely bring perf down which the last line seems to confirm: > fast_io_concat 7967591 ns 7966162 ns 88 items_per_second=125.531M/s This shows that fast_io is slightly slower than the equivalent {fmt} code. Again this is from the fast_io's benchmark results that I hasn't been able to reproduce. 50kB may not seem like much but for comparison, after a recent binary size optimization, the whole {fmt} library is around 57kB when compiled with `-Os -flto`: http://www.zverovich.net/2020/05/21/reducing-library-size.html http://www.zverovich.net/2020/05/21/reducing-library-size.ht... The floating-point benchmark results are even less meaningful. They appear to be based on a benchmark that I wrote to test the worst case Grisu (https://www.cs.tufts.edu/~nr/cs257/archive/florian-loitsch/printf.pdf https://www.cs.tufts.edu/~nr/cs257/archive/florian-loitsch/p...) performance on unrealistic random data with maximum digit count. fast_io compares it to Ryu (https://dl.acm.org/doi/pdf/10.1145/3192366.3192369 https://dl.acm.org/doi/pdf/10.1145/3192366.3192369) where maximum digit count is actually the best case and the performance degrades as the number of digits goes down. A meaningful thing to do would be to use Milo Yip's benchmark instead: https://github.com/miloyip/dtoa-benchmark https://github.com/miloyip/dtoa-benchmark
- bjourne 6y agoHere is my fast io "library": https://github.com/bjourne/c-examples/blob/master/libraries/fastio/fastio.h https://github.com/bjourne/c-examples/blob/master/libraries/... It is very (very!) limited, but also very fast. :) Think I got about 20-50x compared to scanf.