19 ms·
How is GNU `yes` so fast?
- jvolkman 9y ago`yes` (with the backticks) is my favorite "bring the system to its knees right now" shell command.
- AstralStorm 9y agoDoes not do that on modern Linux or especially -ck patch.
- jvolkman 9y agoGood to know. I think the last time I tried it was on a rhel5 or rhel6 variant. However, as avip found out below, it does still render OS X useless within less than a minute (at least on my 2015 MBP).
- ZenoArrow 9y agoWhat does `yes` try to do?
- statictype 9y agoI'm guessing it tries to read the output and return it. Since yes doesn't actually terminate it's going to generate a massive temporary variable to store it's continuous output.
- aexaey 9y agoThis will attempt to open a child shell process with whatever output of "yes" is, interpreted as shell script. But before that, parent shell has to buffer until EOF. With "yes" output being unbound, this means unbound buffering / memory growth. That is, until OOM killer take notice and shut it down. Whole thing will probably take few seconds to a minute (depending on how much free RAM there is vs. how fast it is), peg a single CPU core in the process, and will recover cleanly (except for shell in question being terminated). ...unless system in question has single CPU and large+slow swap. Then yes, bring-it-to-its-knees.
- marcosdumay 9y agoIt may also push everything in memory to swap in the process, what is the real speed killer.
- AstralStorm 9y agoMake sure to have a sane memory limit set in PAM and switch vm.swappiness to lower value than 60.
- AstralStorm 9y agoSingle CPU is fine nowadays. It will be slowish if you ran it with no nice or priority. RAM can be tweaked by preventing Linux heuristic overcommit via sysctl vm.overcommit_memory=2. (0 is only recommended if you do not run broken applications. It so happens that many JS and Java VMs are broken on memory pressure.)
- avip 9y agoWell thanks for killing my mac. People are trying to work here you know.
- danielsamuels 9y agoYou took a command from a comment which said they use it to "bring their system to its knees", ran it, and then complained?
- avip 9y agoI thought that's what internet is for.
- garaetjjte 9y agoHow? Only eats 2GiB and crashes bash with bash: xrealloc: cannot allocate 18446744071562067968 bytes
- likelynew 9y agoWhy is it so slow(compared to the post) in the macbook air. Native yes runs at 26 MiB/s, and GNU yes at 620 MiB/s.
- AstralStorm 9y agoThe CPU and RAM are slower and there is much more kernel overhead. Not to mention pipe buffer size is small on OS X.
- AstralStorm 9y agoCome on people, I ran GNU yes just there, it isn't as slow as OP says but it is a quarter of expected performance.
- aselzer 9y agoIt isn't! yes runs at 7.2 GiB/s on my macbook air. Though I have Linux installed on it instead of Mac OS :)
- umanwizard 9y agoAre you trolling? The commenter obviously meant the macOS version of "yes".
- masklinn 9y agoThe VM subsystem is much slower. Also possibly a not-up-to-date GNU grep, on my (ancient) MBP I get 26MB "native", 780MB GNU and 2.8GB for TFA's C program (2.9 if I replace the malloc by a stack-allocated array, weirdly enough)
- tomsmeding 9y agoProbably alignment issues: stuff on the stack might be aligned by default, in contrast to malloc'd memory.
- ww520 9y agoI would just pre-allocate a static array of "y\n" of size BUFSIZ, write it out in a loop, and call it for the day, skipping the whole malloc and filling loop business. Make the static array BUFSIZ * 1024 to trim the syscalls by a factor of 1000.
- boulos 9y agoReal yes accepts a string to print (so you can have it spit out a full "yes" or "Y" rather than a hardcoded "y").
- ww520 9y agoHave a few pre-written arrays for the common cases: "y", "Y", "n", "N", etc. Those are the fast cases (or the benchmark optimized cases, like, what Volkswagen did). Have another pre-allocated static array to fill in with other input.
- skrebbel 9y ago> (or the benchmark optimized cases, like, what Volkswagen did) This is amazing. Maybe we can turn this into a verb? "I Volkswagened the common cases with precalced buffers".
- kuschku 9y agoThat's how you get sued. Especially because now Ford and GM are also in court, for having done the same as VW.
- dotancohen 9y agoIn the same vein as Adobe sues me for photoshopping a picture or Alphabet sues me for googling my exgirlfriends' names?
- kuschku 9y ago
- fuckemem 9y agoIt would be interesting to test this on Windows too
- kccqzy 9y agoI would think the measurement tool (pv) could be a significant part of the overhead. EDIT: the post I'm replying to has changed in the interim. It was previously talking about overhead.
- oculusthrift 9y agothey are using pv in the original GNU 'yes' measurement too?
- lucozade 9y agoRight, but if that was the limiting factor on throughput with the original pipe, then there's nothing they could do to the bespoke 'yes' to beat it.
- oculusthrift 9y agobut the article was about matching it, and they weren't able to achieve that. i agree it would be a factor if trying to beat it
- notatoad 9y agogood job! you have accurately copied the top comment from the link you just clicked on.
- SteveNuts 9y agoIt's possible that he or she wrote both comments, or someone from Reddit copied the comment from HN.
- deleted 9y ago
- ars 9y agoBut doesn't this make the typical use case (just a few "yes"s needed) slower, since first it has to fill a buffer? I would write() the buffer each time it gets enlarged, in order to improve startup speed. Also: The reddit program has a bug if the size of the buffer is not a multiple of the input text size. And it's increasing the buffer by incrementing one at a time, instead of copying the buffer to itself, reducing the number of loops needed (at cost of slightly more complicated math).
- nostrademons 9y agoIf your only overhead is filling an 8K buffer, I don't think your user is going to care. Taking one microsecond instead of one nanosecond doesn't matter all that much when you're going to lose way more than that in pipes, the kernel, the program you're piping it to, etc.
- statictype 9y agoBut what's the use case for a large volume of continuous output? It feels like we're optimizing for the wrong use case
- pmontra 9y agoMaybe filling a disk or flooding a network connection as in yes | ssh server "cat > /dev/null" But yes take arguments so there might be more use cases: $ man yes NAME yes - output a string repeatedly until killed SYNOPSIS yes [STRING]... yes OPTION DESCRIPTION Repeatedly output a line with all specified STRING(s), or 'y'.
- coldtea 9y ago>But doesn't this make the typical use case (just a few "yes"s needed) slower, since first it has to fill a buffer? If "only few yes's are needed" then the slowdown to produce them will be inconsequential, whether they still fill a buffer in this case or not.
- 9y ago
- ojn 9y agoMeasurements are really noisy, but I seem to get significantly better numbers than that when I use fsplice() on a pre-generated few pages of file data instead.
- AstralStorm 9y agoYes, splice can bypass the pipe buffer in some cases.
- mkj 9y agoYou could make "yes" faster with the tee() syscall. Keep duplicating data from the same fdin (doesn't actually copy) and it becomes entirely zero-copy.
- AstralStorm 9y agoStill gets piped and hits same performance.
- mkj 9y agoIt's 2x times faster for a quick test here. Only copying on the read side?
- readittwice 9y agoOut of interest: Could you please post the code?
- fredmorcos 9y agoDon't think it's possible. The test was so quick it happened in his mind, not as actual proof on his computer.
- mkj 9y agohttps://pastebin.com/jrcJbjU4 https://pastebin.com/jrcJbjU4 I've realised straight tee() is actually wrong - it works fine for piping to something but "zcyes > /dev/null" won't work, it needs vmsplice() like mrb said. core2duo e8400, 4.8 kernel, coreutils 8.25 > yes | dd bs=1M count=10000 of=/dev/null iflag=fullblock 10485760000 bytes (10 GB, 9.8 GiB) copied, 4.1901 s, 2.5 GB/s > ./zcyes | dd bs=1M count=10000 of=/dev/null iflag=fullblock 10485760000 bytes (10 GB, 9.8 GiB) copied, 1.8542 s, 5.7 GB/s
- mrb 9y agoActually yes should use vmsplice(). And pv on the other side of the pipe should use splice(). Now that would be a complete zero-copy I/O path, purely limited by the CPU, not by memory bandwidth. It would benchmark at hundreds of GB/s :)
- Someone 9y agoWith that malloc overhead, I expect GNU yes to be slower when only a few bytes are read from it. So, what's the distribution of #bytes read for runs of 'yes'? If we know that, is GNU 'yes' really faster than the simpler BSD versions? Also, assuming this exercise still is somewhat worhtwhile, could startup time be decreased by creating a static buffer with a few thousand copies of "y\n"? What effect does that have on the size of the binary? I suspect it wouldn't get up much given that you can lose dynamic linking information (that may mean having to make a direct syscall, too).
- bonzini 9y agoUnless you are statically linking, one malloc doesn't significantly affect your startup time. When only a few y's are read, time is going to be dominated by ld.so, by a large margin.
- Sean1708 9y ago> I expect GNU yes to be slower when only a few bytes are read from it Wouldn't that be entirely negligible compared to starting the program in the first place?
- madeofpalk 9y agoBack when I worked at the Genius Bar at Apple Stores I saw a customer come in and talk to a 'Genius' about their MacBook being "slow". After a quick bit of troubleshooting, he just opened up 4 terminal windows an ran yes in all of them, and did some hand wavy explanation about diagnostics.
- noobermin 9y agoI'm curious why he did it. To impress people who work at the apple store? OK, may be I misunderstood, who ran yes, the customer or the genius?
- Vinnl 9y agoI just read this on Wikipedia (https://en.wikipedia.org/wiki/Yes_(Unix)#Uses https://en.wikipedia.org/wiki/Yes_(Unix)#Uses): > In 2006, the yes command received publicity for being a means to test whether or not a user's MacBook is affected by the Intermittent Shutdown Syndrome. By running the yes command twice via Terminal under Mac OS X, users were able to max out their computer's CPU, and thus see if the failure was heat related.
- madeofpalk 9y agoAhh interesting, maybe he did have a legitimate reason for it. I always assumed it was busywork to make the customer feel better.
- jscheel 9y agoNow if I could just prove to Apple that my computer randomly shuts down all the time... problem is, it doesn't appear to be heat related.
- pixelbeat__ 9y agoThe recent commit that sped up GNU yes has a summary of the perf measurements https://github.com/coreutils/coreutils/commit/3521722 https://github.com/coreutils/coreutils/commit/3521722
- exxo_ 9y agoSpeed vs readability: https://github.com/coreutils/coreutils/blob/master/src/yes.c https://github.com/coreutils/coreutils/blob/master/src/yes.c https://github.com/openbsd/src/blob/master/usr.bin/yes/yes.c https://github.com/openbsd/src/blob/master/usr.bin/yes/yes.c
- ceronman 9y agoNormally I think readability is more important than speed. But in this particular case, I think GNU is doing the right thing optimizing the code to the limit. This is the beautiful part of Unix: small tools that do only one thing well. Programs following this philosophy are very good abstractions. They do one very well defined thing so you can use them without having to understand how they work. I have used Unix for years and I've never felt the need to read the source code for `yes`. And because they do a very small thing, even if you need to read them, the overhead of optimization is not that much, for example, the optimized GNU yes is just under 100 LOC if you remove comments and help boilerplate. Yes, it's longer than the BSD version, but it's just a matter of minutes to understands what it does.
- IshKebab 9y agoI totally disagree. Nobody will ever want to use `yes` at 10 GB/s. They will want it to be reliable, and this sort of over-optimisation increases the risk of bugs.
- avar 9y agoI'm having a hard time imagining a case where even a grossly complex 100,000 line implementation of yes(1) couldn't be trivially proven to be correct.
- fredmorcos 9y agoAbsolutely, I once trivially proved a 1000,000 line implementation of return 0 to be correct. I don't know why all comments are bothered by how much overkill this yes implementation is. Maybe they don't hold 100,000 PhDs like we do, am I right?
- akerro 9y agoYears ago I read a similar experiment about max. CPU data flow. Guy was testing how much data can his CPU pass in a second. He was writing it in C, using some Linux optimization, optimizing code for CPU caches, using some magical C vectors that are optimized for such purpose. He got some help from someone working at Google. I tried to find that post but never succeeded. Does anyone here know it?
- tzs 9y agoThe /r/programming discussion of this is interesting [1]. Someone does a Go version and gets the same speed as GNU yes. Someone else tries several languages. This person got the same speed in luajit, and faster in m4 and php. Ruby and perl about 10% slower, python2 about 10% slower still, and python3 about half that. The code is given for all of these, and subsequent comments improved python3 about 50% from his results, but still not up to python2. [1] https://www.reddit.com/r/programming/comments/6gxf02/how_is_gnus_yes_so_fast_xpost_runix/ https://www.reddit.com/r/programming/comments/6gxf02/how_is_...
- dom0 9y agom4 is orders of magnitude slower (notice the different unit). If you write the Python script using bytes (not Unicode), then Python 3 is faster than Python 2, at least for me. Python 3 is 20 % slower than GNU yes and Python 2 35 % slower than GNU yes. e: I think in your last sentence you are comparing results from different computers.
- psittacus 9y agoThis[1] comment seems directly opposite from what you wrote, with Python 3.4 (8.76GiB/s) even outrunning Python 2.7 (8.55 GiB/s). (haven't run the tests myself though) edit: haven't noticed the comment by dom0. [1] https://www.reddit.com/r/programming/comments/6gxf02/comment/diu4l0x https://www.reddit.com/r/programming/comments/6gxf02/comment...
- fredmorcos 9y agoThank you for incorrectly pointing out what can be trivially found in the linked post.
- acomjean 9y agoI had to smile, as this thread is a microcosm of programming language stereotypes: with the python programmers tweaking code to get that extra 10% (nobody mentioned pypy..), someone trying to get javascript to work in a long running program, of course a rust implementation that isn't working well just yet (but we're all rooting for it.. set that compiler flag and rerun).. One comment about perl (turns into a one liner..). Nobody bothering to redo the Lua, Ruby and PHP. And for fun somebody throws down some fortran. what is going on with the m4 code?
- raverbashing 9y agoAnd the question is, do we need yes to be so optimized? Not complaining, I like this kind of analysis But it seems you won't be limited, in a shell script, by the speed you can push y's
- jlarocco 9y agoThat kind of question doesn't make sense for open source code. Somebody wanted to optimize 'yes', so they did. There doesn't need to be a good reason, just like there doesn't need to be a good reason for a person to read a certain book or watch a certain movie, other than they want to do it.
- raverbashing 9y agoThe question was "do we need it? (for a specific use case)" rather than "should we do it?". And it does make sense for Open Source code because the resources are limited hence other features and/or bug fixes might be more important than pushing data at full speed
- jlarocco 9y ago> And it does make sense for Open Source code because the resources are limited hence other features and/or bug fixes might be more important than pushing data at full speed That's not how open source works, though. There's not a group of people obligated to work on the GNU utilities. There's not a central project manager ordering people around telling them what changes to make. Declaring that 'yes' is generally fast enough already doesn't imply that it was fast enough for the person who spent their time optimizing it. Somebody needed (or just wanted) 'yes' to be really fast, so they did the work and submitted the changes back.
- raverbashing 9y ago> There's not a central project manager ordering people around telling them what changes to make. No, but there is review and approval of patches (also the most frequent contributors know where the project is going and there is bug tracking) If one just makes it faster without major benefits that patch is likely to be rejected
- joosters 9y agothe limit isn't the processor, it's how fast memory is. With DDR3-1600, it should be 11.97 GiB/s (12.8 GB/s) I don't understand this reasoning. Why is it being limited to main memory speed? Surely the yes program, the fragments of the OS being used, and the program reading the data, all fit within the L2 cache?
- pandaman 9y agoIn general, only the CPU itself sees the L2 cache. Anything you see on another device (screen, disk, NIC etc) has been flushed out of cache.
- joosters 9y agoSure, but this is a pipe between two tiny processes, hopefully with very little else being run on the computer at the time (otherwise all bets are off for any benchmarking). There's no kind of 'real' I/O going on (in terms of stuff like screen, disk, NIC, and so on) There's no reason that the L2 cache needed to be flushed at any point - the caches are all dealing with physical memory, rather than virtualised address space, so the fact that there are two processes here shouldn't stop the caching from working.
- pandaman 9y agoI am probably way behind the current state of the CPU judging by the downvotes I got so if you are saying there is no reason and the data can be written into a device without leaving the CPU I will just concede my ignorance.
- joosters 9y agoDon't fret about the downvotes, these magic internet points aren't redeemable anywhere :) It's completely possible for the data to not (all) leave the CPU. If the caches are large enough, then the pages full of "y\n" will be still resident in the cache when the next iteration of the program overwrites the same pages again. Then the CPU has no need to send the original page out to main memory.
- du_bing 9y agoI run the command `yes | pv > /dev/null` on my MacBook Pro, it's only 37m/s, is this normal? I am not familiar with the command.
- kjensenxz 9y agoFrom the OP: >OS X just uses an old NetBSD version similar to OpenBSD's >NetBSD's is 139MiB/s, FreeBSD, OpenBSD, DragonFlyBSD have very similar code as NetBSD and are probably identical, It really depends on how fast your memory and processor are. Everything there was benchmarked with an i7-4790 with DDR3-1600, and very very little running in the background.
- qb45 9y agoTwo orders of magnitude difference can't easily be blamed on CPU, in this case it depends on how optimized yes is. I have some older version of coreutils on one machine here and performance is similarly abysmal.
- likelynew 9y agoI am getting 26 MiB/s using native yes, and 620 MiB/s using GNU yes in macbook air.
- peter_retief 9y agoWell now I know what `yes` does (And pv)
- melicerte 9y agoDid you notice PHP outperforms any other scripting languages? Some report that it event beats the GNU yes implementation. After reading here so many unfair critics and pedantic dislike over PHP[1][2][3][4][5][6], I just want to say: STFU. [1] https://news.ycombinator.com/item?id=12706136 https://news.ycombinator.com/item?id=12706136 [2] https://news.ycombinator.com/item?id=3825227 https://news.ycombinator.com/item?id=3825227 [3] https://news.ycombinator.com/item?id=3824881 https://news.ycombinator.com/item?id=3824881 [4] https://news.ycombinator.com/item?id=1823022 https://news.ycombinator.com/item?id=1823022 [5] https://news.ycombinator.com/item?id=1819517 https://news.ycombinator.com/item?id=1819517 [6] https://news.ycombinator.com/item?id=1819413 https://news.ycombinator.com/item?id=1819413 ... Just to name a few.
- deleted 9y ago[deleted]
- dom0 9y agoIt's almost funny that you try to refute, or rather, dismiss, criticism of PHP with "but it's fast", when none of what you cite even mentions that. I just want to say: STFU.
- melicerte 9y agoI don't dismiss criticism. I dismiss unfair critics. What I cite is full of that. And, btw, [6] implies PHP is slow, read on: [quote] It's hard to find a worse example than php, because when many "bad" languages fail in one or the other category, php seems to fail in most categories. * Java has a really repetitive syntax, but it's libraries might make up for that. * C++ with all it's features has bazillion ways to shoot yourself, but it's arguably fast and the syntax is bearable. ... [/quote]
- gjm11 9y agoIt doesn't in fact imply that PHP is slow. What it says about PHP (besides "php seems to fail in most categories" which doesn't imply any single specific failure because most != all) is this: "But php has both an ugly syntax, horrendous stdlib, and fame of security issues." You will notice that none of those specific complaints is about speed.
- fredmorcos 9y ago> or even /dev/zero I don't think you really know what you're talking about.
- acdha 9y agoSince you didn't add anything other than an unwarranted attack on the person you're replying to, this falls under “Avoid gratuitous negativity”: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- fredmorcos 9y agoAlright, let's recap the situation here: Someone spurs a random, obviously incorrect and unfounded piece of information just to make a point about how they use/used huge amounts of data, because big data these days. Then they make it clear that it's just a memory, a less than vivid one: > I agree, all I remember is that when I tried it, /dev/zero sometimes sucked performance-wise Only sometimes yes. Sometimes it didn't suck. But sometimes it did. But sometimes it didn't. > I can't recall the exact circumstances as it was some time ago A long time ago /dev/zero sometimes suck performance-wise, and sometimes it didn't. But sometimes it did. > and could have been on any of Linux/FreeBSD/SunOS/HP-UX/IRIX Ah yes, _could_ have been on any <RANGE FROM THE MOST COMMON UNIX SYSTEM TO THE LEAST COMMON SYSTEM THESE DAYS SO THAT WE INTRODUCE SOME MORE CONFUSION AND ILLUSION INTO THE ARGUMENT>. > perhaps it was the fastest common way at the time? Ah yes, perhaps. Another layer of indirection in the argument. And now the number of possibilities is endless, if you want to find any real evidence of the argument made there, you have to go through hoops and combinations of combinations and run the tests. But wait... > On a recent x64 Linux, /dev/zero seems plenty fast enough now: Ah yes, of course, make sure you're not wrong when it only comes to something others can _easily_ call you out on. Seriously, is HN fostering a culture where it's OK to spew pseudo-intellectual garbage as long as you're being nice? This contradicts how every comment is trying to look like a paper published in a highly acclaimed journal.
- joosters 9y agoOh dear. So sorry that my memory is crap enough to not provide you with the exact details you require. ...every comment is trying to look like a paper published in a highly acclaimed journal It was an anecdote about my previous experience with 'yes' and /dev/zero, it seemed somewhat on-topic. I'm not trying to justify anything or win magic internet points. If you're expecting journal-quality posts, perhaps you should go read a journal and not an internet forum chat.
- sytringy05 9y agoman, I just spent like 8 minutes today writing a python script to use up all the disk space on some servers (part of ops readiness testing) when I could have just used this trick. `yes` will help me on the "see what happens when something uses all the CPU and memory" test case. Thanks Reddit/HN!
- acdha 9y agodd can also be useful for that kind of thing since you can use a source like /dev/urandom to generate random bytes if you're trying to avoid compression and the adjustable block sizes can be optimized for the underlying storage system.
- sytringy05 9y agoI did actually use dd to begin with but it didn't work as I wanted it too right away (can't remember why, probably something SUSE related) and by then I'd used up 6 mins of the 15 mins I gave myself for the job. I also had to do the same test on windows, so python won
- Tepix 9y agoWhy is he using backticks to quote "yes" in the title?
- daveguy 9y agoIt is common in markdown formatting to indicate a command line with backquotes. It tells the markdown compiler to apply special css and possibly syntax highlighting. So, a lot of people who write about shell commands and setup tutorials, or use markdown on something like github will automatically identify it as a command.
- arnaudsm 9y agohttps://m.xkcd.com/619/ https://m.xkcd.com/619/
- dekhn 9y agoclearly, we just need /dev/yes
- e12e 9y agoSounds like a reasonable feature request for systemd.
- mooktakim 9y agoIf anyone, like me, is wondering what "yes" is used for. You can use to pipe "y" to commands that require interactivity, so if you just want to say "y" to all the inputs, you can use "yes" to do this: yes | rm -r large_directory yes | fsck /dev/foo
- euske 9y agoI sometimes do yes "" | cat -n to generate a sequence number.
- cauterize 9y agoTry `seq 1 n`
- JdeBP 9y agoeuske may be using one of the operating systems (one such being mentioned on this very page) where there is no seq utility. JdeBP ~ $seq ksh: seq: not found JdeBP ~ $ Of course, on the one mentioned there is jot.
- moyix 9y agoAnd on bash you can simply use {1..n}: $ echo {1..5} 1 2 3 4 5 If you have Bash 4.0 or above, you can include a step size: $ echo {1..10..2} 1 3 5 7 9
- teapot01 9y agoI know its only an example but why yes | rm -r <> and not rm -rf <>
- x1798DE 9y agoI think rm -rf would also suppress errors in addition to confirmation messages, so presumably if you want a "soft" force...
- crb002 9y agoyes | write <USERNAME> "Don't you hate dialup connections?"
- BenjiWiebe 9y agoI think you mean yes "Don't you hate dialup connections?" | write username
- tobik 9y agoFreeBSD's yes has just been updated because of this. https://github.com/freebsd/freebsd/commit/1d61762ca37c20ab6fa4f5de6e179fa530ec49f2 https://github.com/freebsd/freebsd/commit/1d61762ca37c20ab6f... It's about twice as fast as GNU yes now on my FreeBSD system here.
- pixelbeat__ 9y agoLooks like that may drop some data if you get a short write, possible when writing to pipes etc. Update: They fixed that issue with this follow up https://github.com/freebsd/freebsd/commit/2592fbb8 https://github.com/freebsd/freebsd/commit/2592fbb8
- deleted 9y ago[deleted]
- DonHopkins 9y agoThe proprietary Oracle Solaris 11.2 yes really slowed down when they added DRM and Verified Boot support...
- metaphorm 9y agoI thought this was a fascinating read but it left a serious question lingering in my mind, which is a little out-of-scope for the article, but I hope someone here can address. Why did the GNU developers go to such lengths to optimize the yes program? It's a tiny, simple shell utility that is mostly used for allowing developers to lazily "y" there way through confirm prompts thrown out by other shell scripts. is this a case of optimization "horniness" (for lack of a better word) taken to its most absurd extreme, or is there some use case where making the yes program very fast is actually important?
- kevingranade 9y agoThe stated use case for the perf improvement was "yes(1) may be used to generate repeating patterns of text for test inputs etc., so adjust to be more efficient." Source: https://github.com/coreutils/coreutils/commit/35217221c211f3116f374f305654462195aa634a https://github.com/coreutils/coreutils/commit/35217221c211f3... I've personally used it for generating repeating text and filling disks in system testing, so I appreciate it being faster at those tasks. I also sometimes use it as a signal generator for a hacky load generator, like so: yes | xargs -L 1 -P NUM_PROCESSES -I {} curl SOME_TARGET_URL > /dev/null This doesn't benefit from being faster per se, but I appreciate it using less CPU since I want to give curl as much system resources as possible.
- sequoia 9y agoLet's not forget the most crossplatformest, purest `yes` of them all: https://www.npmjs.com/package/yes https://www.npmjs.com/package/yes # /usr/local/bin/yes | pv > /dev/null 11.5MiB 0:00:09 [1.02MiB/s] [ <=>] # /usr/bin/yes | pv > /dev/null 1.07GiB 0:00:09 [ 142MiB/s] [ <=>] JavaScript wins again!!
- sequoia 9y agoIt's come to my attention that lower numbers are not better here. I have filed a bug, we'll get to the bottom of this shortly. I want to apologize to all our users, this issue does not reflect the values and principals we at Pure JavaScript `yes` hold dear https://github.com/Sequoia/yes/issues/3 https://github.com/Sequoia/yes/issues/3
- peterwwillis 9y agotl;dr someone who doesn't understand how i/o works gets a small insight into how memory and a cpu work and decides "Buffering is the secret" and "You can't out-optimize your hardware" Can we have a new flag for posts by people who don't know what they're doing so I can skip them? I am serious.
- luckydude 9y agoI was not going to post this because hacker news has this ethic (?) of down voting anything that seen as not positive. Perhaps we should have discussion about that, I'm not sure that's a good thing but I'm not in charge here. The top comment is: "It's a shame they didn't finish their kernel, but at least they got yes working at 10GiB/s." which as an OS guy, someone who has been working on Unix for 30+ years, as a guy who was friends with one the QNX kernel guys (they had perhaps the only widely used microkernel that actually delivered), that's hugely amusing and spot on. The GNU guys never really stepped up to being kernel people. Bitch at me all you want, they didn't get there. It's a funny comment, especially coming from reddit.
- mintplant 9y ago> hacker news has this ethic (?) of down voting anything that seen as not positive We must not be reading the same Hacker News... Anyway, the comment you're quoting is just a shallow jab that belittles the GNU developers' work without contributing anything new or meaningful. It's telling that you had to spend two paragraphs to justify cross-posting it here.
- ruleabidinguser 9y agoYou say that and then immediately become an example of what hes talking about. This "shallow jab that contributes nothing new or meaningful" is, in some circles, known as a "joke." I'm continually frustrated by people who think that misinterpreting comments as harmful is a useful activity.
- mintplant 9y agoNot really -- I didn't downvote that comment. And I still dispute the notion that negativity is rare or always shunned on this board: to the contrary, it's so commonplace that an actual rule [0] had to be added to try to sway things in the other direction. Jokes have their place, but bringing up the failure of Hurd in every GNU-related post is banal. And saying they "never really stepped up" to your level as a mighty kernel developer, as if the people who brought us glibc and coreutils lack an understanding of OS internals, just seems rude and curiously out of touch. [0] https://news.ycombinator.com/item?id=9317916 https://news.ycombinator.com/item?id=9317916
- kazinator 9y agoGNU yes is fast because it is coded with the assumption that it's not answering any real question, such as "can I combine this free code with a proprietary program?" or "Would you accept the following monstrous patch to GNU Coreutils /bin/true without a copyright assignment?"
- souprock 9y agoI think we can do better. How about a /proc/bin/yes for this? Like most /proc files, it would appear to be empty. Executing it would involve a fs/binfmt_proc.c file in the kernel source, which would be a handler for this sort of executable. That would get the job done entirely in the kernel.