13 ms·

A Little Story About the `yes` Unix Command

• feelin_googley 9y ago
  k3 while[1;`0:"yes\n"]
• kccqzy 9y ago
  Really? You didn’t even mention reducing system calls? That’s basically what full_write does: try to output a whole buffer with one system call. In your regular program, even with just a normal setvbuf call to set up block buffering would make a huge difference.
• dmitrygr 9y ago
  That last rust example is less readable than modern template-metaprogramming variants of C++. There is something elegant about being able to beat it out on speed with about 30 lines of C
  • a_t48 9y ago
    Simply putting a comment above each function would make it bearable.
    • deleted 9y ago
      [deleted]
    • dmitrygr 9y ago
      not as readable as this faster variant #include <sys/uio.h> //2048 "y\n" s static char __attribute__((aligned(4096))) str[] = "y\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\ny\n"; static struct iovec v[] = { {str, 4096}, //adjust number of these to taste {str, 4096}, {str, 4096}, {str, 4096}, {str, 4096}, {str, 4096}, {str, 4096}, {str, 4096}, {str, 4096}, {str, 4096}, {str, 4096}, {str, 4096}, {str, 4096}, {str, 4096}, {str, 4096}, {str, 4096}, }; int main(int argc, char** argv) { while(1) { //scatter-gather write (on linux there is basically no way for write to write an odd number of characters, except EOF, so we need not check for partial writes (void)writev(1, v, sizeof(v) / sizeof(*v)); } return 0; }
      • kibwen 9y ago
        Bah, that'll never pass code review. How about this: #include <sys/uio.h> #define str(x) #x #define expand(x) str(x) #define y1 y\n #define y2 expand(y1) expand(y1) #define y4 y2 y2 #define y8 y4 y4 #define y16 y8 y8 #define y32 y16 y16 #define y64 y32 y32 #define y128 y64 y64 #define y256 y128 y128 #define y512 y256 y256 #define y1024 y512 y512 #define y2048 y1024 y1024 //2048 "y\n" s static char __attribute__((aligned(4096))) str[] = y2048; static struct iovec v[] = { {str, 4096}, //adjust number of these to taste {str, 4096}, {str, 4096}, {str, 4096}, {str, 4096}, {str, 4096}, {str, 4096}, {str, 4096}, {str, 4096}, {str, 4096}, {str, 4096}, {str, 4096}, {str, 4096}, {str, 4096}, {str, 4096}, {str, 4096}, }; int main(int argc, char** argv) { while(1) { //scatter-gather write (on linux there is basically no way for write to write an odd number of characters, except EOF, so we need not check for partial writes (void)writev(1, v, sizeof(v) / sizeof(*v)); } return 0; }
        • flukus 9y ago
          Both suffer from way to much binary bloat, for small performance hit we can shave off a whole 4Kb. Is anyone going to take a stab at an enterprise edition?
          • taneq 9y ago
            "Enterprise edition" would be the same thing but written in Java and require an Oracle backend.
            • _jal 9y ago
              Don't forget LDAP, SAML and SNMP support. What's the OID for 'y' again?
              • anonfunction 9y ago
                https://github.com/EnterpriseQualityCoding/FizzBuzzEnterpriseEdition https://github.com/EnterpriseQualityCoding/FizzBuzzEnterpris...
  • flukus 9y ago
    And the naive "readable" version is half the speed of python. Makes you wonder how typical that would be in real code?
    • glandium 9y ago
      Fun fact: $ python yes.py | pv -r > /dev/null [11.2MiB/s] $ python3 yes.py | pv -r > /dev/null [4.95MiB/s] As for why the naive rust version is slower, it's because without adding a BufWriter in rust, stdout is line-buffered, so each line emits a write system call, while with python, stdout is buffered. Python 2 emits writes of 4096 bytes, and python 3... 8193 bytes (edit: not a typo, this is 8KiB + 1). That's the likely cause for it being slower. Edit: A minimal version of the naive rust version would be: fn main() { loop { println!("y"); } } On the same machine as with the python tests above, I get: $ ./yes | pv -r > /dev/null [4.81MiB/s] which is actually as slow as python 3, despite doing 4 thousand times more system calls. A version with buffering would look like: use std::io::{stdout,BufWriter,Write}; fn main() { let stdout = stdout(); let mut out = BufWriter::new(stdout.lock()); loop { writeln!(out, "y").unwrap(); } } And produces 129MiB/s on the same machine. And that's strictly doing the same as what the python version does (with a default buffer size of 8KiB, apparently). And fwiw, on the same machine, both GNU yes and the full rust solution from OP do 10.5GiB/s.
      • Bromskloss 9y ago
        > As for why the naive rust version is slower, it's because without adding a BufWriter in rust, stdout is line-buffered, so each line emits a write system call, while with python, stdout is buffered. Python 2 emits writes of 4096 bytes, and python 3... 8193 bytes. That's the likely cause for it being slower. Does it have nothing to do with the fact that string-of-bytes is the default in Python 2, whereas string-of-characters is the default in Python 3? Or is that perhaps related to the explanation you gave? Forcing the byte interpretation, Python 3 is slightly faster than Python 2 for me. Forcing the character interpretation, Python 2 wins, but not by as much as before. Bytes: while True: print(b'y') Characters: while True: print(u'y')
  • agwa 9y ago
    The main function is egregiously overcomplicated. This is how I'd write it instead: fn main() { match env::args_os().nth(1) { None => write(b"y\n"), Some(arg) => { let mut arg = to_bytes(arg); arg.push(b'\n'); write(&arg); } } process::exit(1); } It's just as efficient and far more readable. It avoids Cow, a lambda, and the chain of combinators.
    • sinistersnare 9y ago
      Doesn't this only print once, and not over and over, like `yes` should?
      • agwa 9y ago
        The write function takes care of repeating it. See the complete source code in the blog post.
  • kibwen 9y ago
    > There is something elegant about being able to beat it out on speed with about 30 lines of C What is this referring to? If it's referring to GNU C, the linked C implementation is about 100 non-comment lines, and is three times slower on the author's machine than the Rust version.
    • dmitrygr 9y ago
      see sister comment
      • kibwen 9y ago
        Ah, I'm sorry, I thought that program was meant to be humorous. :P
  • mrighele 9y ago
    As other have pointed out elsewhere what matters here is not much the performance of the language but how much you can avoid calling IO code by making the path as straightforward as possible. You can get comparable performance ( 6.5 GB/s vs 8. GB/s on my machine) with 10 lines of python code, which probably took me the same time to write as the original rust version.
    • e12e 9y ago
      I'm sure a lot of people would like to see the code for comparison. I know I would.
      • mrighele 9y ago
        The code is below. I have to admit that after a few more test I realized that the performance greatly depends on the length of the input, in some cases going as low 2 GB/s and some others reaching the performance of the GNU yes installed on my system. This is probably due to the fact that I build the buffer in a quite naive way. With a bit more work maybe that can be fixed. #!/usr/bin/python import sys yes = "yes\n" print(sys.argv) if len(sys.argv) == 2: yes = sys.argv[1] + "\n" size = 4096 buf = bytes(yes * size, 'utf-8') while True: sys.stdout.buffer.write(buf)
• billsmithaustin 9y ago
  Those performance improvements make the code more complicated too. How fast does the yes command actually need to be?
  • bluedino 9y ago
    This could be applied to something like memset, another deceiving simply function
  • kccqzy 9y ago
    The git commit that sped things up mentioned using yes to generate test data.
  • loeg 9y ago
    Yeah — probably it doesn't matter how fast the yes command is. It will be faster than the program prompting for y/n. FreeBSD's was nice and simple until someone decided to make it faster in r319897.
    • vacri 9y ago
      The yes command will repeat whatever string you give it, so it's useful for load/traffic testing. Speed helps this particular use-case.
      • loeg 9y ago
        That's a possible secondary use, but not really what the command is for. Why not just use /dev/zero or /dev/urandom for generating load?
        • azernik 9y ago
          yes will repeat arbitrary data (including binary).
        • vacri 9y ago
          Because the yes command will repeat whatever string you give it. If you need to parse the incoming load in a certain way, /dev/{zero,urandom} won't be suitable.
• 13of40 9y ago
  I guess a stretch goal would be to make a "shouldi" command that can consume more y's per second than yes can produce. Of course at that point the shell itself would probably become the bottleneck.
  • glandium 9y ago
    The shell can't be a bottleneck. If you run `yes | shouldi`, all the shell does is setup the pipe, give one of its ends as stdin to shouldi, and the other end as stdout to yes.
  • kccqzy 9y ago
    The shell does not even enter into the picture. The shell sets up the pipe and starts your programs. The `yes` program writes its output to stdout which is buffered in kernel and then directly read by your hypothetical `shouldi` program.
  • yjftsjthsd-h 9y ago
    yes >/dev/null ?
• cannikin 9y ago
  Funny, I just learned about this command a couple of days ago as a simple way to max out your CPU. I was trying to drain the battery on my Macbook Pro and running 4 of these at the same time did the trick nicely. Redirected to /dev/null and run in the background: "yes > /dev/null &"
• blibble 9y ago
  all the reddit users from the linked article missed the SIGPIPE trick! you don't need to check the return value from write() as your process will be terminated with SIGPIPE if it tries writing to a closed pipe. saying that, none of them check the return code correctly: if the consumer only reads a byte at a time you could eventually get 'yyyyyyyyyyyyyyyyyy' (without any newlines) quite impressive that so many implementations of "yes" have the same bug :)
  • dmitrygr 9y ago
    please demonstrate that consumer. on linux there is no way to do this without your consumer being a kernel driver that actually [purposefully accepts one byte at a time. all file io and pipes have basically no way to accept one byte only (except EOF)
    • blibble 9y ago
      you wouldn't see it on Linux as pipes are implemented using complete pages (always powers of 2), but there's probably some OS out there with a different implementation where you can set the buffer size to be an odd number, and then you'd see the bug with plain simple cat I didn't say you'd likely see it in practice :)
  • caf 9y ago
    You can't rely on SIGPIPE to terminate it or your program will fail to terminate on error if its output is redirected to a disk file.
  • electrum 9y ago
    write(2) is a blocking call (unless O_NONBLOCK is set), so you should never have a partial write while the descriptor is connected to something valid.
• snikch 9y ago
  I use the yes command to defrost my lunch. I open up a couple of tabs running yes > /dev/null Then place my frozen lunch on the back of my macbook. Give it an hour or so and boom, defrosted.
  • userbinator 9y ago
    Reminds me of http://thedailywtf.com/articles/Just-a-WarmUp http://thedailywtf.com/articles/Just-a-WarmUp , a trick which I also discovered independently many years ago and used to keep my fingers warm while attending lectures in a nearly-unheated room.
    • taneq 9y ago
      Prior art for https://xkcd.com/1172/ https://xkcd.com/1172/ ?
  • Bromskloss 9y ago
    > I open up a couple of tabs So, you're basically thawing your lunch manually. Have you considered the "parallel" command?
    • snikch 9y ago
      Hrm, no I hadn't. I don't know much about this command. Could you give me a (thawing) example?
      • Florin_Andrei 9y ago
        for p in `seq 1 $(sysctl -n hw.ncpu)`; do yes > /dev/null & done Alternatively you could run Boinc with a cancer research project or something - defrost your food AND help the world become a better place.
        • htfy96 9y ago
          $(nproc --all) on Linux
• crehn 9y ago
  Just for fun: GNU true.c: https://github.com/coreutils/coreutils/blob/master/src/true.c https://github.com/coreutils/coreutils/blob/master/src/true.... OpenBSD true.c: http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/usr.bin/true/true.c?rev=1.1&content-type=text/x-cvsweb-markup http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/usr.bin/true/tr...
  • glandium 9y ago
    FWIW, GNU true.c is that way because the GNU policy is for programs to all have --help and --version arguments. Other fun fact, in some UNIXes, true looked(looks?) like: #!/bin/sh # Some multiple line copyright notice from AT&T Yup, that's a shell script with only comments, all of which are a copyright notice for what is, essentially, an empty file. I saw that on Solaris something like 15 years ago. I'm sure it made its way to other flavors of UNIX.
    • kibwen 9y ago
      Now you've got me wondering what it would mean to try and enforce the copyright on an empty file. You'd think there would be some prior art, right? :)
      • perl4ever 9y ago
        Empty files remind me of an episode where I worked. We were supposed to provide a set of files for a court case. But there was a missing zero byte file due to a glitch in processing. So just create one, right? NOPE. The client insisted we find and copy the original empty file from the original media.
        • taneq 9y ago
          You just know it's because, if you just made a new zero-byte file, the opposition in the case would claim that you forged the file and that the evidence was then tainted. Stupid, but your goal here isn't to reproduce the file, it's to stop a stupid argument from convincing stupid people to make a stupid decision.
• pixelbeat 9y ago
  The GNU variant was discussed recently at: https://news.ycombinator.com/item?id=14542938 https://news.ycombinator.com/item?id=14542938 The commit that sped up GNU yes has a summary of the perf measurements: https://github.com/coreutils/coreutils/commit/3521722 https://github.com/coreutils/coreutils/commit/3521722 yes can be used to generate arbitrary repeated data for testing or whatever, so it is useful to be fast
• nodesocket 9y ago
  main(argc, argv) char **argv; { for (;;) printf("%s\n", argc>1? argv[1]: "y"); } Is beautiful, readable, and minimal. The "optimized" Rust version is complicated and over 50 lines of code. At what point does performance optimization go too far?
  • 1_2__4 9y ago
    As a non-Rust developer my first thought upon seeing the optimized version was “On the other hand let’s not visit Rust, ‘tis a silly place.”
    • LEGOlord208 9y ago
      There was a lot of things about Rust that made think it was weird before I actually gave in and tried it. Now I adore Rust. You should try it.
  • azernik 9y ago
    Since it can be used to repeat arbitrary data, it is liable to be used in performance-sensitive tasks (mostly along the lines of pumping dummy data to a program or piece of hardware that's being benchmarked).
  • trungaczne 9y ago
    This is an unfair comparison. For starter this version too is unoptimized, so you need to compare this with the first Rust code, which is about the same size. Moreover a lot of the "bloat" that comes with Rust code isn't because it's more performant. Neither C++ and Rust try to be faster languages (after all, the optimization tricks you can do with your program is limited to what you can do with Assembly and its cousin, C), they try to be safer by providing more abstractions and restrictions (compile time checks).
• Myrmornis 9y ago
  The author says "no magic here" for the C version: for (;;) printf("%s\n", argc>1? argv[1]: "y"); but it's not totally obvious to me whether the argument to printf would be evaluated on every iteration of the for loop or not. Does the compiler know that those don't change, and is the answer to that question fairly basic C knowledge or not?
  • pkaye 9y ago
    The compiler can assume that argc will not change within the loop so it can optimize it. I just looked up the assembly output from gcc and it pulls the argc>1 outside of the loop and replaced the printf with puts. So something like: if argc>1 for (;;) puts(argv[1]) else for (;;) puts("y") The replacing of printf with puts is based on gcc having specific knowledge about the printf library function.
    • ben_bai 9y ago
      This is exactly the OpenBSD implementation.
• jlebrech 9y ago
  maybe it's supposed to be slow, wouldn't a faster 'yes' spam stdin much faster? you only need to hit yes occasionally and faster than a second
• sabujp 9y ago
  tldr; watched pootie tang
• dreta 9y ago
  Can anybody explain to me what's this syntax? This is the first time i see anything like it, and i've been programming in C since i was a teenager. main(argc, argv) char **argv; { }
  • ecma 9y ago
    That's a K&R (Kernighan and Ritchie) style function declaration. Compilers still support it but the version you'd be more familiar with (ANSI C style) has been standard since at least the late 80s IIRC. ANSI C was standardised in 1989 but that process had been in progress for something like 5 years beforehand.
    • colejohnson66 9y ago
      Does it have to do with how, in BCPL (one of C’s ancestors), everything was basically an integer? So everything is an integer unless told otherwise?
      • boomlinde 9y ago
        Specifically, everything is an integer with the auto lifetime (gets discarded with the stack frame) unless otherwise specified. I think most modern compilers will warn on encountering such short hand due to the common error of accidentally declaring integers when you meant to assign a value to a variable you forgot to declare.
  • boomlinde 9y ago
    main and argc are declared as int (technically auto int) by default. argv is declared and given its type between the function head and the function block, which is perfectly legal but quite unusual these days This syntax is probably nice if you use a line oriented editor like ed
    • dreta 9y ago
      I was mostly talking about the odd re-declaration of argv. Everything else is quite clear.
• Annatar 9y ago
  env::args().nth(1).unwrap_or("y".into()); this ridiculously complicated syntax to perform such a simple thing is why I will never accept Rust. What a clumsy, ugly language. I’ll just stick with learning ANSI common LISP, that pays immediate dividends.
  • 0T-T0 9y ago
    The only difference from expletive = 'y' if sys.argc == 1 else sys.argv[2] is the memory safety guards, which happens to be the key feature of Rust
    • Annatar 9y ago
      But what if one doesn’t care about memory safety because one has other languages at one’s disposal, like shell, AWK, Lisp? What benefit does Rust bring then? And at what price? And who is to say that the programmers of Rust built a perfect language that always generates code which is bounds-safe? I question that as I know from experience that no human has ever written 100% correct code 100% of the time; even machines aren’t capable of achieving that. Therefore, I hold that the entire promise of Rust is flawed. I don’t like Rust one bit.
      • steveklabnik 9y ago
        We had a large community discussion about this last year: https://brson.github.io/fireflowers/ https://brson.github.io/fireflowers/
        • cat199 9y ago
          I don't think that theme is bright enough. Perhaps neon green with yellow letters would do the trick?
      • rurban 9y ago
        Problem is that Rust is not memory safe anymore, since they switched from GC to refcounting and added unsafe. Pair that with being no concurrency safety (deadlocks, races) there are only macros left which do make rust attractive. On the other side there are proper languages which do provide all safeties and beat rust or C++ in performance, such as pony. A proper type-system does help (compile-time guaranteed), but you could also add such features to the run-time system (e.g. a GC or safe threading with a single-writer system such as parrot). With rust you don't have any of it, you have to manually add locks or mutexes into your threaded code, and try to avoid unsafe.
• dnel 9y ago
  Interesting, but is there actually a practical benefit of having GigaBytes of the word yes being generated or is this all just optimisation porn?
  • kqr 9y ago
    I thought about that too, but I suspect it may be some sort of race to idle thing.
  • unkown-unknowns 9y ago
    Someone else ITT pointed out a good reason: > yes can be used to generate arbitrary repeated data for testing or whatever, so it is useful to be fast
  • dcosson 9y ago
    Seems like it might even be better if yes was rate limited to a couple of lines per second or less. It would be more than enough for its intended use and when users inevitably run it in a shell to see what it does it wouldn't generate tons of output. It seems like a violation of "do one thing well" to use it for generating data for testing, isn't dd and /dev/zero a better way to do that?
• jwilk 9y ago
  Curiously, yes(1) is not standardized by POSIX. Are there any UNIX systems that don't have it?
• xurukefi 9y ago
  import sys while True: sys.stdout.write("y\n" * 2**16) gives me over 4GiB/s
  • omlettehead 9y ago
    Sure, but you're building strings in memory. It might not be a lot of memory, especially if you're able to run Python, but the native `yes` command can run on the smallest of embedded systems, which is why its speed is impressive.
• 11235813213455 9y ago
  // with node.js: const { Readable } = require('stream'); const ys = Buffer.from('y\n'.repeat(2**16)); class Y extends Readable { _read(size) { this.push(ys.slice(0, size)); } } new Y().pipe(process.stdout);
• naranha 9y ago
  any idea why it is so much faster on fedora 26? $ yes | pv -r > /dev/null [10.3GiB/s]
  • totony 9y ago
    benchmarking is useless when comparing different conditions ie your computer is probably faster than op's
    • naranha 9y ago
      Of course. Though 37MiB/s compared to 10Gi/s probably means that the GNU version used by Fedora has a faster implementation than Apple's version.
      • totony 9y ago
        You're right, my mistake, I didn't check the speed of the OP's
• nurettin 9y ago
  Rust strike force is back, rewriting Unix tools in a sane language
• aleden 9y ago
  This topic comes up every now and then. I thought this post was particularly insightful, "One thing to keep in mind when looking at GNU programs is that they're often intentionally written in an odd style to remove all questions of Unix copyright infringement at the time that they were written. The long-standing advice when writing GNU utilities used to be that if the program you were replacing was optimized for minimizing CPU use, write yours to minimize memory use, or vice-versa. Or in this case, if the program was optimized for simplicity, optimize for throughput. It would have been very easy for the nascent GNU project to unintentionally produce a line-by-line equivalent of BSD yes.c, which would have potentially landed them in the 80/90s equivalent of the Google v.s. Oracle case." https://news.ycombinator.com/item?id=14543640 https://news.ycombinator.com/item?id=14543640
• rurban 9y ago
  Not impressive at all. Basically he had to write a lot of manual buffering code to reach GNU yes throughput. I would suggest to use an infrastructure which already provides proper IO. e.g. perl or clisp. $ perl -C0 -e'print "y\n" x (1024*8) while 1' | pv > /dev/null ^C.4GiB 0:00:11 [6.17GiB/s] [ <=> ] $ yes | pv > /dev/null ^C.3GiB 0:00:07 [6.64GiB/s] [ <=> And with linux-only vmsplice the record stands at 123GB/s https://www.reddit.com/r/unix/comments/6gxduc/how_is_gnu_yes_so_fast/diua761/ https://www.reddit.com/r/unix/comments/6gxduc/how_is_gnu_yes...