4 ms·

Clearly autoconf isn't so unremittingly horrible that he doesn't use it in Varnish :) autoconf is a bit of a Faustian bargain though - it promises a quick and

by djmdjm 16y ago

Clearly autoconf isn't so unremittingly horrible that he doesn't use it in Varnish :) autoconf is a bit of a Faustian bargain though - it promises a quick and easy solution to the labour-intensive and mindlessly boring task of managing all the microscopic differences between (and within!) Unix variants. Unfortunately it brings its own complexity, fragility and bugs along with it. It also encourages a programming style best described as "#ifdef hell" which is just poison to good maintainability.

cperciva 16y ago
Clearly autoconf isn't so unremittingly horrible that he doesn't use it in Varnish :) Autocrap is horrible in two ways: First, it's horrible to work with; and second, it's responsible for making any attempt to avoid it horribly painful. If autocrap didn't exist, people would write operating systems without so many gratuitous incompatibilities. What possible excuse is there for the fact that many linux distributions need a special compiler flag in order to support files greater than 4 GB?
- djmdjm 16y ago
  In this case, I think the egg (autoconf) arrived on the scene well after the chicken (idiotic, gratuitous incompatibilities).
  - davidw 16y ago
    Isn't it a 90ies thing, when there were not just a few different Linux distributions, but several competing Unix systems? Solaris, IRIX, HPUX, AIX, Digital Unix, plus Linux, *BSD, and various and sundry other minor players.
    - jacquesm 16y ago
      There still are several competing UNIX flavors: - bsd and derivates (including OS/X) - linux (64 and 32 bit sub-flavors) - QnX (though very different under the hood) And lots of others. So 'it's an 80's thing' but it will be very hard to get rid of, 'just a few different linux distributions' does not describe the situation accurately. It also isn't rare to find older boxes still chugging away quietly at some task in a corner somewhere running any one of those older operating systems and those sometimes need software compiled for them as well.
      - davidw 16y ago
        What I meant is that autoconf was created in the 1990ies.
    - avar 16y ago
      Those systems are still around, and people are still porting software to them. For instance Git which is a fairly popular package created in 2005 runs on at least: Linux, GNU, GNU/Hurd, GNU/Linux, Tru64 UNIX, GNU/kFreeBSD, SCO UnixWare, SCO UNIX SV, Darwin, various versions of Solaris, Cygwin, *BSD, AIX, IRIX, IRIX64, HP-UX, Windows, MINGW. Older packages like Perl and Emacs compile on a much wider range of systems.
  - cperciva 16y ago
    Absolutely. And I'd say that in the 90s autocrap was probably a net positive. But times have changed. Windows is now more compatible with Linux than a random pair of mid-90s UNIXes were with each other. Now that the major incompatibilities gone, all autocrap is doing is keeping the lesser incompatibilities around.
    - avar 16y ago
      The Windows compilers are more capable than old 90s compilers. But random source code written for Linux is a lot more compatible as-is with AIX, HP-UX, IRIX and others than with Windows. Windows doesn't have forks (needs emulation via threads), second class support for things like symlinks, and isn't POSIX compatible (yes, I know you can get that as a separate install). Porting a Linux program to Windows is like porting it to Novell, or some random mainfram OS. Not another OS in the Unix family.
  - thyrsus 16y ago
    Just yesterday I was trying to get the macutils package built (http://www.ibiblio.org/pub/Linux/utils/compress/macutils.tar.gz http://www.ibiblio.org/pub/Linux/utils/compress/macutils.tar...) and wishing they had used autoconf instead of ad hac #ifdef'ing for each arbitrary environment they had encountered - which never happened to be standard C. The package is apparently unmaintained, and if it weren't for tracking down the license provenances(!), I'd be tempted to apply some autoconf love. Varnish guy also left HP-UX off his list of still-kicking Unix implementations. The continuing exponential growth of computer use hides the fact for many that there are still an enormous number of "non-mainstream" systems still in essential use. Within the last couple weeks, someone on the autoconf list just asked for help supporting AS400. Kids these days. Sheesh. ;-)
jacquesm 16y ago
For the record, the 'varnish guy' is also one of the main people behind FreeBSD: http://people.freebsd.org/~phk/ http://people.freebsd.org/~phk/ Varnish is a pretty good example of how far you can push a unix box in terms of network performance, one site I work for/with serves up billions of images daily using varnish without a single hickup. If phk criticizes something it's usually worth listening to, he really knows his stuff.
- gruseom 16y ago
  I agree. His recent ACM article is an example: provocative and even unpleasant, but making a point which, if correct, could hardly be more important; and I think it is probably correct. What do you think it is that makes Varnish so fast? I've read the article (http://varnish-cache.org/wiki/ArchitectNotes http://varnish-cache.org/wiki/ArchitectNotes) but I'm curious to hear about it from people's experience. Sometimes the stated reasons for something being great are not the same as the real reasons.
  - jacquesm 16y ago
    Using system calls the way they are intended to and building a server the way UNIX is meant to be used. It's a textbook example of how to do a thing like this.
    - gruseom 16y ago
      Fair enough, but I want to know the key principles about how these are intended to be used. Or are you saying I should just read Stevens? The major point I took from the articles is that you should let the kernel manage what's in RAM -- it's going to do it anyway, so don't fight it -- and instead of building complicated paging and caching mechanisms, focus on laying out your data to take advantage of how the kernel already does it. What I want to know is, what are the other such major points? p.s. Good luck tomorrow :)
  - akkartik 16y ago
    Ah, here's the article: http://queue.acm.org/detail.cfm?id=1814327 http://queue.acm.org/detail.cfm?id=1814327 Especially interesting to me because I spent 5 years on memory issues in grad school, steeped in the idea that number of comparisons or number of instructions per unit input is often a red herring. Unfortunately the alternative is a much more complex model to juggle in your head.

by djmdjm 16y ago

cperciva 16y ago

Clearly autoconf isn't so unremittingly horrible that he doesn't use it in Varnish :) Autocrap is horrible in two ways: First, it's horrible to work with; and second, it's responsible for making any attempt to avoid it horribly painful. If autocrap didn't exist, people would write operating systems without so many gratuitous incompatibilities. What possible excuse is there for the fact that many linux distributions need a special compiler flag in order to support files greater than 4 GB?

djmdjm 16y ago
In this case, I think the egg (autoconf) arrived on the scene well after the chicken (idiotic, gratuitous incompatibilities).
- davidw 16y ago
  Isn't it a 90ies thing, when there were not just a few different Linux distributions, but several competing Unix systems? Solaris, IRIX, HPUX, AIX, Digital Unix, plus Linux, *BSD, and various and sundry other minor players.
  - jacquesm 16y ago
    There still are several competing UNIX flavors: - bsd and derivates (including OS/X) - linux (64 and 32 bit sub-flavors) - QnX (though very different under the hood) And lots of others. So 'it's an 80's thing' but it will be very hard to get rid of, 'just a few different linux distributions' does not describe the situation accurately. It also isn't rare to find older boxes still chugging away quietly at some task in a corner somewhere running any one of those older operating systems and those sometimes need software compiled for them as well.
    - davidw 16y ago
      What I meant is that autoconf was created in the 1990ies.
  - avar 16y ago
    Those systems are still around, and people are still porting software to them. For instance Git which is a fairly popular package created in 2005 runs on at least: Linux, GNU, GNU/Hurd, GNU/Linux, Tru64 UNIX, GNU/kFreeBSD, SCO UnixWare, SCO UNIX SV, Darwin, various versions of Solaris, Cygwin, *BSD, AIX, IRIX, IRIX64, HP-UX, Windows, MINGW. Older packages like Perl and Emacs compile on a much wider range of systems.
- cperciva 16y ago
  Absolutely. And I'd say that in the 90s autocrap was probably a net positive. But times have changed. Windows is now more compatible with Linux than a random pair of mid-90s UNIXes were with each other. Now that the major incompatibilities gone, all autocrap is doing is keeping the lesser incompatibilities around.
  - avar 16y ago
    The Windows compilers are more capable than old 90s compilers. But random source code written for Linux is a lot more compatible as-is with AIX, HP-UX, IRIX and others than with Windows. Windows doesn't have forks (needs emulation via threads), second class support for things like symlinks, and isn't POSIX compatible (yes, I know you can get that as a separate install). Porting a Linux program to Windows is like porting it to Novell, or some random mainfram OS. Not another OS in the Unix family.
- thyrsus 16y ago
  Just yesterday I was trying to get the macutils package built (http://www.ibiblio.org/pub/Linux/utils/compress/macutils.tar.gz http://www.ibiblio.org/pub/Linux/utils/compress/macutils.tar...) and wishing they had used autoconf instead of ad hac #ifdef'ing for each arbitrary environment they had encountered - which never happened to be standard C. The package is apparently unmaintained, and if it weren't for tracking down the license provenances(!), I'd be tempted to apply some autoconf love. Varnish guy also left HP-UX off his list of still-kicking Unix implementations. The continuing exponential growth of computer use hides the fact for many that there are still an enormous number of "non-mainstream" systems still in essential use. Within the last couple weeks, someone on the autoconf list just asked for help supporting AS400. Kids these days. Sheesh. ;-)

jacquesm 16y ago

For the record, the 'varnish guy' is also one of the main people behind FreeBSD: http://people.freebsd.org/~phk/ http://people.freebsd.org/~phk/ Varnish is a pretty good example of how far you can push a unix box in terms of network performance, one site I work for/with serves up billions of images daily using varnish without a single hickup. If phk criticizes something it's usually worth listening to, he really knows his stuff.

gruseom 16y ago
I agree. His recent ACM article is an example: provocative and even unpleasant, but making a point which, if correct, could hardly be more important; and I think it is probably correct. What do you think it is that makes Varnish so fast? I've read the article (http://varnish-cache.org/wiki/ArchitectNotes http://varnish-cache.org/wiki/ArchitectNotes) but I'm curious to hear about it from people's experience. Sometimes the stated reasons for something being great are not the same as the real reasons.
- jacquesm 16y ago
  Using system calls the way they are intended to and building a server the way UNIX is meant to be used. It's a textbook example of how to do a thing like this.
  - gruseom 16y ago
    Fair enough, but I want to know the key principles about how these are intended to be used. Or are you saying I should just read Stevens? The major point I took from the articles is that you should let the kernel manage what's in RAM -- it's going to do it anyway, so don't fight it -- and instead of building complicated paging and caching mechanisms, focus on laying out your data to take advantage of how the kernel already does it. What I want to know is, what are the other such major points? p.s. Good luck tomorrow :)
- akkartik 16y ago
  Ah, here's the article: http://queue.acm.org/detail.cfm?id=1814327 http://queue.acm.org/detail.cfm?id=1814327 Especially interesting to me because I spent 5 years on memory issues in grad school, steeped in the idea that number of comparisons or number of instructions per unit input is often a red herring. Unfortunately the alternative is a much more complex model to juggle in your head.