5 ms·

Oh, no, now I have to go dig out some of mine.... The first really big one I wrote was the ~7000 line installer for the Enrust CA and directory, which ran on,

Oh, no, now I have to go dig out some of mine.... The first really big one I wrote was the ~7000 line installer for the Enrust CA and directory, which ran on, well, all Unixes at that time. It didn't initially, of course, but it grew with customer demand. The installation itself wasn't especially complicated, but upgrades were, a little, and this was back when every utility on every Unix had slight variations. Much of the script was figuring out and managing those differences, much was error detection and recovery and rollback, some was a very primitive form of package and dependency management.... DEC's Unix (the other one, not Ultrix) was the most baffling. It took me days to realize that all command line utilities truncated their output at column width. Every single one. Over 30 years later and that one still stands out. Every release of HP-UX had breaking changes, and we covered 6.5 to 11, IIRC. I barely remember Ultrix or the Novell one or Next, or Sequent. I do remember AIX as being weird but I don't remember why. And of course even Sun's three/four OS's had their differences (SunOS pre 4.1.3; 4.1.3; Solaris pre 2; and 2+) but they had great FMs. The best.

emmelaich 2y ago
That column truncation sounds bizarre. Are you sure the terminal didn't have some sort of sideways scroll available?
- dspillett 2y ago
  I think he was meaning that they truncated the lines even when called from a script, with their output going somewhere other than a terminal, not just when run interactively
  - raffraffraff 2y ago
    :O
  - emmelaich 2y ago
    Yep, but I'm curious enough to quiz it. Weirdly, today I ran wish in MacOS Sequoia (15.1.x) and had the (exception) output truncated at terminal width!
    - PeterWhittaker 2y ago
      dspillett was exactly right: ps, e.g., truncated its output at $COLUMNS and there was no horizontal scroll. As suggested above, it did this even when called from a script. The fix was easy, set COLUMNS ridiculously large if DEC Unix, but it took days of WTF apparent UB before I realized how simple was what was happening. It just seemed haphazard: I'd reposition and resize a window so I could run the script in one while manually running the commands in another, get inconsistent results, rinse, repeat... ...and eventually realize the common element in each test was me, and the variations I was introducing were window size. I cursed their engineers for trying to be "helpful" and keep things "pretty".
    - p_l 2y ago
      Because macOS closest relative isn't Free/NetBSD, but OSF/1 which, under few different names, was sold by Digital as Unix for Alpha (there were few rare builds for MIPS too).
      - emmelaich 2y ago
        Interesting, I thought MacOS was basically a FreeBSD variant. But I just tried it again on a resized terminal window and I couldn't reproduce it!
        p_l 2y ago
        OSX took in a bit of FreeBSD and NetBSD to modernise sons areas, but the main thing for 10.0 release involved pulling in the latest OSFMK. Also, several APIs introduced after BSD 4.4 are very visibly missing, pointing to how little was taken from Free/NetBSD
      - skissane 2y ago
        > Because macOS closest relative isn't Free/NetBSD, but OSF/1 What you say here contains some truth (certainly with respect to the kernel), but I doubt that has anything to do with the behaviour the grandparent is reporting–the wish command in Tcl/Tk truncating output at terminal width. That behaviour would be determined by the Tcl/Tk code, nothing inherently to do with the underlying OS. > but OSF/1 which, under few different names, was sold by Digital as Unix for Alpha (there were few rare builds for MIPS too). IBM also briefly sold a port of OSF/1 to IBM mainframes, AIX/ESA: it was available to customers in June 1992, and withdrawn from marketing in June 1993–it was discontinued so quickly due to lack of customer interest, and also because IBM was about to release (in 1994) a UNIX compatibility subsystem for MVS (OpenEdition), which was a more attractive UNIX option for many of their mainframe customers. I believe IBM's abortive Workplace OS – shipped in beta form only as OS/2 PowerPC Edition – was also partially derived from the OSF/1 code base.
  - nikau 2y ago
    If he is talking about osf1/tru64 that's the first one heard of it
throw16180339 2y ago
> DEC's Unix (the other one, not Ultrix) was the most baffling. It took me days to realize that all command line utilities truncated their output at column width. Every single one. Over 30 years later and that one still stands out. Do you mean OSF1/, Digital Unix, or Tru64 Unix?
- PeterWhittaker 2y ago
  Oh, yes, I think it was Digital Unix. IIRC, we toyed with OSF/1, but there wasn't much call for it.
  - p_l 2y ago
    OSF/1, Digital Unix, and Tru64 are the same OS at different points in time. Technically OSF/1 was supposed to be the commercial BSD answer to System V, in practice only several niche vendors used it, plus Digital and NeXT (and through NeXT, Apple which continues the line to this day)
    - PeterWhittaker 2y ago
      Thanks for the clarification. It was so long ago, it’s all a bit hazy. Other than the COLUMNS thing. That is burnt into my memory forever.
banku_brougham 2y ago
Thank you for your service, Im so glad you could share. Id be interested to read more.
raffraffraff 2y ago
I made it to thousands but more like 2000. At least I only had to support Redhat and Ubuntu (modern ones, at that)
PeterWhittaker 2y ago
OK, so JOOC I ran wc against the main binary and supporting libraries for a project I did last year: It's a script to manage linear assured pipelines implemented as a series of containers (an input protocol adapter, one or more filters, an output protocol adapter). The whole thing was intended to be useful for people who aren't necessarily experts in either containers or protocols, but who have an idea of how they want to filter/transform files as they transit the pipeline. It's 6224 lines, so far. There is a top-level binary with sub-functions, sort of like how git [ git options ] < git action> [action options] or systemctl [etc.[ work. There is a sub command to add a new sub command, which creates the necessary libraries and pre-populates function definitions from a template; the template includes short and long usage functions, so that cbap -h or cbap pipeline -h give useful and reasonable advice. There are subcommands for manipulating base images, components (which are images with specific properties for use as containers in the pipelines), and pipelines themselves. A LOT of code is for testing, to make sure that the component and pipeline definitions are correctly formatted. (Pipelines are specified in something-almost-TOML, so there is code to parse toml, convert sections to arrays, etc., while components are specified as simple key=value files, so there is code to parse those, extract LHS and RHS, perform schema validation, etc.). Since pipeline components can share properties, there is code to find common properties in var and etc files, specify component properties, etc. There are a lot user and group and directory and FIFO manipulation functions tailored to the security requirements: When a pipeline is setup, users and groups and SEL types and MCS categories are generated and applied, then mapped into to the service files that start the components (so there is a lot of systemd manipulation as well). Probably the single biggest set of calls are the functions that get/set component properties (which are really container properties) and allow us to use data-driven container definitions, with each property having a get function, a validation function, and an inline (in a pipeline) version, for maximum flexibility. Finally, there is code that uses a lot of bash references to set variables either from files, the environment, or the command line, so that we can test rapidly. It also support four levels of user, from maintainer (people who work on the code itself), developer (people who develop component definitions), integrators (people who build pipelines from components), and operators (people who install pipelines), with the ability to copy and package itself for export to users at any of those levels (there is a lot of data-driven, limited recursive stuff happening therein). Since target systems can be any Linux, it uses makeself to package and extract itself. For example, an integrator can create a pipeline definition, which will produce a makeself file that, when run on the target system, will create all users, groups, directories, FIFOs (the inter-component IPC), apply DAC and MAC, create systemd files, copy images to each user, and launch the pipeline - with a delete option to undo all of that. There is some seccomp in there as well, but we've paused that as we need to find the right balance between allow- and deny- listing. (Yes, I use shellcheck. Religiously. :->)

emmelaich 2y ago

That column truncation sounds bizarre. Are you sure the terminal didn't have some sort of sideways scroll available?

dspillett 2y ago
I think he was meaning that they truncated the lines even when called from a script, with their output going somewhere other than a terminal, not just when run interactively
- raffraffraff 2y ago
  :O
- emmelaich 2y ago
  Yep, but I'm curious enough to quiz it. Weirdly, today I ran wish in MacOS Sequoia (15.1.x) and had the (exception) output truncated at terminal width!
  - PeterWhittaker 2y ago
    dspillett was exactly right: ps, e.g., truncated its output at $COLUMNS and there was no horizontal scroll. As suggested above, it did this even when called from a script. The fix was easy, set COLUMNS ridiculously large if DEC Unix, but it took days of WTF apparent UB before I realized how simple was what was happening. It just seemed haphazard: I'd reposition and resize a window so I could run the script in one while manually running the commands in another, get inconsistent results, rinse, repeat... ...and eventually realize the common element in each test was me, and the variations I was introducing were window size. I cursed their engineers for trying to be "helpful" and keep things "pretty".
  - p_l 2y ago
    Because macOS closest relative isn't Free/NetBSD, but OSF/1 which, under few different names, was sold by Digital as Unix for Alpha (there were few rare builds for MIPS too).
    - emmelaich 2y ago
      Interesting, I thought MacOS was basically a FreeBSD variant. But I just tried it again on a resized terminal window and I couldn't reproduce it!
      - p_l 2y ago
        OSX took in a bit of FreeBSD and NetBSD to modernise sons areas, but the main thing for 10.0 release involved pulling in the latest OSFMK. Also, several APIs introduced after BSD 4.4 are very visibly missing, pointing to how little was taken from Free/NetBSD
    - skissane 2y ago
      > Because macOS closest relative isn't Free/NetBSD, but OSF/1 What you say here contains some truth (certainly with respect to the kernel), but I doubt that has anything to do with the behaviour the grandparent is reporting–the wish command in Tcl/Tk truncating output at terminal width. That behaviour would be determined by the Tcl/Tk code, nothing inherently to do with the underlying OS. > but OSF/1 which, under few different names, was sold by Digital as Unix for Alpha (there were few rare builds for MIPS too). IBM also briefly sold a port of OSF/1 to IBM mainframes, AIX/ESA: it was available to customers in June 1992, and withdrawn from marketing in June 1993–it was discontinued so quickly due to lack of customer interest, and also because IBM was about to release (in 1994) a UNIX compatibility subsystem for MVS (OpenEdition), which was a more attractive UNIX option for many of their mainframe customers. I believe IBM's abortive Workplace OS – shipped in beta form only as OS/2 PowerPC Edition – was also partially derived from the OSF/1 code base.
- nikau 2y ago
  If he is talking about osf1/tru64 that's the first one heard of it

throw16180339 2y ago

> DEC's Unix (the other one, not Ultrix) was the most baffling. It took me days to realize that all command line utilities truncated their output at column width. Every single one. Over 30 years later and that one still stands out. Do you mean OSF1/, Digital Unix, or Tru64 Unix?

PeterWhittaker 2y ago
Oh, yes, I think it was Digital Unix. IIRC, we toyed with OSF/1, but there wasn't much call for it.
- p_l 2y ago
  OSF/1, Digital Unix, and Tru64 are the same OS at different points in time. Technically OSF/1 was supposed to be the commercial BSD answer to System V, in practice only several niche vendors used it, plus Digital and NeXT (and through NeXT, Apple which continues the line to this day)
  - PeterWhittaker 2y ago
    Thanks for the clarification. It was so long ago, it’s all a bit hazy. Other than the COLUMNS thing. That is burnt into my memory forever.

banku_brougham 2y ago

Thank you for your service, Im so glad you could share. Id be interested to read more.

raffraffraff 2y ago

I made it to thousands but more like 2000. At least I only had to support Redhat and Ubuntu (modern ones, at that)

PeterWhittaker 2y ago

OK, so JOOC I ran wc against the main binary and supporting libraries for a project I did last year: It's a script to manage linear assured pipelines implemented as a series of containers (an input protocol adapter, one or more filters, an output protocol adapter). The whole thing was intended to be useful for people who aren't necessarily experts in either containers or protocols, but who have an idea of how they want to filter/transform files as they transit the pipeline. It's 6224 lines, so far. There is a top-level binary with sub-functions, sort of like how git [ git options ] < git action> [action options] or systemctl [etc.[ work. There is a sub command to add a new sub command, which creates the necessary libraries and pre-populates function definitions from a template; the template includes short and long usage functions, so that cbap -h or cbap pipeline -h give useful and reasonable advice. There are subcommands for manipulating base images, components (which are images with specific properties for use as containers in the pipelines), and pipelines themselves. A LOT of code is for testing, to make sure that the component and pipeline definitions are correctly formatted. (Pipelines are specified in something-almost-TOML, so there is code to parse toml, convert sections to arrays, etc., while components are specified as simple key=value files, so there is code to parse those, extract LHS and RHS, perform schema validation, etc.). Since pipeline components can share properties, there is code to find common properties in var and etc files, specify component properties, etc. There are a lot user and group and directory and FIFO manipulation functions tailored to the security requirements: When a pipeline is setup, users and groups and SEL types and MCS categories are generated and applied, then mapped into to the service files that start the components (so there is a lot of systemd manipulation as well). Probably the single biggest set of calls are the functions that get/set component properties (which are really container properties) and allow us to use data-driven container definitions, with each property having a get function, a validation function, and an inline (in a pipeline) version, for maximum flexibility. Finally, there is code that uses a lot of bash references to set variables either from files, the environment, or the command line, so that we can test rapidly. It also support four levels of user, from maintainer (people who work on the code itself), developer (people who develop component definitions), integrators (people who build pipelines from components), and operators (people who install pipelines), with the ability to copy and package itself for export to users at any of those levels (there is a lot of data-driven, limited recursive stuff happening therein). Since target systems can be any Linux, it uses makeself to package and extract itself. For example, an integrator can create a pipeline definition, which will produce a makeself file that, when run on the target system, will create all users, groups, directories, FIFOs (the inter-component IPC), apply DAC and MAC, create systemd files, copy images to each user, and launch the pipeline - with a delete option to undo all of that. There is some seccomp in there as well, but we've paused that as we need to find the right balance between allow- and deny- listing. (Yes, I use shellcheck. Religiously. :->)