4 ms·

Arbitrary or not, the limits had to be there due to small amount of memory computer had back then. I don’t think this is specific to Unix/C culture (unless it w

Arbitrary or not, the limits had to be there due to small amount of memory computer had back then. I don’t think this is specific to Unix/C culture (unless it was what influenced the design of MS-DOS with its limit on the length of file names, for example).

kragen 4y ago
you don't need megabytes to allocate your strings dynamically, microsoft basic did it on the altair in 4k "/bin/sh\0" or "\7\0/bin/sh" is less than 14 bytes, not more, even including a pointer or two the real issue is that anything you try to allocate dynamically can fail, so you need a failure path, and your failure path needs to reliably free all the other dynamically allocated resources early unix kind of sucked at handling failures a fixed-size buffer isn't more efficient but it is simpler ms-dos used cp/m's filesystem, which had fixed-size filename and ext fields, which i think it copied from rt-11, which i think got them from various pdp-8 systems fixed-size string buffers make more sense on a word-oriented machine, or arguably in a filesystem, where pointers require some kind of hack i'd like to blame radix50 but i don't think we can because then the limit would be 9.3 or 6.3 rather than 8.3. there have been some 6.3 filesystems (3 16-bit words per filename with rad50) but cp/m fat (and thus ms-dos) are not among them
- tinus_hn 4y ago
  A great and easy idea, as long as you never need to release anything. Otherwise you end up with holes in your memory and you need to do compaction, and reserve the memory you need to be able to do that.
  - kragen 4y ago
    or you can just have memory fragmentation compaction doesn't require a lot of memory but it's slow and intrusive; it has to know where all your buffer pointers are
    - tinus_hn 4y ago
      Obviously permanently wasting memory on fragmentation is not an option on a device that typically has 24kb memory. Especially not just so you can specify long filenames for the shell, when the filesystem doesn’t even support those filenames.
      - kragen 4y ago
        the alternative they chose was to permanently waste memory on fixed-size struct fields, so i guess it was actually an option malloc on the pdp-11 never compacted of course the filesystem supported 14 characters for the basename, but evidently we're talking about code with a 14-character limit on the pathname, not the basename
        tinus_hn 4y ago
        You don’t need compaction to reuse fixed size structures. I presume their priorities were with creating a working system, not one that answers to the ‘requirements’ of 50 years into the future.
        kragen 4y ago
        you don't need compaction to reuse heap-allocated memory either wasted memory due to heap fragmentation is very similar to wasted memory due to slack space in fixed-size structure fields
- Zardoz84 4y ago
  Perhaps I wrong, but FAT and CP/M filesystems, aren't the same. CP/M filesystem never had directories. When the original FAT even predates QDOS as was used by Microsoft on his BASIC in 1977
  - kragen 4y ago
    hmm, i thought fat came from cp/m, not basic-80, but it looks like i was wrong about that thanks for the correction https://www.informit.com/articles/article.aspx?p=25878&seqNum=3 https://www.informit.com/articles/article.aspx?p=25878&seqNu... fat didn't have subdirectories in qdos or ms-dos pre-2 either
xigoi 4y ago
I'd argue that in practical cases, resizable arrays require less memory than fixed-size arrays, because the actual length is usually going to be shorter than the arbitrary maximum length. My guess is that C encouraged using arbitrary limits by not having first-class support for resizable arrays (or even a non-hacky type-safe way to implement them as a library).
khiqxj 4y ago
the mode of thinking certainly hasnt changed since hardware improved, however
pjmlp 4y ago
Yet the computers designed one decade earlier were much better in security than UNIX culture, go figure.
- aap_ 4y ago
  Maybe because they were bigger?
  - pjmlp 4y ago
    Maybe because UNIX folks had more fun doing their stuff instead of actually do it properly? One decade older hardware was better than newly released PDP-11... Thankfully we have encyclopedias.
    - aap_ 4y ago
      Oh I bet fun also played a role. But I don't even know what exactly you are comparing against the PDP-11 here or and what point, because UNIX development happened on 11s for at least a decade. UNIX first ran on a PDP-11 in 1971 or so and a decade earlier there was only CTSS perhaps to compare it to, which had more memory than a PDP-11 at the time. Later timesharing systems like Multics and TOPS-10 also ran on bigger machines. And let's not forget it all began on a PDP-7, which was even smaller.
- deleted 4y ago
  [deleted]
electroly 4y ago
Agreed, I think it was just a matter of being originally developed at a time when memory was tight and CPU cycles were few. That is, if the original UNIX were developed today with otherwise the same development culture, I'm not sure that we'd see fixed length restrictions. One of GNU's original goals was eliminating arbitrary fixed size limits in UNIX tools, in exchange for using more memory. It's discussed in https://www.gnu.org/gnu/thegnuproject.en.html https://www.gnu.org/gnu/thegnuproject.en.html under "Technical goals". > But it was natural to apply the known standards of good practice to the work—for example, dynamically allocating data structures to avoid arbitrary fixed size limits, and handling all the possible 8-bit codes wherever that made sense. > In addition, we rejected the Unix focus on small memory size, by deciding not to support 16-bit machines (it was clear that 32-bit machines would be the norm by the time the GNU system was finished), and to make no effort to reduce memory usage unless it exceeded a megabyte.
akira2501 4y ago
The other advantage of fixed size buffers is if you want to write the records to disk you can just write it directly. You don't need to serialize or instantiate the strings when reading, you can just copy the bytes straight to or from the file. Looking at things like man utmp.5 should reveal what their thinking was.

kragen 4y ago

you don't need megabytes to allocate your strings dynamically, microsoft basic did it on the altair in 4k "/bin/sh\0" or "\7\0/bin/sh" is less than 14 bytes, not more, even including a pointer or two the real issue is that anything you try to allocate dynamically can fail, so you need a failure path, and your failure path needs to reliably free all the other dynamically allocated resources early unix kind of sucked at handling failures a fixed-size buffer isn't more efficient but it is simpler ms-dos used cp/m's filesystem, which had fixed-size filename and ext fields, which i think it copied from rt-11, which i think got them from various pdp-8 systems fixed-size string buffers make more sense on a word-oriented machine, or arguably in a filesystem, where pointers require some kind of hack i'd like to blame radix50 but i don't think we can because then the limit would be 9.3 or 6.3 rather than 8.3. there have been some 6.3 filesystems (3 16-bit words per filename with rad50) but cp/m fat (and thus ms-dos) are not among them

tinus_hn 4y ago
A great and easy idea, as long as you never need to release anything. Otherwise you end up with holes in your memory and you need to do compaction, and reserve the memory you need to be able to do that.
- kragen 4y ago
  or you can just have memory fragmentation compaction doesn't require a lot of memory but it's slow and intrusive; it has to know where all your buffer pointers are
  - tinus_hn 4y ago
    Obviously permanently wasting memory on fragmentation is not an option on a device that typically has 24kb memory. Especially not just so you can specify long filenames for the shell, when the filesystem doesn’t even support those filenames.
    - kragen 4y ago
      the alternative they chose was to permanently waste memory on fixed-size struct fields, so i guess it was actually an option malloc on the pdp-11 never compacted of course the filesystem supported 14 characters for the basename, but evidently we're talking about code with a 14-character limit on the pathname, not the basename
      - tinus_hn 4y ago
        You don’t need compaction to reuse fixed size structures. I presume their priorities were with creating a working system, not one that answers to the ‘requirements’ of 50 years into the future.
        kragen 4y ago
        you don't need compaction to reuse heap-allocated memory either wasted memory due to heap fragmentation is very similar to wasted memory due to slack space in fixed-size structure fields
Zardoz84 4y ago
Perhaps I wrong, but FAT and CP/M filesystems, aren't the same. CP/M filesystem never had directories. When the original FAT even predates QDOS as was used by Microsoft on his BASIC in 1977
- kragen 4y ago
  hmm, i thought fat came from cp/m, not basic-80, but it looks like i was wrong about that thanks for the correction https://www.informit.com/articles/article.aspx?p=25878&seqNum=3 https://www.informit.com/articles/article.aspx?p=25878&seqNu... fat didn't have subdirectories in qdos or ms-dos pre-2 either

xigoi 4y ago

I'd argue that in practical cases, resizable arrays require less memory than fixed-size arrays, because the actual length is usually going to be shorter than the arbitrary maximum length. My guess is that C encouraged using arbitrary limits by not having first-class support for resizable arrays (or even a non-hacky type-safe way to implement them as a library).

khiqxj 4y ago

the mode of thinking certainly hasnt changed since hardware improved, however

pjmlp 4y ago

Yet the computers designed one decade earlier were much better in security than UNIX culture, go figure.

aap_ 4y ago
Maybe because they were bigger?
- pjmlp 4y ago
  Maybe because UNIX folks had more fun doing their stuff instead of actually do it properly? One decade older hardware was better than newly released PDP-11... Thankfully we have encyclopedias.
  - aap_ 4y ago
    Oh I bet fun also played a role. But I don't even know what exactly you are comparing against the PDP-11 here or and what point, because UNIX development happened on 11s for at least a decade. UNIX first ran on a PDP-11 in 1971 or so and a decade earlier there was only CTSS perhaps to compare it to, which had more memory than a PDP-11 at the time. Later timesharing systems like Multics and TOPS-10 also ran on bigger machines. And let's not forget it all began on a PDP-7, which was even smaller.
deleted 4y ago
[deleted]

electroly 4y ago

Agreed, I think it was just a matter of being originally developed at a time when memory was tight and CPU cycles were few. That is, if the original UNIX were developed today with otherwise the same development culture, I'm not sure that we'd see fixed length restrictions. One of GNU's original goals was eliminating arbitrary fixed size limits in UNIX tools, in exchange for using more memory. It's discussed in https://www.gnu.org/gnu/thegnuproject.en.html https://www.gnu.org/gnu/thegnuproject.en.html under "Technical goals". > But it was natural to apply the known standards of good practice to the work—for example, dynamically allocating data structures to avoid arbitrary fixed size limits, and handling all the possible 8-bit codes wherever that made sense. > In addition, we rejected the Unix focus on small memory size, by deciding not to support 16-bit machines (it was clear that 32-bit machines would be the norm by the time the GNU system was finished), and to make no effort to reduce memory usage unless it exceeded a megabyte.

akira2501 4y ago

The other advantage of fixed size buffers is if you want to write the records to disk you can just write it directly. You don't need to serialize or instantiate the strings when reading, you can just copy the bytes straight to or from the file. Looking at things like man utmp.5 should reveal what their thinking was.