4 ms·

> A static library is nothing but a bundle of object files, wrapped in an ancient and never-quite-standardized archive format. To this day I'm astonished that

by Joker_vD 11mo ago

> A static library is nothing but a bundle of object files, wrapped in an ancient and never-quite-standardized archive format. To this day I'm astonished that it's not just tar. Or pax. Or even cpio! Or literally any file format that has any other use.

xyzzy_plugh 11mo ago
tar/pax are kind of terrible formats. They are hard to implement correctly. I'm glad they are not used more often. cpio is pretty reasonable though. zip is actually pretty great and I've been growing increasingly fond of it over the years.
- Joker_vD 11mo ago
  The thing is, there is always tar(1) even in the most basic of distributions. And everyone uses tar.gz's or .bz2's or whatever for distributing all kinds of things, so tar is pretty ubiquitous. But the moment you want to do some C development, or any binutils-related, nope, install and use ar(1) which is used for literally one single purpose and nothing else. Because reasons.
  - hyperman1 11mo ago
    Im not sure how ar does it, but tar has no centralised directory. The only way to get file 100 is to walk trough the 99 files before. This kills random access speed.
    - Joker_vD 11mo ago
      Ar puts a file called "/" as the first file of the archive. Inside, there is a number N, then a list of N file offsets, and then a list of N null-terminated strings. It's a symbol table of sorts: each null-terminated string is a symbol name, and the corresponding file offset points at the archive header for the object file that contains the symbol. The filenames themselves are not recorded centrally since it's not really needed.
  - kragen 11mo ago
    At the time, ar existed and tar didn't. Tar came later.
    - Joker_vD 11mo ago
      At the time, a.out existed and COFF and ELF didn't. The switch from STABS to DWARF has also never happened, right?
- yjftsjthsd-h 11mo ago
  > tar/pax are kind of terrible formats. They are hard to implement correctly. I'm glad they are not used more often. I'll grant you "kind of terrible", but what's hard to correctly implement about tar? It's just a bunch of files concatenated together with a tiny chunk of metadata stuck on the front of each.
  - electroly 11mo ago
    Having never done it myself, I don't know, but I do know that the "microtar" library I picked up off GitHub is buggy when expanding GNU Tar archives but perfect when expanding its own archives. Correctly creating one valid archive is a lot easier than reliably extracting all valid archives. The code appeared competent, I assume tar just has a bunch of historical baggage that you can get wrong or fail to implement.
sureglymop 11mo ago
Wasn't there first 'ar' as a general purpose archiver and later 'tar', "tape archiver"? Why do you find it surprising that the archive format from that time was used to archive a bunch of files? I wasn't alive but I'm pretty sure ar wasn't only used for this purpose in unix.
- Joker_vD 11mo ago
  Because e.g. a.out didn't survive and was replaced? Several times, even?
  - sureglymop 11mo ago
    That is a good point. Although I can only guess, it intuitively makes sense to me why the executable format would more quickly evolve than the archive format which "just works" still.
- electroly 11mo ago
  > Why do you find it surprising that the archive format from that time was used to archive a bunch of files? It's surprising because we still use it today, not because it was used at the time.
  - 1718627440 11mo ago
    Why would we break backwards compatibility, when it works just fine?
    - electroly 11mo ago
      This response answers too much--why did we invent and adopt tar for all other use cases, then? Why do we not still use ar for its original general archiving purpose in any other situation but this one? Obviously there is some reason! I'm curious what that reason is. You can make regular archives with ar today but we don't--"it still works" wasn't good enough reason for any other archive use case. What are the general archiving requirements that don't exist for object archives that meant ar is still fine for this one use case?

xyzzy_plugh 11mo ago

tar/pax are kind of terrible formats. They are hard to implement correctly. I'm glad they are not used more often. cpio is pretty reasonable though. zip is actually pretty great and I've been growing increasingly fond of it over the years.

Joker_vD 11mo ago
The thing is, there is always tar(1) even in the most basic of distributions. And everyone uses tar.gz's or .bz2's or whatever for distributing all kinds of things, so tar is pretty ubiquitous. But the moment you want to do some C development, or any binutils-related, nope, install and use ar(1) which is used for literally one single purpose and nothing else. Because reasons.
- hyperman1 11mo ago
  Im not sure how ar does it, but tar has no centralised directory. The only way to get file 100 is to walk trough the 99 files before. This kills random access speed.
  - Joker_vD 11mo ago
    Ar puts a file called "/" as the first file of the archive. Inside, there is a number N, then a list of N file offsets, and then a list of N null-terminated strings. It's a symbol table of sorts: each null-terminated string is a symbol name, and the corresponding file offset points at the archive header for the object file that contains the symbol. The filenames themselves are not recorded centrally since it's not really needed.
- kragen 11mo ago
  At the time, ar existed and tar didn't. Tar came later.
  - Joker_vD 11mo ago
    At the time, a.out existed and COFF and ELF didn't. The switch from STABS to DWARF has also never happened, right?
yjftsjthsd-h 11mo ago
> tar/pax are kind of terrible formats. They are hard to implement correctly. I'm glad they are not used more often. I'll grant you "kind of terrible", but what's hard to correctly implement about tar? It's just a bunch of files concatenated together with a tiny chunk of metadata stuck on the front of each.
- electroly 11mo ago
  Having never done it myself, I don't know, but I do know that the "microtar" library I picked up off GitHub is buggy when expanding GNU Tar archives but perfect when expanding its own archives. Correctly creating one valid archive is a lot easier than reliably extracting all valid archives. The code appeared competent, I assume tar just has a bunch of historical baggage that you can get wrong or fail to implement.

sureglymop 11mo ago

Wasn't there first 'ar' as a general purpose archiver and later 'tar', "tape archiver"? Why do you find it surprising that the archive format from that time was used to archive a bunch of files? I wasn't alive but I'm pretty sure ar wasn't only used for this purpose in unix.

Joker_vD 11mo ago
Because e.g. a.out didn't survive and was replaced? Several times, even?
- sureglymop 11mo ago
  That is a good point. Although I can only guess, it intuitively makes sense to me why the executable format would more quickly evolve than the archive format which "just works" still.
electroly 11mo ago
> Why do you find it surprising that the archive format from that time was used to archive a bunch of files? It's surprising because we still use it today, not because it was used at the time.
- 1718627440 11mo ago
  Why would we break backwards compatibility, when it works just fine?
  - electroly 11mo ago
    This response answers too much--why did we invent and adopt tar for all other use cases, then? Why do we not still use ar for its original general archiving purpose in any other situation but this one? Obviously there is some reason! I'm curious what that reason is. You can make regular archives with ar today but we don't--"it still works" wasn't good enough reason for any other archive use case. What are the general archiving requirements that don't exist for object archives that meant ar is still fine for this one use case?