4 ms·

Looks like the LOC folks are also unaware of the existence of HyperCard clones such as SuperCard (a compiler) and subsequently Runtime Revolution (now discontin

Looks like the LOC folks are also unaware of the existence of HyperCard clones such as SuperCard (a compiler) and subsequently Runtime Revolution (now discontinued): before the company stopped selling its end-product cross-platform HyperCard clone there was a brief open source foray, and some folks appear to still be trying to develop an open source XTalk platform (XTalk is the generalized descendant language of HyperTalk): https://openxtalk.org/forum/ https://openxtalk.org/forum/ https://en.wikipedia.org/wiki/XTalk https://en.wikipedia.org/wiki/XTalk ... They also don't seem to know about how files on an HFS filesystem all have separate resource/data forks: it's a distinctly different paradigm from the familiar a-file-is-a-stream-of-bytes convention of Linux/UNIX/NTFS, and some of their problem seems to arise because they're looking at a "flattened" HFS file that has been squished to fit on a non-HFS filesystem. (Modern macOS makes almost no use of resource forks, so they're vestigial in HFS+; not sure APFS even provides them.)

GeekyBear 5y ago
The original version of Windows NT Server allowed you to install Services for Macintosh and act as an AppleTalk file server, so support for resource forks was added to NTFS. https://en.wikipedia.org/wiki/NTFS#Alternate_data_stream_(ADS) https://en.wikipedia.org/wiki/NTFS#Alternate_data_stream_(AD...
- p_l 5y ago
  Alternate Data Streams are pretty interesting, cause I'm pretty sure Services for Macintosh weren't the only users - it's also way more powerful (closest thing on Unix would be Solaris-style extended attributes) and pretty fundamental part of NTFS on-disk structures (in fact, "alternate" data streams differ from main one only by having a name - the default data stream simply has no name, and all data streams are equal in importance)
  - retrac 5y ago
    Extended attributes aren't limited just to Solaris. A variety of *nix file systems have generally supported forks/multiple data streams for some time. FreeBSD has support for arbitrary-sized extended attributes in UFS/ZFS, for example, and implements its access control with them. Linux probably has the most limited form, where the xattr are capped at 4 or 64 kilobytes depending on the underlying FS. I think the big issue for uptake is that it just doesn't fit into the UNIX world. Nothing else is aware of xattrs/forks. The basic file utilities will blindly strip them so you can't just cp or tar such a file, for example. If cp transparently breaks all your files, that's pretty much a show-stopper as anyone who remembers the classic Mac to OS X transition will know.
    - p_l 5y ago
      The reason I mentioned Solaris is because Solaris behaviour is closest to NTFS, whereas linux API is based upon Irix, which is closer in behaviour to OS/2 (OS/2 subsystem was another use case for ADS), as well as implementation on a bunch of other systems where extended attributes were treated as just that, attributes. Comparatively, NTFS is very... flexible navigational database and a bunch of things are just "this is the default parameter set, not exhaustive". I wasn't aware that FreeBSD copied Solaris in this (ZFS extended attributes by default create a sort-of directory object allocated linked from the file, linux supports inlining simplified xattrs for speed)
    - trasz 5y ago
      There is a difference between alternate data streams and extended attributes. Basically, the former is data, the latter is metadata. NTFS and ZFS implement alternate data streams, but in eg FreeBSD those are accessed like extended attributes.
      - p_l 5y ago
        NTFS doesn't really have a difference internally - they are all attributes of the FILE object (everything in NTFS is a FILE or its attribute, including filesystem metadata). Arguably ZFS ZPL has "blessed" data element (I dare you to implement ZFS while replacing DMU/SPA with Ceph or S3 XD - yes it's doable)
  - deleted 5y ago
    [deleted]
kragen 5y ago
The other post linked from the one you're commenting on (https://blogs.loc.gov/thesignal/2020/11/an-archivists-perspective-on-legacy-files/ https://blogs.loc.gov/thesignal/2020/11/an-archivists-perspe...) talks specifically about the problems of preserving and reverse-engineering dual-fork files, so the problem is not what they're unaware of, Charlie. You just aren't aware of the problem they're trying to solve. The LOC, the Internet Archive, and the UK National Archives don't benefit from proprietary compilers for proprietary operating systems like SuperCard; their remit is to preserve these files for centuries or millennia, not until the next update of macOS X. If SuperCard interprets a HyperCard stack differently than original HyperCard did, or SuperCard on a new version of macOS X displays or behaves differently than it did on the previous version, that's a failure of digital preservation. Unless, that is, you notice --- and to notice, you need to emulate the original HyperCard environment and/or be sufficiently familiar with the semantics of the file specification. If https://news.ycombinator.com/item?id=29555959 https://news.ycombinator.com/item?id=29555959 is to be believed, SuperCard is 32-bit-only and so will not run on the most recent version of macOS X, which lack 32-bit support, which if true demonstrates the enormity of the problem. As for streams of bytes, a disk is a stream of bytes, or rather a collection of fixed-size blocks of bytes. HFS is looking at flattened HFS files that have been "squished" to fit on a medium that does not, itself, implement HFS. Therefore anything that you can represent with an HFS dual-fork file can also be represented losslessly in a stream of bytes. (You do of course benefit from understanding the dual-fork structure.) HFS+ actually went further, implementing files with an arbitrary number of forks, like sections in ELF or PE. As GeekyBear points out, NTFS did the same, leading to path traversal vulnerabilities in early Microsoft web servers.
- sokoloff 5y ago
  Wouldn’t an effective (and seemingly easier) digital preservation approach be to virtualize an older Mac OS (System 7 even) and just run HyperCard on it?
  - kragen 5y ago
    Yes, that's one of the things they're doing, but in the screenshots they're using MacOS 9. It's less effective if you want to do things like full-text search, copy and paste, or render the vector graphics (though maybe those were MacDraw and not HyperCard?) at a different pixel resolution than the emulated Macintosh supported, or with antialiasing. There may also be metadata or "forensic" data that isn't visible directly in HyperCard itself, which may be of interest to future historians. Aside from the emulation/undocumented-format problem that the librarians here are suffering from, a really big problem with born-digital archival work for more recent formats is copyright. Although libraries are permitted to make copies for archival work, in many cases the software tools they would need to make those copies don't exist, because those tools would circumvent an effective technological protection measure within the meaning of the DMCA, so writing and distributing them is potentially a crime. So in effect copyright law is privileging the business interests of Harvey Weinstein, Dan Brown, Bob Guccione, and Adam Sandler over the academic interests of scholars, historians, and the Library of Congress. I don't think there's anything wrong with Pee-wee Herman or Amy Poehler making movies, but I don't think it's worth sacrificing the intellectual heritage of humanity to buy them second vacation homes.
soapdog 5y ago
Runtime Revolution is not discontinued, it is alive and well, it just rebranded to https://www.livecode.com https://www.livecode.com many years ago.
pstuart 5y ago
It seems like it would be reasonably straightforward to create a hypercard clone on top of js and the DOM. Or maybe that's already been done? (a lazy search did not reveal this to be the case).
ksherlock 5y ago
The HyperCard stack's resource fork contains icons, sounds, xcmds, xfcns, and probably some other miscellanea. Most of the stack is stored in the data fork. APFS doesn't specifically have resource forks. They're handled as the com.apple.ResourceFork extended attribute (which is also one way OS X exposes them on HFS+ but with HFS+ the fork/xattr always exists; on APFS it has to be explicitly created).
jfrunyon 5y ago
> it's a distinctly different paradigm from the familiar a-file-is-a-stream-of-bytes convention of Linux/UNIX/NTFS Each of those platforms has an analog to HFS forks.

GeekyBear 5y ago

The original version of Windows NT Server allowed you to install Services for Macintosh and act as an AppleTalk file server, so support for resource forks was added to NTFS. https://en.wikipedia.org/wiki/NTFS#Alternate_data_stream_(ADS) https://en.wikipedia.org/wiki/NTFS#Alternate_data_stream_(AD...

p_l 5y ago
Alternate Data Streams are pretty interesting, cause I'm pretty sure Services for Macintosh weren't the only users - it's also way more powerful (closest thing on Unix would be Solaris-style extended attributes) and pretty fundamental part of NTFS on-disk structures (in fact, "alternate" data streams differ from main one only by having a name - the default data stream simply has no name, and all data streams are equal in importance)
- retrac 5y ago
  Extended attributes aren't limited just to Solaris. A variety of *nix file systems have generally supported forks/multiple data streams for some time. FreeBSD has support for arbitrary-sized extended attributes in UFS/ZFS, for example, and implements its access control with them. Linux probably has the most limited form, where the xattr are capped at 4 or 64 kilobytes depending on the underlying FS. I think the big issue for uptake is that it just doesn't fit into the UNIX world. Nothing else is aware of xattrs/forks. The basic file utilities will blindly strip them so you can't just cp or tar such a file, for example. If cp transparently breaks all your files, that's pretty much a show-stopper as anyone who remembers the classic Mac to OS X transition will know.
  - p_l 5y ago
    The reason I mentioned Solaris is because Solaris behaviour is closest to NTFS, whereas linux API is based upon Irix, which is closer in behaviour to OS/2 (OS/2 subsystem was another use case for ADS), as well as implementation on a bunch of other systems where extended attributes were treated as just that, attributes. Comparatively, NTFS is very... flexible navigational database and a bunch of things are just "this is the default parameter set, not exhaustive". I wasn't aware that FreeBSD copied Solaris in this (ZFS extended attributes by default create a sort-of directory object allocated linked from the file, linux supports inlining simplified xattrs for speed)
  - trasz 5y ago
    There is a difference between alternate data streams and extended attributes. Basically, the former is data, the latter is metadata. NTFS and ZFS implement alternate data streams, but in eg FreeBSD those are accessed like extended attributes.
    - p_l 5y ago
      NTFS doesn't really have a difference internally - they are all attributes of the FILE object (everything in NTFS is a FILE or its attribute, including filesystem metadata). Arguably ZFS ZPL has "blessed" data element (I dare you to implement ZFS while replacing DMU/SPA with Ceph or S3 XD - yes it's doable)
- deleted 5y ago
  [deleted]

kragen 5y ago

The other post linked from the one you're commenting on (https://blogs.loc.gov/thesignal/2020/11/an-archivists-perspective-on-legacy-files/ https://blogs.loc.gov/thesignal/2020/11/an-archivists-perspe...) talks specifically about the problems of preserving and reverse-engineering dual-fork files, so the problem is not what they're unaware of, Charlie. You just aren't aware of the problem they're trying to solve. The LOC, the Internet Archive, and the UK National Archives don't benefit from proprietary compilers for proprietary operating systems like SuperCard; their remit is to preserve these files for centuries or millennia, not until the next update of macOS X. If SuperCard interprets a HyperCard stack differently than original HyperCard did, or SuperCard on a new version of macOS X displays or behaves differently than it did on the previous version, that's a failure of digital preservation. Unless, that is, you notice --- and to notice, you need to emulate the original HyperCard environment and/or be sufficiently familiar with the semantics of the file specification. If https://news.ycombinator.com/item?id=29555959 https://news.ycombinator.com/item?id=29555959 is to be believed, SuperCard is 32-bit-only and so will not run on the most recent version of macOS X, which lack 32-bit support, which if true demonstrates the enormity of the problem. As for streams of bytes, a disk is a stream of bytes, or rather a collection of fixed-size blocks of bytes. HFS is looking at flattened HFS files that have been "squished" to fit on a medium that does not, itself, implement HFS. Therefore anything that you can represent with an HFS dual-fork file can also be represented losslessly in a stream of bytes. (You do of course benefit from understanding the dual-fork structure.) HFS+ actually went further, implementing files with an arbitrary number of forks, like sections in ELF or PE. As GeekyBear points out, NTFS did the same, leading to path traversal vulnerabilities in early Microsoft web servers.

sokoloff 5y ago
Wouldn’t an effective (and seemingly easier) digital preservation approach be to virtualize an older Mac OS (System 7 even) and just run HyperCard on it?
- kragen 5y ago
  Yes, that's one of the things they're doing, but in the screenshots they're using MacOS 9. It's less effective if you want to do things like full-text search, copy and paste, or render the vector graphics (though maybe those were MacDraw and not HyperCard?) at a different pixel resolution than the emulated Macintosh supported, or with antialiasing. There may also be metadata or "forensic" data that isn't visible directly in HyperCard itself, which may be of interest to future historians. Aside from the emulation/undocumented-format problem that the librarians here are suffering from, a really big problem with born-digital archival work for more recent formats is copyright. Although libraries are permitted to make copies for archival work, in many cases the software tools they would need to make those copies don't exist, because those tools would circumvent an effective technological protection measure within the meaning of the DMCA, so writing and distributing them is potentially a crime. So in effect copyright law is privileging the business interests of Harvey Weinstein, Dan Brown, Bob Guccione, and Adam Sandler over the academic interests of scholars, historians, and the Library of Congress. I don't think there's anything wrong with Pee-wee Herman or Amy Poehler making movies, but I don't think it's worth sacrificing the intellectual heritage of humanity to buy them second vacation homes.

soapdog 5y ago

Runtime Revolution is not discontinued, it is alive and well, it just rebranded to https://www.livecode.com https://www.livecode.com many years ago.

pstuart 5y ago

It seems like it would be reasonably straightforward to create a hypercard clone on top of js and the DOM. Or maybe that's already been done? (a lazy search did not reveal this to be the case).

ksherlock 5y ago

The HyperCard stack's resource fork contains icons, sounds, xcmds, xfcns, and probably some other miscellanea. Most of the stack is stored in the data fork. APFS doesn't specifically have resource forks. They're handled as the com.apple.ResourceFork extended attribute (which is also one way OS X exposes them on HFS+ but with HFS+ the fork/xattr always exists; on APFS it has to be explicitly created).

jfrunyon 5y ago

> it's a distinctly different paradigm from the familiar a-file-is-a-stream-of-bytes convention of Linux/UNIX/NTFS Each of those platforms has an analog to HFS forks.