3 ms·

Am I missing something here, or did the author forget to discuss "dataless files" after mentioning it in the title?

by jasoneckert 3y ago

mistercow 3y ago
Yeah, I’m wondering that too. That was the hook I was interested in.
- bitwize 3y ago
  It's now common for "You Wouldn't Believe" style clickbait articles to show a screenshot of something that's not in the actual article. Guess it's leaking into other forms of online journalism.
Dig1t 3y ago
Dataless files are interesting! They are a special type of file with a bit set (and an xattr) in the file system and no actual file contents. They all live inside a “sync root” in APFS, where each sync root belongs to a specific file provider like Box, Dropbox, iCloud Drive etc. Whenever the kernel detects access to one of these files it will synchronously “upcall” into the current user context and XPC directly to fileproviderd which will in turn communicate with an installed file provider to download and provide the file (changing it from a dataless file into a normal file). This is called “materializing” the file. It will also emit an event using the FSEvents API. fileproviderd (a launch agent on macOS and a daemon on iOS) watches the FSEvents stream and saves the events in its database. fileproviderd maintains its own sync engine for each provider on the system and will forward events to/from each provider. Dataless files are an interesting example of Apple using its vertical integration to create an end to end solution that others can’t quite make. Since they own the filesystem, kernel, and user space daemons they can make something that works like this. This is part of Apple’s years-long effort to kick 3rd parties out of the kernel. As apart of the M1 transition, kernel extensions were no longer aloud and file provider apps were using kernel extensions to intercept file system events at a low level before. Now that they are no longer able to do that, Apple has provided this replacement.
- tambourine_man 3y ago
  You comment is better than the linked article. Also, I think you meant allowed not aloud
  - Dig1t 3y ago
    Thank you! D'oh! Damnit I hate when I do that, thanks for letting me know. I wish I could still edit the comment..
- shrubble 3y ago
  It sounds like a very much better integrated, more foolproof version of say, the FUSE filesystem with SSH or FTP as the remote side. You would start with a listing of files but then upon access the remote file will be accessed.
  - abwizz 3y ago
    it'll be a day of celebration when apple allows fuse as a backend.
  - TheNewsIsHere 3y ago
    My experience using applications built around the FUSE stack or the File Provider API suggest to me that while your take is generally true, it’s a “grass is always greener” problem. The File Provider API provides a much better admin and user experience, but the storage providers tend to really easily make a confusing mess of it. OneDrive took the time to figure out how to use sync roots in a way that enables you to store your actual data-containing files to an external drive. Dropbox didn’t care about supporting that use case on Macs, so what was once possible with both is now just possible with one. That has been a source of problems for many. On the other hand the migration to OneDrive’s File Provider implementation was a data-thrashing horror show for many. And iCloud Drive doesn’t truly let you decide a persistent sync state for everything. The security implications are better but it’s just the same old mess file syncing has been for a long time :)
    - 3y ago
blueboo 3y ago
It also forgot to discuss Cyberdog, aside from mentioning its name. What is it? I kept rereading the first paragraphs thinking I’d missed it.
- layer8 3y ago
  Contemporary reviews: http://www.atpm.com/2.06/page12.shtml http://www.atpm.com/2.06/page12.shtml https://www.mymac.com/1996/08/cyberdog-v1-0-review/ https://www.mymac.com/1996/08/cyberdog-v1-0-review/

Am I missing something here, or did the author forget to discuss "dataless files" after mentioning it in the title?

mistercow 3y ago

Yeah, I’m wondering that too. That was the hook I was interested in.

bitwize 3y ago
It's now common for "You Wouldn't Believe" style clickbait articles to show a screenshot of something that's not in the actual article. Guess it's leaking into other forms of online journalism.

Dig1t 3y ago

Dataless files are interesting! They are a special type of file with a bit set (and an xattr) in the file system and no actual file contents. They all live inside a “sync root” in APFS, where each sync root belongs to a specific file provider like Box, Dropbox, iCloud Drive etc. Whenever the kernel detects access to one of these files it will synchronously “upcall” into the current user context and XPC directly to fileproviderd which will in turn communicate with an installed file provider to download and provide the file (changing it from a dataless file into a normal file). This is called “materializing” the file. It will also emit an event using the FSEvents API. fileproviderd (a launch agent on macOS and a daemon on iOS) watches the FSEvents stream and saves the events in its database. fileproviderd maintains its own sync engine for each provider on the system and will forward events to/from each provider. Dataless files are an interesting example of Apple using its vertical integration to create an end to end solution that others can’t quite make. Since they own the filesystem, kernel, and user space daemons they can make something that works like this. This is part of Apple’s years-long effort to kick 3rd parties out of the kernel. As apart of the M1 transition, kernel extensions were no longer aloud and file provider apps were using kernel extensions to intercept file system events at a low level before. Now that they are no longer able to do that, Apple has provided this replacement.

tambourine_man 3y ago
You comment is better than the linked article. Also, I think you meant allowed not aloud
- Dig1t 3y ago
  Thank you! D'oh! Damnit I hate when I do that, thanks for letting me know. I wish I could still edit the comment..
shrubble 3y ago
It sounds like a very much better integrated, more foolproof version of say, the FUSE filesystem with SSH or FTP as the remote side. You would start with a listing of files but then upon access the remote file will be accessed.
- abwizz 3y ago
  it'll be a day of celebration when apple allows fuse as a backend.
- TheNewsIsHere 3y ago
  My experience using applications built around the FUSE stack or the File Provider API suggest to me that while your take is generally true, it’s a “grass is always greener” problem. The File Provider API provides a much better admin and user experience, but the storage providers tend to really easily make a confusing mess of it. OneDrive took the time to figure out how to use sync roots in a way that enables you to store your actual data-containing files to an external drive. Dropbox didn’t care about supporting that use case on Macs, so what was once possible with both is now just possible with one. That has been a source of problems for many. On the other hand the migration to OneDrive’s File Provider implementation was a data-thrashing horror show for many. And iCloud Drive doesn’t truly let you decide a persistent sync state for everything. The security implications are better but it’s just the same old mess file syncing has been for a long time :)
  - 3y ago

blueboo 3y ago

It also forgot to discuss Cyberdog, aside from mentioning its name. What is it? I kept rereading the first paragraphs thinking I’d missed it.

layer8 3y ago
Contemporary reviews: http://www.atpm.com/2.06/page12.shtml http://www.atpm.com/2.06/page12.shtml https://www.mymac.com/1996/08/cyberdog-v1-0-review/ https://www.mymac.com/1996/08/cyberdog-v1-0-review/