11 ms·
Chromium: Secretly stores referer and url for downloaded files (2017)
- josteink 8y agoAnd so does wget if this report is correct! How can I verify this? I’ve often wanted this information but without having to rely on external book-keeping.
- anoncake 8y agogetfattr -d <downloaded file>
- callesgg 8y agoxattr -l <downloaded file>
- geofft 8y agoOn most Linux distros you can use the `getfattr` command from the `attr` package, but it's usually not installed by default. http://man7.org/linux/man-pages/man1/getfattr.1.html http://man7.org/linux/man-pages/man1/getfattr.1.html With Python 3.3+, which often is installed by default, you can use os.listxattr() and os.getxattr(). https://docs.python.org/3/library/os.html#linux-extended-attributes https://docs.python.org/3/library/os.html#linux-extended-att...
- aboutruby 8y agoOn macOS: mdls FILENAME `kMDItemWhereFroms` is what you are looking for. And for a list of all the URLs from the current directory: mdls * | grep kMDItemWhereFroms -A 1 | grep http | ruby -ne "puts \$_.strip[1..-2]"
- icebraining 8y agoIt no longer does: https://news.ycombinator.com/item?id=19390748 https://news.ycombinator.com/item?id=19390748
- EE84M3i 8y agoIt's not incredibly clear in the ticket, but the bug seems to be refering to Linux file system extended attributes.
- stefan_ 8y agoAs the "xdg" in the name suggests, this is a Freedesktop thing. Silly to blame Chrome for supporting the very extended attributes the premier Linux desktop project defined.
- agumonkey 8y agototally, IIRC curl does this too and it's extremely useful later on..
- nolok 8y agoBug report's comments says wget does too, and you can't turn it off.
- icebraining 8y agoIt's an old report, they have changed it since: https://security-tracker.debian.org/tracker/CVE-2018-20483 https://security-tracker.debian.org/tracker/CVE-2018-20483 https://lists.gnu.org/archive/html/bug-wget/2018-12/msg00034.html https://lists.gnu.org/archive/html/bug-wget/2018-12/msg00034... After @marcan42 noted this wasn't obvious for users: https://twitter.com/marcan42/status/1077676739877232640 https://twitter.com/marcan42/status/1077676739877232640
- ratling 8y ago"premier Linux desktop project" That's the real yikers to me. The metadata for downloaded files thing is all over the place (i.e. how macs will tell you that you're running something downloaded from the internet and where).
- sitkack 8y agoI find this super useful on a Mac and routinely would dump the urls with xattr. That is before I switched back to FF which runs better on older hardware .
- iscrewyou 8y agoCan you please explain it a little bit more? I am just curious.
- eropple 8y agoBrowsers on OS X store this information in extended file system attribtes (accessible via the xattr command). It's also how the OS knows to prompt you the first time you open an executable--"you downloaded this from googlechrome.com via Safari, are you sure you want to run this?". Calling it "secret", as the article does, seems disingenous. (Further, even if somebody downloads an application in an incog window, it is probably better for the system's security posture to record these xattrs for such a "hey, is this what you actually meant to download and execute?" situation.)
- yasp 8y agoSo, if I send the file to someone else, (1) on Linux and (2) on Mac, does this URL metadata come along, or no? Sounds like no, and the issue is just your device being compromised.
- eropple 8y agoI could be wrong, but my understanding matches yours and that, this data is not ever read back in by common HTTP clients (browsers, curl, etc.).
- icebraining 8y agoThey might if you package the file in some archive format that stores xattrs. For example, tar(1) can store them, although it doesn't by default (you must pass --xattrs).
- deleted 8y ago[deleted]
- hannob 8y agoI reported this to chromium recently, but I wasn't the first, it was marked as a duplicate. I think it should be fixed in latest versions or a fix should come soon. wget had the exact same issue and they recently disabled the attribute storage by default. See e.g. also * https://www.openwall.com/lists/oss-security/2019/01/01/1 https://www.openwall.com/lists/oss-security/2019/01/01/1 * https://lists.gnu.org/archive/html/bug-wget/2018-12/msg00034.html https://lists.gnu.org/archive/html/bug-wget/2018-12/msg00034...
- deleted 8y ago[deleted]
- achille 8y agoThis is a standard feature of many browsers, including Safari. E.g: On OSX you can download a .dmg file or .zip file, and when opening the OS will warn: "XYZ is an application downloaded from the internet. Are you sure you want to open it?". The information about the origin of the file comes from extended attributes. See: https://www.idownloadblog.com/2017/04/20/fix-application-from-internet-gatekeeper/ https://www.idownloadblog.com/2017/04/20/fix-application-fro...
- shawnz 8y agoWindows has the functionality you describe too, but it works only by storing a flag specifying what kind of origin the file has, not specifically what the origin was. Your article seems to indicate that mac OS uses basically the same system as Windows. EDIT: According to some other comments in this thread, I'm wrong. Mac OS does store the whole origin. EDIT 2: Looks like I'm wrong about Windows too, which also stores the whole origin. This actually disagrees with what is written in the bug report, so perhaps it needs to be updated.
- darkpuma 8y agoNo, MacOS stores the origin of the file. Downloaded with Safari: $ xattr -l Downloads/Ethiopian_Airlines_ET-AVJ_takeoff_from_TLV_\(46461974574\).jpg com.apple.lastuseddate#PS: 00000000 D9 76 8A 5C 00 00 00 00 9F E5 89 0D 00 00 00 00 |.v..............| 00000010 com.apple.metadata:kMDItemDownloadedDate: 00000000 62 70 6C 69 73 74 30 30 A1 01 33 41 C1 1D 57 2B |bplist00..3A..W+| 00000010 FF B1 3A 08 0A 00 00 00 00 00 00 01 01 00 00 00 |..:.............| 00000020 00 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00 |................| 00000030 00 00 00 00 13 |.....| 00000035 com.apple.metadata:kMDItemWhereFroms: 00000000 62 70 6C 69 73 74 30 30 A2 01 02 5F 10 7D 68 74 |bplist00..._.}ht| 00000010 74 70 73 3A 2F 2F 75 70 6C 6F 61 64 2E 77 69 6B |tps://upload.wik| 00000020 69 6D 65 64 69 61 2E 6F 72 67 2F 77 69 6B 69 70 |imedia.org/wikip| 00000030 65 64 69 61 2F 63 6F 6D 6D 6F 6E 73 2F 64 2F 64 |edia/commons/d/d| 00000040 32 2F 45 74 68 69 6F 70 69 61 6E 5F 41 69 72 6C |2/Ethiopian_Airl| 00000050 69 6E 65 73 5F 45 54 2D 41 56 4A 5F 74 61 6B 65 |ines_ET-AVJ_take| 00000060 6F 66 66 5F 66 72 6F 6D 5F 54 4C 56 5F 25 32 38 |off_from_TLV_%28| 00000070 34 36 34 36 31 39 37 34 35 37 34 25 32 39 2E 6A |46461974574%29.j| 00000080 70 67 3F 64 6F 77 6E 6C 6F 61 64 5F 10 19 68 74 |pg?download_..ht| 00000090 74 70 73 3A 2F 2F 65 6E 2E 77 69 6B 69 70 65 64 |tps://en.wikiped| 000000A0 69 61 2E 6F 72 67 2F 08 0B 8B 00 00 00 00 00 00 |ia.org/.........| 000000B0 01 01 00 00 00 00 00 00 00 03 00 00 00 00 00 00 |................| 000000C0 00 00 00 00 00 00 00 00 00 A7 |..........| 000000ca com.apple.quarantine: 0083;5c8a76d8;Safari;ADF309D2-762B-4FE2-AEC6-104E019BDBF9
- peterwwillis 8y agoWell, it's invisible, not secret. You can also recover deleted files from a hard drive, but that's not a secret. Both things need to be more widely known, but the fact that they exist is still useful (both for individuals and law enforcement)
- shereadsthenews 8y agoHere is the finest example of privacy derangement syndrome that we will ever see: an open-source program implements an open standard in a way that's completely above-board and it's described as a nefarious scheme.
- XCabbage 8y agoIf an open standard has features that violate user privacy and don't provide sufficient value in exchange to justify it, it's reasonable to discuss violating or reforming that standard for the sake of privacy. The existence of a standard doesn't make the privacy issue go away.
- shereadsthenews 8y agoMy beef is with the adverb "secretly".
- darkpuma 8y agoI don't have an issue with this feature, but to play devil's advocate: it obviously isn't being adequately advertised if it's surprising to so many people on HN. And if many among the audience of HN find it surprising, you can be certain it's even more-so to the general public.
- stupidlogin 8y agoSince the behavior is unexpected and very non-obvious for the average user, I don't see any issue with calling it "secretly".
- shereadsthenews 8y agoYou confused secrecy with ignorance.
- kllrnohj 8y agoNote that per the report GNU wget does this as well. So other well known, well regarded, widely used tools are following this same standard doing the same thing. That doesn't excuse the standard, but it does mean Chromium is just matching platform on this, not setting any particular policy on its own.
- jolmg 8y agoI was curious if Firefox supported this feature, and I found an unassigned issue from 8 years ago: https://bugzilla.mozilla.org/show_bug.cgi?id=665531 https://bugzilla.mozilla.org/show_bug.cgi?id=665531
- josteink 8y agoI hate how (in open-source projects in particular) anything even remotely associatable with security/privacy can be bike-shedded for almost a decade with no actual work done what so ever. Talk about snailing your way to irrelevance. And I say that as a Firefox-user. Sometimes I think Firefox would benefit from a more benevolent leader who just stomped down on issues like this and settled things properly without spending months or years doing so. This is ridiculous.
- JdeBP 8y agoPah! A decade is nothing. * http://jdebp.eu./FGA/dns-srv-record-use-by-clients.html#HTTPShame http://jdebp.eu./FGA/dns-srv-record-use-by-clients.html#HTTP...
- aasasd 8y agoOn MacOS, FF marks files as downloaded from the web for the system. And also, in its standard 'Downloads' dialog you can copy the address of each file and, purportedly, go to the page it was downloaded from (the latter not working for me).
- luizfzs 8y agoIt's missing a NSFW warning on that photo
- ToFab123 8y agoIf that is NSFW allow me to recommend you to go get a new job.
- growt 8y agoI wrote a small os X app based on this, that sorts your downloads in subfolders named like the domain you downloaded the file from (it's just a small shell script in a wrapper so it might run in Linux as well with some modifications): https://github.com/grothkopp/sortDownloads.app https://github.com/grothkopp/sortDownloads.app
- mthoms 8y agoVery cool. I wrote a similar script to copy the source URL to the file "comments" field so it's viewable/sortable in Finder. How did you package your script as an .app like that? Platypus perhaps?
- KirinDave 8y agoIt's interesting that this is considered a bug by Linux users. On the OSX side, populating the file metadata with the URL source has always been looked upon as a feature.
- jolmg 8y agoI'm a Linux user and I don't see it as a bug. Rather, I wish I had that in Firefox.
- fock 8y agoI also once contemplated making a browser extension, actually storing the url in metadata. I'm also not quite sure, how this affects user privacy, as the image content might be far more telling than the origin. Imo this compromises the origin of the file...
- deleted 8y ago[deleted]
- ignoranceprior 8y agoI use Safari, which does the same thing, and I actually find it useful. It's nice to be able to go back and find where you downloaded something from. IMO, complaining that this metadata violates the user's privacy is as silly as complaining that storing EXIF location metadata in JPGs violates privacy. They're both forms of metadata that can be useful in certain situations, and which many users are unaware of. Yeah, there is a technical difference in that EXIF data is stored within the file while this metadata is stored in the file attributes, but I think the analogy holds.
- icebraining 8y agoI agree that they are comparable; both can be violations of privacy. It should be clear to the user that such data is being recorded.
- ignoranceprior 8y agoI concur, and there should also be a simple UI option to disable both features.
- jplayer01 8y agoAgreed. I'd say what would be relevant is whether this data is ever transferred to a third-party like Google. If not, I don't quite see the problem. There's a lot of data I gather about myself, and my having access to this kind of metadata would be fantastic.
- hughes 8y agoApart from whether this is nefarious and/or intended behavior, it seems odd that the bug report specifically uses protection of illegal content as motivation.
- JdeBP 8y agoFor some perspective here: Various WWW tools for OS/2 back in the 1990s also did this, putting the source URL into a .SUBJECT extended attribute. The OS/2 port of wget was also modified to do this. It wasn't in any way secret. The .SUBJECT of a file was visible in its Properties dialogue on the Worksplace Shell desktop. Which one could also use to edit it. People like me wrote other tools for manipulating and viewing these .SUBJECTs, which were also used for file descriptions by 4OS2 and various OS/2 file management and BBS softwares. * https://jdebp.eu./Softwares/os2/ https://jdebp.eu./Softwares/os2/