15 ms·

Windows Is Bloated, Thanks to Adobe’s Extensible Metadata Platform

321 points by Integer 9y ago

blibble 9y ago
I wonder how many hundreds of kilobytes that adds up to in a 20gb windows install
- moolcool 9y ago
  You don't keep a 20gb Windows install in RAM all the time though. Bloat in explorer.exe is the issue here
  - jszymborski 9y ago
    To this point, the files in the install are compressed, and I'm sure XML metadata is the sort of thing that compresses well with the DEFLATE algorithm they likely use.
    - PaulHoule 9y ago
      I discovered this a few months ago, when I went looking for XMP metadata in the filesystem and used the magic number trick to extract it from files of all kinds. I found it is common to find XMP inside media files embedded inside Windows EXE, as well as Linux binaries, JAR, Microsoft Word and other composite formats. Complex media objects frequently use an encapsulation system such as ZIP. When a PNG file is incorporated into a JAR or a Word Document, the XMP content in the file may not be compressed because the archiver may not attempt to compress the png file since it assumes the data is already compressed. XMP is very good from the viewpoint of content creators in terms of having comprehensive metadata incorporated into files so that it does not get out of sync. XMP data is RDF data using an improved version of Dublin Core, IPCC and other industry RDF vocabulary. You can write SPARQL queries right away, plus XMP specifies a way to make an XMP packet based on pre-existing metadata in common industry schemes. The XMP packets can get big, and you sometimes see people make a tiny GIF image (say a transparent pixel GIF) that is bulked up 100x because of bulky metadata. Once you package data for delivery to consumers you want to strip all that stuff out. The XMP spec is here: http://www.adobe.com/devnet/xmp.html http://www.adobe.com/devnet/xmp.html There is some brilliant thinking in there, but also things that will make your head explode such as the method for embedding an XMP packet into a GIF
      - jszymborski 9y ago
        Hmm... would be interesting if we started taking XMP into account when designing compression programs then...
        abdias 9y ago
        You could actually take any ancillary chunks into consideration, ie. chunks starting with a lower-case first letter. These are non-critical/mandatory.
      - TazeTSchnitzel 9y ago
        > When a PNG file is incorporated into a JAR or a Word Document, the XMP content in the file may not be compressed because the archiver may not attempt to compress the png file since it assumes the data is already compressed. PNG can apply DEFLATE to blocks though, right? Does XMP not use it?
        abdias 9y ago
        Deflating can be applied to some chunks, but not at will. The zTXt chunk can be compressed while for example the tEXt chunk cannot. The newer iTXt chunk can vary. The two former are limited in scope and language encoding support, so iTXt is typically used for extended textual data such as XML/XMP etc. But if is saved compressed or not depends on the PNG encoder/host used (there can also be multiple instances of these chunks in the same file). Photoshop for instance saves uncompressed, I guess to give fast access for performance reasons (ie. file viewers using galleries for numerous images while displaying their meta-data).
  - galacticpony 9y ago
    data contained in an exe (or dll) is not necessarily in RAM at all time
- astrobe_ 9y ago
  One could fit about 10 Linux live distros in those 20Gb. Those "few hundreds of kilobytes" are indeed trivial.
  - 21 9y ago
    Windows comes with a ton of builtin drivers. So it will work with a lot of devices out of the box without needing an Internet connection to update.
    - icebraining 9y ago
      Same is true in most Linux distros. My stock Debian kernel comes with 2300+ drivers, and which only take ~130MB.
      - simooooo 9y ago
        None of them are ever for any of my hardware
        cmdrfred 9y ago
        I have the opposite experience myself.
        Something1234 9y ago
        What hardware are you using that the defaults don't work?
        bastawhiz 9y ago
        I've installed the latest Ubuntu in 2016 to find my mouse, wifi, and printer did not work. And my Cinema Display was stuck at 1024x768.
      - pikzen 9y ago
        https://blogs.msdn.microsoft.com/e7/2008/11/19/disk-space/ https://blogs.msdn.microsoft.com/e7/2008/11/19/disk-space/ Bear in mind, that was the Vista days, and Windows 10 now supports even more devices. 800MB of drivers at the time. I would not be surprised if Windows supported by default upwards of 10000 drivers. It works pretty much flawlessly on even somewhat obscure and old hardware. And when your OS is installed on that many consumer devices, and not informally standardized servers, you are going to meet those weird devices one way or the other. Windows drivers may also take up a bit more space individually because of the overhead caused by either the Windows Driver Model or Windows Driver Framework, but that's the price to pay to not have a driver crashing and bringing down your entire system. Yes, Linux, I'm looking at you.
- kevindqc 9y ago
  5 MB. https://gist.githubusercontent.com/riverar/f4a56b91580af1bd3cf4dfacc3733bc3/raw/8f35e3def3173ded72c5715e5af37e830b2a639f/results.txt https://gist.githubusercontent.com/riverar/f4a56b91580af1bd3...
  - asveikau 9y ago
    Haven't looked at the code that generates this list (if available), but that sure looks to me like double-counting going on here. Most files in \Windows\WinSXS are hardlinks. Example pulled at random: D:\analysis\Windows\WinSxS\amd64_microsoft-windows -imageres_31bf3856ad364e35_10.0.15063.0_none_edd17c6c30b4bf9f\imageres.dll ... - Total: 4435 D:\analysis\Windows\System32\imageres.dll ... - Total: 4435 I am willing to bet those are the same file hardlinked and only wastes the 4435 bytes once, verifiable thusly: cmd> fsutil hardlink list \Windows\System32\imageres.dll Windows\WinSxS\amd64_microsoft-windows-imageres_31bf3856ad364e35_6.3.9600.16384_ none_cd7c033dcbdd0cab\imageres.dll Windows\System32\imageres.dll
    - kevindqc 9y ago
      The source seems to be: https://github.com/riverar/eoraptor https://github.com/riverar/eoraptor Seems to check though https://github.com/riverar/eoraptor/blob/master/FileEnumerator.cpp#L19 https://github.com/riverar/eoraptor/blob/master/FileEnumerat...
      - asveikau 9y ago
        As I suspected, that doesn't look hardlink aware. A way to correct for this would be to open the files and de-dupe by (((ULONGLONG)nFileIndexHigh) << 32) | nFileIndexLow in this structure: https://msdn.microsoft.com/en-us/library/windows/desktop/aa363788(v=vs.85).aspx https://msdn.microsoft.com/en-us/library/windows/desktop/aa3... Edit: > Seems to check though https://github.com/riverar/eoraptor/blob/master/FileEnumerator.cpp#L19 https://github.com/riverar/eoraptor/blob/master/FileEnumerat... No it does not, reparse points are used for symbolic links and junctions - not hardlinks.
- ashark 9y ago
  What's the story on that footprint, anyway? We went from (from memory) ~250-350MB Win98 to ~700-800MB XP to ~10-15GB(!?!?!) Win7, and just up from there. Plus the default settings seemed to starting going really crazy with swapspace/caching around the time of Win7. Another 10+GB if you didn't tell it to knock that crap off. Why the sudden, giant shift? They didn't add 10-15x the features, that's for sure.
  - rpeden 9y ago
    I think at least some of it is the Windows on Windows stuff to allow 64 bit machines to run both 32 and 64 bit software. Weren't the 32 but versions of Win7 about half the size of the 64 bit ones? There's still a lot of size growth over time, of course.
  - jug 9y ago
    WinSxS (Windows Side By Side) assemblies were introduced to avoid dll hell by allowing Windows to store multiple versions of installed dll's. So even a minor security patch may leave the former version around because other apps may use/expect it. I think that might add some bloat over time? Also Windows Update installer caches. A ton of Windows updates actually leave their installers around in case you want to uninstall them. That can add up! I've seen it easily get to 1-2 GB.
  - bastawhiz 9y ago
    They did, to some extent. Plug in almost any piece of standard consumer hardware and it'll probably mostly just work without a network connection. All those drivers don't take up zero space, but the benefit when my mom plugs in a printer and it just works makes it worth it.
thinknot 9y ago
Next time, run `optipng -o9 -strip all` on all your png files!
sp332 9y ago
Is it feasible to remove this junk yourself, or will the system freak out about hacked binaries? Would it also complain if I just applied to the PNG files?
- INTPenis 9y ago
  I'm a Linux specialist and a stranger to Windows but logically any modification of binaries should result in security issues. And in a perfect world the external PNG content would also be verified.
  - 21 9y ago
    All Windows binaries are signed. Changing the embedded PNG will void the signature. Not sure what Windows will do if explorer.exe has a bad signature. There is also a Windows system integrity checker service which disallows changes to protected Windows files, and repairs them automatically (using a cached copy).
    - kuschku 9y ago
      Well, then you modify that, too. It’s your computer, you installed the software, you have a license, therefore you own that copy, and can modify it however you wish. (EU Copyright Directive, especially Article 6 and following). Now, the question is, why does Windows not allow me to add signatures that should be considered acceptable by default, why can I not modify my own OS installation?
      - 21 9y ago
        Can you modify Android or iOS system files or add your key? Rooting is not really an answer, because in that case you can also root Windows. I was under the impression that you can only legally modify files for fair use or compatibility purposes, not just because you want so.
        kuschku 9y ago
tomcam 9y ago
You would be surprised at how much cruft in Windows over the years has been directly due to Adobe. I had many bug triage sessions where Windows developers at Microsoft had to work around Adobe problems to keep Windows users happy. I always thought it was unfair and was quite impressed by Microsoft at their willingness to handle this so quietly.
- batter 9y ago
  Doesn't it makes you wonder why MS created that situation on the first place? MS have no one to blame except themself and their 'legacy code'.
  - commandar 9y ago
    It's hard enough to get enterprise customers to stay current when they do provide legacy support. Fighting with IE version compatibility, for example, is still a daily, very real issue for enterprise size organizations. Blame the customers. Microsoft never would have captured the marketshare they have if they didn't cater to them.
    - DaiPlusPlus 9y ago
      Microsoft does deserve some blame for deciding to support backwards-compatibility in-situ instead of introducing proper API versioning, packaging, isolation, and other proven techniques. Win32 wasn't versioned for processes until Windows 7 (using app.manifest) even though the need for such as system was blatantly obvious during the Windows 2000 days.
      - commandar 9y ago
        Sure, that's valid. But most of the real nightmare scenarios I've heard related to backwards compatibility have more to do with third-parties doing things they were never supposed to do. Things like hitting private, undocumented APIs. Or checking the Windows version with a "9x" wildcard, giving us the jump from W8 to W10 over a decade later. Microsoft has made their own mistakes, but supporting the mistakes of third parties has been absolutely vital to them keeping their core customer base.
mih 9y ago
In one of the comments on the page, a reader ran the analysis [1] on a windows installation and reports the bloat size. Total bytes wasted: 5341278 [1] https://gist.githubusercontent.com/riverar/f4a56b91580af1bd3cf4dfacc3733bc3/raw/8f35e3def3173ded72c5715e5af37e830b2a639f/results.txt https://gist.githubusercontent.com/riverar/f4a56b91580af1bd3...
- blibble 9y ago
  amusingly that's about 50% of the size of the webpage itself (2.4mb)
  - delinka 9y ago
    Surely you mean twice the size of the web page.
    - blibble 9y ago
      er, yes!
- deleted 9y ago
  [deleted]
- deleted 9y ago
  [deleted]
- WatchDog 9y ago
  That's 5MB, for the humans among us.
- balls187 9y ago
  I believe that was not a reader, but the author of the article.
  - mih 9y ago
    You're right. My mistake
msimpson 9y ago
Given this bloat resides in the metadata of PNG assets exported from Photoshop, couldn't this affect any operating system? How many applications on Mac OS utilize PNG assets which were exported from Photoshop without any further optimization?
- 0x0 9y ago
  When you add a .png file to an xcode iOS project, it will add a build step to pngcrush the asset automatically. https://developer.apple.com/library/content/qa/qa1681/_index.html https://developer.apple.com/library/content/qa/qa1681/_index...
  - msimpson 9y ago
    Interesting. I did not know that. Would the same hold true outside iOS targets?
    - saagarjha 9y ago
      macOS supports true vector assets, so I'm guessing it reduces the need for such measures. But coming back to your question: I'm not sure, but both iOS and macOS use the same asset filetype, so I'd assume so.
- ghostly_s 9y ago
  I would put good money on 'none'. Apple is way too much of a stickler about MacOS' graphics APIs to not be using custom authoring or optimization routines on assets. Edit: I presumed you're talking about first-party applications only, as you said 'operating system'.
0x0 9y ago
It's not the first time Windows has shipped with shameful metadata. For example, a .wav file shipped with Windows XP appears to be authored with a pirated version of SoundForge: https://web.archive.org/web/20060721090521/http://techrepublic.com.com/5208-11183-0.html?forumID=89&threadID=173539&messageID=1765547 https://web.archive.org/web/20060721090521/http://techrepubl...
- douche 9y ago
  Eh, that sort of thing happens all the time. We have customers that are national banks of foreign countries that are running on unactivated or cracked Windows servers. I would bet that case was a contractor who was using their own equipment. Similar things have bit us in the ass with freelancers that have ripped off stock images and presented them as their own.
  - 0x0 9y ago
    Yeah, but you'd expect the golden master .iso of a major Windows release to be examined in greater detail and not sloppily add bloat like that. It's in great contrast to earlier releases like Windows 95 where blinking dots were removed from the task bar to save a few kilobytes of demand paged memory. https://blogs.msdn.microsoft.com/oldnewthing/20031010-00/?p=42203 https://blogs.msdn.microsoft.com/oldnewthing/20031010-00/?p=...
    - rangibaby 9y ago
      The difference is Win95 had a floppy disk version sold to consumers. 13 disks specially formatted to store an extra 200kb each. https://blogs.msdn.microsoft.com/oldnewthing/20050819-10/?p=34513 https://blogs.msdn.microsoft.com/oldnewthing/20050819-10/?p=...
    - douche 9y ago
      Moore's law has allowed us to be lazy, and things have gotten too complicated for anybody to fully understand everything. I feel like I spend half my time trying to delete old dead code, and I can't keep up.
21 9y ago
I remember an article saying that making a trivial change to Windows requires 5 minutes to change the code and 2 weeks to deal with the aftermath (testing/...) I wonder how easy it actually is to remove this XMP metadata, considering that it could potentially break some application which loads a PNG directly from explorer.exe with a broken PNG parser or something.
Esau 9y ago
It's not just Windows that is bloated; so is macOS, Android, and iOS. The wastefulness annoys me and I don't want to hear its okay because we have tons of disk space and RAM - it is still wasteful. I understand why they kitchen-sink operating systems - its mainly so they can crow about new features when releasing new versions of the OS. But I wish they would offer alternate installs for those of us who are proficient.
- Fifer82 9y ago
  409 grains of sugar please or I am going elsewhere.... Actually....
- flukus 9y ago
  On android I'm still severely limited by disk space and RAM, the waste is very noticeable. It still is on PC for most users too, cheap SSD's are still quite small, there are a lot of new laptops sold with 64GB SSD's and 2GB RAM. Good luck running anything in node.
  - deleted 9y ago
    [deleted]
- zacmps 9y ago
  I agree, it's a lot of the reason I like distros like Arch or Alpine.
- saagarjha 9y ago
  Looks like Linux is what you're looking for…
  - realusername 9y ago
    I would really like to have a Linux-style OS on my smartphone and not the bloated Android Java which looks like the Win32 API re-engineered for smartphones.
asveikau 9y ago
It's great investigative work into Windows binaries, and I hope it gets addressed for the sake of people's disk space, but I think the tone is too harsh and overstated. Example: He cites effects on startup time - but has he considered the existence of virtual memory? When explorer.exe loads and maps the bloat into address space, it doesn't need it in RAM until the first page fault accessing it which likely will not even happen.
- codys 9y ago
  So that is true as long as all the bloat is contiguous. If it is spread out throughout the file (in such a way that bloat doesn't fill an entire page) it will still end up loaded. Or even if it is "unused", that doesn't mean something isn't scanning over it byte-by-byte. In the happy case, yes, virtual memory will save us. But there are a lot of ways we could still end up loading the junk into ram. Also, there are potential runtime costs to it being larger just on disk (need to seek over it, etc).
- withinrafael 9y ago
  Most binaries in Windows are signed. This requires the loader to load each PE section, hash, do some number crunching, and compare the result -- which results in the entire file being loaded into memory before execution. This involves (potentially random?) disk I/O, which can be surprisingly slow on certain platforms (e.g. Xbox One, HoloLens, anything IoT, anything with eMMC).
  - asveikau 9y ago
    Fair point. Another reason code signing is stupid.
zeveb 9y ago
Wow, that's a bloated format. Here it is as XML: <?xpacket begin="?" id="W5M0MpCehiHzreSzNTczkc9d"?> <x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Adobe XMP Core 5.4-c002 1.000000, 0000/00/00-00:00:00 "> <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"> <rdf:Description rdf:about="" xmlns:xmp="http://ns.adobe.com/xap/1.0/"> <xmp:CreatorTool>Picasa</xmp:CreatorTool> </rdf:Description> <rdf:Description rdf:about="" xmlns:mwg-rs="http://www.metadataworkinggroup.com/schemas/regions/" xmlns:stDim="http://ns.adobe.com/xap/1.0/sType/Dimensions#" xmlns:stArea="http://ns.adobe.com/xmp/sType/Area#"> <mwg-rs:Regions rdf:parseType="Resource"> <mwg-rs:AppliedToDimensions rdf:parseType="Resource"> <stDim:w>912</stDim:w> <stDim:h>687</stDim:h> <stDim:unit>pixel</stDim:unit> </mwg-rs:AppliedToDimensions> <mwg-rs:RegionList> <rdf:Bag> <rdf:li rdf:parseType="Resource"> <mwg-rs:Type></mwg-rs:Type> <mwg-rs:Area rdf:parseType="Resource"> <stArea:x>0.680921052631579</stArea:x> <stArea:y>0.3537117903930131</stArea:y> <stArea:h>0.4264919941775837</stArea:h> <stArea:w>0.32127192982456143</stArea:w> <stArea:unit>normalized</stArea:unit> </mwg-rs:Area> </rdf:li> </rdf:Bag> </mwg-rs:RegionList> </mwg-rs:Regions> </rdf:Description> <rdf:Description rdf:about="" xmlns:exif="http://ns.adobe.com/exif/1.0/"> <exif:PixelXDimension>912</exif:PixelXDimension> <exif:PixelYDimension>687</exif:PixelYDimension> <exif:ExifVersion>0220</exif:ExifVersion> </rdf:Description> </rdf:RDF> </x:xmpmeta>  <?xpacket end="w"?> And here it is as SXML (https://en.wikipedia.org/wiki/SXML https://en.wikipedia.org/wiki/SXML): (*TOP* (*PI* |xpacket| "begin=\"?\" id=\"W5M0MpCehiHzreSzNTczkc9d\"") (|adobe:ns:meta/:xmpmeta| (@ (@ (*NAMESPACES* (|adobe:ns:meta/| "adobe:ns:meta/" . |x|))) (|adobe:ns:meta/:xmptk| "Adobe XMP Core 5.4-c002 1.000000, 0000/00/00-00:00:00 ")) " " (|http://www.w3.org/1999/02/22-rdf-syntax-ns#:RDF| (@ (@ (*NAMESPACES* (|http://www.w3.org/1999/02/22-rdf-syntax-ns#| "http://www.w3.org/1999/02/22-rdf-syntax-ns#" . |rdf|)))) " " (|http://www.w3.org/1999/02/22-rdf-syntax-ns#:Description| (@ (@ (*NAMESPACES* (|http://ns.adobe.com/xap/1.0/| "http://ns.adobe.com/xap/1.0/" . |xmp|))) (|http://www.w3.org/1999/02/22-rdf-syntax-ns#:about| "")) " " (|http://ns.adobe.com/xap/1.0/:CreatorTool| "Picasa") " ") " " (|http://www.w3.org/1999/02/22-rdf-syntax-ns#:Description| (@ (@ (*NAMESPACES* (|http://ns.adobe.com/xmp/sType/Area#| "http://ns.adobe.com/xmp/sType/Area#" . |stArea|) (|http://ns.adobe.com/xap/1.0/sType/Dimensions#| "http://ns.adobe.com/xap/1.0/sType/Dimensions#" . |stDim|) (|http://www.metadataworkinggroup.com/schemas/regions/| "http://www.metadataworkinggroup.com/schemas/regions/" . |mwg-rs|))) (|http://www.w3.org/1999/02/22-rdf-syntax-ns#:about| "")) " " (|http://www.metadataworkinggroup.com/schemas/regions/:Regions| (@ (|http://www.w3.org/1999/02/22-rdf-syntax-ns#:parseType| "Resource")) " " (|http://www.metadataworkinggroup.com/schemas/regions/:AppliedToDimensions| (@ (|http://www.w3.org/1999/02/22-rdf-syntax-ns#:parseType| "Resource")) " " (|http://ns.adobe.com/xap/1.0/sType/Dimensions#:w| "912") " " (|http://ns.adobe.com/xap/1.0/sType/Dimensions#:h| "687") " " (|http://ns.adobe.com/xap/1.0/sType/Dimensions#:unit| "pixel") " ") " " (|http://www.metadataworkinggroup.com/schemas/regions/:RegionList| " " (|http://www.w3.org/1999/02/22-rdf-syntax-ns#:Bag| " " (|http://www.w3.org/1999/02/22-rdf-syntax-ns#:li| (@ (|http://www.w3.org/1999/02/22-rdf-syntax-ns#:parseType| "Resource")) " " (|http://www.metadataworkinggroup.com/schemas/regions/:Type|) " " (|http://www.metadataworkinggroup.com/schemas/regions/:Area| (@ (|http://www.w3.org/1999/02/22-rdf-syntax-ns#:parseType| "Resource")) " " (|http://ns.adobe.com/xmp/sType/Area#:x| "0.680921052631579") " " (|http://ns.adobe.com/xmp/sType/Area#:y| "0.3537117903930131") " " (|http://ns.adobe.com/xmp/sType/Area#:h| "0.4264919941775837") " " (|http://ns.adobe.com/xmp/sType/Area#:w| "0.32127192982456143") " " (|http://ns.adobe.com/xmp/sType/Area#:unit| "normalized") " ") " ") " ") " ") " ") " ") " " (|http://www.w3.org/1999/02/22-rdf-syntax-ns#:Description| (@ (@ (*NAMESPACES* (|http://ns.adobe.com/exif/1.0/| "http://ns.adobe.com/exif/1.0/" . |exif|))) (|http://www.w3.org/1999/02/22-rdf-syntax-ns#:about| "")) " " (|http://ns.adobe.com/exif/1.0/:PixelXDimension| "912") " " (|http://ns.adobe.com/exif/1.0/:PixelYDimension| "687") " " (|http://ns.adobe.com/exif/1.0/:ExifVersion| "0220") " ") " ") " ") (*COMMENT* " whitespace padding ") (*PI* |xpacket| "end=\"w\"")) The only terrible thing about the SXML is the preserved-whitespace from the XML (which of course wouldn't exist in pure SXML); otherwise it's much nicer and contains exactly as much information.
pavement 9y ago
Which is funny, because all of the things I hate about Microsoft and Windows have absolutely nothing to do with whether or not they provide bloated binaries, containing PNG images that are bundled with extra XML tags and descriptors. Gee whiz! What a world!
II2II 9y ago
By the sounds of it, this bloat is minor. (Keep in mind, the author is pointing out the two most extreme examples.) Bloat arises from a lot of different places, a lot of which cannot realistically be controlled without drastically affecting user expectations, system performance, and how software is developed. Consider graphics. If you are quadrupling the color depth, you are quadrupling the amount of memory required for graphics resources. Even more fun, if you are doubling the resolution you are quadrupling the amount of memory required for graphics resources. Going back to the olden days would only be an option if they are willing to compromise on the quality of the graphics. At the other end of the spectrum are developers. Should they really be choosing things like the type of an integer to reduce the size of code and data? Old software was often limited due to such considerations. In some cases software used bizarre tricks to reduce bloat, such as cramming data into the unused bits of an address. (Apparently that was common on 68000 based personal computers.) Don't get me wrong, there is a lot of unnecessarily bloated software. Yet I suspect that the vast majority of that bloat exists for very good reasons.
- lefty2 9y ago
  It doesn't sound too minor to me. It was 20% of windows explorer. But the worrying thing is that it's indicative of sloppy software development. If this basic stuff slips through, what else slips through as well?
  - II2II 9y ago
    The analysis was done on \Windows\explorer.exe, and it was noted as being an extreme case. The distinction is important for a couple of reasons. First, it is an extremely small part of Windows as a whole, weighing in at about 4.5 MB on disk (Windows 10 AE). Second, chances are that it is also a very small part of what people think of as Windows Explorer since it also depends upon external libraries. While Windows may suffer from unnecessary bloat, this article is not a very good evidence in that direction. Windows itself is much (much) more than a 4.5 MB binary and the growth of Windows over the years likely has more to do with changes in technology and the market than anything else. I also doubt that this is an indicator of sloppy software development, nor is this basic stuff. It simply indicates that the resources were added to the executable file more-or-less as is. Developers are unlikely to be concerned with the structure of the resources as long as those resources are in a format that is well understood by the software. Graphics designers are unlikely to be concerned with how the embedded data bloats the size of the executable. While stripping the excess data may seem like a basic good practice in retrospect, it is not basic nor is it sloppy in the sense that it doesn't strictly fall in the domain of the two groups responsible for handling the data.
JayXon 9y ago
shameless plug: you can throw a windows pe file (exe, dll, etc.) at leanify and it will remove all the garbage in pngs in that pe file (even those embedded pngs in high res ico file in pe file), and it will also optimize png compression with zopfli. But don't use it on windows system files because modifying those pe files will definitely break the digital signature.
- withinrafael 9y ago
  Author here -- Very neat, saves writing another tool!
wtbob 9y ago
Egad that XML is horrible! Whoever thought that could possibly be a sane format? And to repeat it over and over — it's like a boot stomping on disk space, forever.
- flukus 9y ago
  Adobe doesn't really get xml at all. A couple of times I've had to make software produce InDesign templates (https://spin.atomicobject.com/2017/04/25/dynamic-indesign-templates-xml-data/ https://spin.atomicobject.com/2017/04/25/dynamic-indesign-te...), you have to do string templating because real xml serializers aren't compatible with the "sortof xml" that adobe uses.
  - davidwtbuxton 9y ago
    I did a big project involving InDesign and the XML import a few years ago (CS4 times). Once I learnt to be very careful when editing the templates, it was pretty satisfying. I remember that certain XML tags had to use the exact namespace defined in the Adobe spec, but other than that it all seemed pretty XML compliant. I was using Python / ElementTree, and had to override the namespaces to make sure the exact name was being used. Or something. https://docs.python.org/2/library/xml.etree.elementtree.html#xml.etree.ElementTree.register_namespace https://docs.python.org/2/library/xml.etree.elementtree.html... What other problems did you encounter regarding XML compatibility?
    - flukus 9y ago
      It's honestly too many years ago now to remember, I just remember trying a couple of perfectly reasonable things (maybe attributes, maybe multiple sub lists, I'm not sure) and having it break. But I think it only broke when you created a new template, not when you were using an existing one with new data, or something along those lines. You could make a change and not realize it broke things for a week. It was incredibly temperamental too. I got the initial bare bones demo working and showed the powers that be, but after that a spent a week trying to do it again and it wouldn't work. I one stage me and a colleague went through a tutorial on it in sync and the same steps would work on one computer but not the other. Had we got it working well we might have even saved this particular company (mostly a graphic design one), they were in the advanced stages of circling the drain. It's frustrating when you see the potential of huge productivity improvements but their just out of reach.
fiatjaf 9y ago
Ok, how can a Windows user defend himself from that?
- Analemma_ 9y ago
  You can't. This metadata is in the system binaries, which are cryptographically signed and can't be changed. Hopefully the article spurs Microsoft to fix it though.
squarefoot 9y ago
I can't comment on this issue, but if you want to get an idea of how a company can take over an excellent software and ruin it making it beefier and slower, just take a look at the wonderful snappy gem that was Cool Edit Pro and what it became after being morphed into Adobe Audition.
- Intermernet 9y ago
  I still have an installer and valid license for the last version of Cool Edit Pro. So glad I hung on to it!

321 points by Integer 9y ago

blibble 9y ago

I wonder how many hundreds of kilobytes that adds up to in a 20gb windows install

moolcool 9y ago
You don't keep a 20gb Windows install in RAM all the time though. Bloat in explorer.exe is the issue here
- jszymborski 9y ago
  To this point, the files in the install are compressed, and I'm sure XML metadata is the sort of thing that compresses well with the DEFLATE algorithm they likely use.
  - PaulHoule 9y ago
    I discovered this a few months ago, when I went looking for XMP metadata in the filesystem and used the magic number trick to extract it from files of all kinds. I found it is common to find XMP inside media files embedded inside Windows EXE, as well as Linux binaries, JAR, Microsoft Word and other composite formats. Complex media objects frequently use an encapsulation system such as ZIP. When a PNG file is incorporated into a JAR or a Word Document, the XMP content in the file may not be compressed because the archiver may not attempt to compress the png file since it assumes the data is already compressed. XMP is very good from the viewpoint of content creators in terms of having comprehensive metadata incorporated into files so that it does not get out of sync. XMP data is RDF data using an improved version of Dublin Core, IPCC and other industry RDF vocabulary. You can write SPARQL queries right away, plus XMP specifies a way to make an XMP packet based on pre-existing metadata in common industry schemes. The XMP packets can get big, and you sometimes see people make a tiny GIF image (say a transparent pixel GIF) that is bulked up 100x because of bulky metadata. Once you package data for delivery to consumers you want to strip all that stuff out. The XMP spec is here: http://www.adobe.com/devnet/xmp.html http://www.adobe.com/devnet/xmp.html There is some brilliant thinking in there, but also things that will make your head explode such as the method for embedding an XMP packet into a GIF
    - jszymborski 9y ago
      Hmm... would be interesting if we started taking XMP into account when designing compression programs then...
      - abdias 9y ago
        You could actually take any ancillary chunks into consideration, ie. chunks starting with a lower-case first letter. These are non-critical/mandatory.
    - TazeTSchnitzel 9y ago
      > When a PNG file is incorporated into a JAR or a Word Document, the XMP content in the file may not be compressed because the archiver may not attempt to compress the png file since it assumes the data is already compressed. PNG can apply DEFLATE to blocks though, right? Does XMP not use it?
      - abdias 9y ago
        Deflating can be applied to some chunks, but not at will. The zTXt chunk can be compressed while for example the tEXt chunk cannot. The newer iTXt chunk can vary. The two former are limited in scope and language encoding support, so iTXt is typically used for extended textual data such as XML/XMP etc. But if is saved compressed or not depends on the PNG encoder/host used (there can also be multiple instances of these chunks in the same file). Photoshop for instance saves uncompressed, I guess to give fast access for performance reasons (ie. file viewers using galleries for numerous images while displaying their meta-data).
- galacticpony 9y ago
  data contained in an exe (or dll) is not necessarily in RAM at all time
astrobe_ 9y ago
One could fit about 10 Linux live distros in those 20Gb. Those "few hundreds of kilobytes" are indeed trivial.
- 21 9y ago
  Windows comes with a ton of builtin drivers. So it will work with a lot of devices out of the box without needing an Internet connection to update.
  - icebraining 9y ago
    Same is true in most Linux distros. My stock Debian kernel comes with 2300+ drivers, and which only take ~130MB.
    - simooooo 9y ago
      None of them are ever for any of my hardware
      - cmdrfred 9y ago
        I have the opposite experience myself.
      - Something1234 9y ago
        What hardware are you using that the defaults don't work?
        bastawhiz 9y ago
        I've installed the latest Ubuntu in 2016 to find my mouse, wifi, and printer did not work. And my Cinema Display was stuck at 1024x768.
    - pikzen 9y ago
      https://blogs.msdn.microsoft.com/e7/2008/11/19/disk-space/ https://blogs.msdn.microsoft.com/e7/2008/11/19/disk-space/ Bear in mind, that was the Vista days, and Windows 10 now supports even more devices. 800MB of drivers at the time. I would not be surprised if Windows supported by default upwards of 10000 drivers. It works pretty much flawlessly on even somewhat obscure and old hardware. And when your OS is installed on that many consumer devices, and not informally standardized servers, you are going to meet those weird devices one way or the other. Windows drivers may also take up a bit more space individually because of the overhead caused by either the Windows Driver Model or Windows Driver Framework, but that's the price to pay to not have a driver crashing and bringing down your entire system. Yes, Linux, I'm looking at you.
kevindqc 9y ago
5 MB. https://gist.githubusercontent.com/riverar/f4a56b91580af1bd3cf4dfacc3733bc3/raw/8f35e3def3173ded72c5715e5af37e830b2a639f/results.txt https://gist.githubusercontent.com/riverar/f4a56b91580af1bd3...
- asveikau 9y ago
  Haven't looked at the code that generates this list (if available), but that sure looks to me like double-counting going on here. Most files in \Windows\WinSXS are hardlinks. Example pulled at random: D:\analysis\Windows\WinSxS\amd64_microsoft-windows -imageres_31bf3856ad364e35_10.0.15063.0_none_edd17c6c30b4bf9f\imageres.dll ... - Total: 4435 D:\analysis\Windows\System32\imageres.dll ... - Total: 4435 I am willing to bet those are the same file hardlinked and only wastes the 4435 bytes once, verifiable thusly: cmd> fsutil hardlink list \Windows\System32\imageres.dll Windows\WinSxS\amd64_microsoft-windows-imageres_31bf3856ad364e35_6.3.9600.16384_ none_cd7c033dcbdd0cab\imageres.dll Windows\System32\imageres.dll
  - kevindqc 9y ago
    The source seems to be: https://github.com/riverar/eoraptor https://github.com/riverar/eoraptor Seems to check though https://github.com/riverar/eoraptor/blob/master/FileEnumerator.cpp#L19 https://github.com/riverar/eoraptor/blob/master/FileEnumerat...
    - asveikau 9y ago
      As I suspected, that doesn't look hardlink aware. A way to correct for this would be to open the files and de-dupe by (((ULONGLONG)nFileIndexHigh) << 32) | nFileIndexLow in this structure: https://msdn.microsoft.com/en-us/library/windows/desktop/aa363788(v=vs.85).aspx https://msdn.microsoft.com/en-us/library/windows/desktop/aa3... Edit: > Seems to check though https://github.com/riverar/eoraptor/blob/master/FileEnumerator.cpp#L19 https://github.com/riverar/eoraptor/blob/master/FileEnumerat... No it does not, reparse points are used for symbolic links and junctions - not hardlinks.
ashark 9y ago
What's the story on that footprint, anyway? We went from (from memory) ~250-350MB Win98 to ~700-800MB XP to ~10-15GB(!?!?!) Win7, and just up from there. Plus the default settings seemed to starting going really crazy with swapspace/caching around the time of Win7. Another 10+GB if you didn't tell it to knock that crap off. Why the sudden, giant shift? They didn't add 10-15x the features, that's for sure.
- rpeden 9y ago
  I think at least some of it is the Windows on Windows stuff to allow 64 bit machines to run both 32 and 64 bit software. Weren't the 32 but versions of Win7 about half the size of the 64 bit ones? There's still a lot of size growth over time, of course.
- jug 9y ago
  WinSxS (Windows Side By Side) assemblies were introduced to avoid dll hell by allowing Windows to store multiple versions of installed dll's. So even a minor security patch may leave the former version around because other apps may use/expect it. I think that might add some bloat over time? Also Windows Update installer caches. A ton of Windows updates actually leave their installers around in case you want to uninstall them. That can add up! I've seen it easily get to 1-2 GB.
- bastawhiz 9y ago
  They did, to some extent. Plug in almost any piece of standard consumer hardware and it'll probably mostly just work without a network connection. All those drivers don't take up zero space, but the benefit when my mom plugs in a printer and it just works makes it worth it.

thinknot 9y ago

Next time, run `optipng -o9 -strip all` on all your png files!

sp332 9y ago

Is it feasible to remove this junk yourself, or will the system freak out about hacked binaries? Would it also complain if I just applied to the PNG files?

INTPenis 9y ago
I'm a Linux specialist and a stranger to Windows but logically any modification of binaries should result in security issues. And in a perfect world the external PNG content would also be verified.
- 21 9y ago
  All Windows binaries are signed. Changing the embedded PNG will void the signature. Not sure what Windows will do if explorer.exe has a bad signature. There is also a Windows system integrity checker service which disallows changes to protected Windows files, and repairs them automatically (using a cached copy).
  - kuschku 9y ago
    Well, then you modify that, too. It’s your computer, you installed the software, you have a license, therefore you own that copy, and can modify it however you wish. (EU Copyright Directive, especially Article 6 and following). Now, the question is, why does Windows not allow me to add signatures that should be considered acceptable by default, why can I not modify my own OS installation?
    - 21 9y ago
      Can you modify Android or iOS system files or add your key? Rooting is not really an answer, because in that case you can also root Windows. I was under the impression that you can only legally modify files for fair use or compatibility purposes, not just because you want so.
      - kuschku 9y ago

tomcam 9y ago

You would be surprised at how much cruft in Windows over the years has been directly due to Adobe. I had many bug triage sessions where Windows developers at Microsoft had to work around Adobe problems to keep Windows users happy. I always thought it was unfair and was quite impressed by Microsoft at their willingness to handle this so quietly.

batter 9y ago
Doesn't it makes you wonder why MS created that situation on the first place? MS have no one to blame except themself and their 'legacy code'.
- commandar 9y ago
  It's hard enough to get enterprise customers to stay current when they do provide legacy support. Fighting with IE version compatibility, for example, is still a daily, very real issue for enterprise size organizations. Blame the customers. Microsoft never would have captured the marketshare they have if they didn't cater to them.
  - DaiPlusPlus 9y ago
    Microsoft does deserve some blame for deciding to support backwards-compatibility in-situ instead of introducing proper API versioning, packaging, isolation, and other proven techniques. Win32 wasn't versioned for processes until Windows 7 (using app.manifest) even though the need for such as system was blatantly obvious during the Windows 2000 days.
    - commandar 9y ago
      Sure, that's valid. But most of the real nightmare scenarios I've heard related to backwards compatibility have more to do with third-parties doing things they were never supposed to do. Things like hitting private, undocumented APIs. Or checking the Windows version with a "9x" wildcard, giving us the jump from W8 to W10 over a decade later. Microsoft has made their own mistakes, but supporting the mistakes of third parties has been absolutely vital to them keeping their core customer base.

mih 9y ago

In one of the comments on the page, a reader ran the analysis [1] on a windows installation and reports the bloat size. Total bytes wasted: 5341278 [1] https://gist.githubusercontent.com/riverar/f4a56b91580af1bd3cf4dfacc3733bc3/raw/8f35e3def3173ded72c5715e5af37e830b2a639f/results.txt https://gist.githubusercontent.com/riverar/f4a56b91580af1bd3...

blibble 9y ago
amusingly that's about 50% of the size of the webpage itself (2.4mb)
- delinka 9y ago
  Surely you mean twice the size of the web page.
  - blibble 9y ago
    er, yes!
deleted 9y ago
[deleted]
deleted 9y ago
[deleted]
WatchDog 9y ago
That's 5MB, for the humans among us.
balls187 9y ago
I believe that was not a reader, but the author of the article.
- mih 9y ago
  You're right. My mistake

msimpson 9y ago

Given this bloat resides in the metadata of PNG assets exported from Photoshop, couldn't this affect any operating system? How many applications on Mac OS utilize PNG assets which were exported from Photoshop without any further optimization?

0x0 9y ago
When you add a .png file to an xcode iOS project, it will add a build step to pngcrush the asset automatically. https://developer.apple.com/library/content/qa/qa1681/_index.html https://developer.apple.com/library/content/qa/qa1681/_index...
- msimpson 9y ago
  Interesting. I did not know that. Would the same hold true outside iOS targets?
  - saagarjha 9y ago
    macOS supports true vector assets, so I'm guessing it reduces the need for such measures. But coming back to your question: I'm not sure, but both iOS and macOS use the same asset filetype, so I'd assume so.
ghostly_s 9y ago
I would put good money on 'none'. Apple is way too much of a stickler about MacOS' graphics APIs to not be using custom authoring or optimization routines on assets. Edit: I presumed you're talking about first-party applications only, as you said 'operating system'.

0x0 9y ago

It's not the first time Windows has shipped with shameful metadata. For example, a .wav file shipped with Windows XP appears to be authored with a pirated version of SoundForge: https://web.archive.org/web/20060721090521/http://techrepublic.com.com/5208-11183-0.html?forumID=89&threadID=173539&messageID=1765547 https://web.archive.org/web/20060721090521/http://techrepubl...

douche 9y ago
Eh, that sort of thing happens all the time. We have customers that are national banks of foreign countries that are running on unactivated or cracked Windows servers. I would bet that case was a contractor who was using their own equipment. Similar things have bit us in the ass with freelancers that have ripped off stock images and presented them as their own.
- 0x0 9y ago
  Yeah, but you'd expect the golden master .iso of a major Windows release to be examined in greater detail and not sloppily add bloat like that. It's in great contrast to earlier releases like Windows 95 where blinking dots were removed from the task bar to save a few kilobytes of demand paged memory. https://blogs.msdn.microsoft.com/oldnewthing/20031010-00/?p=42203 https://blogs.msdn.microsoft.com/oldnewthing/20031010-00/?p=...
  - rangibaby 9y ago
    The difference is Win95 had a floppy disk version sold to consumers. 13 disks specially formatted to store an extra 200kb each. https://blogs.msdn.microsoft.com/oldnewthing/20050819-10/?p=34513 https://blogs.msdn.microsoft.com/oldnewthing/20050819-10/?p=...
  - douche 9y ago
    Moore's law has allowed us to be lazy, and things have gotten too complicated for anybody to fully understand everything. I feel like I spend half my time trying to delete old dead code, and I can't keep up.

21 9y ago

I remember an article saying that making a trivial change to Windows requires 5 minutes to change the code and 2 weeks to deal with the aftermath (testing/...) I wonder how easy it actually is to remove this XMP metadata, considering that it could potentially break some application which loads a PNG directly from explorer.exe with a broken PNG parser or something.

Esau 9y ago

It's not just Windows that is bloated; so is macOS, Android, and iOS. The wastefulness annoys me and I don't want to hear its okay because we have tons of disk space and RAM - it is still wasteful. I understand why they kitchen-sink operating systems - its mainly so they can crow about new features when releasing new versions of the OS. But I wish they would offer alternate installs for those of us who are proficient.

Fifer82 9y ago
409 grains of sugar please or I am going elsewhere.... Actually....
flukus 9y ago
On android I'm still severely limited by disk space and RAM, the waste is very noticeable. It still is on PC for most users too, cheap SSD's are still quite small, there are a lot of new laptops sold with 64GB SSD's and 2GB RAM. Good luck running anything in node.
- deleted 9y ago
  [deleted]
zacmps 9y ago
I agree, it's a lot of the reason I like distros like Arch or Alpine.
saagarjha 9y ago
Looks like Linux is what you're looking for…
- realusername 9y ago
  I would really like to have a Linux-style OS on my smartphone and not the bloated Android Java which looks like the Win32 API re-engineered for smartphones.

asveikau 9y ago

It's great investigative work into Windows binaries, and I hope it gets addressed for the sake of people's disk space, but I think the tone is too harsh and overstated. Example: He cites effects on startup time - but has he considered the existence of virtual memory? When explorer.exe loads and maps the bloat into address space, it doesn't need it in RAM until the first page fault accessing it which likely will not even happen.

codys 9y ago
So that is true as long as all the bloat is contiguous. If it is spread out throughout the file (in such a way that bloat doesn't fill an entire page) it will still end up loaded. Or even if it is "unused", that doesn't mean something isn't scanning over it byte-by-byte. In the happy case, yes, virtual memory will save us. But there are a lot of ways we could still end up loading the junk into ram. Also, there are potential runtime costs to it being larger just on disk (need to seek over it, etc).
withinrafael 9y ago
Most binaries in Windows are signed. This requires the loader to load each PE section, hash, do some number crunching, and compare the result -- which results in the entire file being loaded into memory before execution. This involves (potentially random?) disk I/O, which can be surprisingly slow on certain platforms (e.g. Xbox One, HoloLens, anything IoT, anything with eMMC).
- asveikau 9y ago
  Fair point. Another reason code signing is stupid.

zeveb 9y ago

Wow, that's a bloated format. Here it is as XML: <?xpacket begin="?" id="W5M0MpCehiHzreSzNTczkc9d"?> <x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Adobe XMP Core 5.4-c002 1.000000, 0000/00/00-00:00:00 "> <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"> <rdf:Description rdf:about="" xmlns:xmp="http://ns.adobe.com/xap/1.0/"> <xmp:CreatorTool>Picasa</xmp:CreatorTool> </rdf:Description> <rdf:Description rdf:about="" xmlns:mwg-rs="http://www.metadataworkinggroup.com/schemas/regions/" xmlns:stDim="http://ns.adobe.com/xap/1.0/sType/Dimensions#" xmlns:stArea="http://ns.adobe.com/xmp/sType/Area#"> <mwg-rs:Regions rdf:parseType="Resource"> <mwg-rs:AppliedToDimensions rdf:parseType="Resource"> <stDim:w>912</stDim:w> <stDim:h>687</stDim:h> <stDim:unit>pixel</stDim:unit> </mwg-rs:AppliedToDimensions> <mwg-rs:RegionList> <rdf:Bag> <rdf:li rdf:parseType="Resource"> <mwg-rs:Type></mwg-rs:Type> <mwg-rs:Area rdf:parseType="Resource"> <stArea:x>0.680921052631579</stArea:x> <stArea:y>0.3537117903930131</stArea:y> <stArea:h>0.4264919941775837</stArea:h> <stArea:w>0.32127192982456143</stArea:w> <stArea:unit>normalized</stArea:unit> </mwg-rs:Area> </rdf:li> </rdf:Bag> </mwg-rs:RegionList> </mwg-rs:Regions> </rdf:Description> <rdf:Description rdf:about="" xmlns:exif="http://ns.adobe.com/exif/1.0/"> <exif:PixelXDimension>912</exif:PixelXDimension> <exif:PixelYDimension>687</exif:PixelYDimension> <exif:ExifVersion>0220</exif:ExifVersion> </rdf:Description> </rdf:RDF> </x:xmpmeta>  <?xpacket end="w"?> And here it is as SXML (https://en.wikipedia.org/wiki/SXML https://en.wikipedia.org/wiki/SXML): (*TOP* (*PI* |xpacket| "begin=\"?\" id=\"W5M0MpCehiHzreSzNTczkc9d\"") (|adobe:ns:meta/:xmpmeta| (@ (@ (*NAMESPACES* (|adobe:ns:meta/| "adobe:ns:meta/" . |x|))) (|adobe:ns:meta/:xmptk| "Adobe XMP Core 5.4-c002 1.000000, 0000/00/00-00:00:00 ")) " " (|http://www.w3.org/1999/02/22-rdf-syntax-ns#:RDF| (@ (@ (*NAMESPACES* (|http://www.w3.org/1999/02/22-rdf-syntax-ns#| "http://www.w3.org/1999/02/22-rdf-syntax-ns#" . |rdf|)))) " " (|http://www.w3.org/1999/02/22-rdf-syntax-ns#:Description| (@ (@ (*NAMESPACES* (|http://ns.adobe.com/xap/1.0/| "http://ns.adobe.com/xap/1.0/" . |xmp|))) (|http://www.w3.org/1999/02/22-rdf-syntax-ns#:about| "")) " " (|http://ns.adobe.com/xap/1.0/:CreatorTool| "Picasa") " ") " " (|http://www.w3.org/1999/02/22-rdf-syntax-ns#:Description| (@ (@ (*NAMESPACES* (|http://ns.adobe.com/xmp/sType/Area#| "http://ns.adobe.com/xmp/sType/Area#" . |stArea|) (|http://ns.adobe.com/xap/1.0/sType/Dimensions#| "http://ns.adobe.com/xap/1.0/sType/Dimensions#" . |stDim|) (|http://www.metadataworkinggroup.com/schemas/regions/| "http://www.metadataworkinggroup.com/schemas/regions/" . |mwg-rs|))) (|http://www.w3.org/1999/02/22-rdf-syntax-ns#:about| "")) " " (|http://www.metadataworkinggroup.com/schemas/regions/:Regions| (@ (|http://www.w3.org/1999/02/22-rdf-syntax-ns#:parseType| "Resource")) " " (|http://www.metadataworkinggroup.com/schemas/regions/:AppliedToDimensions| (@ (|http://www.w3.org/1999/02/22-rdf-syntax-ns#:parseType| "Resource")) " " (|http://ns.adobe.com/xap/1.0/sType/Dimensions#:w| "912") " " (|http://ns.adobe.com/xap/1.0/sType/Dimensions#:h| "687") " " (|http://ns.adobe.com/xap/1.0/sType/Dimensions#:unit| "pixel") " ") " " (|http://www.metadataworkinggroup.com/schemas/regions/:RegionList| " " (|http://www.w3.org/1999/02/22-rdf-syntax-ns#:Bag| " " (|http://www.w3.org/1999/02/22-rdf-syntax-ns#:li| (@ (|http://www.w3.org/1999/02/22-rdf-syntax-ns#:parseType| "Resource")) " " (|http://www.metadataworkinggroup.com/schemas/regions/:Type|) " " (|http://www.metadataworkinggroup.com/schemas/regions/:Area| (@ (|http://www.w3.org/1999/02/22-rdf-syntax-ns#:parseType| "Resource")) " " (|http://ns.adobe.com/xmp/sType/Area#:x| "0.680921052631579") " " (|http://ns.adobe.com/xmp/sType/Area#:y| "0.3537117903930131") " " (|http://ns.adobe.com/xmp/sType/Area#:h| "0.4264919941775837") " " (|http://ns.adobe.com/xmp/sType/Area#:w| "0.32127192982456143") " " (|http://ns.adobe.com/xmp/sType/Area#:unit| "normalized") " ") " ") " ") " ") " ") " ") " " (|http://www.w3.org/1999/02/22-rdf-syntax-ns#:Description| (@ (@ (*NAMESPACES* (|http://ns.adobe.com/exif/1.0/| "http://ns.adobe.com/exif/1.0/" . |exif|))) (|http://www.w3.org/1999/02/22-rdf-syntax-ns#:about| "")) " " (|http://ns.adobe.com/exif/1.0/:PixelXDimension| "912") " " (|http://ns.adobe.com/exif/1.0/:PixelYDimension| "687") " " (|http://ns.adobe.com/exif/1.0/:ExifVersion| "0220") " ") " ") " ") (*COMMENT* " whitespace padding ") (*PI* |xpacket| "end=\"w\"")) The only terrible thing about the SXML is the preserved-whitespace from the XML (which of course wouldn't exist in pure SXML); otherwise it's much nicer and contains exactly as much information.

pavement 9y ago

Which is funny, because all of the things I hate about Microsoft and Windows have absolutely nothing to do with whether or not they provide bloated binaries, containing PNG images that are bundled with extra XML tags and descriptors. Gee whiz! What a world!

II2II 9y ago

By the sounds of it, this bloat is minor. (Keep in mind, the author is pointing out the two most extreme examples.) Bloat arises from a lot of different places, a lot of which cannot realistically be controlled without drastically affecting user expectations, system performance, and how software is developed. Consider graphics. If you are quadrupling the color depth, you are quadrupling the amount of memory required for graphics resources. Even more fun, if you are doubling the resolution you are quadrupling the amount of memory required for graphics resources. Going back to the olden days would only be an option if they are willing to compromise on the quality of the graphics. At the other end of the spectrum are developers. Should they really be choosing things like the type of an integer to reduce the size of code and data? Old software was often limited due to such considerations. In some cases software used bizarre tricks to reduce bloat, such as cramming data into the unused bits of an address. (Apparently that was common on 68000 based personal computers.) Don't get me wrong, there is a lot of unnecessarily bloated software. Yet I suspect that the vast majority of that bloat exists for very good reasons.

lefty2 9y ago
It doesn't sound too minor to me. It was 20% of windows explorer. But the worrying thing is that it's indicative of sloppy software development. If this basic stuff slips through, what else slips through as well?
- II2II 9y ago
  The analysis was done on \Windows\explorer.exe, and it was noted as being an extreme case. The distinction is important for a couple of reasons. First, it is an extremely small part of Windows as a whole, weighing in at about 4.5 MB on disk (Windows 10 AE). Second, chances are that it is also a very small part of what people think of as Windows Explorer since it also depends upon external libraries. While Windows may suffer from unnecessary bloat, this article is not a very good evidence in that direction. Windows itself is much (much) more than a 4.5 MB binary and the growth of Windows over the years likely has more to do with changes in technology and the market than anything else. I also doubt that this is an indicator of sloppy software development, nor is this basic stuff. It simply indicates that the resources were added to the executable file more-or-less as is. Developers are unlikely to be concerned with the structure of the resources as long as those resources are in a format that is well understood by the software. Graphics designers are unlikely to be concerned with how the embedded data bloats the size of the executable. While stripping the excess data may seem like a basic good practice in retrospect, it is not basic nor is it sloppy in the sense that it doesn't strictly fall in the domain of the two groups responsible for handling the data.

JayXon 9y ago

shameless plug: you can throw a windows pe file (exe, dll, etc.) at leanify and it will remove all the garbage in pngs in that pe file (even those embedded pngs in high res ico file in pe file), and it will also optimize png compression with zopfli. But don't use it on windows system files because modifying those pe files will definitely break the digital signature.

withinrafael 9y ago
Author here -- Very neat, saves writing another tool!

wtbob 9y ago

Egad that XML is horrible! Whoever thought that could possibly be a sane format? And to repeat it over and over — it's like a boot stomping on disk space, forever.

flukus 9y ago
Adobe doesn't really get xml at all. A couple of times I've had to make software produce InDesign templates (https://spin.atomicobject.com/2017/04/25/dynamic-indesign-templates-xml-data/ https://spin.atomicobject.com/2017/04/25/dynamic-indesign-te...), you have to do string templating because real xml serializers aren't compatible with the "sortof xml" that adobe uses.
- davidwtbuxton 9y ago
  I did a big project involving InDesign and the XML import a few years ago (CS4 times). Once I learnt to be very careful when editing the templates, it was pretty satisfying. I remember that certain XML tags had to use the exact namespace defined in the Adobe spec, but other than that it all seemed pretty XML compliant. I was using Python / ElementTree, and had to override the namespaces to make sure the exact name was being used. Or something. https://docs.python.org/2/library/xml.etree.elementtree.html#xml.etree.ElementTree.register_namespace https://docs.python.org/2/library/xml.etree.elementtree.html... What other problems did you encounter regarding XML compatibility?
  - flukus 9y ago
    It's honestly too many years ago now to remember, I just remember trying a couple of perfectly reasonable things (maybe attributes, maybe multiple sub lists, I'm not sure) and having it break. But I think it only broke when you created a new template, not when you were using an existing one with new data, or something along those lines. You could make a change and not realize it broke things for a week. It was incredibly temperamental too. I got the initial bare bones demo working and showed the powers that be, but after that a spent a week trying to do it again and it wouldn't work. I one stage me and a colleague went through a tutorial on it in sync and the same steps would work on one computer but not the other. Had we got it working well we might have even saved this particular company (mostly a graphic design one), they were in the advanced stages of circling the drain. It's frustrating when you see the potential of huge productivity improvements but their just out of reach.

fiatjaf 9y ago

Ok, how can a Windows user defend himself from that?

Analemma_ 9y ago
You can't. This metadata is in the system binaries, which are cryptographically signed and can't be changed. Hopefully the article spurs Microsoft to fix it though.

squarefoot 9y ago

I can't comment on this issue, but if you want to get an idea of how a company can take over an excellent software and ruin it making it beefier and slower, just take a look at the wonderful snappy gem that was Cool Edit Pro and what it became after being morphed into Adobe Audition.

Intermernet 9y ago
I still have an installer and valid license for the last version of Cool Edit Pro. So glad I hung on to it!