6 ms·

WebKit Limit on Retina JPG Image Display

127 points by mikecane 15y ago

orta 15y ago
When working in UIKit developers are warned now to use images that are larger than 1024x1024 because that is the limit on the size if a texture in openGLES. I wouldn't be surprised to see this being doubled for the retina and Apple using that as the hard limit. ( You should avoid creating UIImage objects that are greater than 1024 x 1024 in size. Besides the large amount of memory such an image would consume, you may run into problems when using the image as a texture in OpenGL ES or when drawing the image to a view or layer - http://developer.apple.com/library/ios/#documentation/uikit/reference/UIImage_Class/Reference/Reference.html http://developer.apple.com/library/ios/#documentation/uikit/...)
- jbrennan 15y ago
  I believe that limit was lifted as of the iPhone 3GS (i.e., for OpenGL ES 2.0 devices and greater).
  - daeken 15y ago
    There's also no reason you can't split the image up (e.g. 2048x2048 -> 4x1024x1024) and render it in 4 pieces. Where a hard limitation like that exists, implementations frequently sidestep them.
    - mistercow 15y ago
      If you do this though, make sure you split at JPEG block boundaries. That means that every column should have a width divisible by 8 except the rightmost, and every row should have a height divisible by 8 except the bottom. Otherwise you may see a noticeable seam.
      - jonhohle 15y ago
        You can split the JPEGs at bit mapped images on the device before rendering. At that pointed you wouldn't be limited tithe block boundaries anymore. edit for clarity: don't manipulate images in the JPEG domain.
      - 0x09 15y ago
        Better make that 16 unless your image is greyscale.
  - superrad 15y ago
    I think it's actually just been doubled to 2048x2048. OpenGL ES 2.0 seems to define a 64x64 as the minimum requirement for texture size to meet the spec, which is really not much at all.
mva 15y ago
Another issue, but also there: PNG images above a certain height failed to load on the iPhone 4. We were facing that problem with our mobile testing tool. Converting the images to JPG solved the problem.
- woogley 15y ago
  In case you're wondering why that is .. > JPEG images can be up to 32 megapixels due to subsampling, which allows JPEG images to decode to a size that has one sixteenth the number of pixels. JPEG images larger than 2 megapixels are subsampled—that is, decoded to a reduced size. JPEG subsampling allows the user to view images from the latest digital cameras. Reference: https://developer.apple.com/library/IOs/#documentation/AppleApplications/Reference/SafariWebContent/CreatingContentforSafarioniPhone/CreatingContentforSafarioniPhone.html https://developer.apple.com/library/IOs/#documentation/Apple...
  - ajasmin 15y ago
    Yes. libjpeg can generate scaled previews faster than decoding the whole image. Maybe that's why Apple didn't bother to implement this for png images.
apinstein 15y ago
We ran into this issue while building our HTML5/Mobile Webkit panorama viewer. It took a long time to figure out what was going on, I almost feel that webkit would've been better to just skip the image and log an error to the console, since by resizing your image without your permission typically causes rendering bugs since they literally resize your image down in-line without telling you. We didn't realize it could be circumvented with PNG's, that's an interesting idea, though it certainly hurts on the bandwidth side.
- rorrr 15y ago
  You could just slice it into multiple JPGs. It's probably even better for parallel download.
  - gokhan 15y ago
    Better? Wrong. Please read "Minimize HTTP Requests" here: http://developer.yahoo.com/performance/rules.html http://developer.yahoo.com/performance/rules.html
    - wisty 15y ago
      If you use domain sharding you can get more parellel downloads on older browsers which reduces the problems some due to more requests (parellel request limit on old browsers and unneeded cookies for static resources, but theres still overhead), but that's a separate issue. It's best to avoid the problem altogether by minimizing HTTP Requests (as you suggest).
    - huxley 15y ago
      That's a great rule for general purposes, but it's intended for situations where you have lots of small assets that you are asking the client to download. In this instance we're talking about 2 megapixel images, so the extra HTTP requests are not even going to register in the mix. With modern browsers pulling down 6+ parallel downloads, breaking the image up might make plenty of sense.
    - rorrr 15y ago
bbwharris 15y ago
Had this problem with a large background image (over 2 Megapixels, I believe). The solution that we came up with was to split the image in two, and then use css3 multiple backgrounds on the ipad, iphone. At that time, those were the only culprits.
hetman 15y ago
On a mostly unrelated note, I wish blatanly marketing driven terms like "retina display" would stop getting thrown around like this. Thanks to Apple's latest marketing adjustment for the iPad 3, it turns the phone I've owned for over 2 years now actually posesses a "retina display" (N900 for anyone wondering). At the viewing distance specified by Apple, the human retina is in fact capable of discerning a far higher pixel density anyway.
- brianpan 15y ago
  How is he "throwing around" this term? He's specifically talking about the iPad 3. Apple uses this marketing term to signal to the purchaser that they are going to get a certain pixel density. I see retina display, I know I will get a "good looking" display by Apple's standards. If I trust Apple, it means something. That's the purpose of marketing and branding. So what if your phone also has the same pixel density? Do you have a problem with marketing in general? Is it a problem that Audi makes quattro cars or that Volkswagens have 4motion or BMWs have cars with 'x' in the model number? They only signify 4WD after all. BTW, iPhone 4 is a 3.5 inch 640×960 resolution (326 ppi). N900 is 3.5 inch 800 × 480 resolution (267 ppi). Not that it matters either way.
  - eurleif 15y ago
    >BTW, iPhone 4 is a 3.5 inch 640×960 resolution (326 ppi). But the new iPad is 264 PPI.
    - davidmp 15y ago
      iPad is "retina" at a viewing distance of 15 inches. iPhone is "retina" at a viewing distance of 10 inches. Pixel density is only half the equation.
    - brianpan 15y ago
      I included the screen size, because it's a proxy for working distance, and thus pixel density per viewing angle. http://punchingin.com/chasing-the-6-4-megapixel-unicorn/ http://punchingin.com/chasing-the-6-4-megapixel-unicorn/ http://www.codinghorror.com/blog/2012/03/welcome-to-the-post-pc-era.html http://www.codinghorror.com/blog/2012/03/welcome-to-the-post...
josscrowcroft 15y ago
Some excellent snooping and calculations there!
- pslam 15y ago
  Or simply looking at the source code would have found an authoritative answer.
  - olalonde 15y ago
    Simply? Why don't you post the location of the bug-feature?
jjcm 15y ago
Ran into this a couple days ago. Thanks for the heads up, clarifies some confusion I had.
huhtenberg 15y ago
> That’s 2,150,400 pixels, which is bigger than the 2 megapixel limit above. On the other hand, those dimensions are evenly divisible by 16. JPEG block size is 16, so this must be an alignment related.
yonran 15y ago
Question: does the HTMLImageElement actually contain fewer pixels, or is it just displayed low-res? Could you take your jpeg, paint it on a canvas using script, and then display it full-resolution?
DHowett 15y ago
This bug is sort of funny (in the frustrating way) when you're looking for high-resolution wallpapers for the iPad - At least on InterfaceLIFT on-device, you get a 1024x1024 image when trying to fetch the iPad3,x version.
unwiredben 15y ago
I remember hitting a similar limit in webOS. There's a WebKit image cache setting set either at build time or startup time about how large of an image will be allowed to persist in the internal decoded image cache. When an image exceeds that, the decoder produces a scaled down version that's grown when displayed. We could adjust it on our platform with a setting in /etc/palm/browser.conf, but it wasn't adjustable without making the core file system writable and restarting the system manager.
codezero 15y ago
I tested and confirmed that if you save the JPEG using Progressive formatting, then it displays fine on the iPad. The Apple "Big Hero" image also has progressive JPEG formatting. This doesn't explain why regular JPEGs are treated special though.

127 points by mikecane 15y ago

orta 15y ago

When working in UIKit developers are warned now to use images that are larger than 1024x1024 because that is the limit on the size if a texture in openGLES. I wouldn't be surprised to see this being doubled for the retina and Apple using that as the hard limit. ( You should avoid creating UIImage objects that are greater than 1024 x 1024 in size. Besides the large amount of memory such an image would consume, you may run into problems when using the image as a texture in OpenGL ES or when drawing the image to a view or layer - http://developer.apple.com/library/ios/#documentation/uikit/reference/UIImage_Class/Reference/Reference.html http://developer.apple.com/library/ios/#documentation/uikit/...)

jbrennan 15y ago
I believe that limit was lifted as of the iPhone 3GS (i.e., for OpenGL ES 2.0 devices and greater).
- daeken 15y ago
  There's also no reason you can't split the image up (e.g. 2048x2048 -> 4x1024x1024) and render it in 4 pieces. Where a hard limitation like that exists, implementations frequently sidestep them.
  - mistercow 15y ago
    If you do this though, make sure you split at JPEG block boundaries. That means that every column should have a width divisible by 8 except the rightmost, and every row should have a height divisible by 8 except the bottom. Otherwise you may see a noticeable seam.
    - jonhohle 15y ago
      You can split the JPEGs at bit mapped images on the device before rendering. At that pointed you wouldn't be limited tithe block boundaries anymore. edit for clarity: don't manipulate images in the JPEG domain.
    - 0x09 15y ago
      Better make that 16 unless your image is greyscale.
- superrad 15y ago
  I think it's actually just been doubled to 2048x2048. OpenGL ES 2.0 seems to define a 64x64 as the minimum requirement for texture size to meet the spec, which is really not much at all.

mva 15y ago

Another issue, but also there: PNG images above a certain height failed to load on the iPhone 4. We were facing that problem with our mobile testing tool. Converting the images to JPG solved the problem.

woogley 15y ago
In case you're wondering why that is .. > JPEG images can be up to 32 megapixels due to subsampling, which allows JPEG images to decode to a size that has one sixteenth the number of pixels. JPEG images larger than 2 megapixels are subsampled—that is, decoded to a reduced size. JPEG subsampling allows the user to view images from the latest digital cameras. Reference: https://developer.apple.com/library/IOs/#documentation/AppleApplications/Reference/SafariWebContent/CreatingContentforSafarioniPhone/CreatingContentforSafarioniPhone.html https://developer.apple.com/library/IOs/#documentation/Apple...
- ajasmin 15y ago
  Yes. libjpeg can generate scaled previews faster than decoding the whole image. Maybe that's why Apple didn't bother to implement this for png images.

apinstein 15y ago

We ran into this issue while building our HTML5/Mobile Webkit panorama viewer. It took a long time to figure out what was going on, I almost feel that webkit would've been better to just skip the image and log an error to the console, since by resizing your image without your permission typically causes rendering bugs since they literally resize your image down in-line without telling you. We didn't realize it could be circumvented with PNG's, that's an interesting idea, though it certainly hurts on the bandwidth side.

rorrr 15y ago
You could just slice it into multiple JPGs. It's probably even better for parallel download.
- gokhan 15y ago
  Better? Wrong. Please read "Minimize HTTP Requests" here: http://developer.yahoo.com/performance/rules.html http://developer.yahoo.com/performance/rules.html
  - wisty 15y ago
    If you use domain sharding you can get more parellel downloads on older browsers which reduces the problems some due to more requests (parellel request limit on old browsers and unneeded cookies for static resources, but theres still overhead), but that's a separate issue. It's best to avoid the problem altogether by minimizing HTTP Requests (as you suggest).
  - huxley 15y ago
    That's a great rule for general purposes, but it's intended for situations where you have lots of small assets that you are asking the client to download. In this instance we're talking about 2 megapixel images, so the extra HTTP requests are not even going to register in the mix. With modern browsers pulling down 6+ parallel downloads, breaking the image up might make plenty of sense.
  - rorrr 15y ago

bbwharris 15y ago

Had this problem with a large background image (over 2 Megapixels, I believe). The solution that we came up with was to split the image in two, and then use css3 multiple backgrounds on the ipad, iphone. At that time, those were the only culprits.

hetman 15y ago

On a mostly unrelated note, I wish blatanly marketing driven terms like "retina display" would stop getting thrown around like this. Thanks to Apple's latest marketing adjustment for the iPad 3, it turns the phone I've owned for over 2 years now actually posesses a "retina display" (N900 for anyone wondering). At the viewing distance specified by Apple, the human retina is in fact capable of discerning a far higher pixel density anyway.

brianpan 15y ago
How is he "throwing around" this term? He's specifically talking about the iPad 3. Apple uses this marketing term to signal to the purchaser that they are going to get a certain pixel density. I see retina display, I know I will get a "good looking" display by Apple's standards. If I trust Apple, it means something. That's the purpose of marketing and branding. So what if your phone also has the same pixel density? Do you have a problem with marketing in general? Is it a problem that Audi makes quattro cars or that Volkswagens have 4motion or BMWs have cars with 'x' in the model number? They only signify 4WD after all. BTW, iPhone 4 is a 3.5 inch 640×960 resolution (326 ppi). N900 is 3.5 inch 800 × 480 resolution (267 ppi). Not that it matters either way.
- eurleif 15y ago
  >BTW, iPhone 4 is a 3.5 inch 640×960 resolution (326 ppi). But the new iPad is 264 PPI.
  - davidmp 15y ago
    iPad is "retina" at a viewing distance of 15 inches. iPhone is "retina" at a viewing distance of 10 inches. Pixel density is only half the equation.
  - brianpan 15y ago
    I included the screen size, because it's a proxy for working distance, and thus pixel density per viewing angle. http://punchingin.com/chasing-the-6-4-megapixel-unicorn/ http://punchingin.com/chasing-the-6-4-megapixel-unicorn/ http://www.codinghorror.com/blog/2012/03/welcome-to-the-post-pc-era.html http://www.codinghorror.com/blog/2012/03/welcome-to-the-post...

josscrowcroft 15y ago

Some excellent snooping and calculations there!

pslam 15y ago
Or simply looking at the source code would have found an authoritative answer.
- olalonde 15y ago
  Simply? Why don't you post the location of the bug-feature?

jjcm 15y ago

Ran into this a couple days ago. Thanks for the heads up, clarifies some confusion I had.

huhtenberg 15y ago

> That’s 2,150,400 pixels, which is bigger than the 2 megapixel limit above. On the other hand, those dimensions are evenly divisible by 16. JPEG block size is 16, so this must be an alignment related.

yonran 15y ago

Question: does the HTMLImageElement actually contain fewer pixels, or is it just displayed low-res? Could you take your jpeg, paint it on a canvas using script, and then display it full-resolution?

DHowett 15y ago

This bug is sort of funny (in the frustrating way) when you're looking for high-resolution wallpapers for the iPad - At least on InterfaceLIFT on-device, you get a 1024x1024 image when trying to fetch the iPad3,x version.

unwiredben 15y ago

I remember hitting a similar limit in webOS. There's a WebKit image cache setting set either at build time or startup time about how large of an image will be allowed to persist in the internal decoded image cache. When an image exceeds that, the decoder produces a scaled down version that's grown when displayed. We could adjust it on our platform with a setting in /etc/palm/browser.conf, but it wasn't adjustable without making the core file system writable and restarting the system manager.

codezero 15y ago

I tested and confirmed that if you save the JPEG using Progressive formatting, then it displays fine on the iPad. The Apple "Big Hero" image also has progressive JPEG formatting. This doesn't explain why regular JPEGs are treated special though.