8 ms·

I ran into this headache while writing a web app. I made an image upload widget that provided a preview, and when users selected the "take a picture" option on

by enlyth 7y ago

I ran into this headache while writing a web app. I made an image upload widget that provided a preview, and when users selected the "take a picture" option on their phones, I showed the preview with a blob link and CSS background-image property. The images were showing sideways on some phones. I looked at the EXIF data of those photos and of course Orientation: 90 showed up. It was easy to fix on the backend when processing the images but I struggled to do it in a performant way on the front-end. One solution involved reading the EXIF data with JS and rotating with canvas and .toBlob(), but proved too slow for large 10MB photos as it blocks the main UI thread. One thing I thought of is just reading the orientation and the using CSS transforms to rotate the preview, but I never got around to trying it.

Waterluvian 7y ago
Try web workers! I've been doing all kinds of crazy expensive spatial data validation in many parallel threads. It's amazing. I think off screen canvas in workers is gaining browser support recently.
- inetknght 7y ago
  > Try web workers! NO, please do not try web workers. I don't want any more running on my device than absolutely necessary.
  - BubRoss 7y ago
    You don't want to use all your cores to run things faster?
    - dralley 7y ago
      It would be nice if they only made things faster. In practice it seems like they use all my cores to do more things, things that I don't want or care about them doing.
    - inetknght 7y ago
      Nope. I want things to look plain and simple. I want to use fewer cores for longer battery life.
      - BubRoss 7y ago
        The faster a CPU finishes, the faster the CPU can sleep, which is how you save battery life. The more cores you use the lower the CPU frequency can be, which saves power, since frequency increases do not use power linearly. I think it goes without saying that javascript workers have nothing to do with the design and layout of a webpage.
        inetknght 7y ago
        > The faster a CPU finishes, the faster the CPU can sleep, which is how you save battery life. Yup. > The more cores you use the lower the CPU frequency can be, which saves power, since frequency increases do not use power linearly. More cores in use has little to do with frequency and more to do with heat. More heat means more thermal throttling which lowers frequency. Lower frequency means that the CPU doesn't sleep sooner. > I think it goes without saying that javascript workers have nothing to do with the design and layout of a webpage. Yup. That's exactly why I don't want them. Why should I execute something which doesn't, and shouldn't, have anything to do with rendering page content? Don't get me wrong, I'm fine with using more cores if it's actually beneficial. But every use I've ever seen for a web/service/javascript worker has always been user hostile by taking what should be done on a server and offloading it onto the user's device instead.
        BubRoss 7y ago
        Using all your cores for the same workload would mean it finishes faster or finishes in the same time with significantly lower frequency. It saves power and heat. Your example would mean using more cores for the same amount of time, which makes no sense in this comparison. Do you also buy single core computers to save power? It is clear that what you are saying has nothing to do with javascript features at all and just boils down to not liking bloated web pages.
  - Franciscouzo 7y ago
    That ship has sailed, if you don't want people running arbitrary code on your device, you should disable JavaScript
    - inetknght 7y ago
      Funny enough, I do that
- bschwindHN 7y ago
  If you don't need to actually modify the image data you shouldn't be doing it at all. The correct solution is to specify a rotational transform so the hardware will do the heavy lifting on the GPU where this kind of computation belongs.
RandallBrown 7y ago
Your second approach is almost exactly what I've done in an iPhone video editing app I've been working on to deal with video previews. Rather than reencode the video just to preview it, I can just apply the same transforms (scale, rotation, translation) to the view that's displaying the preview. I then mask that view with another view of the same size so it doesn't go outside the edges. Of course I still need to encode the video with those transforms if I want it to show up in their camera roll later.
paultopia 7y ago
This shows up even in basic websites! My partner, who is an artist, ended up with some of her portfolio broken seemingly at random in different browsers---and for some of it, "up" wasn't visually obvious and she'd rotated some of the images in apps in addition (preview, adobe things, etc.), so there wasn't a good way to just change everything. ended up having to do a ton of work to strip exif data on an image by image basis. Basically entirely because of this ridiculous issue where chrome refuses to respect exif tags.[1] [1] https://stackoverflow.com/questions/42401203/chrome-image-exif-orienation-issue https://stackoverflow.com/questions/42401203/chrome-image-ex...
robocat 7y ago
> One solution involved reading the EXIF data with JS and rotating with canvas and .toBlob() At my previous job I did the same thing, although I never noticed a significant slowdown. I also made the file size smaller since we wanted to have predictable upload times and mitigate excessive usage of storage space. The other reason was that the EXIF data was wierd on some devices and the back end library didn't rotate them correctly.
avip 7y ago
As we speak I need a file upload widget that will reject some images based on exif. What’s your recommendation for client side js exif extraction?
kyle-rb 7y ago
Squoosh.app, a webapp from Google for compressing and resizing images, has an article about using WebAssembly for exactly this purpose: https://developers.google.com/web/updates/2019/02/hotpath-with-wasm https://developers.google.com/web/updates/2019/02/hotpath-wi...
- acomjean 7y ago
  "In squoosh we wrote a JavaScript function that rotates an image buffer by multiples of 90 degrees. While OffscreenCanvas would be ideal for this, it isn't supported across the browsers we were targeting, and a little buggy in Chrome." Wait, I know its a big company, but you're at the same company...
dgritsko 7y ago
Have run into this myself when building the same "preview upload" feature. It's annoying that it's not something that is just handled by the browser, it feels like it should be a supported feature of the "img" tag or something.
deleted 7y ago
[deleted]
acomjean 7y ago
I had this problem with the artist open studio website. Oddly it seems like it wasn't as big a problem until cell phones. ( I honestly couldn't tell which way is up for some images, being abstract.) we ended up using JS library called "croppie", but I'm not sure it helps with large images.
missblit 7y ago
Was rotating with CSS transforms an option? That way you'd only need to read the EXIF data, but wouldn't need to go all the way to rotating the pixels yourself.

by enlyth 7y ago

Waterluvian 7y ago

Try web workers! I've been doing all kinds of crazy expensive spatial data validation in many parallel threads. It's amazing. I think off screen canvas in workers is gaining browser support recently.

inetknght 7y ago
> Try web workers! NO, please do not try web workers. I don't want any more running on my device than absolutely necessary.
- BubRoss 7y ago
  You don't want to use all your cores to run things faster?
  - dralley 7y ago
    It would be nice if they only made things faster. In practice it seems like they use all my cores to do more things, things that I don't want or care about them doing.
  - inetknght 7y ago
    Nope. I want things to look plain and simple. I want to use fewer cores for longer battery life.
    - BubRoss 7y ago
      The faster a CPU finishes, the faster the CPU can sleep, which is how you save battery life. The more cores you use the lower the CPU frequency can be, which saves power, since frequency increases do not use power linearly. I think it goes without saying that javascript workers have nothing to do with the design and layout of a webpage.
      - inetknght 7y ago
        > The faster a CPU finishes, the faster the CPU can sleep, which is how you save battery life. Yup. > The more cores you use the lower the CPU frequency can be, which saves power, since frequency increases do not use power linearly. More cores in use has little to do with frequency and more to do with heat. More heat means more thermal throttling which lowers frequency. Lower frequency means that the CPU doesn't sleep sooner. > I think it goes without saying that javascript workers have nothing to do with the design and layout of a webpage. Yup. That's exactly why I don't want them. Why should I execute something which doesn't, and shouldn't, have anything to do with rendering page content? Don't get me wrong, I'm fine with using more cores if it's actually beneficial. But every use I've ever seen for a web/service/javascript worker has always been user hostile by taking what should be done on a server and offloading it onto the user's device instead.
        BubRoss 7y ago
        Using all your cores for the same workload would mean it finishes faster or finishes in the same time with significantly lower frequency. It saves power and heat. Your example would mean using more cores for the same amount of time, which makes no sense in this comparison. Do you also buy single core computers to save power? It is clear that what you are saying has nothing to do with javascript features at all and just boils down to not liking bloated web pages.
- Franciscouzo 7y ago
  That ship has sailed, if you don't want people running arbitrary code on your device, you should disable JavaScript
  - inetknght 7y ago
    Funny enough, I do that
bschwindHN 7y ago
If you don't need to actually modify the image data you shouldn't be doing it at all. The correct solution is to specify a rotational transform so the hardware will do the heavy lifting on the GPU where this kind of computation belongs.

RandallBrown 7y ago

Your second approach is almost exactly what I've done in an iPhone video editing app I've been working on to deal with video previews. Rather than reencode the video just to preview it, I can just apply the same transforms (scale, rotation, translation) to the view that's displaying the preview. I then mask that view with another view of the same size so it doesn't go outside the edges. Of course I still need to encode the video with those transforms if I want it to show up in their camera roll later.

paultopia 7y ago

This shows up even in basic websites! My partner, who is an artist, ended up with some of her portfolio broken seemingly at random in different browsers---and for some of it, "up" wasn't visually obvious and she'd rotated some of the images in apps in addition (preview, adobe things, etc.), so there wasn't a good way to just change everything. ended up having to do a ton of work to strip exif data on an image by image basis. Basically entirely because of this ridiculous issue where chrome refuses to respect exif tags.[1] [1] https://stackoverflow.com/questions/42401203/chrome-image-exif-orienation-issue https://stackoverflow.com/questions/42401203/chrome-image-ex...

robocat 7y ago

> One solution involved reading the EXIF data with JS and rotating with canvas and .toBlob() At my previous job I did the same thing, although I never noticed a significant slowdown. I also made the file size smaller since we wanted to have predictable upload times and mitigate excessive usage of storage space. The other reason was that the EXIF data was wierd on some devices and the back end library didn't rotate them correctly.

avip 7y ago

As we speak I need a file upload widget that will reject some images based on exif. What’s your recommendation for client side js exif extraction?

kyle-rb 7y ago

Squoosh.app, a webapp from Google for compressing and resizing images, has an article about using WebAssembly for exactly this purpose: https://developers.google.com/web/updates/2019/02/hotpath-with-wasm https://developers.google.com/web/updates/2019/02/hotpath-wi...

acomjean 7y ago
"In squoosh we wrote a JavaScript function that rotates an image buffer by multiples of 90 degrees. While OffscreenCanvas would be ideal for this, it isn't supported across the browsers we were targeting, and a little buggy in Chrome." Wait, I know its a big company, but you're at the same company...

dgritsko 7y ago

Have run into this myself when building the same "preview upload" feature. It's annoying that it's not something that is just handled by the browser, it feels like it should be a supported feature of the "img" tag or something.

deleted 7y ago

[deleted]

acomjean 7y ago

I had this problem with the artist open studio website. Oddly it seems like it wasn't as big a problem until cell phones. ( I honestly couldn't tell which way is up for some images, being abstract.) we ended up using JS library called "croppie", but I'm not sure it helps with large images.

missblit 7y ago

Was rotating with CSS transforms an option? That way you'd only need to read the EXIF data, but wouldn't need to go all the way to rotating the pixels yourself.