3 ms·

I am confused, a quote from Wikipedia: "The Ken Burns effect is a type of panning and zooming effect used in video production from still imagery." Our framewor

I am confused, a quote from Wikipedia: "The Ken Burns effect is a type of panning and zooming effect used in video production from still imagery." Our framework takes a single still image and animates it via panning and zooming while adding 3D parallax. Does adding the 3D parallax not make it the Ken Burns effect anymore? Please let me know in case I am misunderstanding anything. Yes, the examples that we have shown do not zoom as much as common Ken Burns examples. This stems from our framework processing the input at low resolution due to limitations in deep learning and the overall complexity of the problem. As such, there is not enough detail that could be zoomed into. This will be improved in later generations.

jlebar 7y ago
> Does adding the 3D parallax not make it the Ken Burns effect anymore? Yes. Specifically, the Ken Burns effect describes taking a still, 2D image and panning and zooming on the 2D image in a motion picture. Adding 3D parallax makes it something quite different in the eyes of many of us on this forum. It is cool! We would just not use the words "Ken Burns effect" to describe what you've made.
- jchook 7y ago
  On the contrary I think it basically serves the same purpose as the Ken Burns effect, except it tastefully incorporates the natural depth of the image. One might even argue Ken Burns would have paired the panning and zooming with this parallax effect if he could. I look forward to this enhancing basically all photo slideshows for a long time to come.
  - sniklaus 7y ago
    That is a nice way of putting it, thank you for chiming in!
    - mirimir 7y ago
      I know nothing about this. But it seems obvious that, whatever we call your work, Ken Burns' use of panning and zooming is a subset of it.
- sniklaus 7y ago
  Thank you for clarifying, I see where you are coming from! My background is in research and it is fairly common to extend / reuse an existing terminology, but I totally understand and accept your point of view.
jpfaraco 7y ago
I think adding the 3D parallax gives the footage a very unique effect, and quite different from the traditional Ken Burns 2D scale and pan. I would say enough to make it deserve its own name, such as the sniklaus effect. Ken Burns is Ken Burns. This is something quite different, even though it may be inspired by KB.
- sniklaus 7y ago
  I would be happy if it gets its own name in the future. I am voting against "sniklaus" effect though since this was a team effort and I could not have done it without my mentors.
gridspy 7y ago
Would it not be possible to do the image processing on the original image to create several pixel masks which then draw in the high resolution image data? For instance, you take the source image at resolution n*n and do all processing at n/2 or n/4. Then at the moment you are going to composite the finished image you instead draw "pixels" from a source image which contains X,Y not RGB. Then you upscale the output image x2 (or x4) and replace each X,Y index with the relevant pixel from the source image (adding X%2,Y%2 or X%4 to the indexes to return to source resolution).
- sniklaus 7y ago
  That is called guided upsampling and we are in fact doing just that to get a better estimate of the scene geometry. Specifically, we estimate the scene geometry at a resolution of 256x256 pixels (for a 1:1 aspect ratio, it varies for other resolutions) and then use the original image as a guide to upsample this estimate to 1024x1024 pixels. So yes, it could be used to a greater extent but it needs more work to be able to do this.
greggman2 7y ago
There is probably a name for this kind of "taking a 2D image, separating it into 2d cutouts, and panning with parallax". It's an effect that's been around for a while (hand done, not automated as you have it) but it's never been referred to as a "Ken Burns Effect" Googling "How to make a parallax video from 2d pictures" brings up lots of tutorials, none of which mentions "Ken Burns Effect" https://www.google.com/search?q=how+to+make+parallax+video+from+2d+pictures https://www.google.com/search?q=how+to+make+parallax+video+f... Conversly searching for "How to make the Ken Burns Effect" will show that it's not the same effect your software makes https://www.google.com/search?q=how+make+the+ken+burns+effect https://www.google.com/search?q=how+make+the+ken+burns+effec...

jlebar 7y ago

> Does adding the 3D parallax not make it the Ken Burns effect anymore? Yes. Specifically, the Ken Burns effect describes taking a still, 2D image and panning and zooming on the 2D image in a motion picture. Adding 3D parallax makes it something quite different in the eyes of many of us on this forum. It is cool! We would just not use the words "Ken Burns effect" to describe what you've made.

jchook 7y ago
On the contrary I think it basically serves the same purpose as the Ken Burns effect, except it tastefully incorporates the natural depth of the image. One might even argue Ken Burns would have paired the panning and zooming with this parallax effect if he could. I look forward to this enhancing basically all photo slideshows for a long time to come.
- sniklaus 7y ago
  That is a nice way of putting it, thank you for chiming in!
  - mirimir 7y ago
    I know nothing about this. But it seems obvious that, whatever we call your work, Ken Burns' use of panning and zooming is a subset of it.
sniklaus 7y ago
Thank you for clarifying, I see where you are coming from! My background is in research and it is fairly common to extend / reuse an existing terminology, but I totally understand and accept your point of view.

jpfaraco 7y ago

I think adding the 3D parallax gives the footage a very unique effect, and quite different from the traditional Ken Burns 2D scale and pan. I would say enough to make it deserve its own name, such as the sniklaus effect. Ken Burns is Ken Burns. This is something quite different, even though it may be inspired by KB.

sniklaus 7y ago
I would be happy if it gets its own name in the future. I am voting against "sniklaus" effect though since this was a team effort and I could not have done it without my mentors.

gridspy 7y ago

Would it not be possible to do the image processing on the original image to create several pixel masks which then draw in the high resolution image data? For instance, you take the source image at resolution n*n and do all processing at n/2 or n/4. Then at the moment you are going to composite the finished image you instead draw "pixels" from a source image which contains X,Y not RGB. Then you upscale the output image x2 (or x4) and replace each X,Y index with the relevant pixel from the source image (adding X%2,Y%2 or X%4 to the indexes to return to source resolution).

sniklaus 7y ago
That is called guided upsampling and we are in fact doing just that to get a better estimate of the scene geometry. Specifically, we estimate the scene geometry at a resolution of 256x256 pixels (for a 1:1 aspect ratio, it varies for other resolutions) and then use the original image as a guide to upsample this estimate to 1024x1024 pixels. So yes, it could be used to a greater extent but it needs more work to be able to do this.

greggman2 7y ago

There is probably a name for this kind of "taking a 2D image, separating it into 2d cutouts, and panning with parallax". It's an effect that's been around for a while (hand done, not automated as you have it) but it's never been referred to as a "Ken Burns Effect" Googling "How to make a parallax video from 2d pictures" brings up lots of tutorials, none of which mentions "Ken Burns Effect" https://www.google.com/search?q=how+to+make+parallax+video+from+2d+pictures https://www.google.com/search?q=how+to+make+parallax+video+f... Conversly searching for "How to make the Ken Burns Effect" will show that it's not the same effect your software makes https://www.google.com/search?q=how+make+the+ken+burns+effect https://www.google.com/search?q=how+make+the+ken+burns+effec...