6 ms·

VR is the opposite of cinema

83 points by rhema 11y ago

jraines 11y ago
Ken's blog is a consistently refreshing and insightful read. I had the pleasure to meet him last fall and his enthusiasm for and perspective on the future of how we experience art (and everything else, really) is energizing.
manachar 11y ago
I hadn't thought about virtual concerts, plays, ballets, operas, symphonies, etc. The performances could be live or prerecorded, obviously charging more for live. The performers could be captured via some form of 3d capture. The viewer could then have their choice of viewing angle. Think how much fun it would be to watch a musical from the stage! Some artists might even start experimenting with making it interactive. I live far away from most cultural events, and would absolutely adore being able to catch a ballet again without having to get on a plane.
- corysama 11y ago
  > I hadn't thought about virtual concerts, plays, ballets, operas, symphonies, etc. I've watched many made-for-VR surround video productions and several of them have the strong feeling resembling that of a play. Then there's Will Smith's (formerly of Tested) new gig, The Foo Show [1][2] which is like standing on the set of a talk show. It also shows that Max Headroom was even more prescient than we thought ;) > I live far away from most cultural events, and would absolutely adore being able to catch a ballet again without having to get on a plane. VR won't replace going there. Instead, it will replace not going there. That's a big deal. [1] https://www.youtube.com/watch?v=PT3jZyOXqzU https://www.youtube.com/watch?v=PT3jZyOXqzU [2] http://store.steampowered.com/app/411820/ http://store.steampowered.com/app/411820/
- digi_owl 11y ago
  I recall reading about cinemas that were experimenting with streaming live operas etc from major stages around the world. Filmed such that the screen became a stand in for the stage.
  - greggman 11y ago
    I think the issue is there's no one else at the theater. At least in SF they've been trying that for years. I've never been but I'm just guessing attendance is low. Being in a 200-400 person theater with 10 other people watching a concert is not remotely the same experience as being at the concert with 200-400 screaming cheering singing people around you.
Animats 11y ago
Non-interactive VR is what game developers call a "track ride". You're locked to a path. You can look around, and maybe stop, start, shoot at stuff, and solve puzzles, but you can't get off the track. For a long time, game developers struggled with how to tell a story while not locking the player into a track ride. A movie is a story. A good game today is a place you visit. VR track rides are somewhere in between. Getting a story line into a free play game is hard. But it's been done many times now. GTA V is a good example. The GTA games are open, free-play worlds. You're free to just drive around and sightsee, which is fun all by itself. The gameplay "missions" give the player goals and achievements, but there are many ways to achieve a goal. (Amusingly, the way to make the most money in GTA V is to speculate in the stock market, not steal cars. Score one for realism.) The VR crowd now has to face that problem. Otherwise, they'll go down the same rathole as 3D TV and movies. (Yes, Avatar in 3D was good. That's because Cameron is very good at visual world-building. Almost every other 3D movie didn't need 3D or overdid it.)
- skewart 11y ago
  > A good game today is a place you visit. Yes! I think that's a really good insight. I wonder if VR's "killer app" for consumer entertainment will be more closely related to travel than to movies. It's interesting that open world games, like GTA and Skyrim, are increasingly popular. At the same time multisensory-art-experience festivals like Burning Man are increasingly popular too. Being able to explore immaculately crafted worlds, enjoying beautiful sights and sounds, and getting the thrill of dicovery that comes with going some place new would be pretty awesome.
- caseyf7 11y ago
  You make a great point about Avatar. After all these years, it is still the the only real 3D movie. It seemed like we were on the brink of great 3D, but no one else could even match Avatar.
- stcredzero 11y ago
  Non-interactive VR is what game developers call a "track ride". You're locked to a path. You can look around, and maybe stop, start, shoot at stuff, and solve puzzles, but you can't get off the track. Kids a generation hence will think about games like this like we think of "classic" books. (One of them will reinvent the Mark Twain about a classic book being a book everyone says is important, but no one has actually read.) We will further perfect an artistic and intellectual language of interactivity. (What we now call the medium of games.) On top of that, we will start to develop a language around emergence as well. The Minecraft generation will be the first generation starting fully down the path of emergent-world literacy.
TechNewb 11y ago
"Cinema doesn't have sound. Sound doesn't belong in cinema. True directors can tell stories without sound." Cinema has always embraced technological innovations. There is nothing that says cinema has to be a shared experience. Semantics.
- AimHere 11y ago
  But the difference is not just semantic. VR experiences will necessarily lose a lot of cinematography (particularly that regarding shot composition and framing) because the viewer is in control over the camera, and they almost completely lose the ability to do editing too (because cutting between scenes will make the viewer nauseous). Modern films might seem superficially similar to non-interactive VR experiences because they are both throwing images at screens, but there is a huge world of cinematic techniques that everyone has grown up with (and most people won't notice), that will be severely constrained in VR, and it'll be a jarring experience (at least for the people making this stuff, if not the consumers) for a lot of them to be gone. Perlin is likely right that what's likely to happen is that the scenes in non-interactive VR will seem more like theatre than cinema. Sure, the people putting this stuff together will probably come out of VFX houses but people won't be relying on cinematographers, editors and film directors in the same way. I suspect that it'll take a while for both the artistic problems of living with the new constraints, and the technological problems (such as taking live-action 360 footage with parallax) to be ironed out satisfactorily. Interactive VR still looks much like video games, though, and while there will be some measure of constraints with the new medium, other constraints will be lifted (FPSes and flight simulators, for instance, won't need clumsy workarounds to simulate head movement), so the games industry will probably hit the ground running.
  - TechNewb 11y ago
    Agree with everything you're saying here. But the argument reminds me of a similar one, where the established industry leaders try and define what skills and processes go into cinema. Such as no sound vs sound, or film vs digital. The fact that it's story telling on a screen is what in my mind defines cinema. There is cinematography in current virtual storytelling, such as firewatch, life is strange, and grand theft auto. While none of those are VR, they could be adopted to that medium as well. Cinematography is much more than camera angle, and lens choice, imo, especially in the digitally generated world. There are other techniques to lead the viewers attention, not so different than techniques used on a well made imax. Also of interesting note, 360 moving image recording is not new, and has been around since the 1950s. https://en.m.wikipedia.org/wiki/Circle-Vision_360° https://en.m.wikipedia.org/wiki/Circle-Vision_360°
Aelinsaar 11y ago
It's a good point, but on the other hand, maybe it will inspire extra features such as "Pause, and walk around the scene." You could have virtual interviews with actors or people on the set, linked to the timestamp you paused at. I can imagine it being a great way to experience cinema, but I take the author's broader meaning.
- shiro 11y ago
  A common illusion in films is that in one scene, you think you are watching a fixed space/acting/events from just several different cameras. In fact, it is often the case that acting, positioning of actors, backgrounds or even locations, are adjusted for each shot. The acting you see in a master shot is different from you see in a close-up shot in the same scene. It's amazing what editors can do to build a coherent illusion from pieces of footage. So I agree on the original article; VR experience would be similar to plays rather than films. Maybe future films would employ stage-play like effects more (especially if you need to shoot a shot with cameras that allow VR walkthrough, the whole new technique of film shooting need to be developed.)
  - Aelinsaar 11y ago
    I didn't think about any of that, and it's all interesting and relevant. Lots to think about now!
pcl 11y ago
I don't know if I would call that the opposite of cinema, but it's certainly a huge difference. I wonder what this would mwan mean for all the actors who prefer stage to film. Would they shoot up in demand? What would that mean for theater in general?
- shiro 11y ago
  What I like about acting on stage is immediate, live reaction from audience. Film acting largely lacks that, though there are techniques to compensate it. I'd imagine "VR film shooting" would be similar to film acting in that regard. (Except in the case that VR becomes live and bidirectional---that would be interesting.)
  - Starsgen 11y ago
    You could record a dinner theater-style show. Then the audio becomes the cue for expected viewpoint. Most people turn to look at an actor speaking.
    - shiro 11y ago
      Right, recording non-proscenium theater plays would be interesting contents. I like to watch (and often perform) round- or three-sided stages in a small theater. That kind of shows are difficult to capture in traditional camera for it loses the feeling of space surrounding audience.
aaron695 11y ago
Correct. I pay to chill and have an expert give me an exact and defined experience when watching movies. If I want stress, I'll play computer games and get the added benefits of making (stressful) decisions. We've have recorded vision for a century. Yet mp3's still have their massive market. Video changed the radio star, but didn't kill it. Same will happen with VR.
aaron695 11y ago
Correct. I pay to chill and have an expert give me an experience when watching movies. If I want stress, I'll play computer games. We've have recorded vision for a century. Yet mp3's still have their massive market. Video changed the radio star, but didn't kill it. Same will happen with VR.
Terr_ 11y ago
This is not new, this is exactly the same friction that already existed between 2D movies and 2D games. That extra dimension--the sensation of depth--doesn't change the fundamental rift between passive/interactive camera control. Just close one eye while watching/playing to see why...
- cloudmike 11y ago
  I think you're right that the tension between passive/interactive and storytelling/agency isn't new. But the tension is heightened in VR. What's new is that filmmakers and game makers are building VR experiences for the same audience and hardware. They're on the same turf. Outside of VR, filmmakers create for TV/cinema, and game makers create for PC/console/mobile. I think Ken Perlin is suggesting that new discoveries will be made as this collision of worlds happens. The friction you mentioned becomes more visible to a larger number of people with a wider variety of skills and goals.
jordache 11y ago
those 360deg videos suck. I can't believe what content authors would like that.. you have a total lack of control over the narrative, because you can't control what the viewer sees.
- TOMDM 11y ago
  I can't help but liken this sort of attitude to a statement like "why bother crafting beautiful game environments, the player could just stare at their own feet" Sure, you no longer have control of where the viewer is looking, but you do have control over the entire environment in which they now exist in. Changing scenes and guiding the viewers attention are obviously new challenges, but they are hardly insurmountable, and implying that they are enough to destroy the value of the medium is just lazy.
- slavik81 11y ago
  There are tricks you can do to draw attention to things. The Portal and Half-Life 2 Episodes developer commentaries include quite a few discussions of how they designed their environments to nudge players into noticing stuff. For example, in HL2 Episode 1 there's a scene where you're walking across a bridge and a huge helicopter comes up beside you. To ensure that players were looking the right way to be surprised by it, they put an enemy in the distance firing (ineffectually) at the player to attract attention.
craftandhustle 11y ago
I agree with these thoughts based on the current state of 360VR (or non-interactive, if you will) but I'm optimistic about the future changing this perspective. 360VR for narrative storytelling is in a very early, experimental stage. It's difficult in part because it feels inherently gimmicky. A lot of the traditional elements in the craft & technique of filmmaking are unavailable to the storyteller (or, rather, need to be rethought). Closeups, stylized editing, etc. VR directors are slowly figuring out the best ways to draw attention / guide the viewer, transition between shots and, most importantly, keep you immersed in the STORY. But, there's a lot of room for improvement in the technology as well. "Simple" things even — such as tracking the viewer's orientation so that the following shot can be focused where the director wants regardless of where the viewer has "travelled" to. Right now, the mass distribution system's for VR (such as YouTube 360, Vrideo, et al which use Cardboard or GearVR tech), don't currently allow for this. I'd love to see the interactive tools of VR currently used in gaming (like Unity) made available for narrative storytelling. Specifically, it's more of an issue on the distribution side. The explosion in 360VR feels like it was largely due to the ease and accessibility brought on by Google (Cardboard & YouTube 360), now we need these tools to evolve.. and they are, slowly. tl;dr; 360VR for storytelling is a bit gimmicky currently because it's in a very early stage. I'm optimistic about a new wave of cinematic narratives in VR as the technology evolves allowing for more direction/expression of the craft.
erikpukinskis 11y ago
Except different people do see totally different things when they watch the same film. And looking at a painting can be a highly directed experience. All of art is about the give and take of showing and allowing the viewer to see.
Starsgen 11y ago
I like the dinner theater concept applied to VR.
- DonHopkins 11y ago
  Yeah, but have you ever tried to eat with a VR helmet on?
jamesrom 11y ago
This author only considers the active kind of VR. Where the user participates by looking around or controlling some aspect of the environment or story. Why can't I just sit back and watch events unfold before me? Why does everyone assume I want to look around or interact with the story?
deleted 11y ago
[deleted]

83 points by rhema 11y ago

Ken's blog is a consistently refreshing and insightful read. I had the pleasure to meet him last fall and his enthusiasm for and perspective on the future of how we experience art (and everything else, really) is energizing.

manachar 11y ago

I hadn't thought about virtual concerts, plays, ballets, operas, symphonies, etc. The performances could be live or prerecorded, obviously charging more for live. The performers could be captured via some form of 3d capture. The viewer could then have their choice of viewing angle. Think how much fun it would be to watch a musical from the stage! Some artists might even start experimenting with making it interactive. I live far away from most cultural events, and would absolutely adore being able to catch a ballet again without having to get on a plane.

corysama 11y ago
> I hadn't thought about virtual concerts, plays, ballets, operas, symphonies, etc. I've watched many made-for-VR surround video productions and several of them have the strong feeling resembling that of a play. Then there's Will Smith's (formerly of Tested) new gig, The Foo Show [1][2] which is like standing on the set of a talk show. It also shows that Max Headroom was even more prescient than we thought ;) > I live far away from most cultural events, and would absolutely adore being able to catch a ballet again without having to get on a plane. VR won't replace going there. Instead, it will replace not going there. That's a big deal. [1] https://www.youtube.com/watch?v=PT3jZyOXqzU https://www.youtube.com/watch?v=PT3jZyOXqzU [2] http://store.steampowered.com/app/411820/ http://store.steampowered.com/app/411820/
digi_owl 11y ago
I recall reading about cinemas that were experimenting with streaming live operas etc from major stages around the world. Filmed such that the screen became a stand in for the stage.
- greggman 11y ago
  I think the issue is there's no one else at the theater. At least in SF they've been trying that for years. I've never been but I'm just guessing attendance is low. Being in a 200-400 person theater with 10 other people watching a concert is not remotely the same experience as being at the concert with 200-400 screaming cheering singing people around you.

Animats 11y ago

Non-interactive VR is what game developers call a "track ride". You're locked to a path. You can look around, and maybe stop, start, shoot at stuff, and solve puzzles, but you can't get off the track. For a long time, game developers struggled with how to tell a story while not locking the player into a track ride. A movie is a story. A good game today is a place you visit. VR track rides are somewhere in between. Getting a story line into a free play game is hard. But it's been done many times now. GTA V is a good example. The GTA games are open, free-play worlds. You're free to just drive around and sightsee, which is fun all by itself. The gameplay "missions" give the player goals and achievements, but there are many ways to achieve a goal. (Amusingly, the way to make the most money in GTA V is to speculate in the stock market, not steal cars. Score one for realism.) The VR crowd now has to face that problem. Otherwise, they'll go down the same rathole as 3D TV and movies. (Yes, Avatar in 3D was good. That's because Cameron is very good at visual world-building. Almost every other 3D movie didn't need 3D or overdid it.)

skewart 11y ago
> A good game today is a place you visit. Yes! I think that's a really good insight. I wonder if VR's "killer app" for consumer entertainment will be more closely related to travel than to movies. It's interesting that open world games, like GTA and Skyrim, are increasingly popular. At the same time multisensory-art-experience festivals like Burning Man are increasingly popular too. Being able to explore immaculately crafted worlds, enjoying beautiful sights and sounds, and getting the thrill of dicovery that comes with going some place new would be pretty awesome.
caseyf7 11y ago
You make a great point about Avatar. After all these years, it is still the the only real 3D movie. It seemed like we were on the brink of great 3D, but no one else could even match Avatar.
stcredzero 11y ago
Non-interactive VR is what game developers call a "track ride". You're locked to a path. You can look around, and maybe stop, start, shoot at stuff, and solve puzzles, but you can't get off the track. Kids a generation hence will think about games like this like we think of "classic" books. (One of them will reinvent the Mark Twain about a classic book being a book everyone says is important, but no one has actually read.) We will further perfect an artistic and intellectual language of interactivity. (What we now call the medium of games.) On top of that, we will start to develop a language around emergence as well. The Minecraft generation will be the first generation starting fully down the path of emergent-world literacy.

TechNewb 11y ago

"Cinema doesn't have sound. Sound doesn't belong in cinema. True directors can tell stories without sound." Cinema has always embraced technological innovations. There is nothing that says cinema has to be a shared experience. Semantics.

AimHere 11y ago
But the difference is not just semantic. VR experiences will necessarily lose a lot of cinematography (particularly that regarding shot composition and framing) because the viewer is in control over the camera, and they almost completely lose the ability to do editing too (because cutting between scenes will make the viewer nauseous). Modern films might seem superficially similar to non-interactive VR experiences because they are both throwing images at screens, but there is a huge world of cinematic techniques that everyone has grown up with (and most people won't notice), that will be severely constrained in VR, and it'll be a jarring experience (at least for the people making this stuff, if not the consumers) for a lot of them to be gone. Perlin is likely right that what's likely to happen is that the scenes in non-interactive VR will seem more like theatre than cinema. Sure, the people putting this stuff together will probably come out of VFX houses but people won't be relying on cinematographers, editors and film directors in the same way. I suspect that it'll take a while for both the artistic problems of living with the new constraints, and the technological problems (such as taking live-action 360 footage with parallax) to be ironed out satisfactorily. Interactive VR still looks much like video games, though, and while there will be some measure of constraints with the new medium, other constraints will be lifted (FPSes and flight simulators, for instance, won't need clumsy workarounds to simulate head movement), so the games industry will probably hit the ground running.
- TechNewb 11y ago
  Agree with everything you're saying here. But the argument reminds me of a similar one, where the established industry leaders try and define what skills and processes go into cinema. Such as no sound vs sound, or film vs digital. The fact that it's story telling on a screen is what in my mind defines cinema. There is cinematography in current virtual storytelling, such as firewatch, life is strange, and grand theft auto. While none of those are VR, they could be adopted to that medium as well. Cinematography is much more than camera angle, and lens choice, imo, especially in the digitally generated world. There are other techniques to lead the viewers attention, not so different than techniques used on a well made imax. Also of interesting note, 360 moving image recording is not new, and has been around since the 1950s. https://en.m.wikipedia.org/wiki/Circle-Vision_360° https://en.m.wikipedia.org/wiki/Circle-Vision_360°

Aelinsaar 11y ago

It's a good point, but on the other hand, maybe it will inspire extra features such as "Pause, and walk around the scene." You could have virtual interviews with actors or people on the set, linked to the timestamp you paused at. I can imagine it being a great way to experience cinema, but I take the author's broader meaning.

shiro 11y ago
A common illusion in films is that in one scene, you think you are watching a fixed space/acting/events from just several different cameras. In fact, it is often the case that acting, positioning of actors, backgrounds or even locations, are adjusted for each shot. The acting you see in a master shot is different from you see in a close-up shot in the same scene. It's amazing what editors can do to build a coherent illusion from pieces of footage. So I agree on the original article; VR experience would be similar to plays rather than films. Maybe future films would employ stage-play like effects more (especially if you need to shoot a shot with cameras that allow VR walkthrough, the whole new technique of film shooting need to be developed.)
- Aelinsaar 11y ago
  I didn't think about any of that, and it's all interesting and relevant. Lots to think about now!

pcl 11y ago

I don't know if I would call that the opposite of cinema, but it's certainly a huge difference. I wonder what this would mwan mean for all the actors who prefer stage to film. Would they shoot up in demand? What would that mean for theater in general?

shiro 11y ago
What I like about acting on stage is immediate, live reaction from audience. Film acting largely lacks that, though there are techniques to compensate it. I'd imagine "VR film shooting" would be similar to film acting in that regard. (Except in the case that VR becomes live and bidirectional---that would be interesting.)
- Starsgen 11y ago
  You could record a dinner theater-style show. Then the audio becomes the cue for expected viewpoint. Most people turn to look at an actor speaking.
  - shiro 11y ago
    Right, recording non-proscenium theater plays would be interesting contents. I like to watch (and often perform) round- or three-sided stages in a small theater. That kind of shows are difficult to capture in traditional camera for it loses the feeling of space surrounding audience.

aaron695 11y ago

Correct. I pay to chill and have an expert give me an exact and defined experience when watching movies. If I want stress, I'll play computer games and get the added benefits of making (stressful) decisions. We've have recorded vision for a century. Yet mp3's still have their massive market. Video changed the radio star, but didn't kill it. Same will happen with VR.

aaron695 11y ago

Correct. I pay to chill and have an expert give me an experience when watching movies. If I want stress, I'll play computer games. We've have recorded vision for a century. Yet mp3's still have their massive market. Video changed the radio star, but didn't kill it. Same will happen with VR.

Terr_ 11y ago

This is not new, this is exactly the same friction that already existed between 2D movies and 2D games. That extra dimension--the sensation of depth--doesn't change the fundamental rift between passive/interactive camera control. Just close one eye while watching/playing to see why...

cloudmike 11y ago
I think you're right that the tension between passive/interactive and storytelling/agency isn't new. But the tension is heightened in VR. What's new is that filmmakers and game makers are building VR experiences for the same audience and hardware. They're on the same turf. Outside of VR, filmmakers create for TV/cinema, and game makers create for PC/console/mobile. I think Ken Perlin is suggesting that new discoveries will be made as this collision of worlds happens. The friction you mentioned becomes more visible to a larger number of people with a wider variety of skills and goals.

jordache 11y ago

those 360deg videos suck. I can't believe what content authors would like that.. you have a total lack of control over the narrative, because you can't control what the viewer sees.

TOMDM 11y ago
I can't help but liken this sort of attitude to a statement like "why bother crafting beautiful game environments, the player could just stare at their own feet" Sure, you no longer have control of where the viewer is looking, but you do have control over the entire environment in which they now exist in. Changing scenes and guiding the viewers attention are obviously new challenges, but they are hardly insurmountable, and implying that they are enough to destroy the value of the medium is just lazy.
slavik81 11y ago
There are tricks you can do to draw attention to things. The Portal and Half-Life 2 Episodes developer commentaries include quite a few discussions of how they designed their environments to nudge players into noticing stuff. For example, in HL2 Episode 1 there's a scene where you're walking across a bridge and a huge helicopter comes up beside you. To ensure that players were looking the right way to be surprised by it, they put an enemy in the distance firing (ineffectually) at the player to attract attention.

craftandhustle 11y ago

I agree with these thoughts based on the current state of 360VR (or non-interactive, if you will) but I'm optimistic about the future changing this perspective. 360VR for narrative storytelling is in a very early, experimental stage. It's difficult in part because it feels inherently gimmicky. A lot of the traditional elements in the craft & technique of filmmaking are unavailable to the storyteller (or, rather, need to be rethought). Closeups, stylized editing, etc. VR directors are slowly figuring out the best ways to draw attention / guide the viewer, transition between shots and, most importantly, keep you immersed in the STORY. But, there's a lot of room for improvement in the technology as well. "Simple" things even — such as tracking the viewer's orientation so that the following shot can be focused where the director wants regardless of where the viewer has "travelled" to. Right now, the mass distribution system's for VR (such as YouTube 360, Vrideo, et al which use Cardboard or GearVR tech), don't currently allow for this. I'd love to see the interactive tools of VR currently used in gaming (like Unity) made available for narrative storytelling. Specifically, it's more of an issue on the distribution side. The explosion in 360VR feels like it was largely due to the ease and accessibility brought on by Google (Cardboard & YouTube 360), now we need these tools to evolve.. and they are, slowly. tl;dr; 360VR for storytelling is a bit gimmicky currently because it's in a very early stage. I'm optimistic about a new wave of cinematic narratives in VR as the technology evolves allowing for more direction/expression of the craft.

erikpukinskis 11y ago

Except different people do see totally different things when they watch the same film. And looking at a painting can be a highly directed experience. All of art is about the give and take of showing and allowing the viewer to see.

Starsgen 11y ago

I like the dinner theater concept applied to VR.

DonHopkins 11y ago
Yeah, but have you ever tried to eat with a VR helmet on?

jamesrom 11y ago

This author only considers the active kind of VR. Where the user participates by looking around or controlling some aspect of the environment or story. Why can't I just sit back and watch events unfold before me? Why does everyone assume I want to look around or interact with the story?

deleted 11y ago

[deleted]