4 ms·

How does the author justify comments about 16K resolution being optimal, 12 cameras being the minimum required, and 240Hz being ideal?

acdha 4y ago
Are you asking for citations to research papers? It’s briefly explained in context but basically your eyes capture a ton of data and process it quickly (different types of data are processed at different speeds: your brain can react to motion before you recognize color or identify an object). That has millions of years of evolution behind it, and our entire lives are spent training expectations on how that works and especially how it interacts with other senses or what we do. That leads to several hard problems: you don’t just need to generate a high-quality scene, you need to render it twice because your brain is used to the distance between your eyes (16K==8K per eye) and because your position isn’t static it needs to update very quickly when you move because your brain is very used to how that works in real life. You have a budget measured in single digit milliseconds to that or a fair number of people will get motion sickness and many more will notice the lag and find it detracts from their experience. Here’s a good overview: https://www.microsoft.com/en-us/research/uploads/prod/2018/02/perfectillusion.pdf https://www.microsoft.com/en-us/research/uploads/prod/2018/0...
- Dylan16807 4y ago
  > (16K==8K per eye) That's not how Ks work. And your explanation doesn't cover why the minimum number of cameras would be 12.
  - acdha 4y ago
    It’s also not how eyes work because your visual resolution isn’t consistent across the field of view, saccades are a thing, etc. but it’s serviceable enough to get the point that you’re not just rendering a high-resolution scene but two of them requiring different processing for a fair chunk of the pipeline (consider how much geometry is only visible to one of them when you walk around a corner). > And your explanation doesn't cover why the minimum number of cameras would be 12. This is covered in the article. The key thing to understand is that there are multiple tasks using cameras: AR needs to track the world around you, you need eye tracking to handle saccades and conscious direction, and tracking hands, arms, legs, facial expressions, and body position accurately requires multiple cameras with enough parallax to build 3-D models which are accurate enough to generate realistic, stable imagery. People will really notice if things bounce around or deform or if you punt on some of the proven and end up with Meta’s leg problem. The key point to take away here is that AR/VR aren’t one hard problem but a collection of hard problems which all need to be solved for most people to find the experience compelling — all with tight physical constraints.
    - Dylan16807 4y ago
      > This is covered in the article. Nothing explains why 12 specifically. And if 12 is a minimum, what's the optimum? > The key point to take away here is that AR/VR aren’t one hard problem but a collection of hard problems which all need to be solved for most people to find the experience compelling — all with tight physical constraints. And it's a pretty bad point because it only works if you constrain yourself to head-mounted (including battery) standalone devices. The article gives a nod to other ways of doing it, but otherwise ignores them for all its main arguments. And you don't need optimal to be compelling.
      - acdha 4y ago
        > And it's a pretty bad point because it only works if you constrain yourself to head-mounted (including battery) standalone devices. The article gives a nod to other ways of doing it, but otherwise ignores them for all its main arguments. The article isn’t a complete survey of the entire field, it’s talking specifically about the most popular approaches and extrapolating what the author believes a breakthrough popularity device will look like. It does mention other options such as Apple’s rumored support devices or how industrial applications can take advantage of their environment. The challenge here is price: these systems are expensive and getting people to buy them means they either need to be cheap (impossible) or give a lot of value, and companies are assuming that means something you can easily move around in and use for long periods of time (not many people are going to set up a VR room, buy a headset and sensor array, and a pricey PC). While it’s true that it doesn’t need to be perfect, some of the hard problems spoil the illusion - people get sick or suspension of disbelief fails - and it has to be enough better than the widespread mainstream entertainment systems to justify the much greater cost. Currently we haven’t hit the point where many people, even many self-described fans, use these systems heavily and that makes it an expensive hobby.
        anonred 4y ago
        > Currently we haven’t hit the point where many people, even many self-described fans, use these systems heavily How many minutes per day would you consider “heavy usage” given the current entertainment/gaming focus?

acdha 4y ago

Are you asking for citations to research papers? It’s briefly explained in context but basically your eyes capture a ton of data and process it quickly (different types of data are processed at different speeds: your brain can react to motion before you recognize color or identify an object). That has millions of years of evolution behind it, and our entire lives are spent training expectations on how that works and especially how it interacts with other senses or what we do. That leads to several hard problems: you don’t just need to generate a high-quality scene, you need to render it twice because your brain is used to the distance between your eyes (16K==8K per eye) and because your position isn’t static it needs to update very quickly when you move because your brain is very used to how that works in real life. You have a budget measured in single digit milliseconds to that or a fair number of people will get motion sickness and many more will notice the lag and find it detracts from their experience. Here’s a good overview: https://www.microsoft.com/en-us/research/uploads/prod/2018/02/perfectillusion.pdf https://www.microsoft.com/en-us/research/uploads/prod/2018/0...

Dylan16807 4y ago
> (16K==8K per eye) That's not how Ks work. And your explanation doesn't cover why the minimum number of cameras would be 12.
- acdha 4y ago
  It’s also not how eyes work because your visual resolution isn’t consistent across the field of view, saccades are a thing, etc. but it’s serviceable enough to get the point that you’re not just rendering a high-resolution scene but two of them requiring different processing for a fair chunk of the pipeline (consider how much geometry is only visible to one of them when you walk around a corner). > And your explanation doesn't cover why the minimum number of cameras would be 12. This is covered in the article. The key thing to understand is that there are multiple tasks using cameras: AR needs to track the world around you, you need eye tracking to handle saccades and conscious direction, and tracking hands, arms, legs, facial expressions, and body position accurately requires multiple cameras with enough parallax to build 3-D models which are accurate enough to generate realistic, stable imagery. People will really notice if things bounce around or deform or if you punt on some of the proven and end up with Meta’s leg problem. The key point to take away here is that AR/VR aren’t one hard problem but a collection of hard problems which all need to be solved for most people to find the experience compelling — all with tight physical constraints.
  - Dylan16807 4y ago
    > This is covered in the article. Nothing explains why 12 specifically. And if 12 is a minimum, what's the optimum? > The key point to take away here is that AR/VR aren’t one hard problem but a collection of hard problems which all need to be solved for most people to find the experience compelling — all with tight physical constraints. And it's a pretty bad point because it only works if you constrain yourself to head-mounted (including battery) standalone devices. The article gives a nod to other ways of doing it, but otherwise ignores them for all its main arguments. And you don't need optimal to be compelling.
    - acdha 4y ago
      > And it's a pretty bad point because it only works if you constrain yourself to head-mounted (including battery) standalone devices. The article gives a nod to other ways of doing it, but otherwise ignores them for all its main arguments. The article isn’t a complete survey of the entire field, it’s talking specifically about the most popular approaches and extrapolating what the author believes a breakthrough popularity device will look like. It does mention other options such as Apple’s rumored support devices or how industrial applications can take advantage of their environment. The challenge here is price: these systems are expensive and getting people to buy them means they either need to be cheap (impossible) or give a lot of value, and companies are assuming that means something you can easily move around in and use for long periods of time (not many people are going to set up a VR room, buy a headset and sensor array, and a pricey PC). While it’s true that it doesn’t need to be perfect, some of the hard problems spoil the illusion - people get sick or suspension of disbelief fails - and it has to be enough better than the widespread mainstream entertainment systems to justify the much greater cost. Currently we haven’t hit the point where many people, even many self-described fans, use these systems heavily and that makes it an expensive hobby.
      - anonred 4y ago
        > Currently we haven’t hit the point where many people, even many self-described fans, use these systems heavily How many minutes per day would you consider “heavy usage” given the current entertainment/gaming focus?