3 ms·

As far as large lenses and sensors not working well on sensors, I think you'd be surprised just how well large lenses designed for the sensors they belong on ca

As far as large lenses and sensors not working well on sensors, I think you'd be surprised just how well large lenses designed for the sensors they belong on can work. Indeed, by far the reason why photography lenses are huge is because of lens IS and zoom. Making small, light, fast normal lenses with larger stabilized sensors works perfectly fine and can be made very light. The current cameras and lenses on the market work very well, that's not the problem. The problem is that even "small" cameras and lenses are huge in comparison to what can be integrated into a robot. I use a couple M4/3 cinema/studio cameras for vision projects because they're the smallest affordable cameras with controllable interchangeable lenses, and even with their "small" size they are at the upper limit of anything I could hope to fit in a practical robot. After all, humans have two huge sensors (slightly bigger than full-frame) with f/3 lenses, and it works perfectly fine. The human eye is vastly more complex and capable than a fixed plane imager and capable of much more complex focus behavior than any existing lenses are. Camera-eye comparisons are appealing, but the focus behavior of them is really very different. However, modern autofocus has progressed so much in this time-frame that it has solved almost all of the issues of SfF Canon has done a very nice job with their autofocus, but I will believe they have solved depth perception with phase detection when they apply it to their industrial cameras. Canon makes very nice RV-series parts recognition cameras that do a very good job with challenging materials, but they use structured light. In theory, phase-detection SfF would be perfect for these cameras but they haven't done it. It is true that modern cameras actually do move the focus and re-calculate depth, which might give the impression that if you wanted to calculate depth you would actually need to move focus. But in reality, this is done in order to correct for the small misalignment in different lenses, as well as for the fact autofocus motors do not have accurate encoders and will very frequently miss steps. But a modern camera already knows how much the image plane needs to actually shift before even engaging the autofocus motor at all. The minor problem is that you generally want depth plus RGB, which is obviously solved by adding a second camera but then you face basically the same size and overlap issues that stereo cameras bring. Not entirely a free lunch. The other problem here is political, namely that SfF via phase-detection is (at least for now) entirely in the hands of Canon and Sony, two companies (Sony in particular) with track records related to robotics and machine vision users that can best be described as ranging from "somewhat uninterested" to "useless" to "self-sabotaging". Unless one of them commits to making an actual product and ships it, we will remain in a world where phase-detection SfF is technically possible but entirely out of reach.

sudosysgen 6y ago
>The current cameras and lenses on the market work very well, that's not the problem. The problem is that even "small" cameras and lenses are huge in comparison to what can be integrated into a robot. I use a couple M4/3 cinema/studio cameras for vision projects because they're the smallest affordable cameras with controllable interchangeable lenses, and even with their "small" size they are at the upper limit of anything I could hope to fit in a practical robot. But this is largely because modern lenses are seriously bloated, with IS for older cameras that have non-stabilized sensors, bloated focusing mechanisms to make floating elements work, bloated zooming mechanisms that make everything so much worse, and so on. If we want to design a lens for robotics, we really don't need much more than a double-gauss normal lens with a unitary focusing mechanism. One of the lenses I use when I need to kludge something in a project is exactly that, a 1970s Minolta 55 f/1.7 lens for a full-frame image circle. Despite having a large 32mm aperture, it has a diameter of 56mm and a length of 44mm. I don't know if that's too large for your robotics applications, but it isn't very big and it's much smaller than the LiDAR sensors I've worked with. You could likely make this even smaller with modern techniques. It would also cost about 40$ to build. >The human eye is vastly more complex and capable than a fixed plane imager and capable of much more complex focus behavior than any existing lenses are. Camera-eye comparisons are appealing, but the focus behavior of them is really very different. My point was moreso that in pure optical sizes, the human eye is bigger and takes up more volume than the optical system I'm proposing. But as far as focusing behaviours, if you were to mathematically model how the human eye gets depth information to drive the focus cycle its quite similar to a phase detection system, with the phase information being extracted through the parallax. >Canon has done a very nice job with their autofocus, but I will believe they have solved depth perception with phase detection when they apply it to their industrial cameras. Canon makes very nice RV-series parts recognition cameras that do a very good job with challenging materials, but they use structured light. In theory, phase-detection SfF would be perfect for these cameras but they haven't done it. I think this can be explained quite simply. In a fully controlled environment, using structured light but cheap lenses and small sensors is much more cost-effective. Why would they use expensive APS-C+ sized sensors, that they don't seem to be all that good at making, when you can get away with very cheap parts? >The minor problem is that you generally want depth plus RGB, which is obviously solved by adding a second camera but then you face basically the same size and overlap issues that stereo cameras bring. Not entirely a free lunch. But this is precisely the point of OSPDAF or split-pixel phase detection. You get both depth and RGB at the same time, because the RGB sensor doubles as a phase detection sensor. It's not like old cameras where you had to use a mirror that sent light to the dedicated phase detection sensor. As for political problems, I agree. Canon won't ever sell you their DPAF sensors. Sony, even though they are really bad at robotics, should in theory be willing to sell anyone their sensors, and it should be possible to make a prototype. Their tech in this would be limited relatively to Canon, though. But, to the point, a solution to a problem that's proprietary is still a solution to the problem.

sudosysgen 6y ago

>The current cameras and lenses on the market work very well, that's not the problem. The problem is that even "small" cameras and lenses are huge in comparison to what can be integrated into a robot. I use a couple M4/3 cinema/studio cameras for vision projects because they're the smallest affordable cameras with controllable interchangeable lenses, and even with their "small" size they are at the upper limit of anything I could hope to fit in a practical robot. But this is largely because modern lenses are seriously bloated, with IS for older cameras that have non-stabilized sensors, bloated focusing mechanisms to make floating elements work, bloated zooming mechanisms that make everything so much worse, and so on. If we want to design a lens for robotics, we really don't need much more than a double-gauss normal lens with a unitary focusing mechanism. One of the lenses I use when I need to kludge something in a project is exactly that, a 1970s Minolta 55 f/1.7 lens for a full-frame image circle. Despite having a large 32mm aperture, it has a diameter of 56mm and a length of 44mm. I don't know if that's too large for your robotics applications, but it isn't very big and it's much smaller than the LiDAR sensors I've worked with. You could likely make this even smaller with modern techniques. It would also cost about 40$ to build. >The human eye is vastly more complex and capable than a fixed plane imager and capable of much more complex focus behavior than any existing lenses are. Camera-eye comparisons are appealing, but the focus behavior of them is really very different. My point was moreso that in pure optical sizes, the human eye is bigger and takes up more volume than the optical system I'm proposing. But as far as focusing behaviours, if you were to mathematically model how the human eye gets depth information to drive the focus cycle its quite similar to a phase detection system, with the phase information being extracted through the parallax. >Canon has done a very nice job with their autofocus, but I will believe they have solved depth perception with phase detection when they apply it to their industrial cameras. Canon makes very nice RV-series parts recognition cameras that do a very good job with challenging materials, but they use structured light. In theory, phase-detection SfF would be perfect for these cameras but they haven't done it. I think this can be explained quite simply. In a fully controlled environment, using structured light but cheap lenses and small sensors is much more cost-effective. Why would they use expensive APS-C+ sized sensors, that they don't seem to be all that good at making, when you can get away with very cheap parts? >The minor problem is that you generally want depth plus RGB, which is obviously solved by adding a second camera but then you face basically the same size and overlap issues that stereo cameras bring. Not entirely a free lunch. But this is precisely the point of OSPDAF or split-pixel phase detection. You get both depth and RGB at the same time, because the RGB sensor doubles as a phase detection sensor. It's not like old cameras where you had to use a mirror that sent light to the dedicated phase detection sensor. As for political problems, I agree. Canon won't ever sell you their DPAF sensors. Sony, even though they are really bad at robotics, should in theory be willing to sell anyone their sensors, and it should be possible to make a prototype. Their tech in this would be limited relatively to Canon, though. But, to the point, a solution to a problem that's proprietary is still a solution to the problem.