4 ms·

This is a nice proof of concept, but it's not anywhere production-ready. For instance, this apple harvester is using UR5 robot arm. It's a very research-friend

by krasin 5y ago

This is a nice proof of concept, but it's not anywhere production-ready. For instance, this apple harvester is using UR5 robot arm. It's a very research-friendly robot that I like to work with, but it's nowhere ready for outdoors. Its reach is just shy of 1 meter -- not practical for apples as well. Their choice to place an Intel RealSense camera on the wrist of the robot means that they have a USB cable running through the robot. My experience shows that even heavy-duty cables only work for hundreds of hours in such a mode and then require a replacement. The soft gripper that they use is a good choice for the task, but it depends on compressed air, which means an extra tube running through the robot arm - usually it means even more maintenance required. Still, this approach can be productionized, but the end result might look very different from this prototype.

skinner927 5y ago
What kind of data transmission line can be used for this type of environment?
- krasin 5y ago
  Many robot arms provide robust RS-485 and (in some cases) Ethernet and have all wires plumbed inside the arm (so, little to no maintenance). In certain non-critical cases (like, agriculture), wireless can be used too, for instance, when there's a strong urge to transfer more video data than the internal link allows for.
swiley 5y ago
It's a huge improvement over where we were. I remember being told that picking fruit at all was very hard/impossible for robots even in a lab.
mysterydip 5y ago
Would wifi or bluetooth have enough bandwidth to transfer camera data? That would remove the high maintenance cable from the equation at least.
- krasin 5y ago
  Specifically RealSense sends too much uncompressed depth data (>1 Gbit/s). A better approach would be to encapsulate perception and ML model inference in the end effector and only send robot commands over existing wired links. Wireless is still not a bad idea, but needs to be used as a last resort.
- alaties 5y ago
  These kinds of cameras produce pretty high volumes of data, since it's picture + 3D dot cloud per frame. My guess would be even 1G over wifi wouldn't be enough throughput without some compression engine in place.
- angry_octet 5y ago
  The data rate is very high for wifi, basically it reduces the frame rate and hence increases the delay in feedback. Putting the processor at the end of the actuator or sensor arm puts large stresses and vibrations on the electronics, I'd avoid it. Industrial and armoured USB cabling exist, including 360 degree couplers[1], but obviously it is specialised. Industrial Ethernet is better (e.g. GigE Vision) from an integration pov. [1] https://www.rotarx.com/en/slip-rings/usb-slip-rings/ https://www.rotarx.com/en/slip-rings/usb-slip-rings/
hosh 5y ago
Do you know if these robots are designed for use with existing orchards, or do they require planting and cultivating the orchards in a way to make it easier for the robots? (I remember reading about cultivating trees in a way to make it easier for humans to pick, and something about making it easier for robots to pick. The downside is that it requires a lot more effort at pruning and maintaining the trees, when adjusting for wind, sun, and water).
- krasin 5y ago
  It's safe to say that orchards are not changing anytime soon. These robots are not useful yet, and to demonstrate their utility they need to work on existing orchards. Only once everyone is using robots for apple picking, we might see some new orchards deciding on a denser layout / higher trees.
neolog 5y ago
> hundreds of hours Does that apply to cables designed for use in robots? eg https://okielectriccable.com/okis-robot-cables/ https://okielectriccable.com/okis-robot-cables/
- krasin 5y ago
  These cables would work for power and maybe even Ethernet. The problem with USB is that the protocol is designed with no data loss in mind. A single bit flip due to EM interference or bad cable, and the camera firmware (or the host driver) hangs and require a restart. Intel RealSense and Microsoft Kinect Azure cameras all have to be plugged into USB hubs which allow power cycling. Otherwise, it's just not practical. And even then, after a few hundreds of hours the frequency of restarts it degrades even further. I might sound a bit dramatic, it's mostly that my scars are too fresh.
  - qot 5y ago
    > The problem with USB is that the protocol is designed with no data loss in mind USB employs two error checking methods to ensure that data is sent correctly: https://web.archive.org/web/20130615145009/http://www.totalphase.com/solutions/wp/debugging_usb/ https://web.archive.org/web/20130615145009/http://www.totalp...
    - krasin 5y ago
      Yes. Error checking but not error correction. Bit flips are detected, but they are usually handled with a panic on one of the sides. In Ethernet, on the other hand, packet loss is routinely handled, and higher level protocols can work when physical media is less than perfect.
      - qot 5y ago
        No, if a bit flip is detected the packet is simply resent: > If there is a data error and the CRC check fails, however, the receiver will not reply with an ACK, and the transmitter is required to resend the data with the same toggle bit. The transmitter will continue to resend the same data with the same toggle bit until the receiver ACKs its reception.
hinkley 5y ago
My love of solving problems and my love of nature end up at loggerheads every time automated farming comes up. Farm robots always seem to point out to me the 'white tower syndrome' that researchers fall into. Dirt and robots is like the old joke about spherical cows. Guess what, plants grow where it rains consistently. Plants therefore grow in mud, and sometimes we throw poop at them on purpose. Sooner or later it will start raining just as you are at the farthest point on the property from shelter, and early season work often coincides with freezing rain or hail. And the more you avoid working in the wet, the more you have to deal with dust and grit.
- hinkley 5y ago
  > And the more you avoid working in the wet, the more you have to deal with dust and grit. Incidentally, part of my evolution toward gardening in the rain is realizing I'd rather be wet than sneezing, coughing, or getting dust in my eyes. Rain is also more pleasant than sweating like a pig. Luckily I'm not dealing with clay so I can bend some rules about digging in wet soil.
goler 5y ago
Yes. Abundant Robotics has a more developed solution that is better suited to the task, I think. It uses a vacuum effector. https://youtu.be/-PtqZA2enkQ?t=62 https://youtu.be/-PtqZA2enkQ?t=62
Animats 5y ago
This is a nice proof of concept, but it's not anywhere production-ready. Yes. Nor is it the first. Here's one from 2019, and much faster. [1] Another academic one from 2017.[2] A second one from 2017.[3] Another one from 2016.[4] Eventually, someone is going to get this right. Someone who's good at machinery design. Most of these have dangling wires or duct tape showing, or are far too slow. A good practical question is "Can you pressure-wash it?" This also applies to food-preparation robots. The computer vision part seems to be a solved problem. All of these can find apples visually. Few people are really good at mechanical design. I've mentioned that I restore old Teletype machines. Those are very well designed. All the parts are replaceable. They're reliable, even though complicated. They have few wear problems. Very few of the parts are fragile. They are unharmed by minor jamming problems. They do need annual lubrication. There's a complex balance of complexity, manufacturability, maintainability, and cost in mechanical design. Two people, Walter Krum and Ed Kleinschmidt, designed all the good ones of the 20th century. Many others tried, and produced inferior machines. This is more recognized in the gun industry. There are few good new gun designs. Every few decades, someone makes a good new design, and that gets copied and modified. Well known examples include the 1911A1, by John Browning, the AK-47, by Mikhail Kalashnikov, and the Glock, by Gaston Glock. All demonstrate a mastery of the materials used and a good understanding of the production processes. It's not just the form. [1] https://youtu.be/aijzVv6UeLQ https://youtu.be/aijzVv6UeLQ [2] https://youtu.be/SwE-LPS01lk https://youtu.be/SwE-LPS01lk [3] https://youtu.be/UaL3UxUclKY https://youtu.be/UaL3UxUclKY [4] https://youtu.be/TBcWZcjXr-I?t=59 https://youtu.be/TBcWZcjXr-I?t=59
- hinkley 5y ago
  I recall a strawberry picker from around... 2014? If memory serves, they achieved speed by reducing harvest %.

by krasin 5y ago

skinner927 5y ago

What kind of data transmission line can be used for this type of environment?

krasin 5y ago
Many robot arms provide robust RS-485 and (in some cases) Ethernet and have all wires plumbed inside the arm (so, little to no maintenance). In certain non-critical cases (like, agriculture), wireless can be used too, for instance, when there's a strong urge to transfer more video data than the internal link allows for.

swiley 5y ago

It's a huge improvement over where we were. I remember being told that picking fruit at all was very hard/impossible for robots even in a lab.

mysterydip 5y ago

Would wifi or bluetooth have enough bandwidth to transfer camera data? That would remove the high maintenance cable from the equation at least.

krasin 5y ago
Specifically RealSense sends too much uncompressed depth data (>1 Gbit/s). A better approach would be to encapsulate perception and ML model inference in the end effector and only send robot commands over existing wired links. Wireless is still not a bad idea, but needs to be used as a last resort.
alaties 5y ago
These kinds of cameras produce pretty high volumes of data, since it's picture + 3D dot cloud per frame. My guess would be even 1G over wifi wouldn't be enough throughput without some compression engine in place.
angry_octet 5y ago
The data rate is very high for wifi, basically it reduces the frame rate and hence increases the delay in feedback. Putting the processor at the end of the actuator or sensor arm puts large stresses and vibrations on the electronics, I'd avoid it. Industrial and armoured USB cabling exist, including 360 degree couplers[1], but obviously it is specialised. Industrial Ethernet is better (e.g. GigE Vision) from an integration pov. [1] https://www.rotarx.com/en/slip-rings/usb-slip-rings/ https://www.rotarx.com/en/slip-rings/usb-slip-rings/

hosh 5y ago

Do you know if these robots are designed for use with existing orchards, or do they require planting and cultivating the orchards in a way to make it easier for the robots? (I remember reading about cultivating trees in a way to make it easier for humans to pick, and something about making it easier for robots to pick. The downside is that it requires a lot more effort at pruning and maintaining the trees, when adjusting for wind, sun, and water).

krasin 5y ago
It's safe to say that orchards are not changing anytime soon. These robots are not useful yet, and to demonstrate their utility they need to work on existing orchards. Only once everyone is using robots for apple picking, we might see some new orchards deciding on a denser layout / higher trees.

neolog 5y ago

> hundreds of hours Does that apply to cables designed for use in robots? eg https://okielectriccable.com/okis-robot-cables/ https://okielectriccable.com/okis-robot-cables/

krasin 5y ago
These cables would work for power and maybe even Ethernet. The problem with USB is that the protocol is designed with no data loss in mind. A single bit flip due to EM interference or bad cable, and the camera firmware (or the host driver) hangs and require a restart. Intel RealSense and Microsoft Kinect Azure cameras all have to be plugged into USB hubs which allow power cycling. Otherwise, it's just not practical. And even then, after a few hundreds of hours the frequency of restarts it degrades even further. I might sound a bit dramatic, it's mostly that my scars are too fresh.
- qot 5y ago
  > The problem with USB is that the protocol is designed with no data loss in mind USB employs two error checking methods to ensure that data is sent correctly: https://web.archive.org/web/20130615145009/http://www.totalphase.com/solutions/wp/debugging_usb/ https://web.archive.org/web/20130615145009/http://www.totalp...
  - krasin 5y ago
    Yes. Error checking but not error correction. Bit flips are detected, but they are usually handled with a panic on one of the sides. In Ethernet, on the other hand, packet loss is routinely handled, and higher level protocols can work when physical media is less than perfect.
    - qot 5y ago
      No, if a bit flip is detected the packet is simply resent: > If there is a data error and the CRC check fails, however, the receiver will not reply with an ACK, and the transmitter is required to resend the data with the same toggle bit. The transmitter will continue to resend the same data with the same toggle bit until the receiver ACKs its reception.

hinkley 5y ago

My love of solving problems and my love of nature end up at loggerheads every time automated farming comes up. Farm robots always seem to point out to me the 'white tower syndrome' that researchers fall into. Dirt and robots is like the old joke about spherical cows. Guess what, plants grow where it rains consistently. Plants therefore grow in mud, and sometimes we throw poop at them on purpose. Sooner or later it will start raining just as you are at the farthest point on the property from shelter, and early season work often coincides with freezing rain or hail. And the more you avoid working in the wet, the more you have to deal with dust and grit.

hinkley 5y ago
> And the more you avoid working in the wet, the more you have to deal with dust and grit. Incidentally, part of my evolution toward gardening in the rain is realizing I'd rather be wet than sneezing, coughing, or getting dust in my eyes. Rain is also more pleasant than sweating like a pig. Luckily I'm not dealing with clay so I can bend some rules about digging in wet soil.

goler 5y ago

Yes. Abundant Robotics has a more developed solution that is better suited to the task, I think. It uses a vacuum effector. https://youtu.be/-PtqZA2enkQ?t=62 https://youtu.be/-PtqZA2enkQ?t=62

Animats 5y ago

This is a nice proof of concept, but it's not anywhere production-ready. Yes. Nor is it the first. Here's one from 2019, and much faster. [1] Another academic one from 2017.[2] A second one from 2017.[3] Another one from 2016.[4] Eventually, someone is going to get this right. Someone who's good at machinery design. Most of these have dangling wires or duct tape showing, or are far too slow. A good practical question is "Can you pressure-wash it?" This also applies to food-preparation robots. The computer vision part seems to be a solved problem. All of these can find apples visually. Few people are really good at mechanical design. I've mentioned that I restore old Teletype machines. Those are very well designed. All the parts are replaceable. They're reliable, even though complicated. They have few wear problems. Very few of the parts are fragile. They are unharmed by minor jamming problems. They do need annual lubrication. There's a complex balance of complexity, manufacturability, maintainability, and cost in mechanical design. Two people, Walter Krum and Ed Kleinschmidt, designed all the good ones of the 20th century. Many others tried, and produced inferior machines. This is more recognized in the gun industry. There are few good new gun designs. Every few decades, someone makes a good new design, and that gets copied and modified. Well known examples include the 1911A1, by John Browning, the AK-47, by Mikhail Kalashnikov, and the Glock, by Gaston Glock. All demonstrate a mastery of the materials used and a good understanding of the production processes. It's not just the form. [1] https://youtu.be/aijzVv6UeLQ https://youtu.be/aijzVv6UeLQ [2] https://youtu.be/SwE-LPS01lk https://youtu.be/SwE-LPS01lk [3] https://youtu.be/UaL3UxUclKY https://youtu.be/UaL3UxUclKY [4] https://youtu.be/TBcWZcjXr-I?t=59 https://youtu.be/TBcWZcjXr-I?t=59

hinkley 5y ago
I recall a strawberry picker from around... 2014? If memory serves, they achieved speed by reducing harvest %.