3 ms·

Two decades ago, I was trying to use classical machine vision to tell the difference between cut and uncut grass, to guide a self-driving lawnmower. I conclude

Two decades ago, I was trying to use classical machine vision to tell the difference between cut and uncut grass, to guide a self-driving lawnmower. I concluded that it couldn't be done with classical machine vision, and that this "neural network" nonsense wasn't going to catch on. Very slow, computationally inefficient, full of weirdos making grandiose claims about "artificial intelligence" without the results to back it up, and they couldn't even explain how their own stuff worked. These days - you want to find the boundary between cut and uncut grass, even though lighting levels can change and cloud cover can change and shadows can change and reflections can change and there's loads of types of grass and grass looks different depending on the angle you look from? Just label some data and chuck a neural network at it, no problemo.

justmarc 2y ago
Funnily now with with the advent of GPS+RTK lawnmower robots, fancy AI is not even needed anymore. They follow a very exact, pre-determined patterns and paths, and do a great job.
- michaelt 2y ago
  Yeah GPS+RTK was what I went with in the end. Didn't work as well as I'd hoped back in those days though, as you could lose carrier lock if you got too close to trees (or indeed buildings), and our target market was golf courses which tend to have a lot of trees. And in those days a dual-frequency RTK+IMU setup was $20k or more, which is expensive for a lawnmower.
  - justmarc 2y ago
    No tool is perfect for every job. That said, the positioning of the RTK unit is crucial. Possibly look for a mower which can work with multiple RTK units, or reposition your existing one for better coverage. I find that even though signals get significantly weaker under trees, mine still works wonderfully in a complex large garden scenario. It will depend on your exact unit/model, as well as their firmware and how it chooses to deal with these scenarios.
madaxe_again 2y ago
That, and you can now train the perfect lawnmower in an entirely virtual environment before dropping it into a physical body. You do your standard GAN thing, have a network that is dedicated to creating the gnarliest lawn mowing problems possible, bang through a few thousand generations of your models, and then hone the best of the best. There are some really astonishing examples that have been published this last year or so - like learning to control a hand from absolute first principles, and perfecting it. This is all pretty much automated by nvidia’s toolkits, and you can do it cheaply on rented hardware before dropping your pretrained model into cheap kit - what a time to be alive.
- blagie 2y ago
  FYI: A comment like this one is more helpful with links. There's one below with a few. If you happen to read this, feel free to respond, or to hit "edit" and add them.
blagie 2y ago
> These days - you want to find the boundary between cut and uncut grass, even though lighting levels can change and cloud cover can change and shadows can change and reflections can change and there's loads of types of grass and grass looks different depending on the angle you look from? Just label some data and chuck a neural network at it, no problemo. If only. Having been faced with the same problem in the real world: 1) There isn't a data bank of millions of images of cut / uncut grass 2) If there were, there's always the possibility of sample bias. E.g. all the cut photos happen to have been taken early in the day, of uncut late in the day, and we get a "time-of-day" detector. Sample bias is oddly common in vision data sets, and machine learning can look for very complex sample bias 3) With something like a lawnmower, you don't want it to kill people or run over flowerbeds. There can be actual damages. It's helpful to be able to understand and validate things. Most machine vision algorithms I actually used in projects (small n) made zero use of neural networks, and 100% of classical algorithms I understand. Right now, the best analogy to NLP is BERT. At that point, neural techniques were helpful for some tasks, and achieved stochastically interesting performance, but were well below the level of general uses, and 95% of what I wanted to do used classical NLP. IF I had a large data set AND could do transfer training from BERT AND didn't need things to work 100% of the time, BERT was great. Systems like DALL-e and the reverse are moving us in the right direction. Once we're at GPT / Claude / etc.-level performance, life will be different, and there's a light at the end of the tunnel. For now, though, the ML machine is still a pretty limited way to go. Think of it this way. What's cheaper: 1) A consulting project for a human expert in machine vision (tens or hundreds of thousands of dollars) 2) Hiring cheap contractors to build out a massive dataset of photos of grass (millions of dollars)
- krisoft 2y ago
  > What's cheaper If you don’t have the second how can you trust the first? Without the dataset to test on your human experts will deliver you slop and be confident about it. And you will only realise the many ways their hand finessed algorithms fail once you are trying to field the algorithm. > With something like a lawnmower, you don't want it to kill people or run over flowerbeds. Best to not mix concerns though. Not killing people with an automatic lawnmover is about the right mechanical design, appropriately selected slow speed, and bumper sensors. None of this is an AI problem. We don’t have to throw out good engineering practices just because the product uses AI somewhere. It is not an all or nothing thing. The flowerbed avoidance question might or might not be an AI problem depending on design decisions. > Hiring cheap contractors to build out a massive dataset of photos of grass (millions of dollars) I think that you are over estimating the effort here. The database doesn’t have to be so huge. Transfer learning and similar techniques reduced the data requirements by a lot. If all you want is a grass height detector you can place stationary cameras in your garden, collect a bunch of data and automatically label them based on when you moved the grass. That will obviously only generalise to your garden, but if this is only a hobby project maybe that is all you want? If this is a product you intend to sell for the general public then of course you need access to a lot of different gardens to test it on. But that is just the nature of product testing anyway.
fakedang 2y ago
What is classical machine vision? For an image recognition problem, wouldn't you use a conventional neural network? (And I might be a bit outdated here, considering that CNNs were used for image recognition a decade ago).

justmarc 2y ago

Funnily now with with the advent of GPS+RTK lawnmower robots, fancy AI is not even needed anymore. They follow a very exact, pre-determined patterns and paths, and do a great job.

michaelt 2y ago
Yeah GPS+RTK was what I went with in the end. Didn't work as well as I'd hoped back in those days though, as you could lose carrier lock if you got too close to trees (or indeed buildings), and our target market was golf courses which tend to have a lot of trees. And in those days a dual-frequency RTK+IMU setup was $20k or more, which is expensive for a lawnmower.
- justmarc 2y ago
  No tool is perfect for every job. That said, the positioning of the RTK unit is crucial. Possibly look for a mower which can work with multiple RTK units, or reposition your existing one for better coverage. I find that even though signals get significantly weaker under trees, mine still works wonderfully in a complex large garden scenario. It will depend on your exact unit/model, as well as their firmware and how it chooses to deal with these scenarios.

madaxe_again 2y ago

That, and you can now train the perfect lawnmower in an entirely virtual environment before dropping it into a physical body. You do your standard GAN thing, have a network that is dedicated to creating the gnarliest lawn mowing problems possible, bang through a few thousand generations of your models, and then hone the best of the best. There are some really astonishing examples that have been published this last year or so - like learning to control a hand from absolute first principles, and perfecting it. This is all pretty much automated by nvidia’s toolkits, and you can do it cheaply on rented hardware before dropping your pretrained model into cheap kit - what a time to be alive.

blagie 2y ago
FYI: A comment like this one is more helpful with links. There's one below with a few. If you happen to read this, feel free to respond, or to hit "edit" and add them.

blagie 2y ago

> These days - you want to find the boundary between cut and uncut grass, even though lighting levels can change and cloud cover can change and shadows can change and reflections can change and there's loads of types of grass and grass looks different depending on the angle you look from? Just label some data and chuck a neural network at it, no problemo. If only. Having been faced with the same problem in the real world: 1) There isn't a data bank of millions of images of cut / uncut grass 2) If there were, there's always the possibility of sample bias. E.g. all the cut photos happen to have been taken early in the day, of uncut late in the day, and we get a "time-of-day" detector. Sample bias is oddly common in vision data sets, and machine learning can look for very complex sample bias 3) With something like a lawnmower, you don't want it to kill people or run over flowerbeds. There can be actual damages. It's helpful to be able to understand and validate things. Most machine vision algorithms I actually used in projects (small n) made zero use of neural networks, and 100% of classical algorithms I understand. Right now, the best analogy to NLP is BERT. At that point, neural techniques were helpful for some tasks, and achieved stochastically interesting performance, but were well below the level of general uses, and 95% of what I wanted to do used classical NLP. IF I had a large data set AND could do transfer training from BERT AND didn't need things to work 100% of the time, BERT was great. Systems like DALL-e and the reverse are moving us in the right direction. Once we're at GPT / Claude / etc.-level performance, life will be different, and there's a light at the end of the tunnel. For now, though, the ML machine is still a pretty limited way to go. Think of it this way. What's cheaper: 1) A consulting project for a human expert in machine vision (tens or hundreds of thousands of dollars) 2) Hiring cheap contractors to build out a massive dataset of photos of grass (millions of dollars)

krisoft 2y ago
> What's cheaper If you don’t have the second how can you trust the first? Without the dataset to test on your human experts will deliver you slop and be confident about it. And you will only realise the many ways their hand finessed algorithms fail once you are trying to field the algorithm. > With something like a lawnmower, you don't want it to kill people or run over flowerbeds. Best to not mix concerns though. Not killing people with an automatic lawnmover is about the right mechanical design, appropriately selected slow speed, and bumper sensors. None of this is an AI problem. We don’t have to throw out good engineering practices just because the product uses AI somewhere. It is not an all or nothing thing. The flowerbed avoidance question might or might not be an AI problem depending on design decisions. > Hiring cheap contractors to build out a massive dataset of photos of grass (millions of dollars) I think that you are over estimating the effort here. The database doesn’t have to be so huge. Transfer learning and similar techniques reduced the data requirements by a lot. If all you want is a grass height detector you can place stationary cameras in your garden, collect a bunch of data and automatically label them based on when you moved the grass. That will obviously only generalise to your garden, but if this is only a hobby project maybe that is all you want? If this is a product you intend to sell for the general public then of course you need access to a lot of different gardens to test it on. But that is just the nature of product testing anyway.

fakedang 2y ago

What is classical machine vision? For an image recognition problem, wouldn't you use a conventional neural network? (And I might be a bit outdated here, considering that CNNs were used for image recognition a decade ago).