7 ms·

In grad school I was using stereo video cameras to measure fish. I wonder if a model like this could do it accurately from frame grabs from a single feed now. A

In grad school I was using stereo video cameras to measure fish. I wonder if a model like this could do it accurately from frame grabs from a single feed now. And of course an AI to identify fish, even if was just which sections of video had/did not have fish, not even doing the species level ID, would have saved a ton of time. We had a whole workshop on various monitoring technologies and the take home from the various video tools is that having highly trained grad students and/or techs watch and analyze the video is extremely slow and expensive. I haven't worked with video in a while now, but I wonder if any labs are doing more automated identification these days. It feels like the kind of problem that is probably completely solvable if the right tech gets applied.

GaggiX 3y ago
You can definitely train a model to identify fish, to be honest you don't really have to train a whole model, there are tons of models trained on millions of images, you can just extract the embeddings from those models and train a single matrix to project them to the different classes and it will work very well.
CGamesPlay 3y ago
Definitely not with this model, because it’s impossible to tell based on the distance alone. Is the fish 34cm away and 34cm long or 30cm away and 30cm long? The fish is floating in a transparent medium, so reference points aren’t even useful as calibration.
godelski 3y ago
Very likely. Tbh, I think there are a lot of domain tasks where if you added a machine learning expert to the team that success and progress would be a lot higher. But to be fair, there are are a lot of people that can do ML but not a lot of people that have a deep understanding. The difference matters for real world tasks when the difference between dataset performance and generalization performance matter. And it's all too common that works that are SOTA are more difficult to generalize, but this is high variance.
bigger_cheese 3y ago
Are the fish always the same color/is their color distinct from the background. I work at an industrial plant we have been able to measure a lot of things simply by analyzing the pixels in the video. For example one application we have a camera pointed down at a conveyor belt. The conveyor belt is one color and objects on the belt are a distinct different color. - we just count how many pixels in a given frame are a specific color/brightness. Then you can easily work out how much of the conveyor belt has material on it in any given frame. So if you are tying to work out what section of a video has fish in it you could count how many pixels are a different color to the normal background color.

GaggiX 3y ago

You can definitely train a model to identify fish, to be honest you don't really have to train a whole model, there are tons of models trained on millions of images, you can just extract the embeddings from those models and train a single matrix to project them to the different classes and it will work very well.

CGamesPlay 3y ago

Definitely not with this model, because it’s impossible to tell based on the distance alone. Is the fish 34cm away and 34cm long or 30cm away and 30cm long? The fish is floating in a transparent medium, so reference points aren’t even useful as calibration.

godelski 3y ago

Very likely. Tbh, I think there are a lot of domain tasks where if you added a machine learning expert to the team that success and progress would be a lot higher. But to be fair, there are are a lot of people that can do ML but not a lot of people that have a deep understanding. The difference matters for real world tasks when the difference between dataset performance and generalization performance matter. And it's all too common that works that are SOTA are more difficult to generalize, but this is high variance.

bigger_cheese 3y ago

Are the fish always the same color/is their color distinct from the background. I work at an industrial plant we have been able to measure a lot of things simply by analyzing the pixels in the video. For example one application we have a camera pointed down at a conveyor belt. The conveyor belt is one color and objects on the belt are a distinct different color. - we just count how many pixels in a given frame are a specific color/brightness. Then you can easily work out how much of the conveyor belt has material on it in any given frame. So if you are tying to work out what section of a video has fish in it you could count how many pixels are a different color to the normal background color.