From my 5 minutes understanding of the problem: Non deep learning solution Step 0: you should have some training data. Draw it < I believe 50 of each class you

From my 5 minutes understanding of the problem: Non deep learning solution Step 0: you should have some training data. Draw it < I believe 50 of each class you want to detect should be enough to start, but you can think about doubling it> First you need look at the opencv library, specifically in the findcontours function https://docs.opencv.org/2.4/modules/imgproc/doc/structural_analysis_and_shape_descriptors.html?highlight=findcontours#findcontours https://docs.opencv.org/2.4/modules/imgproc/doc/structural_a... - I'm linking to the c++ api but the python has the exact same function. Make sure your contours are clean of rubbish data because this can affect the outcome of the classifier. With the extracted attributes of each individual class you pass on this data to a multi class classifier (http://scikit-learn.org/stable/modules/multiclass.html http://scikit-learn.org/stable/modules/multiclass.html) Parameterize your model, Split your data into training and testing ... yada yada ... With the generated model, you can receive individual detected contours and it will give you the fitting probability to each class, use it to feed an api which provides data to a react component that renders html. >> ps: Ah you need an image capture app which reads the image in real time, pass the filters and detect contours to then pass it to the classifier. >> ps2: I'm available for projects :P