If you don't know how to set up a dataset, it's probably too early for you to worry about performance and efficiency. If you haven't, already, I'd suggest to l

If you don't know how to set up a dataset, it's probably too early for you to worry about performance and efficiency. If you haven't, already, I'd suggest to learn some general machine learning, including how to use logistic regression, random forests and SVMs. Keras is certainly capable of what you want to do, at least from your description. One way is to interpret the colors as grayscale images, that would be the fastest option. If however the 16 colors are actually from a palette, it may be better to convert the image to three channels, r/g/b. And if the 16 colors are 16 entirely different things, like 0 - Water, 1 - sand, 2 - earth and so on, you could even turn one 16 color image into 16 images with two colors (1 bit), and get a better model. Again, getting into machine learning or deep learning is not as easy as reading the Keras documentation. You need to understand the basics first.