Please advise me about this Computer Vision task

I have some magnets with images on it. There is only a small set of different images (lets say 20 images), and they will be aligned in fixed chess boar (still not shown in the picture).

I have already an algorithm to extract each single magnet card from the board, applying a perspective transformation to the original image to prevent perspective distortion.

I would like to hear your advices in order to detect each single image on the board, I mean, detect if the magnet is: a panda, a rabbit, a dog, a carrot... since my main objective is to analyze the image and extract a matrix containing all the board elements.

My first attempt was very basic: guess the image according to the average color. It was not very robust since there are several images with similar average colors (specially those frozen cards), and light tinting can change color a lot.

Would you be so kind to point me in the right direction to extract a matrix containing all the images on the board? I don't need a specific implementation, but instead the concept of the steps that I should follow or techniques to be aplied to the main image in order to obtain a robust (and not too complex) algorithm.

I'm gonna implement it using OpenCV, but I guess that it would be the same using any other computer vision libraries.

Thanks a lot for your time!