I'm new to computer vision and I want to solve the task of recognizing the game units of the game Clash Royale in the screenshot. Briefly, there are about 70 different types of gaming units belonging to two teams (they differ a little in colors and some are visible in front, others with backs). I want to find game units on the screenshot and classify their type of unit (then health and the team).

image description

What are the best tools for task like this? What libraries will I use to simply teach the model? How many teaching examples do I need to learn the model? It seems that the quality of the screenshots is quite good and the images are clear. To what size should I reduce the screenshoot to get a good model and its speed? Maybe someone had a similar experience?

I think about CNN or a lot of HAAR cascades for each of the types of units, but I would like to receive advice.

  • $\begingroup$ Not sure if machine learning is needed here. Correct me if I'm wrong, but all of the unit models are known beforehand? If so, since there is a 1:1 match, you can do this with non-ml algorithms. Since the sizes are known as well, you can use a cascading window with the relevant size. Should be very fast for each frame. You can even downsample the image if you want to speed up computation, you'll probably still get good results. $\endgroup$ – Mickey Jul 14 '18 at 19:51
  • $\begingroup$ @Mickey Thanks for your comment. Not for all units in the game package are pre-assembled from individual tiles images. and in fact I found this task interesting for myself to try and learn computer vision. $\endgroup$ – Tolsi Jul 15 '18 at 9:57

Machine learning doesn't seem needed, since the image of each unit is always identical. One approach is to obtain a clean image of each unit, and then use template matching to find all locations where the template occurs in the screenshot. You might be able to check all locations in the screenshot whether they exactly match the template image (you will probably want to mask out the part where the health appears, as that will obscure the image). This can be done very efficiently.

To recover the health, you might also be able to do template matching on the numbers 0-10.

The way to tell what resolution you can downsize to is empirically: you experiment with different resolutions and measure the accuracy for each. There's probably no way to predict without trying it.

If you want to try machine learning, you could try a CNN, or better yet, you could try retraining the YOLOv3 object detector for your particular images (see, e.g., the YOLO project page or here). You will probably need a larger training set this way, and you'll have to do a bunch of manual annotation of sample screenshots.

Tool and library recommendations are off-topic here.

  • $\begingroup$ thanks for your answer! not so simple - with effects or damage units have different color effects, which does not just give you a 1: 1 pixel look. In addition, I would like to receive recognition on screenshots of any size that it is not possible to achieve this way. Now I have a couple of hundred images and plan to retrain a Tensorflow Object Detection model. Although I can try YOLO, thanks. $\endgroup$ – Tolsi Jul 16 '18 at 14:53
  • $\begingroup$ Having even a small labeled dataset in 150 images, the darknet53 network with YOLO v3 showed good results! It's just amazing! $\endgroup$ – Tolsi Jul 17 '18 at 7:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.