I want to automatically track heads in CCTV records.

Sometimes I need to re-identify heads because of multiple heads crossing each other. So I continually need to extract features from tracked heads so that I will be able to re-identify them when needed. Those tracked heads are seen from any angle.

What would be the best features to describe heads?

EDIT: Precisions

  • I work with multiple cameras (15) in differents positions generally placed from 0.5 to 20 meters to the people. Those cameras are inside a building, the luminosity is quite good (but I also would like to work with IR during the night, but for now working with light), cameras are 2048x1536 @ 20fps with color.
  • Cameras are barely overlapping
  • For now, I use https://github.com/Russell91/TensorBox to detect heads accurately from any angle and it works very well (I have almost the same angle and exposition than in TensorBox readme image).
  • As TensorBox is very slow (20ms->70ms/image on gpu) I use it only when there is motion and only 2times/sec max. I use optical flow (calcOpticalFlowPyrLK from openCV) to follow heads between two head detections.
  • I don't have any information on people. When there is someone new, I create a new identity. Then I try to follow him on the camera network and save the path he followed.
  • I am not able to use face detectors for re-identification as cameras do not necessarily see them from the front. But this could improve re-identification.
  • So there are two different cases to handle with re-identification:
    • Multiple persons overlap
    • Person X is moving to another camera

I can easily track persons' heads for now, but I cannot identify them easily because I cannot distinguish them. I think I will need a simple multi-class classifier, but I don't know what would be the best features to describe persons' heads that are moving in any directions.

Thanks for your help!

  • $\begingroup$ Are these color images? Have you considered using some kind of color histogram map for the head? This might fail if the person's orientation changes, though (e.g., they turn their head; they move from the field of vision of one camera to that of another camera that's facing a different direction). $\endgroup$ – D.W. Nov 20 '16 at 18:41
  • $\begingroup$ These are colored images yes. It is indeed an idea, I already thought about it, but this can result in very poor results depending on the number of persons in the building. I think I will combine multiple things such as color histograms, biometric recognition, SIFT/SURF/... features but I am not sure what would be the best features to describe a head. A human could describe a head by having certain colors, a certain haircut, a certain proportion, a certain face... I even thought about modelizing heads in 3D and matching the nearest one. What would you recommend? $\endgroup$ – Dubrzr Nov 20 '16 at 19:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.