2
$\begingroup$

I'm looking for a computer vision algorithm or method that can tell if the camera is moving in a video. Or maybe an alternate way of telling if the background is moving. I have a lot of videos and only want to keep the ones where the camera is stationary.

All videos feature a person talking directly to the camera (the person is mostly stationary throughout the video). I want to keep videos where the camera is stationary, but there are a lot of videos where the person is holding the camera and walking or the camera is moving around too much

I plan to run some algorithm one by one on a collection of videos to determine which video IDs have a moving camera in them.

$\endgroup$
2
$\begingroup$

Use background detection algorithm, say Gaussian Mixture-based background detection (present in OpenCv) and then make some assumption - say backround takes more than 60% of image.
Then convert frames to color model which keeps luminance as separate component. Compare consecutive frames with small tolerance per color per pixel and bigger tolerance per light (to take into account shadows). If your camera is not really stationary, check by movement algorithm if frame is shifted by some distance (camera may move slightly even if it stands still), in that case compare it with shifted frame after detecting shift.

Now gather frames with some tolerance, say 95% of frames match background in at least 60% of pixels with slight shifts (it depends on distance from camera to background).
If it matches, camera was still, if not then either parameters were too big or it was moving.
Adjust parameters, done.

$\endgroup$
2
$\begingroup$

This is an experimental area, so expect that you will need to try something, see how well it works, and then adjust to address error cases.

Here is a possible approach, that's a variation on what Evil suggests:

You could use optical flow on the video, to find the direction in which the pixels are moving. Use background detection to detect which pixels are part of the background.

Then if all (or most) of the non-background pixels in the image are moving, and all in basically the same direction (though not necessarily the same amount, as some objects may be near and some far), conclude that the camera is moving. Otherwise, conclude that the camera is still.

This is something you could evaluate, to see how often it gets the right answer, and for the cases that it's wrong, if there are commonalities to the failures.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.