identify abrupt frame changes in a video

Question

The frames of live streams or videos, in general, do not flow smoothly. A sports game would have multiple cameras recording the game and hence, there will be camera switches in the stream. I am detecting objects and tracking them as long as possible. When there are full changes in the video stream, I wish to reset object tracking and do a full detection.

I am looking for methods to efficiently identify when a camera switch (or anything of that kind) happens.

What algorithms are available to efficiently identify abrupt changes in a video?

Answer 1

Easiest is to capture video with an h.264 encoder, which uses key frames to identify switches. (It’s a bit more complicated, h264 will handle situations where frames come alternatively fro two different source, for example), but extracting that information from a video stream is easier than doing it yourself).

Source code for decoders is available, and a decoder will also give you useful information about movement, and things that become covered or uncovered.

Answer 2

One approach: Use object tracking, to measure the change between each frame and the previous frame. For instance, you might build a metric based on how far each object moved and how many objects disappeared/appeared. If this metric is large, then suspect a camera switch.

It's likely that when there is a camera switch, the tracking algorithm will totally fail and won't be able to match any object in the prior frame to the current frame, so all objects in the prior frame will have disappeared and all objects in the current frame will be treated as just appearing. This is a strong signal of a camera switch.