3
$\begingroup$

I recently found this music video. While the music is mislabeled, the video part caught my eye. It seems to be transitioning between two images based on shape? I think its a fade-in/fade-out with the images being streched and transformed, but it seems to be specific on the shape in the image. This may be manual, but anybody have ideas on how one might go about something like this automatically? I was thinking it could be possible if you find the "flow" of an image by assigning pixels vectors based on their color compared to their neighbor's colors.

$\endgroup$
1
$\begingroup$

The first effect that appears at the video is a hue change. Later this is morphing, which is a feature based operation.

It seems like some similarity between images is used to set the morphing points (looking at the consecutive changes the points that do not move are chosen separately per transition).

The part of comparing two images (your last sentence) is used to make transition smoother, otherwise it would resemble warp. This is known as the Temporal Coherence, another technique includes Structural Similarity. Techniques yielding the best results are semi-automatic or manual due to a big complexity involved in finding smooth, minimal transition well perceived by people and also lack of good formula evaluating "how natural" transition looks like. The optimal transition calculated as minimal sum of changes does not yield nice looking morphing.

Sometimes a motion blur is added on top of the morphing (between patches) to make transition look better.

$\endgroup$
  • $\begingroup$ Very helpful, thank you! You wouldn't happen to know about a decent way to automate selecting control points? I tried finding some, but methods are mostly for human faces. Thinking of machine learning for something like this, not sure how I'd auto-verify for training though. $\endgroup$ – Vedvart1 Jan 21 '17 at 3:43
  • $\begingroup$ I have added two ideas of automatic morphing (not limited to the faces). As far as I know, it is still ongoing research to develop fully automatic method that works without manual post-processing. For now it does matter what you are morphing to pick the best automatic method. For the rest the standard trick is in blurring and other techniques hidding imperfections. $\endgroup$ – Evil Jan 21 '17 at 4:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.