1
$\begingroup$

I have a video that record the drawing sequence with a pencil by a painter. video screen I want to reconstruction the sequence from the video. And I have google some keywords like "drawing sequence reconstruction", "sketch reconstruction", and have searched like "moving object detection" but cannot found what I want to find.

I want to know how to detect and recognize the drawing between different frames. I have thought the "line detect" algorithm may work but fail. And pixel comparison seems to be bad idea. Because that the new drawing may overlap the old ones.

Could you give me some information that I can read or some keywords to search? Sorry about that because I am not familiar with Computer Image Processing.

$\endgroup$
  • 1
    $\begingroup$ Some kind of constants would be helpful - what is the framerate? Is it heavily compressed - what is the output format (what do you want to get). Can you erase something on the board? Try to avoid scene change detection or background detection - very heavy and you have static drawing. But if you decide to do it - take simple time buffer, ignore variance and other heavy methods. If you think about Hough transform - please don't or at least take short version. For variance - if you must - take online algorithm $\endgroup$ – Evil Jan 26 '16 at 16:01
  • $\begingroup$ Frame rate would be 30 fps and video format is avi. I want to output the drawing sequence by time. I cannot erase on board, it is drawing on paper. $\endgroup$ – Jason Jan 26 '16 at 16:07
  • 1
    $\begingroup$ Ok. So this hand is not virtual pointer? Essentially if you cannot erase, you can assume that drawn pixels are there, when they are overlapped - just ignore, and when they appear compare to last known frame - by kniwn I mean once in the buffer they will be your backround. So after appearing you have to check if they are fresh or stored in history $\endgroup$ – Evil Jan 26 '16 at 16:17
1
$\begingroup$

You should have a look at background segmentation algorithms. The hand is moving over the rather static drawing. One very simple approach would be to store the last N images and compute the variance of the color for each pixel. The pixels with a low variance will be your painting (and the canvas you draw an) so a subsequent thresholding should seperate the drawing from the canvas.

$\endgroup$
1
$\begingroup$

I assume you have clean background like in the picture:
0) you take first frame - this is your initial data.
1) take next frame, then subtract previous from it - here white pixels are freshly drawn and black are unchanged.
If your frames are from compressed video - threshold them to avoid taking blurred parts.

You have not stated if stylus (pencil) is there - probably pixels are to be erased - in such case pixels under stylus should be flagged as overlapped and the drawn pixels are to be retreived from last nit overlapped frames.

After this operations you have pure pixels - 8-connected flood fill from changed pixels (if drawing was not continous - one per component).
Line detection is computationally expensive and works for lines / segments, but hand drawing gives poor results.

$\endgroup$
  • $\begingroup$ Actually, the test video will like imgur.com/FeN1IPy. And about overlap, I still cannot understand. Because if overlapped, they appear the same like the last frame. $\endgroup$ – Jason Jan 26 '16 at 16:49
  • $\begingroup$ I mean hand overlaps lines, the lines drawn will be checked in history (buffer of past frames) whether they are new drawn or existed earlier. Oh, this complicates a bit - shadows, gloves... Is it possible to take additional light to remove shadows? Could you use color (like blue or red) for a pencil? $\endgroup$ – Evil Jan 26 '16 at 17:34
  • $\begingroup$ Ok, in that case the background detection seems to be good option. Could you edit your question to update description, put image/video as example, and change tags? This is not connected to graphs, but is to computer graphics / image processing. The goal is to separate drawn pixels from the video? $\endgroup$ – Evil Jan 26 '16 at 17:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.