I am reading this paper Handy AR.

As suggested I am adding certain quotes from paper to get basic idea of what is it about :

  1. Intro

    In this paper, we present and evaluate a method to use a user’s bare outstretched hand in the same way a cardboard AR marker would be used, enabling spontaneous tangible user interfaces in mobile settings


  1. Adaptive Hand Segmentation

    Given a captured frame, every pixel is categorized to be either a skin-color pixel or a non-skin-color pixel. An adaptive skin color-based method 17 is used to segment the hand region. According to the generalized statistical skin color model 14, each pixel is determined to be in the hand region if the skin color likelihood is larger than a constant threshold.
    In order to adapt the skin color model to the illumination change, a color histogram of the hand region is learned for each frame and accumulated with the ones from the previous n frames (n = 5 works well in practice). Then the probability of skin color is computed by combining the general skin color model and the adaptively learned histogram.

  2. Accurate Fingertip detection

    Fingertips are detected from the contour of a hand using a curvature-based algorithm similar to the one described in [2]. We then fit the curvature points to ellipses in order to increase the accuracy. The curvature of a contour point is measured on multiple scale levels in order to detect fingertips with various sizes as follows: The points with higher curvature values than a threshold (on any scale level) are selected as candidates for fingertips.
    Since the detection algorithm may produce false positives of fingertips for initial detection, we choose the most frequently detected points above the center of the hand for a certain number of consecutive frames as our final fingertips. Thus, for initial detection, the hand has to be held fingers up, which is the most convenient and by far the most common pose anyway. After fingertips have been detected, we eliminate false positives by tracking a successful configuration over time.

  3. FingerTip Tracking

    Once fingertips are detected, we track them based on matching the newly detected fingertips to the previously tracked fingertips. Similar to 21, we track the fingertip trajectory by a matching algorithm that minimizes the displacement of pairs of fingertips over two frames. In addition, we use our knowledge about the centroid of the hand blob to effectively handle large movements of the hand as follows: The matching cost is minimized as the formula I mentioned below.

I have a couple of doubts in this :
1. In the 2nd paragraph of the 2nd points of steps:

After fingertips have been detected, we eliminate false positives by tracking a successful configuration over time.

What does 'successful configuration over time' mean? If we already are tracking a successful configuration , then why are there false positives.

2.In the next section, Fingertrip tracking has been explained. From what I understood , it has two parts

  • Using a matching algorithm
  • Using centroid to effectively handle large movenments.

My doubt is in the second part, I am not able to understand properly how does it handle large movenments and also the correctness proof of that statement.

Formula for fi+1 Formula for centroid

where fi fi+1 are the sets of N fingertips at the ith and i + 1th frame and
Ci , Ci+1 are centroids at ith and i + th frame.

Citation -

I got this from 1 paper published by Taehee Lee, Tobias Hollerer in 2007 11th IEEE International Symposium on Wearable Computers

  1. Handy AR
  1. What does 'successful configuration over time' mean?... why are there false positives.
  2. I am not able to understand properly how does it handle large movements

they are repeatedly scanning a similar form, the "hand blob", so any outliers from the usual successful input will hopefully be ignored. (maybe if someone nearby passes you something and it thinks their hand is yours for a few frames)

the paper says they register the "hand blob" or centroid of skin color as an easier tracking target for large movements. the "one time calibration" is probably about where your hand is, and it's predominant shade, in relation to the device. then it makes sure the fingers are there and orientates using the thumb. there is no difference to the formula for large movements except where Ci+1 actually is compared to Ci.

I really am looking forward to real time interaction with augmented reality. are there many more papers you know of like this? id love to read them too.

Edit: hello again raphael. this is someone of my lvl of understanding and the attempt has been made to fill all of the exceedingly strict criteria for a simple question. this guy is interested, like many are, but has been told off and deleted by you lot before. i can tell.

  • $\begingroup$ How is this an answer? $\endgroup$ – Raphael Nov 11 '16 at 1:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.