Segmenting a human body point cloud into limbs

Question

I have a human body point cloud taken through a Kinect (v1). Now, I want to segment the point cloud into different parts, namely hands, lower arms, upper arms, torso, upper legs, lower legs, feet and head.

Here's the method I'm currently using:

• I get the approximate skeleton from the point cloud using OpenNI
• I take those points from the point cloud that are at a distance d from the line that is the bone (obtained from OpenNI). This distance d is currently set differently for different parts, based on observations only.

The problem with the above method is that it's not very robust. The parameter d can be different for people with different body types and also this is prone to errors since at the joints, the points within the circle of radius d will lie in both the parts.

Is there a better way of segmenting the point cloud?

---

Edit: This is what I get from OpenNI

The blue portion is the lower arms segmented using the above algorithm.

Answer 1

Suggestion: for each point $P$ in the point cloud, find which bone it is nearest to, and associate it with that bone. In other words, find which point $Q$ on the skeleton is closest to $P$, and associate $P$ with $Q$. Now associate $Q$ with a particular part of the skeleton (e.g., arm, leg, etc.); that will let you associate $P$ with a particular part of body (e.g., arm, leg, etc.). Do this for each point in the point cloud.

Try that -- it's a very simple approach, and it might just work.

Answer 2

I would try the following:

for each joint $j$:

1. $p_1 \gets$ closest extreme point to $j$ in direction $\vec{d}$
2. $p_2 \gets$ closest extreme point to $j$ in direction $-\vec{d}$
3. Draw vector $\vec{p_1p_2}$
4. Let $\vec{N}, -\vec{N}$ be two normal vectors of $\vec{p_1p_2}$
5. Draw two rays $a_1$, $a_2$ in the direction of $\vec{N}$ and $-\vec{N}$ respectively.
6. Let $i_1$ and $i_2$ be the points belong to another area (such as gray area or another body part) which are closest to the origin of $a_1$ and $a_2$ respectively.
7. $p_3 \gets$ closest extreme point to $i_1$ in direction $\vec{f}$
8. $p_4 \gets$ closest extreme point to $i_1$ in direction $-\vec{f}$
9. $p_5 \gets$ closest extreme point to $i_2$ in direction $-\vec{g}$
10. $p_6 \gets$ closest extreme point to $i_2$ in direction $-\vec{g}$
11. Mark the area $(p_1, p_2, p_3, p_4, p_5, p_6)$ as body part.

I am not familiar with KINECT nor OPENNI, but given the green circles, representing joints, I would use above algorithm.