# Segmenting a human body point cloud into limbs

I have a human body point cloud taken through a Kinect (v1). Now, I want to segment the point cloud into different parts, namely hands, lower arms, upper arms, torso, upper legs, lower legs, feet and head.

Here's the method I'm currently using:

• I get the approximate skeleton from the point cloud using OpenNI
• I take those points from the point cloud that are at a distance d from the line that is the bone (obtained from OpenNI). This distance d is currently set differently for different parts, based on observations only.

The problem with the above method is that it's not very robust. The parameter d can be different for people with different body types and also this is prone to errors since at the joints, the points within the circle of radius d will lie in both the parts.

Is there a better way of segmenting the point cloud?

Edit: This is what I get from OpenNI

The blue portion is the lower arms segmented using the above algorithm.

• The green lines inside the point cloud, is that from OpenNI? Can't you use joints to discriminate body parts? – padawan Jan 15 '16 at 11:21
• @cagirici yes, those green lines are bones of the skeleton from OpenNI. OpenNI has this getLimb() method that gives me one particular bone corresponding to a limb – Ranveer Jan 15 '16 at 11:23
• The white part is the point cloud and I need the part of the point cloud that corresponds to a particular limb. – Ranveer Jan 15 '16 at 11:24
• What distance measures of points vs lines have you tried? There are several. – Raphael Jan 15 '16 at 15:54
• Just the perpendicular point-line distance. – Ranveer Jan 15 '16 at 15:55

## 2 Answers

Suggestion: for each point $P$ in the point cloud, find which bone it is nearest to, and associate it with that bone. In other words, find which point $Q$ on the skeleton is closest to $P$, and associate $P$ with $Q$. Now associate $Q$ with a particular part of the skeleton (e.g., arm, leg, etc.); that will let you associate $P$ with a particular part of body (e.g., arm, leg, etc.). Do this for each point in the point cloud.

Try that -- it's a very simple approach, and it might just work.

• This is way easier and more accurate: imgur.com/fldeugJ Thanks a ton :) – Ranveer Jan 17 '16 at 17:36

I would try the following:

for each joint $j$:

1. $p_1 \gets$ closest extreme point to $j$ in direction $\vec{d}$
2. $p_2 \gets$ closest extreme point to $j$ in direction $-\vec{d}$
3. Draw vector $\vec{p_1p_2}$
4. Let $\vec{N}, -\vec{N}$ be two normal vectors of $\vec{p_1p_2}$
5. Draw two rays $a_1$, $a_2$ in the direction of $\vec{N}$ and $-\vec{N}$ respectively.
6. Let $i_1$ and $i_2$ be the points belong to another area (such as gray area or another body part) which are closest to the origin of $a_1$ and $a_2$ respectively.
7. $p_3 \gets$ closest extreme point to $i_1$ in direction $\vec{f}$
8. $p_4 \gets$ closest extreme point to $i_1$ in direction $-\vec{f}$
9. $p_5 \gets$ closest extreme point to $i_2$ in direction $-\vec{g}$
10. $p_6 \gets$ closest extreme point to $i_2$ in direction $-\vec{g}$
11. Mark the area $(p_1, p_2, p_3, p_4, p_5, p_6)$ as body part.

I am not familiar with KINECT nor OPENNI, but given the green circles, representing joints, I would use above algorithm.

• Can you illustrate via a figure? Also, what's $d$, $f$ and $g$ here? – Ranveer Jan 15 '16 at 13:26
• They are the vectors perpendicular to the green line connecting two green dots. – padawan Jan 15 '16 at 13:27
• Oh, so you're basically trying to create a bounding box between the two joints and getting the points which are in it, if I'm not wrong? – Ranveer Jan 15 '16 at 13:39
• @ranveer indeed, that is my solution – padawan Jan 15 '16 at 15:00
• That seems way better than what I was using. Plus, if I can find the extreme points, that can make the bounding box size adjust for changes in body type. I'm going to keep the question open for a while, for other answers, though I doubt I'll find something as simple and accurate as this :) – Ranveer Jan 15 '16 at 15:13