I have a human body point cloud taken through a Kinect (v1). Now, I want to segment the point cloud into different parts, namely hands, lower arms, upper arms, torso, upper legs, lower legs, feet and head.

Here's the method I'm currently using:

  • I get the approximate skeleton from the point cloud using OpenNI
  • I take those points from the point cloud that are at a distance d from the line that is the bone (obtained from OpenNI). This distance d is currently set differently for different parts, based on observations only.

The problem with the above method is that it's not very robust. The parameter d can be different for people with different body types and also this is prone to errors since at the joints, the points within the circle of radius d will lie in both the parts.

Is there a better way of segmenting the point cloud?

Edit: This is what I get from OpenNI

enter image description here

The blue portion is the lower arms segmented using the above algorithm.

  • $\begingroup$ The green lines inside the point cloud, is that from OpenNI? Can't you use joints to discriminate body parts? $\endgroup$
    – padawan
    Jan 15, 2016 at 11:21
  • $\begingroup$ @cagirici yes, those green lines are bones of the skeleton from OpenNI. OpenNI has this getLimb() method that gives me one particular bone corresponding to a limb $\endgroup$
    – Ranveer
    Jan 15, 2016 at 11:23
  • $\begingroup$ The white part is the point cloud and I need the part of the point cloud that corresponds to a particular limb. $\endgroup$
    – Ranveer
    Jan 15, 2016 at 11:24
  • $\begingroup$ What distance measures of points vs lines have you tried? There are several. $\endgroup$
    – Raphael
    Jan 15, 2016 at 15:54
  • $\begingroup$ Just the perpendicular point-line distance. $\endgroup$
    – Ranveer
    Jan 15, 2016 at 15:55

2 Answers 2


Suggestion: for each point $P$ in the point cloud, find which bone it is nearest to, and associate it with that bone. In other words, find which point $Q$ on the skeleton is closest to $P$, and associate $P$ with $Q$. Now associate $Q$ with a particular part of the skeleton (e.g., arm, leg, etc.); that will let you associate $P$ with a particular part of body (e.g., arm, leg, etc.). Do this for each point in the point cloud.

Try that -- it's a very simple approach, and it might just work.

  • $\begingroup$ This is way easier and more accurate: imgur.com/fldeugJ Thanks a ton :) $\endgroup$
    – Ranveer
    Jan 17, 2016 at 17:36

I would try the following:

for each joint $j$:

  1. $p_1 \gets$ closest extreme point to $j$ in direction $\vec{d}$
  2. $p_2 \gets$ closest extreme point to $j$ in direction $-\vec{d}$
  3. Draw vector $\vec{p_1p_2}$
  4. Let $\vec{N}, -\vec{N}$ be two normal vectors of $\vec{p_1p_2}$
  5. Draw two rays $a_1$, $a_2$ in the direction of $\vec{N}$ and $-\vec{N}$ respectively.
  6. Let $i_1$ and $i_2$ be the points belong to another area (such as gray area or another body part) which are closest to the origin of $a_1$ and $a_2$ respectively.
  7. $p_3 \gets$ closest extreme point to $i_1$ in direction $\vec{f}$
  8. $p_4 \gets$ closest extreme point to $i_1$ in direction $-\vec{f}$
  9. $p_5 \gets$ closest extreme point to $i_2$ in direction $-\vec{g}$
  10. $p_6 \gets$ closest extreme point to $i_2$ in direction $-\vec{g}$
  11. Mark the area $(p_1, p_2, p_3, p_4, p_5, p_6)$ as body part.

I am not familiar with KINECT nor OPENNI, but given the green circles, representing joints, I would use above algorithm.

  • $\begingroup$ Can you illustrate via a figure? Also, what's $d$, $f$ and $g$ here? $\endgroup$
    – Ranveer
    Jan 15, 2016 at 13:26
  • $\begingroup$ They are the vectors perpendicular to the green line connecting two green dots. $\endgroup$
    – padawan
    Jan 15, 2016 at 13:27
  • $\begingroup$ Oh, so you're basically trying to create a bounding box between the two joints and getting the points which are in it, if I'm not wrong? $\endgroup$
    – Ranveer
    Jan 15, 2016 at 13:39
  • $\begingroup$ @ranveer indeed, that is my solution $\endgroup$
    – padawan
    Jan 15, 2016 at 15:00
  • $\begingroup$ That seems way better than what I was using. Plus, if I can find the extreme points, that can make the bounding box size adjust for changes in body type. I'm going to keep the question open for a while, for other answers, though I doubt I'll find something as simple and accurate as this :) $\endgroup$
    – Ranveer
    Jan 15, 2016 at 15:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.