1
$\begingroup$

Examples like

  • swinging both of their hands
  • squat
  • walking around
  • jump around
  • etc

Is it possible to perform this in real time under crowded scene without depth information?

I can find out the "action level" if I knew what kind of action the person are doing, but can't find a fast solution to find out their action level if I don't know what kind of action they are doing.

Method I tried

First

Use adaptive background segmentation solutions like mog and mog2 of opencv, count the proportions of the foreground, but this solution cannot measure how fast the person is doing. There exist some frequencies patterns for the change of the foreground proportions, but it is hard to measure the wave of the frequencies reliable since the patterns may from A to Z.

Second

Use the multi-pose algorithm and tracking algorithm to compare the different positions of each pose. I measure out the pose of the person, and then compare the difference of the last keypoints, not good enough to measure how fast the people moving their body either.

pseudo codes

total_diff = abs(kpts_1 - kpts_2).sum()

Third(Just an idea, haven't tried)

Store n images(ex: 100), classify the action of the person by a cnn model, then measure their action level. This should work, but it can't work with crowded scenes.

Fourth(Just an idea, haven't tried)

Collect clips of different movement, give the activity level for different video clips manually, train a 3D/2D convolution models to classify the activity level. This will need a lot of videos, not a practical solution.

Do any research paper trying to solve this issue? What are the keywords I should try? Thanks

The keywords I tried:

Computer vision, measure human action speed

Improve this question
$\endgroup$
3
  • $\begingroup$ If you can perform people detection and tracking, you can estimate the apparent speed (in the direction perpendicular to the viewing axis) by considering the displacement between two frames, calibrated with the height of an average person. The speed in the viewing direction is a function of the change in height but also the focal length. $\endgroup$
    – user16034
    Nov 1, 2022 at 20:05
  • $\begingroup$ @Yves Daoust Thanks, this solution can work for actions like walking/jumping, but it don't work for motion like raising hands(the person stand at the same place) $\endgroup$
    – StereoMatching
    Nov 1, 2022 at 21:26
  • 1
    $\begingroup$ This question is too large. $\endgroup$
    – user16034
    Nov 2, 2022 at 7:44

1 Answer 1

Reset to default
0
$\begingroup$

Sure. Use any method for detecting the pose of the person (or keypoints for their hands, arms, etc.) in every frame. Then, it is trivial to measure the "speed" of movement, in pixels per second: this is the distance between the two keypoint locations in two frames, divided by the time between the two frames. This measures the speed in the plane orthogonal to the vector from the camera to the person (i.e., it ignores the component of the speed in the direction towards or away from the camera).

Converting pixels per second to a more reasonable unit, like meters per second, will require a bit more work. One reasonable approach is to assume a specific height for the person, based on the average height of people, and use that as your calibration. Another way is to estimate the distance to the person (using machine learning models for monocular depth estimation, or using a RGBD camera, or using a binocular camera), and then use the camera geometry to convert a pixel distance to distance in meters.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.