I have the following messy chart;

As you can see some features are very stable through the time series (post-2011) like Extraversion. Is there an algorithm to remove features that do not have much variation compared to the other features?

  • $\begingroup$ Is there are reason that "compute the sample range of each time series and drop the ones with max-minus-min below some threshold" is not acceptable? $\endgroup$ Aug 7 '20 at 17:11
  • $\begingroup$ Thank you. Is there any way to calculate this threshold? It seems arbitrary. $\endgroup$
    – Sy Ker
    Aug 7 '20 at 19:44
  • $\begingroup$ If you just have the one dataset, I would recommend just producing several charts with different threshold values and using the chart that makes you happiest. $\endgroup$ Aug 7 '20 at 22:58
  • 1
    $\begingroup$ An approach from a different angle may be simpler: What are the five features featuring most variance? $\endgroup$
    – greybeard
    Aug 9 '20 at 1:52

Step 1. Identify a measure of variation for a feature.

Step 2. Remove all features for which this measure is below some threshold (or is below that of msot other features).

There are many measures of variability that you could use in step 1: standard deviation, interquartile range, range (max minus min), median absolute deviation, and more. Any of these are probably reasonable.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.