I have logs of activities without labels, which describe whether an activity is normal or not. Assuming that normal behaviors will follow a Gaussian distribution, I fit Gaussian distributions on dataset. I utilized this to generate a synthetic dataset with abnormal patterns. Then, according to the literature review, I generate labels by confidence intervals. After generating labels, I separated train/test dataset. I utilized pdf of gaussian distribution as a feature and learned a simple decision trees.

Depending on dataset, sometimes, this approaches produce 99.5% precision and recall, almost perfect classification.

I felt like that the approach of generating synthetic dataset, labelling, and feature extraction is related in some sense. I am not sure how much I can rely on this trained classifier. If I did anything incorrect, could anyone suggest how to build more credibility on classifiers?

  • $\begingroup$ What do you want to achieve? Are you trying to do anomaly detection? $\endgroup$ – Pål GD Aug 1 '17 at 16:14
  • $\begingroup$ Yes, I want to achieve anomaly detection. I was thinking about applying unsupervised algorithms and measuring the performance after confidence intervals. However, the performance was not good. Thus, I tried to use generated labels and apply supervised algorithms, which leaded to better results. The main issue is that it seems there is a strong relationship between my label generation (confidence interval) and feature extraction (pdf of distribution). If I apply a decision tree, it seems it will generate a pretty similar rules like confidence intervals. I was wondering if it is valid approach $\endgroup$ – pippp Aug 3 '17 at 20:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.