How to generate response variable during machine learning?

Question

I am analyzing data of insurance companies and for some reasons the data that was provided doesn't have any response variable of whether an insurance claim is legit or suspicious.

Are there any ways to generate the required response variable from the data that is available to us. I can think of clustering (hierarchical or K-means) as a means to identify clusters. But I am not sure if the clusters developed are true representation of my unreported response variable.

Also, Let me know of any other methods to generate response variable when there is none.

Answer 1

No. You can't. You need ground truth. You're asking "if I don't know which claims are fraudulent, can an algorithm somehow determine that for me?" The answer of course is no: the algorithm doesn't know anything more than you do about insurance fraud -- if anything, it knows even less.

This is one of the challenges with using machine learning. You typically need a lot of labelled data: data that has been labelled with ground truth (e.g., whether the insurance claim is legit or fraudulent). Supervised learning is data-driven: given some instances where we know the correct answer, it tries to figure out some rules to use to classify other instances where we don't know the answer. For this to work, it needs a training set of instances where we do know the correct answer.

Now there are some techniques that can be helpful. You might be able to use active learning to try to figure out claims to manually label, to try to reduce the number of examples you need to label. You might be able to use clustering to cluster the instances, and then hand-label a few instances from each cluster. But ultimately, you'll need some labelled instances to train on and evaluate the accuracy of your method.

See also How to identify labels in unsupervised learning? and Why is automatically labeling data in unsupervised learning hard? and What exactly is the difference between supervised and unsupervised learning? and What would be a decent threshold for classification problem?

Answer 2

Let me know of any other methods to generate response variable when there is none

Anomaly detection techniques could be what you're looking for (but you don't "generate response variable when there is none").

If you can assume that for the available dataset ($m$ examples, $n$ features):

$$x^{(1)}, \dots x^{(m)} \in \mathbb{R}^{n}$$

the features are gaussian:

$$x_i \sim N(\mu_i, \sigma_i^2)$$

you can compute a Density Estimation ($p(x)$):

$$p(x) = \prod_{j=1}^n p(x_j; \mu_j, \sigma_j^2)$$

where

$$p(x_j; \mu_j, \sigma_j^2) = \frac{1}{\sqrt{2 \pi} \sigma_j} e^{-\frac{(x - \sigma)^2}{2 \sigma^2}}$$

(this is the formula for independent features but, in practice, it works quite well even if the features are not independent)

and classify a claim as being fraudulent when $p(x) < \epsilon$.

The problem is that you need a good $\epsilon$ (threshold value).

You can split your data in:

• a training set. Ideally this shouldn't contain anomalous (fraudulent) data points, but if a few of them are present it isn't a big problem.
• a cross validation set. This should contain some known anomalous elements.

As you can see this is not unsupervised anomaly detection: you assume that the available data are legit and you add some examples of anomalous data for the cross validation set (of course you can also examine carefully the data and mark just a few examples).

The key points are:

1. for a random sample of insurance records a good share is probably legit (this is the reason that anomaly detection is frequently used for fraud detection);
2. you just need a very small number of examples which are known to be fraudulent (to be placed into the cross training set).

The training set is used to compute $\mu_i$, $\sigma_i$ and thus the density estimation. The "optimal" value of $\epsilon$ is the value that maximizes some evaluation metrics on the cross validation set. A simple evaluation metric is F1-score (data is very skewed and accuracy is not good).

This method can be adapted to non-gaussian features and features that have have some correlation with each other.

For a good introduction and further details take a look at week 7 of Andrew Ng's Machine Learning Class (and to Alex's notes).

---

There are also completely unsupervised anomaly detection techniques (they make the implicit assumption that normal instances are far more frequent than anomalies): k-NN Global Anomaly Score, Local outlier factor... (see Anomaly Detection : A Survey by Varun Chandola, Arindam Banerjee, Vipin Kumar).

Anyway for your kind of problem semi-supervised anomaly detection seems more promising.