Random Forest closed loop

Question

I'm using the Random Forest algorithm for classification. I have some variable to use as features in input, but I was wondering if I can use the output of the classification itself as input.

Suppose, for example, that I want to classify people in two categories : "criminal" or "honest". In order to do that, I have a vector of three features: the name of the city where the person is born, the number of people that committed crimes in the city where the person is born, and the number of people that live in that city. When I run the algorithm, I will get the classification results $y_1,y_2,...,y_n$ for each input vector $x_1,x_2,...,x_n$.

Now, it would be wrong to add to the input vector a new feature "number of people from the same city who have been classified by the algorithm as criminal"?

My purpose is to run several instances of the algorithm on different datasets and then share the informations gathered in each subset between the different instances. Does it can make sense?

Answer 1

This is a cascade classifier. Of course you can do that, but the main disadvantage (that you have to keep in mind) is that your error propagates along your cascade classifiers. So even if you have a very good classifier somewhere towards the end of your cascade, if you feed it with weak features (i.e., the output of the previous layer which was not so good) you should not expect to get a reasonable answer. There is generally another common problem with this method and that's the well-known overfitting effect. However, since you are using different subsets of data to train each classifier it shouldn't be a problem in your case I guess.