# Dimension Reduction - Which feature should remove to reduce the dimension of the matrix

Let's suppose that we have the following 2 tables: If we want to reduce the dimension by one(in every table) which feature we should remove and why ?

I am confused about the way that i should work generally in this type of problems.

• For the left table i thought to check the combinations per 2 features and after i do this i understand that i should remove the x2 , because if i remove the x1 then (0,0) gives the value y = 0 or (y = 1) that means that we need another one feature to understand the value of y -- The same and with x3.It seems like underfeeting .For the right table the x1 characteristic i thought that have some noise in the data and for this reason the values of x1 is so far from y. So i choose to remove x1. Could someone answer to me if my work in this -particular- problem was correct and if not why ? – Emily Serone Jun 16 '19 at 21:26

## 1 Answer

In the first table, X2 is an unnecessary feature. You can check that for any row, the value of Y that you obtain after reversing the value of X2 is always unaffected. Instead, X1 and X3 are both individually necessary; this can be seen for example by comparing the first row with the fifth row (for X1) and with the second row (for X3).

In the second table, X1 is an unnecessary feature. The value of Y is solely determined by whether X2 is large or small; say, larger or smaller than 0.5. Hence a linear classifier based on X2 can predict Y with full accuracy. Instead, no linear classifier based on X1 only can correctly predict Y; for example, the value of Y is 1 for X1=0.6931, becomes 0 for X1=0.7465, and becomes 1 again for X1=2.7361.

For the first type of problem, you can look at each feature individually and check whether the results indeed depend on it or not, by comparing corresponding rows as I did in the example.

For the second type of problem, you can look for non-monotonicity (as I did in the example with the X1 feature), in which case you can exclude the non-monotone feature, at least if you are looking for a linear model.