I am doing a prediction assignment as part of a machine learning course using loans data. I have just done some exploratory data analysis on my dataset of just over 9000 rows. There are 11 variables which I need to use to predict the output. Whilst exploring my correlation matrix, I have discovered that 5 pairs are strongly correlated with each other (corr_coeff > 0.5) while 24 pairs have a correlation of less than 0.1.
Can anyone tell me what implications this might have on my further analysis?
I'm thinking of using PCA to lessen the collinearity problem and get the features. Do I run the PCA only on the variables which are strongly correlated or on all of them? Then use something like decision tree/GradientBoostingClassifier - the dataset size maybe too small for a neural network to learn. would that be a recommended approach?
Or is it better to do something else with the strongly correlated variables (corr = 0.54..0.71) before proceeding with any of the classifiers? thank you. machine-learning