2
$\begingroup$

I'm implementing the ID3 algorithm (Iterative Dichotomiser 3). I have an attribute which happens to be continuous like 12.21, 3.01, etc. AND have missing values which are marked as "NA".

How I'm discretizing the data: I'm finding the optimal split which results in the max information gain. How I'm dealing with missing values: I will use the most probable attribute value to replace the "?".

Of course I can do either process in both ways, and this is where my confusion arises. Is there a correct way in handling this?

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ @pierop It would be a good idea to edit your question to include the expansion of the acronym. In general, you should avoid assuming that everybody knows the same set of acronyms as you. Some, such as "RAM" are near-universal among people who use computers, but anything domain-specific is likely to be unknown by many people and lead to confusion. $\endgroup$
    – David Richerby
    Commented Apr 4, 2016 at 15:30
  • $\begingroup$ Sure thing! that was a mistake on my part. $\endgroup$
    – Edqu3
    Commented Apr 4, 2016 at 15:39
  • $\begingroup$ @EvilJS I don't know about interpolation but can the Mean be used? $\endgroup$
    – Edqu3
    Commented Apr 4, 2016 at 15:40
  • $\begingroup$ In this case an even better solution would have been to add a link to a description of the algorithm. I added one for you -- just so you know for the future. $\endgroup$
    – D.W. ♦
    Commented Apr 4, 2016 at 20:52
  • 1
    $\begingroup$ Indeed! I was thinking of music files; my cursor already hovered over "close"! $\endgroup$
    – Raphael
    Commented Apr 5, 2016 at 16:59

2 Answers 2

Reset to default
0
$\begingroup$

I would like to propose paper about ID3 and it's successors, generally about Decision Tree Algorithms.

Using Mean, Median, Mode etc. is very tempting and it works to some degree, but of course the outcome depends on values inserted to missing (NA) data.

Mean has nice property in many statistics that it just acts like missing value, but increases weight of other ones (since it changes nothing, other values are counted with +1/N weight).
But in decision trees the effect is bigger, changing the classifier, so there is one big idea - apply all possible missing values :-/.

There are also three easier techniques:

  • apply mean and do not care
  • reconstruct data to fit classifier better (very often trial and error, but due to discretization of continous data, only values that differ by multiplicity of $\epsilon$ are to be checked)
  • try to reconstruct data

The last one should yield the best results, but it is not always possible, and still these are not exact values.

If you can predict the most probable value and replace missing ones - this is the best way to do it.

Improve this answer
$\endgroup$
1
$\begingroup$

Take a look at the C4.5 algorithm. C4.5 is a successor to ID3. It is more complex. However, Wikipedia lists as one of its advantages over ID3 that C4.5 can handle data with missing attribute values. So, you might take a look at how C4.5 handles missing attribute values -- or even simply use C4.5 rather than ID3.

Improve this answer
$\endgroup$
1
  • $\begingroup$ The C4.5 and C5.0 (faster successor of C4.5) are described in paper I provided. And the case of continous / missing values handled by C4.5 are exactly the same how OP handles it, with one difference, if possible values are known or can be approximated giving more information, this is preferable way over ommiting them. $\endgroup$
    – Evil
    Commented Apr 5, 2016 at 23:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.