As far as I understand from PCY (Park, Chen, and Yu) algo is that the algo uses hashing during the first pass to reduce the number of CANDIDATE pairs that are considered in the second pass. I have 2 concerns.

1/ How can I chose a good hash function? since I noticed that if we chose a bad func, it turns out to make many "false noisy" bucket which is the home of many failed (pair) candidates.

For example: (3,5) and (6,10) are not frequent itemsets but what if h (3,5) = h (6,10) and make that bucket reach the support threshold = 2. (given {3} , {5}, {6}, {10} are all singleton itemsets already)

2/ In the filtering phase of the algo, I guess each candidate also has to loop over all the baskets to verify if it satisfy the threshold right? so the filtering part also thrashing disk as h*ll right?

Thank you all.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.