1
$\begingroup$

I'm trying to think of a way to do approximate joins between 2 rows in a data table. Like for example lets say I have a row where the person name is "John Doe" but then another data table uses a different person format like "Doe, John." Now suppose I can put them both to a standard format say "John Doe." Sweet so I have a way to reference them and let's say there's other data correlated with "John Doe" is there any algorithms out there to infer whether these 2 John Does are the same given the other data columns? You can see the other data columns as features as well, because I know FB has an algorithm to stop duplicates for the same person and StackOverflow will merge accounts if it thinks they're being used by the same person. Any suggestions?

$\endgroup$
1
1
$\begingroup$

I am not able to clearly understand your use case, but from what I infer I believe if you are trying to de-dupe data on other features, the approach would be to use LSH(Locality Sensitive Hashing) you can see the following links to get an overview. hope this helps.

https://stackoverflow.com/questions/12952729/how-to-understand-locality-sensitive-hashing

https://www.cs.utah.edu/~jeffp/teaching/cs5955/L6-LSH.pdf

https://en.wikipedia.org/wiki/Locality-sensitive_hashing

$\endgroup$
1
  • $\begingroup$ Sort of, this is useful, but I was thinking more like if there's 2 datasets of names, being able to infer if the same person is in both. $\endgroup$ – user8714896 Feb 4 '20 at 3:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.