0
$\begingroup$

I have some sources of data, and each one of this sources have a cost per request. I know wich data attributes each one of these sources have, and what are the best sources for each attribute.

Given a list of attributes, I have to decide the best group of datasources to consume, thinking about Quality of the data first, and secondly about costs.

Its common to make requests to multiple datasources, getting just some attributes from each one, but i will give a simple example below:

Say I have something like this:

DATASOURCE A costs $0.50 returns NAME (good), BIRTHDAY (good), ADDRESS (good)

DATASOURCE B costs $1 and returns NAME (good), MONTHLY INCOME (good), ADDRESS (medium)

DATASOURCE C costs $0.70 returns MONTHLY INCOME (good), NAME (medium)

If i want NAME, it would be better to consume DATASOURCE A (quality and cost of NAME).

If i want MONTHLY INCOME, it would be better to consume DATASOURCE C. (quality and cost of MONTHLY INCOME)

If i want NAME and MONTHLY INCOME, now it would be better to consume DATASOURCE B (because NAME and MONTHLY INCOME are GOOD).

But if i want NAME, MONTHLY INCOME and ADDRESS, I need to consume DATASOURCE A (for NAME and ADDRESS) but will consume DATASOURCE B too for MONTHLY INCOME.

Im trying to make some algorithm that makes sense to do it, but its getting very messy hehe. I would love any help with the logic. Maybe in javascript, java, c# or anything else I could read and have good idea about the best way to solve this.

Any help will be apreciated.

edit

some things to note:

I will have only 3 types of Quality here, and always prefer the best quality, so If i know any datasource has quality GOOD for the attribute, i can filter all other datasources with quality lower than Good for this attribute.

I know that in my case, we will have something like 5 or 10 datasources at all, and only 2 or 3 will have the same attribute.

Improve this question
$\endgroup$
  • 1
    $\begingroup$ If you want NAME, MONTHLY INCOME and ADDRESS, why isn't the best answer to use DATASOURCE A and DATASOURCE C? That costs only \$0.50+0.70, instead of \$0.50+1.00 as in your proposed solution, and provides GOOD quality for all three attributes. What am I missing? $\endgroup$ – D.W.♦ May 16 '18 at 6:43
  • $\begingroup$ oh my god, you are right haha D.W. will edit it as soon as I get to my PC $\endgroup$ – Jean Robert May 16 '18 at 10:51
  • 3
    $\begingroup$ This problem is called weighted minimum set cover. It's NP-hard, so expect any exact algorithm to take exponential time on at least some inputs. $\endgroup$ – j_random_hacker May 16 '18 at 10:58
  • $\begingroup$ some things to note: I will have only 3 types of Quality here, and always prefer the best quality, so If i know any datasource has quality GOOD for the attribute, i can filter all other datasources with quality lower than Good for this attribute. I know that in my case, we will have something like 5 or 10 datasources at all, and only 2 or 3 Will return the same attribute $\endgroup$ – Jean Robert May 16 '18 at 11:29
  • $\begingroup$ i found this java code and it seems to solve my problem: ideone.com/hUY3Am $\endgroup$ – Jean Robert May 16 '18 at 19:34
0
$\begingroup$

What you're asking reminds me of some of the work I did a Linear Programming course and made me think of the Hungarian Algorithm. I hope this helps!

Improve this answer
$\endgroup$
  • $\begingroup$ thanks. Ive looked about Hungarian algorithm, but it seems to solve the problem only if i could take only one Attribute from each datasource. In my case, the more atributtes i can get from the same datasource better $\endgroup$ – Jean Robert May 16 '18 at 12:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.