12
$\begingroup$

I am trying to understand the difference between these two but it looks like as if they are calculated the same way. What is the difference?

R precision:

R precision is the precision at the Rth position in the ranking of results for a query that has R relevant documents.

Precision at K:

Precision at K is calculate for only K documents. Documents ranked lower than K are ignored.

$\endgroup$
  • $\begingroup$ It could be the same thing. People don't always use the same letter for a given parameter, or the exact same grammar for an expression. $\endgroup$ – Yuval Filmus Dec 21 '16 at 14:29
  • 1
    $\begingroup$ What is the context here? Where did you find these two definitions? $\endgroup$ – Raphael May 1 '17 at 0:16
10
$\begingroup$

R-Precision

R-Precision is defined as $\frac{r}{R}$, which is the ratio between all the relevant documents retrieved until the rank that equals the number of relevant documents you have in your collection in total ($r$), to the total number of relevant documents in your collection $R$.

Suppose in your collection there are 100 documents in total, 30 of which are relevant ($R = 30$), the rest irrelevant. So you retrieve the first 30 documents (because 30 are relevant in total in your collection) and, say, 10 of them are relevant ($r = 10$). Your R-Precision is then $\frac{10}{30} = \frac{1}{3}$.

Precision@K

Precision@K is defined as the precision at rank $k$, which is more intuitive if you know about the definition of precision itself. Suppose your system retrieves 10 documents, where the relevant ones are at ranks ${1,3,6,7,9}$ and you want to know how 'well' your system does until rank 5 ('well' depends on the objective of your system of course but here we'll say we want high precision at low ranks). So you compute your Precision@5, which is the ratio of the number of relevant documents until (including) rank 5 to the total number of documents until $k$ (this number is $k$ itself). Your Precision@K where $K = 5$ is therefore $\frac{2}{5}$.

Youc an see that R-Precision is a measure that allows for assessment of your entire system whereas Precision@K really only takes the documents until $K$ into account. Depending on what you want to measure, either of them may be suitable.

$\endgroup$
2
$\begingroup$

In addition to what @simonst told, I'm gonna add these:

Disadvantage of Precision@K:

Precision at K has this disadvantage that the total number of relevant documents in the collection has a strong influence on this metric.
For example, a perfect system, could only achieve a precision@20 of 0.4, if there were only 8 documents relevant to an information need.


R-Precision solves that issue:

Actually R-Precision is the same as Precision@X where X is the total number of relevant documents in the collection. A perfect system could score 1 for each query, if R-Precision is used.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.