Skip to Content

Precision-at-K (Top-K Retrieval)

When to use

Score a RAG pipeline, recommender system or search quality — only the top-K results matter to the user.

Analogy

Job interviewer reading the first 5 resumes. Precision-at-5 is the share of those 5 that are actually qualified candidates.

Data-flow diagram

   Top-5 results
   [relevant, relevant, irrelevant, relevant, irrelevant]
   precision_at_5 = 3 / 5 = 0.60

Deep explanation

For ranked outputs only the top-K results matter; precision-at-K is the cleanest signal. Recall-at-K is the symmetric question of how many truly relevant items were returned. For RAG K is usually 3 to 10; for recommenders K is 20 to 100. Always log K explicitly in your report.

Examples

Example 1

def precision_at_k(relevant, retrieved, k):
    return sum(x in relevant for x in retrieved[:k]) / k
rel = {'a','c','e'}
pred = ['a','b','c','d','e']
print('P@3:', precision_at_k(rel, pred, 3))
print('P@5:', precision_at_k(rel, pred, 5))

Pass relevant as a set of true IDs and retrieved as a ranked list of IDs.

Example 2

def recall_at_k(relevant, retrieved, k):
    return sum(x in relevant for x in retrieved[:k]) / max(len(relevant), 1)
print('R@3:', recall_at_k(rel, pred, 3))

Recall-at-K answers: are we missing relevant items — vital for completeness audits.

Example 3

def apk(actual, predicted, k=5):
    score, hits = 0.0, 0
    for i, p in enumerate(predicted[:k]):
        if p in actual:
            hits += 1
            score += hits / (i + 1)
    return score / max(len(actual), 1)
print('MAP@5:', apk({'a','c'}, ['a','b','c','d','e'], 5))

MAP@K averages per-user precision weighted by rank — the canonical ranking metric.

Common mistake

Computing Precision-at-K from a list that includes duplicates — the duplicate is counted as relevant multiple times and inflates the numerator. Deduplicate first.

Key takeaway

P@K and R@K measure how the top of a ranked list looks. Always fix K explicitly and use MAP@K for user averages.

Production Failure Playbook

Failure scenario 1: relevance-set-empty

Failure scenario 2: duplicate-counter-bias