IoU, precision–recall and mAP for object detection

Classification has a simple scorecard: was the label right? Object detection is harder, because a model must say what is in an image, where it is, and how confident it is, often for several objects at once. The metrics that summarise this (IoU, precision-recall, average precision and mAP) look intimidating but build on each other in a clear sequence.

Step 1: is a box correct? Intersection over Union

A detection is a bounding box plus a class and a confidence score. To decide whether a predicted box matches a real one (the ground truth), we measure how much they overlap.

IoU = area of overlap / area of union.

• IoU of 1 means a perfect match; 0 means no overlap.
• A threshold turns it into a yes/no: a common choice is IoU of at least 0.5, meaning "overlaps by half or more of the combined area".

def iou(a, b):
    # boxes as (x1, y1, x2, y2)
    x1, y1 = max(a[0], b[0]), max(a[1], b[1])
    x2, y2 = min(a[2], b[2]), min(a[3], b[3])
    inter = max(0, x2 - x1) * max(0, y2 - y1)
    area = lambda r: (r[2] - r[0]) * (r[3] - r[1])
    union = area(a) + area(b) - inter
    return inter / union if union > 0 else 0.0

Step 2: classify each detection

For one class, sort all predictions by confidence, highest first. Then each prediction is:

• a true positive if it matches an unmatched ground-truth box of the same class with IoU above the threshold (each real object can be matched only once),
• otherwise a false positive (a wrong box, a duplicate, or a poor localisation).

Real objects that no prediction matched are false negatives (misses).

Step 3: precision and recall as you walk down the list

Go down the ranked list one detection at a time and keep running counts:

• Precision: of the detections so far, what share were true positives?
• Recall: of all real objects, what share have been found so far?

Early in the list, precision is usually high (confident detections are mostly right) and recall is low. As you continue, recall rises and precision typically falls. Plotting precision against recall gives the precision-recall curve for that class.

Step 4: average precision (AP)

AP summarises the curve in one number: roughly the area under the precision-recall curve. A detector that keeps high precision while recall grows has a large area. Different benchmarks compute the area slightly differently (for example interpolating at fixed recall points), so quote the protocol you used.

Step 5: mean average precision (mAP)

mAP is the average of AP across classes. Two details matter when reading a reported mAP:

• The IoU threshold. "mAP at 0.5" is more forgiving of sloppy boxes than a metric averaged over several thresholds (for example 0.5 to 0.95 in steps), which rewards precise localisation.
• Averaging. Are classes weighted equally, and were small, medium and large objects reported separately?

Without those details, two mAP numbers are not comparable.

What the numbers do not tell you

• Per-class behaviour. A healthy mAP can hide a class the model never gets right. Look at AP per class.
• Size effects. Small objects are harder; report by size if it matters to your use.
• Threshold choice. In deployment you must pick a confidence threshold. mAP integrates over all thresholds, so it does not tell you how the model behaves at the one you will use. Look at precision and recall at that threshold.
• Error types. Was a false positive a duplicate, a mislocalised box, a wrong class, or background? The fix differs for each.

Practical habits

1. Visualise predictions on real images, with ground truth, before trusting numbers.
2. Use non-maximum suppression to remove duplicate boxes, and know its settings.
3. Evaluate on data that resembles deployment: lighting, camera angle, image quality.
4. Check the annotation quality. Inconsistent or wrong labels cap what any metric can show.
5. Report the protocol: IoU thresholds, image size, confidence cut-off, dataset split.

A checklist

• Which IoU threshold(s) are used, and why?
• Is AP computed per class, and are weak classes visible?
• Are results reported at the operating threshold used in production?
• Are error types analysed, not only totals?
• Does the test set resemble the real deployment?