Why a model's confidence is often wrong: calibration and ECE explained

When a classifier says "92 percent sure", it is making a promise. Calibration asks a simple question: is that promise kept? A model can be accurate and still make promises it cannot keep, and that matters most where people act on the number.

What calibration means

Take every case where the model reports about 90 percent confidence. If the model is well calibrated, about 90 percent of those cases are correct. If only 70 percent are, the model is overconfident. If 98 percent are, it is underconfident.

Here is an illustrative example, with made up numbers. A model gives 0.9 confidence on 1,000 cases and is right on 700 of them. Its accuracy in that group is 70 percent, but it claimed 90 percent. A reader who trusts the number will overestimate how often the model is right.

Calibration is a property of the confidence scores, not of the predictions themselves. Two models can make exactly the same predictions and have very different calibration.

Why it matters in medical imaging

In medicine, a probability is read as risk. A score of 0.95 on a mammogram invites a different reaction from a score of 0.6. If the scores are not calibrated, the numbers look precise but mislead.

Calibration also matters for everything built on top of confidence. Triage rules, thresholds for referral, and systems that defer uncertain cases to a human all assume that a higher score means a more reliable answer. For the deferral idea, see selective prediction in medical AI.

See it: the reliability diagram

A reliability diagram groups predictions by confidence and plots, for each group, how often the model was actually right. The horizontal axis is the confidence, the vertical axis is the observed accuracy. A perfectly calibrated model follows the diagonal. Points below the diagonal mean the model is overconfident in that range. Points above mean it is underconfident.

The diagram is more informative than any single number, because it shows where the problem is. A model can be well calibrated at low confidence and badly overconfident at high confidence, which is the range where people tend to trust it.

Measure it: expected calibration error

Expected calibration error, or ECE, summarizes the reliability diagram in one number. The recipe is short:

1. Split the predictions into bins by confidence, for example ten bins of equal width.
2. In each bin, compute the average confidence and the accuracy.
3. Take the absolute gap between the two.
4. Average the gaps across bins, weighting each bin by the share of samples it contains.

A lower ECE is better, and zero means the confidence matches the accuracy in every bin.

Three traps when you use ECE

• The number depends on the binning. Changing the number of bins, or using equal width versus equal count bins, changes ECE. State your choice, and do not compare ECE values computed with different settings.
• A calibrated model can still be useless. Consider a model that gives every case the overall disease rate as its probability. It can look very well calibrated and tells you nothing about any individual patient. Always report discrimination, such as accuracy or AUC, next to calibration.
• ECE is noisy on small test sets. With few samples in a bin, its estimate is unstable. Standard ECE also looks only at the confidence of the predicted class, so it says nothing about the probabilities given to the other classes.

Proper scoring rules, such as the Brier score and the negative log likelihood, reward accuracy and calibration together, and are a useful complement to ECE.

Modern networks tend to be overconfident

In a widely cited study, Guo and colleagues found that modern neural networks are poorly calibrated, unlike networks from a decade earlier. They also showed that a very simple fix, temperature scaling, works well on most of the datasets they tested.

Temperature scaling divides the network's output scores by a single number, the temperature, which is fitted on held out data. It softens the confidence without changing which class the model picks, so accuracy stays exactly the same. That is why it is the first thing to try.

A checklist for your paper

• Fit any calibration step, such as the temperature, on validation data. Report results on a separate test set.
• Show the reliability diagram, not only ECE, and state the binning you used.
• Report accuracy or AUC together with calibration.
• Test on data from another source if you can, such as another hospital or scanner. Calibration can degrade when the data change.
• Do not claim "well calibrated" from a single metric on a single dataset.

Calibration is the first step toward a model that can say how sure it is, and mean it. The next question is what the model is unsure about, which is the subject of aleatoric versus epistemic uncertainty. For the full map of methods and tests, see uncertainty quantification explained for researchers.