What a Confidence Heatmap Tells You

A confidence score is a promise. When a model outputs 0.83 on a finding, it is implicitly claiming that findings at this score level are correct about 83 percent of the time. If that claim is not kept, the number is not a confidence value: it is a number between zero and one. In a clinical context, a false promise about uncertainty is not a minor technical deficiency. It is a failure of the instrument's basic function.

Calibration, defined

A model is calibrated when its stated confidence matches its empirical accuracy across the distribution. If you take all the predictions where the model said it was 70 percent confident, and you find that roughly 70 percent of those were correct, the model is well-calibrated at that operating point. Calibration can be measured using reliability diagrams: predictions are grouped into confidence bins and the observed accuracy in each bin is plotted against the stated confidence. A perfectly calibrated model lies on the diagonal. Most neural networks lie above it, meaning they are overconfident. Some, particularly those trained on small datasets with heavy augmentation, show the opposite pattern. Neither is acceptable as the output of a clinical instrument.

Why deep networks tend towards overconfidence

The dominant explanation is the softmax output layer. Softmax applies an exponential transformation to logits before normalising, which concentrates probability mass towards the highest-scoring class. A small difference in raw logit values can produce a very large difference in the softmax output, which reads as high confidence even when the underlying signal is weak. Training on cross-entropy loss compounds this: the loss decreases as the probability assigned to the correct class increases, which gives the optimiser a continuous incentive to push confidence towards 1.0. By the time a model is thoroughly trained on a large dataset, its raw outputs are reliably overconfident, and something must be done about that before those outputs reach a clinician.

The reliability diagram

The most informative tool for understanding a model's calibration is the reliability diagram, sometimes called a calibration plot. Predictions on a held-out set are sorted into bins, typically ten to fifteen equally spaced confidence intervals. Within each bin, you compute the mean predicted confidence and the mean observed accuracy. The resulting plot shows you immediately where the model is overconfident or underconfident, and by how much. The Expected Calibration Error, the weighted average of the gap between confidence and accuracy across bins, gives you a single scalar to track. For Verityn, we track ECE per-modality and per-finding-class across every model version, because a model that is well-calibrated on consolidation may be systematically overconfident on interstitial patterns.

Saying you are 90 percent confident and being right 67 percent of the time is not a rounding error. It is a different instrument from the one you described.

The heatmap is evidence, not illustration

A confidence heatmap in a clinical tool has to satisfy a faithfulness requirement that marketing heatmaps do not. It is not enough to show a visually plausible region of activation. The highlighted region must correspond to the image features that actually drove the prediction. Post-hoc attribution methods, Grad-CAM and its variants, LIME, SHAP for vision, all have known failure modes: they can highlight regions that correlate with the correct class in the training distribution without reflecting the causal path from pixel to output. For Verityn, we require that heatmaps pass a region-occlusion test: masking the highlighted region must cause a statistically significant drop in the model's confidence for that finding. If the heatmap highlights the right costophrenic angle but masking that region does not change the output, the heatmap is decorative, and we do not ship it.

What a trustworthy heatmap shows

• The region that, when removed, most reduces the model's output confidence for the flagged finding.
• Spatial resolution appropriate to the modality, not smoothed beyond clinical usefulness.
• A stated confidence interval alongside the heatmap, not a point estimate alone.
• A distinction between high-confidence regions and moderate-confidence regions using a consistent colour scale.
• Version control alongside the model checkpoint, so findings from different model versions remain traceable.
• A bounded footprint when the underlying detection is focal: a nodule heatmap does not expand to cover a whole lobe.

Why this matters clinically

A radiologist reading with a second-reader tool is forming a modified interpretation. They see the image, they see the flag, and they see where the model was looking. If the heatmap is misleading, the radiologist's attention is being misdirected at the moment they are deciding whether to agree with the flag. This is a qualitatively different problem from a model that is simply wrong on some fraction of cases. A wrong flag the radiologist can dismiss on clinical grounds. A wrong heatmap may shift their visual attention before they have had the chance to form their own view. The heatmap must be honest about what the model saw, not a plausible-looking overlay.

Calibration and heatmap faithfulness are not features added to Verityn after the core product was built. They are the core product. A model that classifies accurately on aggregate but provides dishonest uncertainty and misleading attribution is not clinical decision support. It is a confident noise generator. We hold every model version to a calibration and faithfulness gate before it reaches any clinical reader, and we publish those results in our validation summary.