My Model Cheated: How Grad-CAM Exposed a 95% Accuracy Lie

The Project

Adarsh Sriuma trained a car damage classifier achieving 95.08% validation accuracy with ResNet-18, only to discover via Grad-CAM that the model relied on background cues, not car features.

Why This Matters

The model’s high accuracy was an illusion created by dataset bias: damaged cars had cluttered backgrounds while undamaged cars had clean ones. This highlights a critical gap between ideal models (which should focus on relevant features) and real-world implementations where data biases can lead to catastrophic failures in production. The cost of such errors could be significant in applications like autonomous driving or insurance claims.

Key Insights

• “95.08% validation accuracy, 2025”: A misleading metric caused by dataset bias in car damage classification.
• “Background bias in image classification”: The model learned to detect messy backgrounds instead of car damage.
• “Grad-CAM used by Adarsh Sriuma”: Visualization technique exposed the model’s reliance on irrelevant features.

Practical Applications

• Use Case: Image classification systems where background context is irrelevant (e.g., medical imaging, satellite analysis).
• Pitfall: Relying solely on accuracy metrics without visualization tools like Grad-CAM to validate model behavior.

References:

• https://dev.to/tsadarsh/my-model-cheated-how-grad-cam-exposed-a-95-accuracy-lie-2n81
• https://github.com/lplenka/coco-car-damage-detection-dataset
• https://github.com/yamaerenay/100-images-of-top-50-car-brands