← Back to blog

Are all AI models equal?

Technology

If they all use “AI,” why would results differ so much?

Every vendor says “we have AI.” But a model that scores 90% in a clean lab can fall apart on a wet, shadowed road at 50 km/h. Here’s what actually separates a model you can trust from one you can’t.

In our last piece 'Understanding AI for road damage detection', we opened the black box: every detection has a class, a confidence score, and a bounding box you can check. Fair question next — if the machinery is the same, are all models equal?

They are not. Three systems can all say “AI-powered road inspection” and produce wildly different data. The difference isn’t the acronym. It’s how much hard work went into making the model survive the real world. Because a demo on a sunny, clean test strip tells you almost nothing about a rainy Tuesday in November. Same “AI,” radically different results. The gap is the work: the data, the annotations, and the tuning behind the model.

What makes a real road so hard for a model

A road is a highly complex, messy visual environment, and every variable is a chance to be wrong. Serious research deliberately trains on this chaos — raw field imagery with the shadows and glare left in.

Angles

The same crack looks different from every approach and lane position.

Weather

Rain, glare, and a low winter sun all rewrite the image.

Shadows & reflections

A hard shadow line or a wet patch mimics a crack — a classic false positive.

Texture & occlusion

Leaves, tyre marks, patches, and worn paint clutter the frame.

And when a model trained in one place moves to a new region or camera setup, performance degrades unless it’s adapted. Collecting the right images, labelling them, retraining — that unglamorous work is what separates a robust model from a fragile demo version.

Won’t all that detection just bury me in data?

Only if the model is blunt. The fix is granularity: a well-trained model doesn’t just say “crack here,” it grades how bad. With severity and registration type attached to every detection, you filter your whole network down to only, for example, the Severity 4 and 5 cracks and send crews there first — 400,000 data points become a one-click priority list. But that grading isn’t free: it takes extensive multi-tier annotation. It’s the layer that turns raw detections into decisions.

Metric A Premature AI A Mature AI
How big? Approximate area — “something is here” Real geometry — length, width, area
How bad? At best the category type “hole, crocodile cracking, etc.” “3 m longitudinal crack, 5 mm wide — Severity 2”
Tools to filter data No easy tooling. You drown in unstructured data. Filtering with just a few clicks so you can prioritise work orders.

A box tells you where. Can it tell you what’s important?

This is where models genuinely diverge. Extend a model from bounding boxes to instance segmentation — classifying each pixel, not just drawing a rectangle — and it can measure real geometry.

Three questions that tell a good model from a good demo

1

“What happens when it rains — or when there’s a hard shadow?”

A good answer describes training on messy, real-world imagery — not a promise that it's never wrong.

2

“How much data was this trained on, and does it know my roads?”

Models degrade across regions and camera setups without adaptation. Listen for local data and retraining.

3

“Does it just draw a box, or measure size and severity?”

Boxes locate. Segmentation measures. Severity lets you filter. Only the last two make a maintenance plan.