Skip to main content
Back to blog

6 min readEnglish

Techtonics: measuring pothole depth with a stereo camera

At GDG AI Hack Milan we put a Luxonis OAK-D Pro on a vehicle. It finds potholes and measures their depth against a road plane fitted on every frame.

#Computer Vision#Edge AI#Spatial Computing#Robotics#Embedded Systems

Also available in Italian: Versione in italiano.

Potholes cost cities a lot of money. Asphalt cracks under heavy traffic, heat cycles and rain, and small cracks grow into holes that damage cars.

Most cities fix roads after someone complains. A driver breaks a wheel rim or has an accident, reports it, an inspector visits days later, and a crew fills the hole with cold-mix asphalt. By then the damage around it has grown.

On May 8 to 10, 2026, my teammates Mirco Bonomo, Assylbek Bugybay and I took part in GDG AI HACK 2026(opens in a new tab) in Milan. GDG on Campus PoliMi and GDG Cloud Milano organized it at the Randstad Box. We entered the “See Beyond” track sponsored by Luxonis(opens in a new tab) and built Techtonics, a camera system that runs on a vehicle, finds potholes and measures how deep they are in real time.

The Techtonics team at work during GDG AI Hack Milan with the Luxonis camera on the desk

Why 2D detection is not enough

Most pothole detectors run a small 2D object detection model (usually YOLO or SSD) on RGB frames from a dashcam. A 2D bounding box can show where something looks wrong, but it fails in three ways:

  1. Shadows and stains. An oil stain, a fresh asphalt patch or the shadow of a branch can look exactly like a pothole in a photo. With the sun at different angles, a single camera gives too many false positives.
  2. No depth. A bounding box cannot tell a 2 mm scuff from a 12 cm hole that can break a suspension strut.
  3. No way to rank repairs. A city repairs a limited number of holes per day. To decide which ones come first, it needs numbers: volume, maximum depth and area.

We needed depth. A stereo camera gives it: from the disparity between two views we can rebuild the road surface in metric units (X,Y,ZX, Y, Z).

Doing everything on the camera

Sending high-resolution video from hundreds of buses to a data center does not work. Mobile upload bandwidth is limited and expensive, the round trip is too slow to warn the driver, and the video would contain faces and license plates.

So the camera does all the work: neural inference, depth and plane fitting.

Real-life detection screen showing pothole bounding box, 3D coordinate localization, and measured depth

We used the Luxonis OAK-D Pro. It has a 12 MP RGB sensor, a pair of global-shutter mono cameras at a fixed baseline for stereo, and an infrared dot projector for surfaces with little texture. Its VPU runs the neural network and computes the disparity map on the device, without using the host CPU.

The pipeline has four steps:

  1. Detect. A neural network on the camera finds candidate damage regions (ROI) in the RGB frames.
  2. Measure. The pipeline samples the stereo depth in and around each ROI to compute the depth of the hole.
  3. Score. It combines depth and area into a severity score from 1 to 10.
  4. Report. It sends the coordinates, the severity and a timestamp, so the city can log the damage.

Measuring depth on a moving vehicle

Depth was the hard part. On a bench you can read the distance ZZ from the depth map inside the bounding box and be done. On a moving vehicle that does not work.

A vehicle pitches, rolls and bounces on its suspension. Roads go up and down hills and are higher in the middle so water drains off. A pothole 5 meters ahead does not lie on a plane perpendicular to the lens.

Depth estimation diagram showing the reference road plane fitted outside the ROI and 95th percentile depth inside

We wanted the depth of the hole below the road, not the distance from the camera. We got it in three steps.

Step 1: fit the road plane

We do not assume the road is flat. We take the depth pixels just outside the bounding box. They belong to the intact asphalt around the hole, so they tell us where the road surface is.

We fit a plane to them:

z=a⋅u+b⋅v+cz = a \cdot u + b \cdot v + c

where (u,v)(u, v) are pixel coordinates and zz is the depth. A least-squares fit over the pixels around the box gives aa, bb and cc. We refit them on every frame, so the plane follows the vibration of the vehicle, the slope of the road and the tilt of the camera.

Step 2: clean the depth map

Stereo depth on asphalt is noisy. Reflections and the uniform texture of gravel confuse block matching. Inside the bounding box we apply a 3×33 \times 3 median filter to the depth buffer. It removes isolated bad pixels and keeps the edges sharp.

Step 3: take a robust depth

Inside the filtered ROI, the depth of the hole at each pixel is its distance from the fitted plane:

Δz(u,v)=zmeasured(u,v)−zplane(u,v)\Delta z(u, v) = z_{\text{measured}}(u, v) - z_{\text{plane}}(u, v)

The single deepest pixel is often noise, a reflection from a puddle or a loose stone. We take the 95th percentile of Δz\Delta z inside the ROI instead of the maximum. That number stays stable from frame to frame.

The demo

In 48 hours we wrote the pipeline, cleaned it up, tested it around the venue and wrapped it in a real-time tracking app.

In the live demo the system:

Here is the demo video (also on YouTube(opens in a new tab)):

The video is hosted on YouTube. Nothing is loaded until you press play: then YouTube (Google) receives your IP address and may store data on your device. Privacy policy · Watch on YouTube (opens in a new tab)

Davide Bertoni, Mirco Bonomo, and Assylbek Bugybay at GDG AI Hack Milan 2026

What I learned

In 48 hours there is no time for abstractions you might need later. You build what works, find out where the sensor fails, and fix that.

It was a lot of fun with Mirco and Assylbek, from calibrating the OAK-D Pro on a makeshift bench to checking the plane fitting math at 4 AM on espresso. Techtonics showed that a stereo camera on a city bus could map road damage while the bus does its normal route.

Thanks to GDG on Campus PoliMi, GDG Cloud Milano and Luxonis for the event and for lending us the hardware.