Techtonics: Real-Time Pothole Depth Estimation with Edge Spatial AI

Road deterioration is a multi-billion dollar municipal liability. Across Italy and urban centers worldwide, asphalt degrades under heavy traffic, thermal cycles, and rainwater seepage. What begins as microscopic fissures quickly evolves into vehicle-damaging craters.

The standard institutional approach to road maintenance is reactive: a commuter ruins a wheel rim or suffers an accident, files a complaint with the municipality, a municipal inspector inspects the site days later, and a maintenance crew is dispatched with temporary cold-mix asphalt. By the time an intervention occurs, secondary damage has multiplied.

During the weekend of May 8–10, 2026, my teammates Mirco Bonomo, Assylbek Bugybay, and I competed in the GDG AI HACK 2026(opens in a new tab) in Milan, hosted at the Randstad Box and organized by GDG on Campus PoliMi and GDG Cloud Milano. Competing in the “See Beyond” track sponsored by Luxonis(opens in a new tab), we set out to build Techtonics: a self-contained, edge-first spatial computer vision system capable of identifying road surface anomalies and measuring their volumetric depth in real time directly on moving vehicles.

The Limitation of Flat 2D Vision

Most automated pothole detection projects rely on conventional monocular 2D object detection models—typically a lightweight YOLO or SSD architecture trained on RGB dashboard camera frames. While 2D bounding boxes can localize visual anomalies, they suffer from fundamental failure modes when applied to municipal civil engineering:

1. Illumination and Shadow Artifacts: A dark oil stain, a newly laid asphalt patch, or a tree branch shadow cast on a bright afternoon often exhibits the exact same visual chromatic signature as an open pothole. Monocular detectors produce intolerable false-positive rates under varying solar angles.
2. Lack of Volumetric Measurement: A 2D bounding box can tell you that an area looks irregular, but it cannot determine whether it is a cosmetic 2-millimeter surface scuff or an emergency-grade 12-centimeter crater capable of snapping a suspension strut.
3. No Ground Truth for Priority Scoring: Municipalities operate with finite daily maintenance budgets. To prioritize triage, road managers need quantifiable metrics: physical volume, maximum depth, and surface area.

To solve this, we needed spatial perception: capturing synchronous stereo disparity to reconstruct the road topography in physical metric units ($X, Y, Z$).

Edge Architecture: Luxonis OAK-D Pro

Streaming high-resolution video streams from hundreds of city buses or municipal utility vehicles to a cloud datacenter for central processing is infeasible. Cellular uplink bandwidth is constrained and costly, round-trip latency prevents immediate driver alert feedback, and continuous video streaming raises privacy concerns regarding pedestrian faces and vehicle license plates.

The solution is an edge-first architecture where all neural inference, depth calculation, and geometric plane fitting happen directly on the camera hardware.

For Techtonics, we built upon the Luxonis OAK-D Pro. The camera pairs an onboard 12-megapixel RGB sensor with a synchronized stereo pair of global-shutter monochrome cameras spaced at a fixed baseline, supported by an active infrared dot projector for low-texture environments. The onboard VPU executes neural network graphs and computes disparity maps at hardware speed without touching host CPU cycles.

Our runtime pipeline operates in four coordinated stages:

1. Detect: An edge neural network scans the incoming RGB video feed to isolate candidate road damage regions of interest (ROI).
2. Measure: The stereo depth stream is sampled around and within the identified ROI to compute the metric physical depth.
3. Score: A composite damage severity index (1 to 10) is calculated by correlating depth against bounding area.
4. Report: Telemetry payloads containing spatial coordinates, severity metrics, and localized timestamps are emitted for municipal logging.

Solving Depth Estimation: Dynamic Plane Fitting

The central algorithmic challenge was accurate depth estimation. In a stationary lab setup, calculating depth within a bounding box is straightforward: you simply read the distance $Z$ reported by the stereo depth map. On a moving vehicle, however, this naive assumption collapses.

Vehicles constantly pitch, roll, and bounce on their suspensions. Furthermore, roads are rarely level; they incline along hills and crown laterally for water drainage. A pothole 5 meters ahead does not sit on a plane orthogonal to the camera lens.

To measure genuine surface depression rather than distance from the sensor, we developed a dynamic planar regression model:

Step 1: Estimating the Road Surface Reference Plane

Rather than assuming an ideal horizontal plane, we sample the metric depth pixels located immediately outside the pothole’s bounding box. Because these pixels represent the undamaged asphalt surface surrounding the defect, they serve as the local ground truth.

We fit a planar equation:

where $(u, v)$ are the image pixel coordinates and $z$ represents the physical depth. Using least-squares regression over the outer perimeter pixels, the coefficients $a$, $b$, and $c$ continuously model the immediate road surface angle, automatically compensating for vehicle vibration, road camber, and camera tilt.

Step 2: Denoising Disparity Data

Stereo disparity maps frequently suffer from high-frequency noise and speckle artifacts, especially on asphalt where specular reflections or uniform gravel texture challenge block-matching algorithms. Within the candidate bounding box, we apply a $3 \times 3$ median filter across the depth buffer to eliminate isolated disparity dropouts without blurring edge boundaries.

Step 3: Outlier-Resistant Depth Extraction

Within the filtered ROI, the true pothole depth is the signed orthogonal deviation from the computed reference surface plane:

To safeguard the metric against residual sensor artifacts, reflective puddle glints, or stray gravel at the bottom of the crater, we discard the absolute maximum and extract the 95th percentile depth inside the ROI. This yields a stable, repeatable measurement of the defect’s true penetration into the roadbed.

Prototype & Live Demonstration

Over 48 hours of rapid prototyping, code refactoring, and field testing around the venue, we packaged the algorithm into a real-time tracking application.

During our live evaluation, the system successfully demonstrated real-time detection and millimeter-accurate localization:

• Identifying pothole candidates with $72\%$ detection confidence.
• Real-time 3D spatial positioning: $x = -23.69\,\text{mm}$, $y = -81.01\,\text{mm}$, $z = 818.00\,\text{mm}$.
• Real-time severity classification and depth logging (e.g. measuring sub-surface drop down to $-181\,\text{mm}$ and assigning a calibrated severity score).

You can see a demo of the detection pipeline in action in our project demonstration video below (also available on YouTube(opens in a new tab)):

Reflections from the Hackathon

Hackathons force an intense compression of engineering cycles. In 48 hours, you cannot afford architectural self-indulgence or over-engineered abstractions. You must build what works, understand where your sensors fail, and solve the physics of the problem directly.

Working alongside Mirco and Assylbek was an exhilarating technical exercise. From calibrating the OAK-D Pro optics on our makeshift workbench to verifying planar regression math at 4 AM fueled by espresso, Techtonics demonstrated that combining spatial computer vision with edge hardware can turn standard municipal vehicles into continuous, proactive guardians of civil infrastructure.

Special thanks to the organizers at GDG on Campus PoliMi, GDG Cloud Milano, and Luxonis for hosting a fantastic event and providing cutting-edge spatial AI hardware to push to its limits.