University of Florida study finds repeated patterns can distort stereo-camera depth estimates

University of Florida researchers say simple repeated patterns can make stereo-camera systems misjudge how far away an obstacle is, a potential problem for vehicles and robots that use those cameras to navigate. In a report dated September 22, the university described tests spanning commercial cameras, depth-estimation software and a moving vehicle at a controlled facility. The findings identify a way visual surroundings could disrupt obstacle detection, but they do not show how often that happens in everyday operation.

The underlying study was submitted to arXiv on September 14 and is scheduled for presentation at the ACM Conference on Computer and Communications Security in November. Its authors report that a pattern made an obstacle appear up to 20 metres farther away or 12 metres closer in one ZED2 camera setup. Those figures describe that setup; they are not measurements of errors across vehicles in service.

Why a repeated pattern can change a distance reading

A stereo system uses two cameras to estimate depth from corresponding parts of their images. If the software associates the wrong regions, its calculated distance changes. The study’s authors attribute the vulnerability to the way camera pixels sample a scene and to calibration errors, which can bias the matching process when a scene contains repeated features. That helps explain why a regular visual pattern can cause trouble even without a specially trained attack on an AI model.

The team evaluated two conventional stereo-matching algorithms, three deep-learning depth models and a model that combines stereo cameras with LiDAR. It also tested ZED2 and Intel RealSense D435 cameras. The range of systems matters because the reported effect was not confined to one software approach, although the tests cannot establish that every stereo-camera product responds in the same way.

What the vehicle tests showed

For outdoor experiments, the researchers mounted cameras on a ground vehicle and projected patterns onto a surface or the back of a van. With a car travelling at about 15 km/h past the patterned van, they report false obstacle readings lasting more than half a second in both daytime and nighttime conditions. The physical moving-vehicle tests reached about 15 km/h; tests up to 40 km/h took place in the CARLA driving simulator.

The researchers say an erroneous depth reading could prompt emergency braking through an autonomous-driving framework. In the university’s account, a projected checkerboard-like pattern on the back of a vehicle made part of it appear closer to the perception system, enough to trigger an automatic response such as braking. These demonstrations show a possible route from a camera error to a vehicle response in a controlled setting. They do not establish that ordinary deployed vehicles have braked or crashed because of striped fences or other repeated patterns.

Sara Rampazzi, who led the research effort, said the patterns need not be placed by an attacker. ‘These things can also happen naturally, so it’s not a matter of imagining a sophisticated attacker. It becomes a safety problem,’ she told the university. That distinction matters for operators of vehicles, drones and ground robots: a sensing weakness could be relevant to routine surroundings as well as deliberate interference. The reported tests do not quantify how frequently natural scenes cause such failures.

Proposed defense and remaining questions

The authors propose detecting suspicious depth discrepancies and suppressing the wrong estimate. With a conventional algorithm, they report reducing the error below 0.5 metre in 96.5% of 200 real-world stereo image pairs. For a deep-learning model, they report error below 0.1 metre in all 200 KITTI image pairs tested. The latter result comes from dataset images, not a deployed vehicle; neither figure is a guarantee of performance across cameras and operating conditions.

Earlier, independently authored research from Nanyang Technological University described erroneous stereo depth estimates caused by a different physical camouflage attack. It provides precedent for studying physical interference with stereo vision, rather than a replication of the Florida team’s measurements with simple repeated patterns. The University of Florida study adds tests of patterns across several sensing and software approaches and reports both simulated and controlled physical experiments.

The authors say their moving real-world experiments used a single projected pattern on flat or nearly flat surfaces. Their results therefore leave open how the proposed defense performs with changing patterns, varied surfaces and deployed systems. The planned November conference presentation is the next stated milestone. For now, the measured errors and braking response should be read as findings from the reported setups, while the prevalence and consequences of similar errors in regular service remain unmeasured here.