Computer Vision Object Detection for Airport Surface Traffic

Computer vision fills a critical safety gap at small airports where radar can't see ground traffic.

· 10 min read
Cover illustration for “Computer Vision Object Detection for Airport Surface Traffic”
Situational Awareness · October 10, 2026 · 10 min read · 2,183 words

Airport surface traffic awareness is a distinct sensing problem, and computer vision object detection has emerged as its primary solution precisely because the airports that need it most, the small and uncontrolled ones, have never had another form of oversight to fall back on.

Why airport surface traffic is hard to see

Most people assume that if an aircraft is tracked in the sky, it is tracked on the ground too, but it isn't. Radar reads transponders, and transponders exist on aircraft, not on baggage carts, not on a ground crew member walking across a taxiway, and not on every light aircraft that might be taxiing without one. The moment a surveillance problem moves from the air to the pavement, an entire category of object becomes invisible to the systems built to track airspace.

The physical scene itself resists easy sensing. None of these four problems is a generic computer vision challenge borrowed from some other industry. Each one is a property of the airport environment itself, and each demands a solution built specifically for that environment rather than adapted from a system designed to watch a parking lot or a warehouse floor.

What happens at airports with no surface watch

Diagram: Who Bears the Risk: Runway Incursions at Uncontrolled Airports. Visualizes: Show the structural concentration of runway incursion risk at non-towered airports.

The consequences of that sensing gap are not spread evenly across the national airspace system. U.S. runway incursion data shows the risk piles up at uncontrolled airports, and the most dangerous categories of incursion are growing faster than the overall incursion count. That concentration has a structural explanation, not an anecdotal one. Runway incursion data analyzed for non-towered airports establishes the mechanism directly: at a towered airport, a controller watches the surface, corrects deviations in real time, and records them as they unfold; at a non-towered airport, the entire burden shifts onto voluntary pilot action, radio calls on CTAF that no regulation requires anyone to make, visual scanning that no one verifies actually happened, and self-separation that no external authority enforces.

The FAA's own taxonomy for runway incursions makes the gap explicit. The agency recognizes three types: pilot deviation, vehicle or pedestrian deviation, and operational incident. Pilot deviations already make up close to two-thirds of all U.S. runway incursions nationally. At non-towered airports you can't have an operational incident at all, so pilot deviation ends up as close to the universal failure mode.

The fatal-accident record shows how concentrated this risk really is. Near-collision reports filed by pilots point in the same direction: three-quarters of them involve operations within ten nautical miles of a non-controlled aerodrome, and near-collisions occur twice as often when the pilot involved had no prior radio warning of the other traffic. A March 2025 audit by the DOT Inspector General found that the agency lacks an integrated approach for analyzing runway incursion data and identifying risk, and the FAA itself may not yet have a complete picture of what is driving the trend it is trying to manage. None of this reflects carelessness on the part of individual pilots. It reflects a system that assigns pilots a surveillance and separation job that a controller would otherwise share, without ever verifying that the job gets done.

What computer vision does when it watches a runway

Object detection for airport surfaces is best understood as a pipeline with distinct stages, not a single algorithm that magically "sees" a runway. A 2026 study published in Sensors demonstrates the full pipeline using YOLOv11 for detection paired with Norfair for multi-object tracking, running from video acquisition through object detection, multi-object tracking, and motion analysis, and ending in annotated video, event logs, and activity timelines as final output.

Detection is the first stage, and it works by having a trained neural network examine each video frame and draw a bounding box around anything it recognizes, aircraft, ground vehicles, personnel, assigning each box a class label and a confidence score. Tracking comes next, and this is where a single detection in a single frame becomes something more useful: a trajectory. The output stage turns all of this into something a person can actually use: an annotated live feed for human review, structured logs for incident investigation, and timeline visualizations useful for reconstructing an event after the fact.

Fixed cameras bring their own engineering demands that a pipeline has to account for. The Airport-YOLO study, published in IET Image Processing in 2026, introduces an architecture built specifically for fixed-camera airport scenes, adding an Edge Information Detection Module to enhance small-object feature representation, a Selective Feature Fusion Module to optimize feature aggregation, and a multi-branch detection head, all aimed at the small-object and occlusion failures that standard detectors mishandle in exactly this kind of environment. What all of this adds up to is a capability no human observer can match under any circumstances: continuous, consistent, logged attention across an entire airport surface at once, with no degradation from fatigue, no blind spot from looking the wrong direction, and no gap in the record afterward.

Why small and occluded objects are the hard problem

The failure mode that actually threatens safety is failing to detect a person, a small vehicle, or a distant aircraft whose footprint in the camera frame amounts to a handful of pixels, not a system mistaking a widebody jet for something else. Airport-YOLO's authors frame small-object detection and occlusion as the two defining challenges of fixed-camera airport scenes: a ground crew member standing near a large aircraft may occupy only a handful of pixels in a wide-area frame, and a detector trained primarily on larger objects produces weak or absent feature signals for cases that small. Occlusion compounds the problem on any active ramp, where aircraft, jetways, vehicles, and personnel constantly overlap one another from a single fixed camera angle, hiding part or all of an object the system needs to see.

Weather and lighting compound the small-object detection difficulty. None of this amounts to a fundamental barrier. It is an active engineering problem, and architectures like Airport-YOLO exist specifically because researchers are closing the gap. What it means for any airport operator evaluating a system is that a vendor's performance numbers from a controlled test environment do not automatically transfer to a specific runway. A system has to be validated against the actual lighting, camera geometry, and traffic mix of the airport where it will be installed, not just against a benchmark dataset.

Camera placement and sensor fusion

Software can only detect what the cameras actually capture, and that makes camera placement, coverage, and sensor selection a system design problem as much as a computer vision problem. No detection architecture, however well trained, compensates for an angle no camera was ever pointed at.

Digital tower architectures described in industry implementation literature typically combine several sensor types. High-definition daylight cameras supply the base video feed when visibility is ordinary. Infrared cameras extend detection into fog, night, and precipitation, when daylight cameras degrade; digital tower documentation notes that infrared feeds improve low-visibility operations and that AI algorithms use them to detect objects and help prevent runway incursions. Eurocontrol's documentation of remote tower implementations describes 360-degree panoramic images built from high-resolution daylight and infrared cameras, combined with augmented reality overlays that show you additional information about aircraft and vehicles in real time. Millimeter-wave radar, referenced in descriptions of foreign object debris detection systems, adds a sensing modality that continues to function in weather conditions that defeat optical cameras.

Latency is not a flexible design parameter in this environment. Digital tower specifications require that the image of the remote aerodrome reach the controller's display in under one second in the worst case, a single requirement that governs camera compression settings, network architecture, and processing pipeline design all at once, because a detection arriving a few seconds late is a detection arriving too late to matter. For a small, uncontrolled airport, this entire design problem is more tractable than it is at a major terminal: the surface area to cover is smaller, traffic density is lower, and a modest array of cameras with overlapping fields of view can achieve full runway and taxiway coverage at a fraction of the infrastructure cost a major hub would require.

System outputs and usage

A detection pipeline only matters if what it produces is structured, timely, and something a person can act on. Raw video, however complete, is not the same thing as situational awareness.

Well-designed implementations layer several kinds of output on top of the raw feed. That distinction, between a camera that records an event for someone to review later and a system that flags a developing conflict while there's still time to act on it, is what separates a documentation tool from a safety tool. The 2026 Sensors study demonstrates both CSV logs and Gantt chart-based activity timelines as outputs alongside the annotated video, creating a searchable record that supports post-event review and formal incident investigation. Eurocontrol's remote tower documentation describes augmented reality overlays that display aircraft identification, speed vectors, and conflict alerts directly on the camera view, reducing the cognitive load on a controller by placing the data right where the controller is already looking.

The significance of this output layer is sharpest at a non-towered airport equipped with a digital ATC tower. The annotated feed, for that airport, is the first time anyone has had a real-time, logged picture of what is actually happening on its surface.

Computer vision inside a digital ATC tower for small airports

Computer vision object detection is the sensing layer that makes a digital ATC tower possible at facilities where a manned tower was never going to happen, not a product that stands on its own at a small airport. The FAA has updated its own nomenclature to reflect this, moving from "Remote Tower" to "Digital Tower" specifically to remove the implication that the facility has to sit off-airport, which opens the door to an on-site system, a genuinely remote configuration, or a hybrid of the two depending on what a given airport actually needs and can staff.

Roughly 90 percent of U.S. public-use airports run with no tower at all, and if you count every U.S. airport, that figure climbs past 97 percent. For that population, the economics of a digital tower built on a computer vision sensing layer look fundamentally different from the economics of a conventional staffed tower. The Contract Tower Program requires a benefit-cost ratio of at least 1.0 for full funding, with a cost-share option for airports that fall short, but a great many small uncontrolled airports fail that test, so conventional tower service locks them out. A computer vision sensing layer also supports functions beyond primary surface surveillance at the same airport: foreign object debris detection, catching debris on runways and taxiways before it causes damage, and perimeter security, an application already demonstrated in research on security fence inspection using object detection. The technology described throughout this article is what makes this class of solution work at all; the architecture speaks for itself without needing embellishment.

What the technology still cannot do reliably

None of this amounts to a finished technology, and a sophisticated buyer deserves a straight account of where the limits currently sit. Small-object detection performance still degrades at distance and in poor lighting, and a system that performs well in midday testing can produce meaningfully more missed detections at dusk or in low-contrast conditions unless both the training data and the camera hardware were configured with those scenarios in mind from the start. Occlusion remains genuinely unsolved in single-camera configurations. Airport-YOLO's own development was motivated precisely by the inadequacy of existing detectors on small and occluded objects. The state of the art as of 2026 is still actively working on a problem it has not finished solving. The 2026 multi-task learning review notes that few implementations integrate deep learning into civilian airport surface surveillance, so the operational track record at real airports is thinner than simulation and benchmark-dataset results alone would suggest. Novel or unexpected objects, an animal on the runway, an unfamiliar ground vehicle, a damaged aircraft in an unusual configuration, can be misclassified or missed outright if nothing resembling them appeared in the training data.

The right response to these limitations is to design around the limitations that exist today: sensor redundancy that pairs infrared with daylight cameras, alert thresholds calibrated to each airport's specific traffic mix, and human review of flagged events rather than a fully automated response with no person in the loop. The broader policy backdrop adds urgency to that design work. The GAO reported that, as of September 2024, more than three-quarters of the FAA's 138 air traffic control systems were unsustainable or potentially unsustainable, and Congress appropriated funding in July 2025 to back the Brand New Air Traffic Control System initiative. That is a policy environment built for exactly this kind of sensing technology at exactly the airports that have gone without it the longest. The deployment case for computer vision at uncontrolled airports has never depended on the technology being perfect. It depends on the technology being better than the nothing that currently watches those runways, and on the evidence assembled here, it already is.

Sources

  1. Computer Vision-Based Airport Turnaround Monitoring Using YOLOv11, Multi-Object Tracking, and Motion-Based Passenger and Baggage Activity Detection
  2. Airport‐YOLO: A Fixed‐Camera Solution for Airport Object Detection - Zhao - 2026 - IET Image Processing - Wiley Online Library
  3. Multi‐Task Learning for Airport Surface Surveillance: A Review - Fu - 2026 - Expert Systems - Wiley Online Library
  4. Security Fence Inspection at Airports Using Object Detection
  5. ASRS Database Report Set - Runway Incursions

More in Situational Awareness