The ability to detect and recognize buildings is important to a variety of vision applications operating in outdoor urban environments. These include landmark recognition, assisted and autonomous navigation, image-based rendering, and 3D scene modeling. The problem of detecting multiple planar surfaces from a single image has been solved with this technology.

Building Facades Detected in urban scenes. Not all of the final clusters of plane support points correspond to true building facades. Some clusters correspond to building roofs, some to reflections of building facades in windows, and some to walls inside buildings.
As any given image can be generated by an infinite number of 3D surfaces, when only a single image is available, some assumptions about the geometric properties of the scene must be made in order to recover the surface geometry. Most urban building facades have surface markings due to doors, windows, bricks, and blocks. As such, each building facade generally consists of two sets of parallel lines, where lines in the first set intersect lines in the second set at right angles. It is well known that the perspective image of a collection of parallel scene edges intersects at a single point in the image, known as the vanishing point. Thus, the image of a building facade may be identified by locating regions in the image covered by pairs of intersecting edges, where each edge is oriented in the direction of one of two vanishing points.

Image line segments are first located, and then the vanishing points of these segments are determined. Groups of short segments are combined into longer segments while maintaining alignment with the associated vanishing points. Next, the intersections of line segments associated with pairs of vanishing points are used to generate local support for planar facades at different orientations. The plane support points are then clustered using an algorithm that requires no knowledge of the number of clusters or of their spatial proximity. Finally, building facades are identified by fitting vanishing-point-aligned quadrilaterals to the clustered support points. The main contribution of this approach is its improved performance over existing approaches while placing no constraints on the facades in terms of their number or orientation, and minimal constraints on the length of the detected line segments.

Image line segments that have been labeled according to vanishing point provide an initial cue to segmenting planar regions in the image. Under the assumption that intersecting edges in the scene are coplanar and orthogonal, every pair of nearby, nonparallel, vanishing point-aligned image line segments defines the local surface orientation of the scene point that projects to the segment intersection point in the image. For two local image regions to be images of the same plane, the pairs of intersecting line segments in each of the two regions should be labeled with the same two vanishing points. It was determined to cluster pairs of intersecting line segments that have identical vanishing point label pairs.

Not all pairs of vanishing points define the orientation of a plane that can be easily detected in an image. Vanishing directions that are close to parallel correspond to planes that are highly foreshortened: their normals are nearly perpendicular to the camera line of sight, and their image consists of line pairs that are nearly parallel and very dense. These line segments will be very difficult to accurately detect. Although building facades may occur at these orientations, what is more common is that two nearly parallel vanishing directions correspond to edges on two different, nonparallel planes. Hence, to label the intersections of lines aligned with a pair of vanishing points, the mean angle between their pairs of intersecting line segments must be sufficiently large.

The figure shows additional examples of using the algorithm to detect building facades in urban environments. As shown in these examples, good results were obtained on images of a number of complex buildings. As seen in the figure, not all of the final clusters of plane support points correspond to true building facades. Some clusters correspond to building roofs, some to reflections of building facades in windows, and some clusters correspond to walls inside of buildings. These false facades can easily be filtered out based on their small size when compared to the larger facades that are detected.

This work was done by Philip David of the Army Research Laboratory. ARL-0077

This Brief includes a Technical Support Package (TSP).
Detecting Planar Surfaces in Outdoor Urban Environments

(reference ARL-0077) is currently available for download from the TSP library.

Don't have an account? Sign up here.