IP Internet

Image-based rendering techniques have received much attention in the computer graphics community in the past decade. These methods are based on the 7D plenoptic function that specifies all possible light rays of a dynamic scene and allows the synthesis of arbitrary views by subsampling the 7D space. For practical systems, this space is usually reduced to 5 (plenoptic modeling), 4 (light fields), or 3 (concentric mosaics) degrees of freedom. On this page, we focus on concentric mosaics captured by a circularly moving camera. These 3D data sets describe scene light intensity as a function of radius, rotation angle, and vertical elevation. Dependent on the radius of the concentric camera motion and the camera's field of view arbitrary new views can be reconstructed within a certain range. However, for the view synthesis, it is assumed that all object points have the same distance from the center of rotation. Deviations of the real scene geometry from this assumption lead to interpolation artifacts and vertical distortions. These artifacts can be reduced by a perspective depth correction based on the distances of the 3D points from the camera. Unfortunately, scene geometry is usually not known and its estimation is still a hard vision problem.

In general, the basic problem of depth estimation from a set of 2D images is the correspondence search. Given a single point in one of the images its correspondences in the other images need to be detected. Depending on the algorithm two or more point correspondences as well as the camera geometry are used to estimate the depth of that point. However, for complex real scenes the correspondence detection problem is still not fully solved. Especially in the case of homogeneous regions, occlusions or noise it still faces many difficulties. It is now generally recognized that using more than two images can dramatically improve the quality of reconstruction. This is particularly interesting for the depth analysis of concentric mosaics which consist of a very large number of views of a scene.

Moche sequence, circular moving camera. Left: first frame. Right: image cube representation, horizontal slice for fixed Y-coordinate.

One method for the simultaneous consideration of all available views is Epipolar Image (EPI) analysis. An epipolar image can be thought of being a horizontal slice (or plane) in the so called image cube that can be constructed by collating all images of a sequence. Each EPI represents a single horizontal line (Y=constant) of all available camera views. For a linear camera movement parallel to the horizontal axis of the image plane, all projections of 3D object points remain in the same EPI throughout the entire sequence. Thus, the EPI represents the trajectories of object points. If the camera is moved equidistantly, the path of an arbitrary 3D point becomes a straight line, called EPI line. The slope of the line represents the depth. The principle of EPI analysis is the detection of all EPI-lines (and their slopes) in all available EPIs.

The advantage of the EPI analysis algorithm is the joint detection of point correspondences for all available views. Occlusions as well as homogeneous regions can be handled efficiently. The big disadvantage of the algorithm is its restriction to linear equidistant camera movements which prevents its usage for the analysis of concentric mosaics. One idea to overcome this problem is the piecewise linear EPI analysis where small segments of the object point trajectory are approximated by straight lines. This approach can also be applied to circular camera movements but significantly reduces the amount of reference images and thus robustness of the 3D reconstruction.

We have proposed a new concept called image cube trajectory (ICT) analysis that overcomes this restriction and is able to jointly exploit all available views also for circular camera configurations. For the special case of an inwards facing, circularly moving camera, we derived the analytical shape of the almost sinusoidal object point trajectories and proposed a new ICT matching method for robust depth estimation. For concentric mosaics, where the rotating camera usually faces in tangential or outwards normal direction, modifications of the ICT calculation and the optimal occlusion compatible ordering scheme are necessary.

Image Cube Trajectory Analysis

Camera configurations for different cases. left (case 1) concentric mosaic (inwards pointing camera), middle (case 2) concentric mosaic (tangential direction) right (case 3) concentric mosaic (normal direction).

For the proposed ICT analysis algorithm we suggest an inverse approach to the conventional way of EPI analysis where usually in a first step the EPI lines are detected in the EPI using some kind of segmentation algorithm. In a second step, the corresponding depth is reconstructed from the slopes of the lines. In contrast, our idea is to specify an ICT for an assumed 3D point by determining its parameters from its assumed 3D position. We call this the reference ICT. In a second step we check if this assumption is valid and the reference ICT fits to the image cube. This is done by evaluating color constancy along the entire trajectory in the image cube for different parameter sets. From the best matching ICT the 3D position of the corresponding object point is derived. For a concentric circular camera motion an arbitrary 3D point may be described in terms of its radius to the center of rotation R, its rotation angle phi and its height y. The ICTs have a well defined structure in the image cube which depends on these 3 parameters. The structure can be exploited to define an efficient and simple occlusion compatible 3D search strategy.

X and Y coordinates for points with varying radius. Left) trajectory for concentric mosaics with inwards facing camera, middle) tangentially facing camera, and right) outwards facing camera.

Experimental Results

One frame of a synthetic sequence with many homogeneous regions (left). Reconstructed depth map (right).

Image Cube Trajectory Analysis (ICT-Analysis)

Image Cube Trajectory Analysis

Experimental Results

Publications

Contact