Multi-View Image and Geometry Processing for 3D Cinematography (08w5070)

Objectives

Several early 3D cinematography systems have already demonstrated
partial 3D cinematography capability in various forms and under
different names -virtualized reality, free- viewpoint video, 3D video,
etc. So far, most of these projects have focused on the computer
systems and engineering aspects involved in the implementation. There
is a need for a forum where researchers active in the field could
focus on the mathematical foundations, and determine what new
mathematical results are needed to advance the state-of-the-art in 3D
cinematography systems to the next generation.

Recent advances have now clearly shown the promises of 3D
cinematography stystems allowing multiple-camera capture, processing,
transmission and rendering of 3D models of real scenes in real time.
Yet, many problems remain to be solved before such systems can be
transposed from blue screen studios to the real world. Problems are
both theoretical and practical:

- Scaling issues. How many cameras should be used to fully sample a
complex, dynamic scene with multiple actors? How accurately should
the cameras be synchronized and calibrated?

- Representation issues. How can the geometry and texture of a scene
be separately extracted and represented independently of the
original viewpoints, and at which levels of details?

- Modeling issues. Which information can be realiably extracted from
video streams to allow temporally consistent 3D reconstruction? How
precisely can the geometry and texture of a scene be recovered in
real world situations? How can prior knowledge about the scene
geometry and appearance be used?

- Compression of multi-view video streams. How can the massive
amounts of multiple video and geometry streams generated by 3D
cinematography be repfresented in compact form for efficient
storage, transmission, and remote visualization?

- Implementation issues. How can the massive amounts of multiple video
and geometry streams generated by 3D cinematography be processed and
rendered efficiently? How can such processing be performed on-line
at video frame rate, without sacrificing quality?

- Human factors and aesthetic issues. How will end users navigate
within 3D cinematographic scenes? Will 3D cinematography produce
exact copies of the real world Or will it evolve into a more
elaborate, yet to be discovered, art form?

Those are difficult and fascinating questions which will no doubt
generate new research directions in computer vision, in graphics, and
in the underlaying mathematics, for many years to come. One goal of
this workshop is to identify and predict these advances.

On one hand this workshop will bring together researchers who have
built three-dimen-sional cinematography systems, to present their work,
show their latest results, share insights on current issues and new
research directions, and discuss real world applications of their
work. Another important goal for this workshop is to produce a
state-of-the art report on the mathematical foundations of the field,
to be published in a major journal, including recomendations for
funding, and a comprehensive list of applications areas with
measurable benefits to society.

Some topics to be discuss in this workshop:

- Camera and recording technology for 3D video, including the
synchronization, optical modeling and calibration of camera arrays

- Projection and display technology for 3D cinema and 3D television,
including auto-stereoscopic displays

- Geometric modeling of dynamic scenes in four dimensions - dynamic
surfels, voxels, mesh, depth images, NURBS surfaces, subdivision
surfaces, level sets, etc.

- Bayesian modeling and priors for geometry, texture and motion
recovery from multiple video cameras

- Multi-resolution 3D modeling from videos captured at different image
resolutions

- Motion tracking and temporal coherence of 3D reconstruction

- Video-based rendering, view morphing and video interpolation in
dynamic scenes

- Distributed and embedded algorithms for real-time geometry and video
processing

- Multiple-view video compression

- Applications of 3D cinematography in games, virtual reality,
medicine, television, education, etc.