Volker Blanz
Volker Blanz |
|
|
New homepage at Max-Planck Institut fuer Informatik, Saarbruecken
318kB pdf (low resolution) or 2.8MB pdf (high resolution) or 12MB ps (gzip)
Click here for a short movie (mpeg 5MB).
In this
paper, a new technique for modeling textured 3D faces is introduced. 3D faces can
either be generated automatically from one or more photographs, or modeled directly
through an intuitive user interface. Users are assisted in two key problems of
computer aided face modeling. First, new face images or new 3D face models can be
registered automatically by computing dense one-to-one correspondence to an internal face
model. Second, the approach regulates the naturalness of modeled faces avoiding
faces with an ``unlikely'' appearance.
Starting from an example set of 3D face models, we derive a Morphable Face Model by transforming the shape and texture of the examples into a vector space representation. New faces and expressions can be modeled by forming linear combinations of the prototypes. Shape and texture constraints derived from the statistics of our example faces are used to guide manual modeling or automated matching algorithms.
In this framework, it is easy to control complex facial attributes, such as gender, attractiveness, body weight, or facial expressions. Attributes are automatically learned from a set of faces rated by the user, and can then be applied to classify and manipulate new faces.
Given a single photograph of a face, we can estimate its 3D shape, its orientation in space and the illumination conditions in the scene. Starting from a rough estimate of size, orientation and illumination, our algorithm optimizes these parameters along with the face's internal shape and surface colour to find the best match to the input image. The face model extracted from the image can be rotated and manipulated in 3D.
Example: The figure shows an application of our approach. Matching a morphable model atomatically to a single sample image (1) of a face results in a 3D shape (2) and a texture map estimate. The texture estimate can be improved by additional texture extraction (4). The 3D model is rendered back into the image after changing facial attributes, such as gaining (3) and loosing weight (5), frowning (6), or being forced to smile (7).
Even though we can easily recognize most 3D objects from all viewpoints, there may still be significant differences between views in terms of reaction time and error rate in recognition and naming experiments. Moreover, if subjects are asked to form a mental image of an object, to take a photograph or to select the view they consider most informative, they clearly prefer some views to others. These preferred views are called canonical views. In an interactive computer graphics experiment, we assessed canonical views for objects by allowing participants to actively rotate realistically shaded 3D models on the screen in realtime.
In 3D object recognition, measures of distance between two 2D images can be used to decide whether these images show two different objects or two different views of the same object. In my Diploma thesis,I considered a particular distance measure using spatial filters, and investigated self-optimization processes for these filters based on large numbers of computer-generated images of 3D models. We have compared the algorithm's recognition performance with results from a support vector machine on an image database of chairs ( Blanz, V.; Schölkopf, B.; Bülthoff, H.; Burges, C.; Vapnik, V.; & Vetter, T. 1996. Comparison of view--based object recognition algorithms using realistic 3D models. In: C. von der Malsburg, W. von Seelen, J. C. Vorbrüggen, and B. Sendhoff (eds.): Artificial Neural Networks - ICANN'96. Springer Lecture Notes in Computer Science Vol. 1112, Berlin, 251-256. (6 pages, 441 K) ); for benchmarking, the set of images of rendered chair models is available on our ftp server.
Back to Psychophysics Department
Last modified: November, 13th 1998