1.    And-Or Graph Representation

     We proposed a three-layered And-Or graph model for the representation and computation. Each And-node represents a sub-configuration (composite template). It is either substantiated by a leaf-node, or expanded to a number of Or-nodes jointly with certain spatial relations. Each Or-node is a switching pointing to a number of alternative sub-configurations.

2.    Diagram of the Model and Algorithm

      We extend the well-known active appearance model (AAM) with two additional layers: (i). A face component layer, which refines the global AAM (PCA) model with more detailed representations in 6-zones for the six facial components: two eyebrows, two eyes, nose and mouth. Each component has a set of diverse representations for the various types of eyes, mouths, noses and their topological configurations, such as open and close states. The representation for each component within its zone is a local AAM model with a variable number of landmarks. The selection of the representation is modulated by the grammatical rules through hidden "switch" variables; (ii). A face skin layer, which further refines the 6 component zones with sketch curves for the subtle differences in eye-lid, eye-shade, nostril, lips etc. In this layer, it divides the face skin into 9 zones with a learned dictionary of sketch primitives to represent possible skin marks and wrinkles. We adopt various prior models for sketches in these 15 zones and the number of sketch curves changes depending on the complexity of the faces.

      A coarse-to-fine strategy is introduced for the analysis and synthesis stages of the algorithm. The hidden variables are inferred from the lowest resolution layer to the highest layer. In which layer the algorithm stops depends on the resolution of observed image and its minimum description length (MDL).

3.    Reconstruction and sketching results over multiple resolutions

      With the proposed model, we are able to represent the faces with multiple resolutions. By replacing the intensity templates with symbolic templates, we can have the sketches drawn automatically. The possible applications of our model include recognition, carton sketch in non-photorealistic rendering, tracking and low-bit face communication in wireless platforms.

publication:

Z.J. Xu, H. Chen, S.C. Zhu, and J. Luo, "A Composite Template for Human Face Modeling and Sketch ", IEEE Trans. on Pattern Analysis and Machine Intelligence, vol.30, no.6, 955-969, 2008. [pdf]

reference:

T.F.Cootes, G.J. Edwards and C.J.Taylor "Active appearance models".
Proceedings of European Conference of Computer Vision (ECCV) 1998.