Center for Image and Vision Science
  Current Projects

Learning Active Basis Model for Object Detection and Recognition

Ying-Nian Wu, Zhangzhang Si, Haifeng Gong, C. Fleming and Song-Chun Zhu
We proposes an active basis model, a shared sketch algorithm, and a compu- tational architecture of sum-max maps for representing, learning, and recognizing deformable templates. In our generative model, a deformable template is in the form of an active basis, which consists of a small number of Gabor wavelet elements at selected locations and orientations. These elements are allowed to slightly perturb their locations and orientations before they are linearly combined to generate the observed image. The active basis model, in particular, the locations and the orientations of the basis elements, can be learned from training images by the shared sketch algorithm .... [project page]

From Image Parsing to Painterly Rendering

Mingtian Zhao and Song-Chun Zhu
We present a semantics-driven approach for stroke-based painterly rendering, based on recent image parsing techniques. Image parsing integrates segmentation for regions, sketching for curves, and recognition for object categories. In an interactive manner, we decompose an input image into a hierarchy of its constituent components in a parse tree representation with occlusion relations among the nodes in the tree. To paint the image, we build a brush dictionary containing a large set (760) of brush examples of four shape/appearance categories, which are collected from professional artists, then we select appropriate brushes from the dictionary and place them on the canvas guided by the image semantics included in the parse tree, with each image component and layer painted in various styles. During this process, the scene and object categories also determine the color blending and shading strategies for inhomogeneous synthesis of image details ... [project page]

Learning Animated Basis Model for Action Detection & Recognition

Benjamin Z. Yao and Song-Chun Zhu
We present an animated basis model that is learnable from cluttered real-world videos. In our generative model, an action template is a sequence of image templates each of which consists of a set of shape and motion primitives (Gabor bases and optical-flow patches) at selected orientations and locations. These primitives are allowed to slightly perturb their locations and orientations to account for spatial deformations. We use a semi-supervised learning procedure to learn from weakly labeled videos with cluttered background.... [project page]

Layered Graph Matching with Composite Cluster Sampling

Liang Lin , Xiaobai Liu and Song-Chun Zhu
We study a framework of layered graph matching for integrating graph partition and matching with graph editing. The objective is to find an unknown number of corresponding graph structures in two images. We extract discriminative local primitives from both images and construct a candidacy graph whose vertices are match candidates (i.e. a pair of primitives) and whose edges are either negative for mutual exclusion or a positive for mutual consistence. Then we pose layered graph matching as a multi-coloring problem on the candidacy graph. We adapt a composite cluster sampling algorithm to work with both positive and negative edges.

A Hierarchical and Contextual Model for Aerial Image Parsing

Jake Porway, Qiongchen Wang and Song-Chun Zhu
We present a hierarchical and contextual model for aerial image understanding. Our model organizes objects (cars, roofs, roads, trees, parking lots) in aerial scenes into hierarchical groups whose appearances and configurations are determined by statistical constraints (e.g. relative position, relative scale, etc.). Our hierarchy is a non-recursive grammar for objects in aerial images comprised of layers of nodes that can each decompose into a number of different configurations. This allows us to generate and recognize a vast number of scenes with relatively few rules. We present a minimax entropy framework for learning the statistical constraints between objects and show that this learned context allows us to rule out unlikely scene configurations and hallucinate undetected objects during inference.

  Previous Projects
Explict/Implicit Image Manifolds
Scene Modeling, Recognition, Graph Grammar
   K. Shi, S. C. Zhu    H. Chen, J. Porway, S. C. Zhu
Perceptual Scale Space Theory*
Face, Hair, Clothes Modeling, and Sketching*
   Z. J. Xu, Y. Wang, S. Bahrami, S. C. Zhu, Y. N. Wu    H. Chen, Z. J. Xu, S. C. Zhu, C. Liu, H. Shum
2D and 3D Shape:
Modeling, Representation and Recognition
Image Parsing:
Segmentation + Grouping + Recognition
   Z. Q. Liu, S. C. Zhu, A. Dubinskiy    Z. Tu, S. C. Zhu, X. R. Chen, A. Yuille
Texture Analysis and Synthesis*
Natural Image Statistics, Generic Visual Learning*
   S. C. Zhu, Y. N. Wu    S. C. Zhu, D. Mumford
Textons, Primal Sketch, Gestalt Fields*
Textured and Complex Motion Modeling*
   C. Guo, S. C. Zhu, Y. N. Wu    Y. Wang, S. C. Zhu
Markov Chain Monte Carlo and Search*
Performance Bound Analysis*
   S. C. Zhu, Z. Tu, A. Barbu, R. Maciuca    A. Yuille, J. Coughlan, S. C. Zhu
Graph Partition by Swendsen-Wang Cuts
3D Reconstruction and Parsing*
   A. Barbu, S. C. Zhu    F. Han, S. C. Zhu
Stereo and Shape from Shading:
From Primal Sketch to 2.5D Sketch
Vision for the Blind and Visually Impaired
   A. Barbu, F. Han, S. C. Zhu    H. Chen, Z. J. Xu, S. C. Zhu, C. Liu, H. Shum
Partial Differential Equations in Vision*
Lighting Models and Lightons, KGBR
   S. C. Zhu    F. Han, C. Guo, S. C. Zhu, A. Yuille
Inference on Graphs, Belief Propagation, CCCP
   A. Yuille   
Address: 8145 Math Sciences Bldg Telephone: 310-206-7721