Center for Image and Vision Science



Current Projects 

Learning Active Basis Model for Object Detection and Recognition
YingNian Wu, Zhangzhang Si, Haifeng Gong, C. Fleming and SongChun Zhu
We proposes an active basis model, a shared sketch algorithm, and a compu
tational architecture of summax maps for representing, learning, and recognizing deformable templates. In our generative model, a deformable template is in the form of
an active basis, which consists of a small number of Gabor wavelet elements at selected
locations and orientations. These elements are allowed to slightly perturb their locations
and orientations before they are linearly combined to generate the observed image. The
active basis model, in particular, the locations and the orientations of the basis elements,
can be learned from training images by the shared sketch algorithm .... [project page] 



From Image Parsing to Painterly Rendering
Mingtian Zhao and SongChun Zhu
We present a semanticsdriven approach for strokebased painterly rendering, based on recent image parsing techniques. Image parsing integrates segmentation for regions, sketching for curves, and recognition for object categories. In an interactive manner, we decompose an input image into a hierarchy of its constituent components in a parse tree representation with occlusion relations among the nodes in the tree. To paint the image, we build a brush dictionary containing a large set (760) of brush examples of four shape/appearance categories, which are collected from professional artists, then we select appropriate brushes from the dictionary and place them on the canvas guided by the image semantics included in the parse tree, with each image component and layer painted in various styles. During this process, the scene and object categories also determine the color blending and shading strategies for inhomogeneous synthesis of image details ... [project page]




Learning Animated Basis Model for Action Detection & Recognition
Benjamin Z. Yao and SongChun Zhu
We present an animated basis model that is learnable from cluttered realworld
videos. In our generative model,
an action template is a sequence of image templates each of
which consists of a set of shape and motion primitives (Gabor
bases and opticalflow patches) at selected orientations
and locations. These primitives are allowed to slightly
perturb their locations and orientations to account for spatial
deformations. We use a semisupervised learning procedure to learn from weakly labeled videos with cluttered background.... [project page] 


Layered Graph Matching with Composite Cluster Sampling
Liang Lin , Xiaobai Liu and SongChun Zhu
We study a framework of layered graph matching for integrating graph partition and matching with graph editing. The objective is to find an unknown number of corresponding graph structures in two images. We extract discriminative local primitives from both images and construct a candidacy graph whose vertices are match candidates (i.e. a pair of primitives) and whose edges are either negative for mutual exclusion or a positive for mutual consistence. Then we pose layered graph matching as a multicoloring problem on the candidacy graph. We adapt a composite cluster sampling algorithm to work with both positive and negative edges. 


A Hierarchical and Contextual Model for Aerial Image Parsing
Jake Porway, Qiongchen Wang and SongChun Zhu
We present a hierarchical and contextual model for aerial image understanding. Our
model organizes objects (cars, roofs, roads, trees, parking lots) in aerial scenes into hierarchical groups
whose appearances and configurations are determined by statistical constraints (e.g. relative position,
relative scale, etc.). Our hierarchy is a nonrecursive grammar for objects in aerial images comprised
of layers of nodes that can each decompose into a number of different configurations. This allows us to
generate and recognize a vast number of scenes with relatively few rules. We present a minimax entropy
framework for learning the statistical constraints between objects and show that this learned context allows
us to rule out unlikely scene configurations and hallucinate undetected objects during inference. 

Previous Projects



Explict/Implicit Image Manifolds 


Scene Modeling, Recognition, Graph Grammar 
K. Shi,
S. C. Zhu


H. Chen,
J. Porway, S. C. Zhu 


Perceptual Scale Space Theory* 


Face, Hair, Clothes Modeling, and Sketching* 
Z. J. Xu, Y. Wang, S. Bahrami, S. C. Zhu,
Y. N. Wu 

H. Chen,
Z. J. Xu, S. C. Zhu,
C. Liu, H. Shum 


2D and 3D Shape:
Modeling, Representation and Recognition* 


Image Parsing:
Segmentation + Grouping + Recognition* 
Z. Q. Liu,
S. C. Zhu, A. Dubinskiy 

Z. Tu,
S. C. Zhu, X. R. Chen, A. Yuille 


Texture Analysis and Synthesis* 


Natural Image Statistics, Generic Visual Learning* 
S. C. Zhu,
Y. N. Wu 

S. C. Zhu,
D. Mumford 


Textons, Primal Sketch, Gestalt Fields* 


Textured and Complex Motion Modeling* 
C. Guo,
S. C. Zhu, Y. N. Wu 

Y. Wang, S. C. Zhu 


Markov Chain Monte Carlo and Search* 


Performance Bound Analysis* 
S. C. Zhu,
Z. Tu, A. Barbu, R. Maciuca 

A. Yuille,
J. Coughlan, S. C. Zhu 


Graph Partition by SwendsenWang Cuts 


3D Reconstruction and Parsing* 
A. Barbu,
S. C. Zhu 

F. Han,
S. C. Zhu 


Stereo and Shape from Shading:
From Primal Sketch to 2.5D Sketch* 


Vision for the Blind and Visually Impaired 
A. Barbu, F. Han, S. C. Zhu 

H. Chen,
Z. J. Xu, S. C. Zhu,
C. Liu, H. Shum 


Partial Differential Equations in Vision* 


Lighting Models and Lightons, KGBR 
S. C. Zhu 

F. Han, C. Guo,
S. C. Zhu, A. Yuille 


Inference on Graphs, Belief Propagation, CCCP 



A. Yuille 



Address: 8145 Math Sciences Bldg

Telephone: 3102067721

