VCLA provides research opportunities for Undergraduate and Master's students at UCLA to work together with our Ph.D students and Postdocs.
See our Project Bulletin.

Vision with Robot Autonomy and Language

Learning Virtual Grasp with Failed Demonstrations via Bayesian Inverse Reinforcement Learning

Xu Xie, Changyang Li, Chi Zhang, Yixin Zhu, and Song-Chun Zhu

VRGym: A Virtual Testbed for Physical and Interactive AI

Xu Xie, Hangxin Liu, Zhenliang Zhang, Yuxing Qiu, Feng Gao, Siyuan Qi, Yixin Zhu, and Song-Chun Zhu

Self-Supervised Incremental Learning for Sound Source Localization in Complex Indoor Environment

Hangxin Liu*, Zeyu Zhang*, Yixin Zhu, and Song-Chun Zhu
*Equal contributors

Human-centric Indoor Scene Synthesis Using Stochastic Grammar

Siyuan Qi, Yixin Zhu, Siyuan Huang, Chenfanfu Jiang, and Song-Chun Zhu

Interactive Robot Knowledge Patching using Augmented Reality

Hangxin Liu*, Yaofang Zhang*, Wenwen Si, Xu Xie, Yixin Zhu, and Song-Chun Zhu
*Equal contributors

Feeling the Force: Integrating Force and Pose for Fluent Discovery through Imitation Learning to Open Medicine Bottles

Mark Edmonds*, Feng Gao*, Xu Xie, Hangxin Liu, Siyuan Qi, Yixin Zhu, Brandon Rothrock, and Song-Chun Zhu
*Equal contributors

Learning Social Affordance Grammar from Videos: Transferring Human Interactions to Human-Robot Interactions

Tianmin Shu, Xiaofeng Gao, Michael S. Ryoo, and Song-Chun Zhu

Learning Social Affordance for Human-Robot Interaction

Tianmin Shu, M. S. Ryoo, and Song-Chun Zhu

Robot Learning with a Spatial, Temporal, and Causal And-Or Graph

Caiming Xiong, Nishant Shukla, Wenlong Xiong, and Song-Chun Zhu

Robot Learning from Demonstration on a Unified Representation

Caiming Xiong, Nishant Shukla, Pablo Garcia Kilroy, Mun Wai Lee, and Song-Chun Zhu

Joint Video and Text Parsing for Understanding Events and Answering Queries

Kewei Tu, Meng Meng, Mun Wai Lee, Tae Eun Choe, and Song-Chun Zhu

VRKitchen: an Interactive 3D Virtual Environment for Task-oriented Learning

Xiaofeng Gao, Ran Gong, Tianmin Shu, Xu Xie, Shu Wang, and Song-Chun Zhu

High-Fidelity Grasping in Virtual Reality using a Glove-based System

Hangxin Liu*, Zhenliang Zhang*, Xu Xie, Yixin Zhu, Yue Liu, Yongtian Wang, and Song-Chun Zhu
*Equal contributors

Mirroring without Overimitation: Learning Functionally Equivalent Manipulation Actions

Hangxin Liu, Chi Zhang, Yixin Zhu, and Song-Chun Zhu

Unsupervised Learning using Hierarchical Models for Hand-Object Interactions

Xu Xie*, Hangxin Liu*, Mark Edmonds, Feng Gao, Siyuan Qi, Yixin Zhu, Brandon Rothrock, and Song-Chun Zhu
*Equal contributors

A Glove-based System for Studying Hand-Object Manipulation via Joint Pose and Force Sensing

Hangxin Liu*, Xu Xie*, Matt Millar*, Mark Edmonds, Feng Gao, Yixin Zhu, Veronica Santos, Brandon Rothrock, and Song-Chun Zhu
*Equal contributors

A Virtual Reality Platform for Dynamic Human-Scene Interaction

Jenny Lin*, Xingwen Guo*, Jingyu Shao*, Chenfanfu Jiang, Yixin Zhu, and Song-Chun Zhu
*Equal contributors

Restricted Visual Turing Test for Deep Scene and Event Understanding

Hang Qi, Tianfu Wu, Mun Wai Lee, and Song-Chun Zhu

Task Learning through Visual Demonstration and Situated Dialogue

Changsong Liu, Joyce Y. Chai, Nishant Shukla, and Song-Chun Zhu

Topic discovery and story segmentation for broadcast news by Swendsen-Wang Cuts

Weixin Li, Jungseock Joo, Hang Qi, and Song-Chun Zhu

I2T: Image Parsing to Text Generation

Benjamin Yao, Xiong Yang, Liang Lin, Mun Wai Lee, and Song-Chun Zhu

 

Vision

Hoilistc++ Scene Understanding: Single-view 3D Holistic Scene Parsing and Human Pose Estimation with Human-Object Interaction and Physical Commensense
Yixin Chen*, Siyuan Huang*, Tao Yuan Siyuan Qi, Yixin Zhu, and Song-Chun Zhu
RAVEN: A Dataset for Relational and Analogical Visual rEasoNing
Chi Zhang*, Feng Gao*, Baoxiong Jia, Yixin Zhu, and Song-Chun Zhu
Cooperative Holistic Scene Understanding: Unifying 3D Object, Layout, and Camera Pose Estimation
Siyuan Huang, Siyuan Qi, Yinxue Xiao, Yixin Zhu, Ying Nian Wu,and Song-Chun Zhu
Holistic 3D Scene Parsing and Reconstruction from a Single RGB Image
Siyuan Huang, Siyuan Qi, Yixin Zhu, Yinxue Xiao, Yuanlu Xu,and Song-Chun Zhu
*Equal contributors
Tracking Occluded Objects and Recovering Incomplete Trajectories by Reasoning about Containment Relations and Human Actions
Wei Liang, Yixin Zhu, and Song-Chun Zhu
Inferring Forces and Learning Human Utilities From Videos
Yixin Zhu*, Chenfanfu Jiang*, Yibiao Zhao, Demetri Terzopoulos, and Song-Chun Zhu
*Equal contributors
What is Where: Inferring Containment Relations from Videos
Wei Liang, Yibiao Zhao, Yixin Zhu, and Song-Chun Zhu
Mining And-Or Graphs for Graph Matching and Object Discovery

Quanshi Zhang, Ying Nian Wu, and Song-Chun Zhu

Learning FRAME Models Using CNN Filters

Yang Lu, Song-Chun Zhu, and Ying Nian Wu

Learning Near-Optimal Cost-Sensitive Decision Policies for Object Detection

Tianfu Wu, and Song-Chun Zhu

Concurrent Action Detection with Structural Prediction
Modeling 4D Human-Object Interactions for Event and Object Recognition

Ping Wei, Yibiao Zhao, Nanning Zheng, and Song-Chun Zhu

Discriminatively Trained And-Or Tree Models for Object Detection

Xi Song, Tianfu Wu, Yunde Jia, and Song-Chun Zhu

Weakly Supervised Learning for Attribute Localization in Outdoor Scenes

Shuo Wang, Jungseock Joo, Yizhou Wang, and Song-Chun Zhu

Cost-Sensitive Top-down/Bottom-up Inference for Multiscale Activity Recognition

Mohamed R. Amer, Dan Xie, Mingtian Zhao, Sinisa Todorovic, and Song-Chun Zhu

Animated Templates for Modelling and Detecting Human Actions

Benjamin Yao, Zicheng Liu, Xiaohan Nie, and Song-Chun Zhu

Human Parsing using Stochastic And-Or Grammars and Rich Appearances

Brandon Rothrock, and Song-Chun Zhu

Configurable, Photorealistic Image Rendering and Ground Truth Synthesis by Sampling Stochastic Grammars Representing Indoor Scenes
Chenfanfu Jiang*, Yixin Zhu*, Siyuan Qi*, Siyuan Huang*, Lap-Fai Yu, Demetri Terzopoulos, and Song-Chun Zhu
*Equal contributors
Inferring Hidden Statuses and Actions
in Video by Causal Reasoning
Amy Fire, and Song-Chun Zhu

Cooperative Training of Descriptor and Generator Networks

Jianwen Xie, Yang Lu, Song-Chun Zhu, and Ying Nian Wu

Alternating Back-Propagation for Generator Network

Tian Han, Yang Lu, Song-Chun Zhu, and Ying Nian Wu

Joint Inference of Groups, Events and Human Roles in Aerial Videos

Tianmin Shu, Dan Xie, Brandon Rothrock, Sinisa Todorovic, and Song-Chun Zhu

Understanding Tools: Task-Oriented Object Modeling, Learning and Recognition

Yixin Zhu*, Yibiao Zhao*, and Song-Chun Zhu
*Equal contributors

Modeling Occlusion by Discriminative AND-OR Structures

Bo Li, Wenze Hu, Tianfu Wu, and Song-Chun Zhu

Inferring "Dark Matter" and "Dark Energy" from Videos

Dan Xie, Sinisa Todorovic, and Song-Chun Zhu

Cosegmentation and Cosketch by Unsupervised Learning

Jifeng Dai, Ying Nian Wu, Jie Zhou, and Song-Chun Zhu

Human Attribute Recognition By Rich Appearance Dictionary

Jungseock Joo, Shuo Wang, and Song-Chun Zhu

Bottom-up / Top-down Inference Processes -- a Numerical Answer

Tianfu Wu, and Song-Chun Zhu

Image Parsing via Stochastic Scene Grammar

Yibiao Zhao, and Song-Chun Zhu

Video Primal Sketch

Z. Han, Z. Xu, and S.-C. Zhu

A Dynamic Model for Face Aging Simulation

Jinli Suo, Song-Chun Zhu, Shiguang Shan, and Xilin Chen

A Hierarchical and Contextual Model for Aerial Image Parsing

Jacob Porway, Qiongchen Wang, and Song-Chun Zhu

 

Cognition

 

Learning

Art