Vision motion analysis is not only one of the basic skills for the survival of human beings, but also an important issue in computer vision. Natural scenes include various types of motions driven by various stochastic processes, how to express these vision motion patterns, how to model, learn and infer them has long been a challenge problem in computer vision. Our mission is to build the largest manually annotated video dataset, and to propose the most creditable vision motion theory in the world, and to examine our theory in some applications.
1. Manually Annotated Video Dataset The object of this project is to build the largest manually annotated video dataset in the world, based on the 'manually annotated image dataset', to provide material and benchmark for research in video and motion parsing. This dataset will include hierarchical parsing information for each frame, changes and events between frames and hierarchical story structure on time line.
2. Complex Motion and Event Parsing and Synthesis Theory Using the video dataset as material, based on the spatial-temporal And-Or graph representation, we can model and learn vision motion. Then, we can use learned And-Or graph to synthesis hierarchical story. During learning, we will model low level dynamics, photometry and topology changes, as well as high level semantic story.
On low level, we will use Primal Sketch model to extract vision patterns in frames, use learning and inference algorithms like DDMCMC, MRF, Swendsen-Wang cuts to model the complex motion containing topological, scale, photometric and geometric changes. Based on the low level modeling, we can define and learning high level semantic events and story from annotated dataset.
3. Video Repairing, Composing and Stylization Synthesize a high resolution image of the scene from video of continuous observations of the same scene. Repair and colorize old or historical video. Generate cartoon from photometric video.
Motion and Dynamics Analysis, Information Retrieval
Anthroponomical or anthropometric body motion analysis, single and group motion analysis in sport videos.
4. Event Recognition, Video Surveillance and Target Tracking Pedestrians and vehicles surveillance in transportation, behaviors and events surveillance in important site like banks.