Primal Sketch

Marr proposed the primal sketch as an intermediate representation in his representational framework for deriving shape information from images. This representation is the stage between the original image and the 2.5-D sketch. The primal sketch was supposed to be a first level inner representation of generic images, in terms of image primitives, such as bars, edges, terminators, etc. However, despite many inspiring observations, Marr provided neither an explicit mathematical model nor a rigorous definition for the dictionary of visual primitives.

Figure 1. Marr's Representational Framework

We propose a mathematical theory of primal sketch and define sketchability. The theory consists of four components:

（ 1 ） The center of our theory is a primal sketch model for natural images, which integrates the MRF and wavelet theories;

（ 2 ） A sketching pursuit process, which combines the matching pursuit procedure (Mallat and Zhang, 1993) for the sketchable part by adding one base at a time, and the filter pursuit procedure (Zhu, Wu, and Mumford 1997) for the non-sketchable part by adding one filter at a time;

（ 3 ） A definition of sketchability;

（ 4 ） Learning a dictionary of primitives (or textons) in image sketch.

        In plain words, we summarize our model as:

（ 1 ） Automatically separate the image into "sketchable" (structures) and "non-sketchable" (textures) parts by a "sketching pursuit process" which computes "sketchability";

（ 2 ） Model the structures by a generative model (Wavelet-like model) with a dictionary of "visual primitives" learned from natural images;

（ 3 ） Model textures by a descriptive model (Markov Random Field model) - Julesz Ensemble model.

（ 4 ） Model the spatial relationship of the structures by a descriptive model - Gestalt Fields.

Figure 2 is o ne example of our primal sketch model .

a). input image	b). sketching pursuit process	c). sketches
f). synthesized image	e). synthesized texture	d). sketch image