Funded Grants

Invariant Representation of Objects in the Human Visual System: Combining Computational Modeling and Brain Mapping

A major achievement of the visual system is its ability to perform invariant object recognition. The image of a given object changes due to variations in the viewing conditions, such as location, viewing direction, or illumination. The visual system can compensate for these changes and treat different images as representing an unchanging object. We propose to build upon recent advances in brain imaging and computational modeling to investigate and model aspects of invariant object representation, focusing initially on the problem of shift invariance.

Recent results using fMRl have revealed a high order visual region, located at the lateral aspect of the human occipital lobe (LO) which is involved in object recognition. Initial results implicate LO in shift invariance and perhaps other aspects of invariant recognition. Surprisingly, LO is similarly activated by images of whole objects and by objects broken into large fragments. These results suggest that the object representation in LO is based on object fragments rather than complete objects.

In parallel, recent computational models of object recognition have proposed a new scheme of object representation in which objects are represented by collections of overlapping, position and size invariant object fragments. These models are attractive in that they are consistent with physiological and psychophysical data, and at the same time propose efficient methods for invariant recognition and object classification.

This parallel development in computational modeling and neuroimaging results provides a fertile ground for multi-disciplinary research in which predictions from computational object recognition models could be tested experimentally and then the model will be modified in accordance with experimental results.

The research proposed here will be an interplay between fMRl imaging of object representations in human cortex and computational models. Imaging experiments will be designed and conducted by the group of R. Malach. The series of experiments will be focused on testing the hypothesis of fragment based object representation, and on quantitative estimation of the range of sizes and degree of overlap of object fragments in low and high level visual areas. This will be achieved through controlled manipulation of object fragments size and type, including the comparison of natural parts vs. random fragments, and the type of scrambling used. The relative importance of fragment representations will be compared for a variety of object categories such as familiar and unfamiliar objects, faces etc.

Modeling work will be conducted by the groups of S. Ullman and M. Tsodyks. In the computational modeling, objects will be represented using overlapping fragments that are more complex then elemental features such as an oriented edge but simpler than complete objects. The initial goal of the simulations will be to demonstrate that a relatively small number of such patterns can allow the system to differentiate between a much larger number of complex images, in a shift and size invariant manner. The model will also be used to differentiate between objects and textures based on their constituent fragments-such a contrast has been found in neuroimaging of human LO. We will use the model to investigate what are the crucial image components that underlie the preferential object activation in LO.

It is expected that this interdisciplinary research will lead to an in depth analysis of the biological feasibility and computational power of a new type of object representation and will therefore contribute to our understanding of object recognition which is undoubtedly one of the most intriguing and challenging aspects of human vision.