ERC FUNDED PROJECTS

Despite their amazing success, we believe that computer vision algorithms have only scratched the surface of what can be done in terms of modeling and understanding our world from images. We believe that novel image analysis techniques will be a major enabler and driving force behind next-generation technologies, enhancing everyday life and opening up radically new possibilities. And we believe that the key to achieving this is to develop algorithms for reconstructing and analyzing the 3D structure of our world. In this project, we will focus on three lines of research: A) We will develop algorithms for 3D reconstruction from standard color cameras and from RGB-D cameras. In particular, we will promote real-time-capable direct and dense methods. In contrast to the classical two-stage approach of sparse feature-point based motion estimation and subsequent dense reconstruction, these methods optimally exploit all color information to jointly estimate dense geometry and camera motion. B) We will develop algorithms for 3D shape analysis, including rigid and non-rigid matching, decomposition and interpretation of 3D shapes. We will focus on algorithms which are optimal or near-optimal. One of the major computational challenges lies in generalizing existing 2D shape analysis techniques to shapes in 3D and 4D (temporal evolutions of 3D shape). C) We will develop shape priors for 3D reconstruction. These can be learned from sample shapes or acquired during the reconstruction process. For example, when reconstructing a larger office algorithms may exploit the geometric self-similarity of the scene, storing a model of a chair and its multiple instances only once rather than multiple times. Advancing the state of the art in geometric reconstruction and geometric analysis will have a profound impact well beyond computer vision. We strongly believe that we have the necessary competence to pursue this project. Preliminary results have been well received by the community.

2 000 000 €

Start date: 2015-09-01, End date: 2020-08-31

4D reconstruction, the camera-based dense dynamic scene reconstruction, is a grand challenge in computer graphics and computer vision. Despite great progress, 4D capturing the complex, diverse real world outside a studio is still far from feasible. 4DRepLy builds a new generation of high-fidelity 4D reconstruction (4DRecon) methods. They will be the first to efficiently capture all types of deformable objects (humans and other types) in crowded real world scenes with a single color or depth camera. They capture space-time coherent deforming geometry, motion, high-frequency reflectance and illumination at unprecedented detail, and will be the first to handle difficult occlusions, topology changes and large groups of interacting objects. They automatically adapt to new scene types, yet deliver models with meaningful, interpretable parameters. This requires far reaching contributions: First, we develop groundbreaking new plasticity-enhanced model-based 4D reconstruction methods that automatically adapt to new scenes. Second, we develop radically new machine learning-based dense 4D reconstruction methods. Third, these model- and learning-based methods are combined in two revolutionary new classes of 4DRecon methods: 1) advanced fusion-based methods and 2) methods with deep architectural integration. Both, 1) and 2), are automatically designed in the 4D Real World Reconstruction Loop, a revolutionary new design paradigm in which 4DRecon methods refine and adapt themselves while continuously processing unlabeled real world input. This overcomes the previously unbreakable scalability barrier to real world scene diversity, complexity and generality. This paradigm shift opens up a new research direction in graphics and vision and has far reaching relevance across many scientific fields. It enables new applications of profound social pervasion and significant economic impact, e.g., for visual media and virtual/augmented reality, and for future autonomous and robotic systems.

1 977 000 €

Start date: 2018-09-01, End date: 2023-08-31

One of the greatest challenges in exploiting the electron spin for information processing is that it is not a conserved quantity like the electron charge. In addition, spin lifetimes are rather short and correspondingly coherence is quickly lost. This challenge culminates in the coherent manipulation and detection of information from a single spin. Except in a few special systems, so far, single spins cannot be manipulated coherently on the atomic scale, while spin coherence times can only be measured on spin ensembles. A new concept is needed for coherence measurements on arbitrary single spins. Here, the principal investigator (PI) will combine a novel time- and spin-resolved low-temperature scanning tunneling microscope (STM) with the concept of pulsed electron paramagnetic resonance. With this unique and innovative setup, he will be able to address long-standing problems, such as relaxation and coherence times of arbitrary single spin systems on the atomic scale as well as individual spin interactions with the immediate surroundings. Spin readout will be realized through the detection of the absolute spin polarization in the tunneling current by a superconducting tip based on the Meservey-Tedrow-Fulde effect, which the PI has recently demonstrated for the first time in STM. For the coherent excitation, a specially designed pulsed GHz light source will be implemented. The goal is to better understand the spin dynamics and coherence times of single spin systems as well as the spin interactions involved in the decay mechanisms. This will have direct impact on the feasibility of quantum spin information processing with single spin systems on different decoupling surfaces and their scalability at the atomic level. A successful demonstration will enhance the detection limit of spins by several orders of magnitude and fill important missing links in the understanding of spin dynamics and quantum computing with single spins.

2 469 136 €

Start date: 2016-07-01, End date: 2021-06-30

Sensory perception involves the discrimination and binding of multiple modalities of sensory input. This is especially evident in the somatosensory system where different modalities of sensory input, including thermal and mechanosensory, are combined to generate a unified percept. The neural mechanisms of multisensory binding are unknown, in part because sensory perception is typically studied within a single modality in a single brain region. I propose a multi-level approach to investigate thermo-tactile processing in the mouse forepaw system from the primary sensory afferent neurons to thalamo-cortical circuits and behaviour. The mouse forepaw system is the ideal system to investigate multisensory binding as the sensory afferent neurons are well investigated, cell type-specific lines are available, in vivo optogenetic manipulation is possible both in sensory afferent neurons and central circuits and we have developed high-resolution somatosensory perception behaviours. We have previously shown that mouse primary somatosensory forepaw cortical neurons respond to both tactile and thermal stimuli and are required for non-noxious cooling perception. With multimodal neurons how, then, is it possible to both discriminate and bind thermal and tactile stimuli? I propose 3 objectives to address this question. We will first, perform functional mapping of the thermal and tactile pathways to cortex; second, investigate the neural mechanisms of thermo-tactile discrimination in behaving mice; and third, compare neural processing during two thermo-tactile binding tasks, the first using passively applied stimuli, and the second, active manipulation of thermal objects. At each stage we will perform cell type-specific neural recordings and causal optogenetic manipulations in awake and behaving mice. Our multi-level approach will provide a comprehensive investigation into how the brain performs multisensory perceptual binding: a fundamental yet unsolved problem in neuroscience.

1 999 877 €

Start date: 2016-09-01, End date: 2021-08-31