ERC FUNDED PROJECTS

4D reconstruction, the camera-based dense dynamic scene reconstruction, is a grand challenge in computer graphics and computer vision. Despite great progress, 4D capturing the complex, diverse real world outside a studio is still far from feasible. 4DRepLy builds a new generation of high-fidelity 4D reconstruction (4DRecon) methods. They will be the first to efficiently capture all types of deformable objects (humans and other types) in crowded real world scenes with a single color or depth camera. They capture space-time coherent deforming geometry, motion, high-frequency reflectance and illumination at unprecedented detail, and will be the first to handle difficult occlusions, topology changes and large groups of interacting objects. They automatically adapt to new scene types, yet deliver models with meaningful, interpretable parameters. This requires far reaching contributions: First, we develop groundbreaking new plasticity-enhanced model-based 4D reconstruction methods that automatically adapt to new scenes. Second, we develop radically new machine learning-based dense 4D reconstruction methods. Third, these model- and learning-based methods are combined in two revolutionary new classes of 4DRecon methods: 1) advanced fusion-based methods and 2) methods with deep architectural integration. Both, 1) and 2), are automatically designed in the 4D Real World Reconstruction Loop, a revolutionary new design paradigm in which 4DRecon methods refine and adapt themselves while continuously processing unlabeled real world input. This overcomes the previously unbreakable scalability barrier to real world scene diversity, complexity and generality. This paradigm shift opens up a new research direction in graphics and vision and has far reaching relevance across many scientific fields. It enables new applications of profound social pervasion and significant economic impact, e.g., for visual media and virtual/augmented reality, and for future autonomous and robotic systems.

1 977 000 €

Start date: 2018-09-01, End date: 2023-08-31

"Digital 3D models are gaining more and more importance in diverse application fields ranging from computer graphics, multimedia and simulation sciences to engineering, architecture, and medicine. Powerful technologies to digitize the 3D shape of real objects and scenes are becoming available even to consumers. However, the raw geometric data emerging from, e.g., 3D scanning or multi-view stereo often lacks a consistent structure and meta-information which are necessary for the effective deployment of such models in sophisticated down-stream applications like animation, simulation, or CAD/CAM that go beyond mere visualization. Our goal is to develop new fundamental algorithms which transform raw geometric input data into augmented 3D models that are equipped with structural meta information such as feature aligned meshes, patch segmentations, local and global geometric constraints, statistical shape variation data, or even procedural descriptions. Our methodological approach is inspired by the human perceptual system that integrates bottom-up (data-driven) and top-down (model-driven) mechanisms in its hierarchical processing. Similarly we combine algorithms operating on different levels of abstraction into reconstruction and modeling networks. Instead of developing an individual solution for each specific application scenario, we create an eco-system of algorithms for automatic processing and interactive design of highly complex 3D models. A key concept is the information flow across all levels of abstraction in a bottom-up as well as top-down fashion. We not only aim at optimizing geometric representations but in fact at bridging the gap between reconstruction and recognition of geometric objects. The results from this project will make it possible to bring 3D models of real world objects into many highly relevant applications in science, industry, and entertainment, greatly reducing the excessive manual effort that is still necessary today."

2 482 000 €

Start date: 2014-03-01, End date: 2019-02-28

The cell cortex is a highly dynamic layer of crosslinked actin filaments and myosin molecular motors beneath the cell membrane. It plays a central role in large scale rearrangements that occur inside cells. Many molecular mechanisms contribute to cortex structure and dynamics. However, cell scale physical properties of the cortex are difficult to grasp. This is problematic because for large scale rearrangements inside a cell, such as coherent flow of the cell cortex, it is the cell scale emergent properties that are important for the realization of such events. I will investigate how the actomyosin cytoskeleton behaves at a coarse grained and cellular scale, and will study how this emergent active behaviour is influenced by molecular mechanisms. We will study the cell cortex in the one cell stage C. elegans embryo, which undergoes large scale cortical flow during polarization and cytokinesis. We will combine theory and experiment. We will characterize cortex structure and dynamics with biophysical techniques such as cortical laser ablation and quantitative photobleaching experiments. We will develop and employ novel theoretical approaches to describe the cell scale mechanical behaviour in terms of an active complex fluid. We will utilize genetic approaches to understand how these emergent mechanical properties are influenced by molecular activities. A central goal is to arrive at a coarse grained description of the cortex that can predict future dynamic behaviour from the past structure, which is conceptually similar to how weather forecasting is accomplished. To date, systematic approaches to link molecular scale physical mechanisms to those on cellular scales are missing. This work will open new opportunities for cell biological and cell biophysical research, by providing a methodological approach for bridging scales, for studying emergent and large-scale active mechanical behaviours and linking them to molecular mechanisms.

1 500 000 €

Start date: 2011-12-01, End date: 2017-08-31

"The first decade of Algorithmic Mechanism Design (AMD) concentrated, very successfully, on the design of truthful mechanisms for the allocation of resources among agents with private preferences. Truthful mechanisms are ones that incentivize rational users to report their preferences truthfully. Truthfulness, however, for all its theoretical appeal, suffers from several inherent limitations, mainly its high communication and computation complexities. It is not surprising, therefore, that practical applications forego truthfulness and use simpler mechanisms instead. Simplicity in itself, however, is not sufficient, as any meaningful mechanism should also have some notion of fairness; otherwise agents will stop using it over time. In this project I plan to develop an innovative AMD theoretical framework that will go beyond truthfulness and focus instead on the natural themes of simplicity and fairness, in addition to computational tractability. One of my primary goals will be the design of simple and fair poly-time mechanisms that perform at near optimal levels with respect to important economic objectives such as social welfare and revenue. To this end, I will work toward providing precise definitions of simplicity and fairness and quantifying the effects of these restrictions on the performance levels that can be obtained. A major challenge in the evaluation of non-truthful mechanisms is defining a reasonable behavior model that will enable their evaluation. The success of this project could have a broad impact on Europe and beyond, as it would guide the design of natural mechanisms for markets of tens of billions of dollars in revenue, such as online advertising, or sales of wireless frequencies. The timing of this project is ideal, as the AMD field is now sufficiently mature to lead to a breakthrough and at the same time young enough to be receptive to new approaches and themes."

1 394 600 €

Start date: 2013-11-01, End date: 2018-10-31