Project acronym COMET
Project foundations of COmputational similarity geoMETtry
Researcher (PI) Michael Bronstein
Host Institution (HI) UNIVERSITA DELLA SVIZZERA ITALIANA
Call Details Starting Grant (StG), PE6, ERC-2012-StG_20111012
Summary "Similarity is one of the most fundamental notions encountered in problems practically in every branch of science, and is especially crucial in image sciences such as computer vision and pattern recognition. The need to quantify similarity or dissimilarity of some data is central to broad categories of problems involving comparison, search, matching, alignment, or reconstruction. The most common way to model a similarity is using metrics (distances). Such constructions are well-studied in the field of metric geometry, and there exist numerous computational algorithms allowing, for example, to represent one metric using another by means of isometric embeddings.
However, in many applications such a model appears to be too restrictive: many types of similarity are non-metric; it is not always possible to model the similarity precisely or completely e.g. due to missing data; some objects might be mutually incomparable e.g. if they are coming from different modalities. Such deficiencies of the metric similarity model are especially pronounced in large-scale computer vision, pattern recognition, and medical imaging applications.
The ambitious goal of this project is to introduce a paradigm shift in the way we model and compute similarity. We will develop a unifying framework of computational similarity geometry that extends the theoretical metric model, and will allow developing efficient numerical and computational tools for the representation and computation of generic similarity models. The methods will be developed all the way from mathematical concepts to efficiently implemented code and will be applied to today’s most important and challenging problems in Internet-scale computer vision and pattern recognition, shape analysis, and medical imaging."
Summary
"Similarity is one of the most fundamental notions encountered in problems practically in every branch of science, and is especially crucial in image sciences such as computer vision and pattern recognition. The need to quantify similarity or dissimilarity of some data is central to broad categories of problems involving comparison, search, matching, alignment, or reconstruction. The most common way to model a similarity is using metrics (distances). Such constructions are well-studied in the field of metric geometry, and there exist numerous computational algorithms allowing, for example, to represent one metric using another by means of isometric embeddings.
However, in many applications such a model appears to be too restrictive: many types of similarity are non-metric; it is not always possible to model the similarity precisely or completely e.g. due to missing data; some objects might be mutually incomparable e.g. if they are coming from different modalities. Such deficiencies of the metric similarity model are especially pronounced in large-scale computer vision, pattern recognition, and medical imaging applications.
The ambitious goal of this project is to introduce a paradigm shift in the way we model and compute similarity. We will develop a unifying framework of computational similarity geometry that extends the theoretical metric model, and will allow developing efficient numerical and computational tools for the representation and computation of generic similarity models. The methods will be developed all the way from mathematical concepts to efficiently implemented code and will be applied to today’s most important and challenging problems in Internet-scale computer vision and pattern recognition, shape analysis, and medical imaging."
Max ERC Funding
1 495 020 €
Duration
Start date: 2012-10-01, End date: 2017-09-30
Project acronym COMPCAMERAANALYZ
Project Understanding Designing and Analyzing Computational Cameras
Researcher (PI) Anat Levin
Host Institution (HI) WEIZMANN INSTITUTE OF SCIENCE
Call Details Starting Grant (StG), PE6, ERC-2010-StG_20091028
Summary Computational cameras go beyond 2D images and allow the extraction of more dimensions from the visual world such as depth, multiple viewpoints and multiple illumination conditions. They also allow us to overcome some of the traditional photography challenges such as defocus blur, motion blur, noise and resolution. The increasing variety of computational cameras is raising the need for a meaningful comparison across camera types. We would like to understand which cameras are better for specific tasks, which aspects of a camera make it better than others and what is the best performance we can hope to achieve.
Our 2008 paper introduced a general framework to address the design and analysis of computational cameras. A camera is modeled as a linear projection in ray space. Decoding the camera data then deals with inverting the linear projection. Since the number of sensor measurements is usually much smaller than the number of rays, the inversion must be treated as a Bayesian inference problem accounting for prior knowledge on the world.
Despite significant progress which has been made in the recent years, the space of computational cameras is still far from being understood.
Computational camera analysis raises the following research challenges: 1) What is a good way to model prior knowledge on ray space? 2) Seeking efficient inference algorithms and robust ways to decode the world from the camera measurements. 3) Evaluating the expected reconstruction accuracy of a given camera. 4) Using the expected reconstruction performance for evaluating and comparing camera types. 5) What is the best camera? Can we derive upper bounds on the optimal performance?
We propose research on all aspects of computational camera design and analysis. We propose new prior models which will significantly simplify the inference and evaluation tasks. We also propose new ways to bound and evaluate computational cameras with existing priors.
Summary
Computational cameras go beyond 2D images and allow the extraction of more dimensions from the visual world such as depth, multiple viewpoints and multiple illumination conditions. They also allow us to overcome some of the traditional photography challenges such as defocus blur, motion blur, noise and resolution. The increasing variety of computational cameras is raising the need for a meaningful comparison across camera types. We would like to understand which cameras are better for specific tasks, which aspects of a camera make it better than others and what is the best performance we can hope to achieve.
Our 2008 paper introduced a general framework to address the design and analysis of computational cameras. A camera is modeled as a linear projection in ray space. Decoding the camera data then deals with inverting the linear projection. Since the number of sensor measurements is usually much smaller than the number of rays, the inversion must be treated as a Bayesian inference problem accounting for prior knowledge on the world.
Despite significant progress which has been made in the recent years, the space of computational cameras is still far from being understood.
Computational camera analysis raises the following research challenges: 1) What is a good way to model prior knowledge on ray space? 2) Seeking efficient inference algorithms and robust ways to decode the world from the camera measurements. 3) Evaluating the expected reconstruction accuracy of a given camera. 4) Using the expected reconstruction performance for evaluating and comparing camera types. 5) What is the best camera? Can we derive upper bounds on the optimal performance?
We propose research on all aspects of computational camera design and analysis. We propose new prior models which will significantly simplify the inference and evaluation tasks. We also propose new ways to bound and evaluate computational cameras with existing priors.
Max ERC Funding
756 845 €
Duration
Start date: 2010-12-01, End date: 2015-11-30
Project acronym CompDB
Project The Computational Database for Real World Awareness
Researcher (PI) Thomas NEUMANN
Host Institution (HI) TECHNISCHE UNIVERSITAET MUENCHEN
Call Details Consolidator Grant (CoG), PE6, ERC-2016-COG
Summary Two major hardware trends have a significant impact on the architecture of database management systems (DBMSs): First, main memory sizes continue to grow significantly. Machines with 1TB of main memory and more are readily available at a relatively low price. Second, the number of cores in a system continues to grow, from currently 64 and more to hundreds in the near future.
This trend offers radically new opportunities for both business and science. It promises to allow for information-at-your-fingertips, i.e., large volumes of data can be analyzed and deeply explored online, in parallel to regular transaction processing. Currently, deep data exploration is performed outside of the database system which necessitates huge data transfers. This impedes the processing such that real-time interactive exploration is impossible. These new hardware capabilities now allow to build a true computational database system that integrates deep exploration functionality at the source of the data. This will lead to a drastic shift in how users interact with data, as for the first time interactive data exploration becomes possible at a massive scale.
Unfortunately, traditional DBMSs are simply not capable to tackle these new challenges.
Traditional techniques like interpreted code execution for query processing become a severe bottleneck in the presence of such massive parallelism, causing poor utilization of the hardware. I pursue a radically different approach: Instead of adapting the traditional, disk-based approaches, I am integrating a new just-in-time compilation framework into the in-memory database that directly exploits the abundant, parallel hardware for large-scale data processing and exploration. By explicitly utilizing cores, I will be able to build a powerful computational database engine that scales the entire spectrum of data processing - from transactional to analytical to exploration workflows - far beyond traditional architectures.
Summary
Two major hardware trends have a significant impact on the architecture of database management systems (DBMSs): First, main memory sizes continue to grow significantly. Machines with 1TB of main memory and more are readily available at a relatively low price. Second, the number of cores in a system continues to grow, from currently 64 and more to hundreds in the near future.
This trend offers radically new opportunities for both business and science. It promises to allow for information-at-your-fingertips, i.e., large volumes of data can be analyzed and deeply explored online, in parallel to regular transaction processing. Currently, deep data exploration is performed outside of the database system which necessitates huge data transfers. This impedes the processing such that real-time interactive exploration is impossible. These new hardware capabilities now allow to build a true computational database system that integrates deep exploration functionality at the source of the data. This will lead to a drastic shift in how users interact with data, as for the first time interactive data exploration becomes possible at a massive scale.
Unfortunately, traditional DBMSs are simply not capable to tackle these new challenges.
Traditional techniques like interpreted code execution for query processing become a severe bottleneck in the presence of such massive parallelism, causing poor utilization of the hardware. I pursue a radically different approach: Instead of adapting the traditional, disk-based approaches, I am integrating a new just-in-time compilation framework into the in-memory database that directly exploits the abundant, parallel hardware for large-scale data processing and exploration. By explicitly utilizing cores, I will be able to build a powerful computational database engine that scales the entire spectrum of data processing - from transactional to analytical to exploration workflows - far beyond traditional architectures.
Max ERC Funding
1 918 750 €
Duration
Start date: 2017-06-01, End date: 2022-05-31
Project acronym COMPECON
Project Complexity and Simplicity in Economic Mechanisms
Researcher (PI) Noam NISAN
Host Institution (HI) THE HEBREW UNIVERSITY OF JERUSALEM
Call Details Advanced Grant (AdG), PE6, ERC-2016-ADG
Summary As more and more economic activity is moving to the Internet, familiar economic mechanisms are being deployed at unprecedented scales of size, speed, and complexity. In many cases this new complexity becomes the defining feature of the deployed economic mechanism and the quantitative difference becomes a key qualitative one. A well-studied example of such situations is how the humble single-item auction suddenly becomes a billion-times repeated online ad auction, or even becomes a combinatorial auction with exponentially many possible outcomes. Similar complexity explosions occur with various markets, with information dissemination, with pricing structures, and with many other economic mechanisms.
The aim of this proposal is to study the role and implications of such complexity and to start developing a coherent economic theory that can handle it. We aim to identify various measures of complexity that are crucial bottlenecks and study them. Examples of such complexities include the amount of access to data, the length of the description of a mechanism, its communication requirements, the cognitive complexity required from users, and, of course, the associated computational complexity. On one hand we will attempt finding ways of effectively dealing with complexity when it is needed, and on the other hand, attempt avoiding complexity, when possible, replacing it with "simple" alternatives without incurring too large of a loss.
Summary
As more and more economic activity is moving to the Internet, familiar economic mechanisms are being deployed at unprecedented scales of size, speed, and complexity. In many cases this new complexity becomes the defining feature of the deployed economic mechanism and the quantitative difference becomes a key qualitative one. A well-studied example of such situations is how the humble single-item auction suddenly becomes a billion-times repeated online ad auction, or even becomes a combinatorial auction with exponentially many possible outcomes. Similar complexity explosions occur with various markets, with information dissemination, with pricing structures, and with many other economic mechanisms.
The aim of this proposal is to study the role and implications of such complexity and to start developing a coherent economic theory that can handle it. We aim to identify various measures of complexity that are crucial bottlenecks and study them. Examples of such complexities include the amount of access to data, the length of the description of a mechanism, its communication requirements, the cognitive complexity required from users, and, of course, the associated computational complexity. On one hand we will attempt finding ways of effectively dealing with complexity when it is needed, and on the other hand, attempt avoiding complexity, when possible, replacing it with "simple" alternatives without incurring too large of a loss.
Max ERC Funding
2 026 706 €
Duration
Start date: 2017-05-01, End date: 2022-04-30
Project acronym COMPLEX REASON
Project The Parameterized Complexity of Reasoning Problems
Researcher (PI) Stefan Szeider
Host Institution (HI) TECHNISCHE UNIVERSITAET WIEN
Call Details Starting Grant (StG), PE6, ERC-2009-StG
Summary Reasoning, to derive conclusions from facts, is a fundamental task in Artificial Intelligence, arising in a wide range of applications from Robotics to Expert Systems. The aim of this project is to devise new efficient algorithms for real-world reasoning problems and to get new insights into the question of what makes a reasoning problem hard, and what makes it easy. As key to novel and groundbreaking results we propose to study reasoning problems within the framework of Parameterized Complexity, a new and rapidly emerging field of Algorithms and Complexity. Parameterized Complexity takes structural aspects of problem instances into account which are most significant for empirically observed problem-hardness. Most of the considered reasoning problems are intractable in general, but the real-world context of their origin provides structural information that can be made accessible to algorithms in form of parameters. This makes Parameterized Complexity an ideal setting for the analysis and efficient solution of these problems. A systematic study of the Parameterized Complexity of reasoning problems that covers theoretical and empirical aspects is so far outstanding. This proposal sets out to do exactly this and has therefore a great potential for groundbreaking new results. The proposed research aims at a significant impact on the research culture by setting the grounds for a closer cooperation between theorists and practitioners.
Summary
Reasoning, to derive conclusions from facts, is a fundamental task in Artificial Intelligence, arising in a wide range of applications from Robotics to Expert Systems. The aim of this project is to devise new efficient algorithms for real-world reasoning problems and to get new insights into the question of what makes a reasoning problem hard, and what makes it easy. As key to novel and groundbreaking results we propose to study reasoning problems within the framework of Parameterized Complexity, a new and rapidly emerging field of Algorithms and Complexity. Parameterized Complexity takes structural aspects of problem instances into account which are most significant for empirically observed problem-hardness. Most of the considered reasoning problems are intractable in general, but the real-world context of their origin provides structural information that can be made accessible to algorithms in form of parameters. This makes Parameterized Complexity an ideal setting for the analysis and efficient solution of these problems. A systematic study of the Parameterized Complexity of reasoning problems that covers theoretical and empirical aspects is so far outstanding. This proposal sets out to do exactly this and has therefore a great potential for groundbreaking new results. The proposed research aims at a significant impact on the research culture by setting the grounds for a closer cooperation between theorists and practitioners.
Max ERC Funding
1 421 130 €
Duration
Start date: 2010-01-01, End date: 2014-12-31
Project acronym COMPMUSIC
Project Computational models for the discovery of the world's music
Researcher (PI) Francesc Xavier Serra Casals
Host Institution (HI) UNIVERSIDAD POMPEU FABRA
Call Details Advanced Grant (AdG), PE6, ERC-2010-AdG_20100224
Summary Current IT research does not respond to the world's multi-cultural reality. It could be argued that we are imposing the paradigms of our market-driven western culture also on IT and that current IT research results will only facilitate the access of a small part of the world’s information to a small part of the world's population. Most IT research is being carried out with a western centred approach and as a result, our data models, cognition models, user models, interaction models, ontologies, … are all culturally biased. This fact is quite evident in music information research, since, despite the world's richness in musical cultures, most of the research is centred on CDs and metadata of our western commercial music. CompMusic wants to break this huge research bias. By approaching musical information modelling from a multicultural perspective it aims at advancing our state of the art while facilitating the discovery and reuse of the music produced outside the western commercial context. But the development of computational models to address the world’s music information richness cannot be done from the West looking out; we have to involve researchers and musical experts immersed in the different cultures. Their contribution is fundamental to develop the appropriate multicultural musicological and cognitive frameworks from which we should then carry our research on finding appropriate musical features, ontologies, data representations, user interfaces and user centred approaches. CompMusic will investigate some of the most consolidated non-western classical music traditions, Indian (hindustani, carnatic), Turkish-Arab (ottoman, andalusian), and Chinese (han), developing the needed computational models to bring their music into the current globalized information framework. Using these music cultures as case studies, cultures that are alive and have a strong influence in current society, we can develop rich information models that can take advantage of the existing information coming from musicological and cultural studies, from mature performance practice traditions and from active social contexts. With this approach we aim at challenging the current western centred information paradigms, advance our IT research, and contribute to our rich multicultural society.
Summary
Current IT research does not respond to the world's multi-cultural reality. It could be argued that we are imposing the paradigms of our market-driven western culture also on IT and that current IT research results will only facilitate the access of a small part of the world’s information to a small part of the world's population. Most IT research is being carried out with a western centred approach and as a result, our data models, cognition models, user models, interaction models, ontologies, … are all culturally biased. This fact is quite evident in music information research, since, despite the world's richness in musical cultures, most of the research is centred on CDs and metadata of our western commercial music. CompMusic wants to break this huge research bias. By approaching musical information modelling from a multicultural perspective it aims at advancing our state of the art while facilitating the discovery and reuse of the music produced outside the western commercial context. But the development of computational models to address the world’s music information richness cannot be done from the West looking out; we have to involve researchers and musical experts immersed in the different cultures. Their contribution is fundamental to develop the appropriate multicultural musicological and cognitive frameworks from which we should then carry our research on finding appropriate musical features, ontologies, data representations, user interfaces and user centred approaches. CompMusic will investigate some of the most consolidated non-western classical music traditions, Indian (hindustani, carnatic), Turkish-Arab (ottoman, andalusian), and Chinese (han), developing the needed computational models to bring their music into the current globalized information framework. Using these music cultures as case studies, cultures that are alive and have a strong influence in current society, we can develop rich information models that can take advantage of the existing information coming from musicological and cultural studies, from mature performance practice traditions and from active social contexts. With this approach we aim at challenging the current western centred information paradigms, advance our IT research, and contribute to our rich multicultural society.
Max ERC Funding
2 443 200 €
Duration
Start date: 2011-07-01, End date: 2017-06-30
Project acronym COMPUSAPIEN
Project Computing Server Architecture with Joint Power and Cooling Integration at the Nanoscale
Researcher (PI) David ATIENZA ALONSO
Host Institution (HI) ECOLE POLYTECHNIQUE FEDERALE DE LAUSANNE
Call Details Consolidator Grant (CoG), PE6, ERC-2016-COG
Summary The soaring demand for computing power in the last years has grown faster than semiconductor technology evolution can sustain, and has produced as collateral undesirable effect a surge in power consumption and heat density in computing servers. Although computing servers are the foundations of the digital revolution, their current designs require 30-40% of the energy supplied to be dissipated in cooling. The remaining energy is used for computation, but their complex many-core designs produce very high operating temperatures. Thus, operating all the cores continuously at maximum performance levels results in system overheating and failures. This situation is limiting the benefits of technology scaling.
The COMPUSAPIEN proposal aims to completely revise the current computing server architecture. In particular, inspired by the mammalian brain, this proposal targets to design a disruptive three-dimensional (3D) computing server architecture that overcomes the prevailing worst-case power and cooling provisioning paradigm for servers. This new 3D server design champions a heterogeneous many-core architecture template with an integrated on-chip microfluidic fuel cell network for joint cooling delivery and power supply. Also, it will include a novel predictive controller based on holistic power-temperature models, which exploit the server software stack to achieve energy-scalable computing capabilities. Because of its integrated electronic-electrochemical architecture design, COMPUSAPIEN is clearly a high-risk high-reward proposal that will bring drastic energy savings with respect to current server design approaches, and will guarantee energy scalability in future server architectures. To realize this vision, COMPUSAPIEN will develop and integrate breakthrough innovations in heterogeneous computing architectures, cooling-power subsystem design, combined microfluidic power delivery and temperature management in computers.
Summary
The soaring demand for computing power in the last years has grown faster than semiconductor technology evolution can sustain, and has produced as collateral undesirable effect a surge in power consumption and heat density in computing servers. Although computing servers are the foundations of the digital revolution, their current designs require 30-40% of the energy supplied to be dissipated in cooling. The remaining energy is used for computation, but their complex many-core designs produce very high operating temperatures. Thus, operating all the cores continuously at maximum performance levels results in system overheating and failures. This situation is limiting the benefits of technology scaling.
The COMPUSAPIEN proposal aims to completely revise the current computing server architecture. In particular, inspired by the mammalian brain, this proposal targets to design a disruptive three-dimensional (3D) computing server architecture that overcomes the prevailing worst-case power and cooling provisioning paradigm for servers. This new 3D server design champions a heterogeneous many-core architecture template with an integrated on-chip microfluidic fuel cell network for joint cooling delivery and power supply. Also, it will include a novel predictive controller based on holistic power-temperature models, which exploit the server software stack to achieve energy-scalable computing capabilities. Because of its integrated electronic-electrochemical architecture design, COMPUSAPIEN is clearly a high-risk high-reward proposal that will bring drastic energy savings with respect to current server design approaches, and will guarantee energy scalability in future server architectures. To realize this vision, COMPUSAPIEN will develop and integrate breakthrough innovations in heterogeneous computing architectures, cooling-power subsystem design, combined microfluidic power delivery and temperature management in computers.
Max ERC Funding
1 999 281 €
Duration
Start date: 2017-06-01, End date: 2022-05-31
Project acronym COMPUTED
Project Computational User Interface Design
Researcher (PI) Antti Olavi Oulasvirta
Host Institution (HI) AALTO KORKEAKOULUSAATIO SR
Call Details Starting Grant (StG), PE6, ERC-2014-STG
Summary PROBLEM: Despite extensive research on human-computer interaction (HCI), no method exists that guarantees the optimal or even a provably good user interface (UI) design. The prevailing approach relies on heuristics and iteration, which can be costly and even ineffective, because UI design often involves combinatorially hard problems with immense design spaces, multiple objectives and constraints, and complex user behavior.
OBJECTIVES: COMPUTED establishes the foundations for optimizing UI designs. A design can be automatically optimized to given objectives and constraints by using combinatorial optimization methods that deploy predictive models of user behavior as objective functions. Although previous work has shown some improvements to usability, the scope has been restricted to keyboards and widgets. COMPUTED researches methods that can vastly expand the scope of optimizable problems. First, algorithmic support is developed for acquiring objective functions that cover the main human factors in a given HCI task. Second, formal analysis of decision problems in UI design allows combating a broader range of design tasks with efficient and appropriate optimization methods. Third, a novel interactive UI optimization paradigm for UI designers promotes fast convergence to good results even in the face of uncertainty and incomplete knowledge.
IMPACT: Combinatorial UI optimization offers a strong complement to the prevailing design approaches. Because the structured search process has a high chance of finding good solutions, optimization could improve the quality of interfaces used in everyday life. Optimization can also increase cost-efficiency, because reference to optimality can eliminate fruitless iteration. Moreover, because no preknowledge of UI design is required, even novices will be able to design great UIs. Even in “messy,” less well-defined problems, it may support designers by allowing them to delegate the solving of well-known sub-problems.
Summary
PROBLEM: Despite extensive research on human-computer interaction (HCI), no method exists that guarantees the optimal or even a provably good user interface (UI) design. The prevailing approach relies on heuristics and iteration, which can be costly and even ineffective, because UI design often involves combinatorially hard problems with immense design spaces, multiple objectives and constraints, and complex user behavior.
OBJECTIVES: COMPUTED establishes the foundations for optimizing UI designs. A design can be automatically optimized to given objectives and constraints by using combinatorial optimization methods that deploy predictive models of user behavior as objective functions. Although previous work has shown some improvements to usability, the scope has been restricted to keyboards and widgets. COMPUTED researches methods that can vastly expand the scope of optimizable problems. First, algorithmic support is developed for acquiring objective functions that cover the main human factors in a given HCI task. Second, formal analysis of decision problems in UI design allows combating a broader range of design tasks with efficient and appropriate optimization methods. Third, a novel interactive UI optimization paradigm for UI designers promotes fast convergence to good results even in the face of uncertainty and incomplete knowledge.
IMPACT: Combinatorial UI optimization offers a strong complement to the prevailing design approaches. Because the structured search process has a high chance of finding good solutions, optimization could improve the quality of interfaces used in everyday life. Optimization can also increase cost-efficiency, because reference to optimality can eliminate fruitless iteration. Moreover, because no preknowledge of UI design is required, even novices will be able to design great UIs. Even in “messy,” less well-defined problems, it may support designers by allowing them to delegate the solving of well-known sub-problems.
Max ERC Funding
1 499 790 €
Duration
Start date: 2015-04-01, End date: 2020-03-31
Project acronym Con Espressione
Project Getting at the Heart of Things: Towards Expressivity-aware Computer Systems in Music
Researcher (PI) Gerhard Widmer
Host Institution (HI) UNIVERSITAT LINZ
Call Details Advanced Grant (AdG), PE6, ERC-2014-ADG
Summary What makes music so important, what can make a performance so special and stirring? It is the things the music expresses, the emotions it induces, the associations it evokes, the drama and characters it portrays. The sources of this expressivity are manifold: the music itself, its structure, orchestration, personal associations, social settings, but also – and very importantly – the act of performance, the interpretation and expressive intentions made explicit by the musicians through nuances in timing, dynamics etc.
Thanks to research in fields like Music Information Research (MIR), computers can do many useful things with music, from beat and rhythm detection to song identification and tracking. However, they are still far from grasping the essence of music: they cannot tell whether a performance expresses playfulness or ennui, solemnity or gaiety, determination or uncertainty; they cannot produce music with a desired expressive quality; they cannot interact with human musicians in a truly musical way, recognising and responding to the expressive intentions implied in their playing.
The project is about developing machines that are aware of certain dimensions of expressivity, specifically in the domain of (classical) music, where expressivity is both essential and – at least as far as it relates to the act of performance – can be traced back to well-defined and measurable parametric dimensions (such as timing, dynamics, articulation). We will develop systems that can recognise, characterise, search music by expressive aspects, generate, modify, and react to expressive qualities in music. To do so, we will (1) bring together the fields of AI, Machine Learning, MIR and Music Performance Research; (2) integrate theories from Musicology to build more well-founded models of music understanding; (3) support model learning and validation with massive musical corpora of a size and quality unprecedented in computational music research.
Summary
What makes music so important, what can make a performance so special and stirring? It is the things the music expresses, the emotions it induces, the associations it evokes, the drama and characters it portrays. The sources of this expressivity are manifold: the music itself, its structure, orchestration, personal associations, social settings, but also – and very importantly – the act of performance, the interpretation and expressive intentions made explicit by the musicians through nuances in timing, dynamics etc.
Thanks to research in fields like Music Information Research (MIR), computers can do many useful things with music, from beat and rhythm detection to song identification and tracking. However, they are still far from grasping the essence of music: they cannot tell whether a performance expresses playfulness or ennui, solemnity or gaiety, determination or uncertainty; they cannot produce music with a desired expressive quality; they cannot interact with human musicians in a truly musical way, recognising and responding to the expressive intentions implied in their playing.
The project is about developing machines that are aware of certain dimensions of expressivity, specifically in the domain of (classical) music, where expressivity is both essential and – at least as far as it relates to the act of performance – can be traced back to well-defined and measurable parametric dimensions (such as timing, dynamics, articulation). We will develop systems that can recognise, characterise, search music by expressive aspects, generate, modify, and react to expressive qualities in music. To do so, we will (1) bring together the fields of AI, Machine Learning, MIR and Music Performance Research; (2) integrate theories from Musicology to build more well-founded models of music understanding; (3) support model learning and validation with massive musical corpora of a size and quality unprecedented in computational music research.
Max ERC Funding
2 318 750 €
Duration
Start date: 2016-01-01, End date: 2020-12-31
Project acronym CONVEXVISION
Project Convex Optimization Methods for Computer Vision and Image Analysis
Researcher (PI) Daniel Cremers
Host Institution (HI) TECHNISCHE UNIVERSITAET MUENCHEN
Call Details Starting Grant (StG), PE6, ERC-2009-StG
Summary Optimization methods have become an established paradigm to address most Computer Vision challenges including the
reconstruction of three-dimensional objects from multiple images, or the tracking of a deformable shape over time. Yet, it has
been largely overlooked that optimization approaches are practically useless if they do not come with efficient algorithms to
compute minimizers of respective energies. Most existing formulations give rise to non-convex energies. As a consequence,
solutions highly depend on the choice of minimization scheme and implementational (initialization, time step sizes, etc.), with
little or no guarantees regarding the quality of computed solutions and their robustness to perturbations of the input data.
In the proposed research project, we plan to develop optimization methods for Computer Vision which allow to efficiently
compute globally optimal solutions. Preliminary results indicate that this will drastically leverage the power of optimization
methods and their applicability in a substantially broader context. Specifically we will focus on three lines of research: 1) We
will develop convex formulations for a variety of challenges. While convex formulations are currently being developed for
low-level problems such as image segmentation, our main effort will focus on carrying convex optimization to higher level
problems of image understanding and scene interpretation. 2) We will investigate alternative strategies of global optimization
by means of discrete graph theoretic methods. We will characterize advantages and drawbacks of continuous and discrete
methods and thereby develop novel algorithms combining the advantages of both approaches. 3) We will go beyond convex
formulations, developing relaxation schemes that compute near-optimal solutions for problems that cannot be expressed by
convex functionals.
Summary
Optimization methods have become an established paradigm to address most Computer Vision challenges including the
reconstruction of three-dimensional objects from multiple images, or the tracking of a deformable shape over time. Yet, it has
been largely overlooked that optimization approaches are practically useless if they do not come with efficient algorithms to
compute minimizers of respective energies. Most existing formulations give rise to non-convex energies. As a consequence,
solutions highly depend on the choice of minimization scheme and implementational (initialization, time step sizes, etc.), with
little or no guarantees regarding the quality of computed solutions and their robustness to perturbations of the input data.
In the proposed research project, we plan to develop optimization methods for Computer Vision which allow to efficiently
compute globally optimal solutions. Preliminary results indicate that this will drastically leverage the power of optimization
methods and their applicability in a substantially broader context. Specifically we will focus on three lines of research: 1) We
will develop convex formulations for a variety of challenges. While convex formulations are currently being developed for
low-level problems such as image segmentation, our main effort will focus on carrying convex optimization to higher level
problems of image understanding and scene interpretation. 2) We will investigate alternative strategies of global optimization
by means of discrete graph theoretic methods. We will characterize advantages and drawbacks of continuous and discrete
methods and thereby develop novel algorithms combining the advantages of both approaches. 3) We will go beyond convex
formulations, developing relaxation schemes that compute near-optimal solutions for problems that cannot be expressed by
convex functionals.
Max ERC Funding
1 985 400 €
Duration
Start date: 2010-09-01, End date: 2015-08-31