ERC FUNDED PROJECTS

Parameterized mathematical models have been central to the understanding and design of communication, networking, and radar systems. However, they often lack the ability to model intricate interactions innate in complex systems. On the other hand, data-driven approaches do not need explicit mathematical models for data generation and have a wider applicability at the cost of flexibility. These approaches need labelled data, representing all the facets of the system interaction with the environment. With the aforementioned systems becoming increasingly complex with intricate interactions and operating in dynamic environments, the number of system configurations can be rather large leading to paucity of labelled data. Thus there are emerging networks of systems of critical importance whose cognition is not effectively covered by traditional approaches. AGNOSTIC uses the process of exploration through system probing and exploitation of observed data in an iterative manner drawing upon traditional model-based approaches and data-driven discriminative learning to enhance functionality, performance, and robustness through the notion of active cognition. AGNOSTIC clearly departs from a passive assimilation of data and aims to formalize the exploitation/exploration framework in dynamic environments. The development of this framework in three applications areas is central to AGNOSTIC. The project aims to provide active cognition in radar to learn the environment and other active systems to ensure situational awareness and coexistence; to apply active probing in radio access networks to infer network behaviour towards spectrum sharing and self-configuration; and to learn and adapt to user demand for content distribution in caching networks, drastically improving network efficiency. Although these cognitive systems interact with the environment in very different ways, sufficient abstraction allows cross-fertilization of insights and approaches motivating their joint treatment.

2 499 595 €

Start date: 2017-10-01, End date: 2022-09-30

Recurrent neural networks (RNNs) are general parallel-sequential computers. Some learn their programs or weights. Our supervised Long Short-Term Memory (LSTM) RNNs were the first to win pattern recognition contests, and recently enabled best known results in speech and handwriting recognition, machine translation, etc. They are now available to billions of users through the world's most valuable public companies including Google and Apple. Nevertheless, in lots of real-world tasks RNNs do not yet live up to their full potential. Although universal in theory, in practice they fail to learn important types of algorithms. This ERC project will go far beyond today's best RNNs through novel RNN-like systems that address some of the biggest open RNN problems and hottest RNN research topics: (1) How can RNNs learn to control (through internal spotlights of attention) separate large short-memory structures such as sub-networks with fast weights, to improve performance on many natural short-term memory-intensive tasks which are currently hard to learn by RNNs, such as answering detailed questions on recently observed videos? (2) How can such RNN-like systems metalearn entire learning algorithms that outperform the original learning algorithms? (3) How to achieve efficient transfer learning from one RNN-learned set of problem-solving programs to new RNN programs solving new tasks? In other words, how can one RNN-like system actively learn to exploit algorithmic information contained in the programs running on another? We will test our systems existing benchmarks, and create new, more challenging multi-task benchmarks. This will be supported by a rather cheap, GPU-based mini-brain for implementing large RNNs.

2 500 000 €

Start date: 2017-10-01, End date: 2022-09-30

Linear Parameter-Varying (LPV) systems are flexible mathematical models capable of representing Nonlinear (NL)/Time-Varying (TV) dynamical behaviors of complex physical systems (e.g., wafer scanners, car engines, chemical reactors), often encountered in engineering, via a linear structure. The LPV framework provides computationally efficient and robust approaches to synthesize digital controllers that can ensure desired operation of such systems - making it attractive to (i) high-tech mechatronic, (ii) automotive and (iii) chemical-process applications. Such a framework is important to meet with the increasing operational demands of systems in these industrial sectors and to realize future technological targets. However, recent studies have shown that, to fully exploit the potential of the LPV framework, a number of limiting factors of the underlying theory ask a for serious innovation, as currently it is not understood how to (1) automate exact and low-complexity LPV modeling of real-world applications and how to refine uncertain aspects of these models efficiently by the help of measured data, (2) incorporate control objectives directly into modeling and to develop model reduction approaches for control, and (3) how to see modeling & control synthesis as a unified, closed-loop system synthesis approach directly oriented for the underlying NL/TV system. Furthermore, due to the increasingly cyber-physical nature of applications, (4) control synthesis is needed in a plug & play fashion, where if sub-systems are modified or exchanged, then the control design and the model of the whole system are only incrementally updated. This project aims to surmount Challenges (1)-(4) by establishing an innovative revolution of the LPV framework supported by a software suite and extensive empirical studies on real-world industrial applications; with a potential to ensure a leading role of technological innovation of the EU in the high-impact industrial sectors (i)-(iii).

1 493 561 €

Start date: 2017-09-01, End date: 2022-08-31

Deep neural networks (DNNs) have led to dramatic improvements of the state-of-the-art for many important classification problems, such as object recognition from images or speech recognition from audio data. However, DNNs are also notoriously dependent on the tuning of their hyperparameters. Since their manual tuning is time-consuming and requires expert knowledge, recent years have seen the rise of Bayesian optimization methods for automating this task. While these methods have had substantial successes, their treatment of DNN performance as a black box poses fundamental limitations, allowing manual tuning to be more effective for large and computationally expensive data sets: humans can (1) exploit prior knowledge and extrapolate performance from data subsets, (2) monitor the DNN's internal weight optimization by stochastic gradient descent over time, and (3) reactively change hyperparameters at runtime. We therefore propose to model DNN performance beyond a blackbox level and to use these models to develop for the first time: 1. Next-generation Bayesian optimization methods that exploit data-driven priors to optimize performance orders of magnitude faster than currently possible; 2. Graybox Bayesian optimization methods that have access to -- and exploit -- performance and state information of algorithm runs over time; and 3. Hyperparameter control strategies that learn across different datasets to adapt hyperparameters reactively to the characteristics of any given situation. DNNs play into our project in two ways. First, in all our methods we will use (Bayesian) DNNs to model and exploit the large amounts of performance data we will collect on various datasets. Second, our application goal is to optimize and control DNN hyperparameters far better than human experts and to obtain: 4. Computationally inexpensive auto-tuned deep neural networks, even for large datasets, enabling the widespread use of deep learning by non-experts.

1 495 000 €

Start date: 2017-01-01, End date: 2021-12-31