Secrets of the PoC proposal evaluation process

18 January 2019
Cover image of Secrets of the PoC proposal evaluation process

Proof of Concept grants, worth up to €150,000 each, are open to ERC grant holders who want to explore the commercial or societal potential of their work.

How did you get into this line of work? And why?

When I started working in the area of technology transfer I was increasingly invited to judge business idea competitions. I also worked as a consultant on the submission of grant proposals under the EU's 7th Framework Programmes for Research and Innovation. So for me it was a natural progression to work on evaluating EU grant proposals and a way to transfer the knowledge I had developed.

As for the ERC, eventually, due to my experience in technology transfer, I was called by the ERC Scientific Council to participate in the evaluation of the PoC proposals - where the project aims to move towards the production of a product, service or innovation.

It's always interesting working on PoCs. It's very exciting to see new ideas as to how science can make a difference to our lives. Working as an evaluator of grants that involve technology transfer and are given by an international body also helps with my day job. It is perceived in a positive way by my university, and once a year when all the evaluators meet for a general briefing by the ERC, I get to meet some extremely interesting people.

Does it take a lot of time and effort?

Each proposal has about 8 pages, and can be quite dense so it can take about a day per proposal. On average I get about 12 to 13 each time I am asked to participate.

The most I ever had was 36 proposals to evaluate. But since the expert pool widened, the average number fell greatly. Also, occasionally I have to abstain from reviewing some proposals due to a potential conflict of interest, e.g. I know the applicant personally.

Once the proposals have all been evaluated, I do a cross check to ensure fair marking.

Each PoC project is evaluated by at least 5 evaluators. Each evaluator uses three main criteria: a) Excellence in Innovation potential; b) Impact and; c) Quality and efficiency of the implementation plan.
Each project must achieve a pass mark in each criteria from at least 3 evaluators.

What is your overall aim as an evaluator?

In general I aim to judge the ability of the applicant to develop the project. I need to see that the researcher is serious about the project and details how they envisage its final output will get to market. The researcher doesn’t have to have everything figured out but they do need to indicate that they have thought about the main issues.

I don't judge the scientific potential of the proposal – this is assumed to have been established, as the applicant already has an ERC grant - but its innovation potential.

What do you see as a "good application"?

First I must say that nothing is set in stone, the subject matter of each application can be quite different so there is no "one size fits all" approach – except for the general need to state the problem, the solution proposed and the beneficiaries of the solution. However I do see a number of aspects that tend to recur in successful applications.

These include detailing the economic or social benefits of the project – how the idea will help a specific sector or group of people with a specific problem. Is it faster, is it quicker? Is it more innovative than existing solutions? Other elements include estimating the related financial profit or inherent value, a testing phase, and an exploration of intellectual property right concerns. Lack of a competitive analysis poses a major risk to the success of the project. As does lack of risk analysis!

Market analysis is also essential. Another aspect is market validation. It's obvious to me, and a real positive, when a researcher has already asked for market feedback on his or her proposal - from for example a hospital or institution. It indicates that the researcher is quite serious about their proposal and that there are actual potential users of its output.

A "go to market" roadmap is important – how they are going to get the technology to the market. It might include the initial steps in the project, a business plan and details of ideas or plans concerning licencing, or spin off agreements.

I also think it's important to have a balanced team in place to carry out the PoC project, for example including an expert in commercialisation. The researcher can't rely purely on the research team built up for his or her ERC grant. Ideally the team should have clear structure and an explanation of how it will coordinate.

What applications fill you with dread?

Applications in which it is clear that the researcher has made a breakthrough but they do not know how to apply it are really difficult to evaluate.

Unfortunately some applications seem to be submitted in an effort to get more funding for basic research, i.e. to generate more research data. This is not the purpose of the PoC. We need to know the impact for society in general, not just how it will generate another research paper, employ an additional PHD student or expand on the original project.

If an applicant's proposal initially fails, revised attempts are welcome, but when an applicant submits the exact same proposal a second time, evaluators really don't appreciate it – it just indicates that their feedback on the first attempt was not taken into account.

Sometimes the proposal is written in very standard "proposal" language or text, for example when it is ghost-written by a consulting firm or with the help of a university that generates a lot of proposals. These are not specific to the researcher's proposal and often sacrifice substance for style. Of course the researcher might need some support with business language – terms such as benchmarking, framework etc – but they have to make their application personal.

What differences do you see between the applications from the different domains, if any?

For applications in the ICT domain, I generally like to see that the product is at a prototype stage.

Projects in the medical field often have a very long term horizon and won't get to market in one year. Regulatory approval is often required and can take time. Instead such projects may aim to raise the interest of a pharmaceutical company.

For projects in the Humanities field a common problem is that the researcher does not indicate how he or she will translate the project into something that is sustainable in the medium to long term (instead of for example only having a one off event as a project goal). Materials used for a showcase event could perhaps be transformed into digital content and used for training purposes over a longer period (if applicable in the research domain).

The researcher could for example identify sponsors or organisations already working in their respective field to help pay for a training platform. I would advise the researcher to look in particular to their host institution for advice and support in these areas.

Researchers sometimes subcontract aspects of the PoC project. What are your feelings about that?

There are not that many scientists that are good in commercial application – if they were, they would have gone into the business field! While some professors feel they are so smart that they can do it all, others don't have the time or the motivation – so they subcontract certain aspects of the PoC project.

The amount of subcontracting which is acceptable really depends on the type of proposal. Some items that I have seen successfully subcontracted include IPR strategy, market research and testing/validation.

For more information, and how to apply, visit our Proof of Concept page.
The 2019 application deadlines are: 22 January 2019, 25 April 2019, and 19 September 2019

Maria.JPG

Maria Oliveira is Director of U.Porto Innovation, and was invited fellow of the Office of Technology Development (OTD) of Boston University. She has more than 15 years’ experience working in technology transfer and innovation management, in Portugal and abroad, and in 2012 she was appointed as the Executive Director of UTEN Portugal, a component of the UT Austin|Portugal Program. She is invited regularly to evaluate Proof of Concept proposals and technology commercialisation plans for the European Commission, The EIT Health and national and international panels for several Universities.