Difference between revisions of "Main Page"

From mn.ifi.proposalfailures
Jump to: navigation, search
(Final words)
Line 8: Line 8:
 
=== Generic reviewer statements from NFR proposal rejections ===
 
=== Generic reviewer statements from NFR proposal rejections ===
 
''Note: the headlines below are just my effort to categorize the comments; this is not direct advice to write text addressing them into the respective section of a proposal!''
 
''Note: the headlines below are just my effort to categorize the comments; this is not direct advice to write text addressing them into the respective section of a proposal!''
 +
 +
All plain text, per bullet, is directly copy+pasted from a proposal review document.
  
 
'''TO BE REMOVED BEFORE DISSEMINATING WIDER:'''
 
'''TO BE REMOVED BEFORE DISSEMINATING WIDER:'''
  
'''Proposal names after the statements!!'''
+
'''Proposal names after the statements below!!'''
  
'''This is currently based on the following proposals: RAMIFY, OCARINA, CATRINA, IC-RAIN, DiReQT, MOROCCO, MOROCCOv2  (will be replaced with just a number)'''
+
'''This is currently based on 7 NFR proposals. RAMIFY, OCARINA, CATRINA, IC-RAIN, DiReQT, MOROCCO, MOROCCOv2  (NAMES TO BE REMOVED)'''
  
 
===== Excellence (the research idea itself, methods, ..) =====
 
===== Excellence (the research idea itself, methods, ..) =====
*The quantitative goals defined in the proposal seem realistic, but somewhat vaguely specified not very ambitious (20% improvement over state of the art does not seem a ground-breaking result, nor it is clear what metrics will be used to quantify the improvement). [RAMIFY]
+
*'''Unclear, not enough detail, not ground-breaking enough:'''
*The proposed research directions are clearly innovative, but the goals seem somewhat incremental rather than truly ground-breaking. [RAMIFY]
+
**The proposal also mentions analytic evaluations, but does not become specific there. [OCARINA]
*The proposal does not consider alternative Future Internet approaches. [OCARINA]
+
**In a few cases, the concrete results, e.g., simulation model, prototype etc. are not given in sufficient detail. [CATRINA]
*The proposal also mentions analytic evaluations, but does not become specific there. [OCARINA]
+
**The quantitative goals defined in the proposal seem realistic, but somewhat vaguely specified not very ambitious (20% improvement over state of the art does not seem a ground-breaking result, nor it is clear what metrics will be used to quantify the improvement). [RAMIFY]
*Research methods are not well defined. [CATRINA]
+
**The proposal can be strengthened though by better explaining its groundbreaking nature and generalisability. [DiReQT]
*In a few cases, the concrete results, e.g., simulation model, prototype etc. are not given in sufficient detail. [CATRINA]
+
**The proposed research directions are clearly innovative, but the goals seem somewhat incremental rather than truly ground-breaking. [RAMIFY]
*The scientific method limits itself to the experimental space, complementary analytic alternatives are not considered. Scientific achievements are expected more on experimental developments rather than on theoretical aspects. [IC-RAIN]
+
**On the one hand, the existence of a solid set of initial results reduces the risk that the project fails in achieving its planned goals. On the other hand, though, it makes the proposal somewhat incremental, reducing its creativity and originality. [RAMIFY]
*How the project will differentiate from other approaches is not described in detail. [IC-RAIN]
+
**Some more details on how some of the questions are going to be tackled might have been useful. [DiReQT]
*On the one hand, the existence of a solid set of initial results reduces the risk that the project fails in achieving its planned goals. On the other hand, though, it makes the proposal somewhat incremental, reducing its creativity and originality. [RAMIFY]
+
*'''Problems with methods:'''
*Some more details on how some of the questions are going to be tackled might have been useful. [DiReQT]
+
**The scientific method limits itself to the experimental space, complementary analytic alternatives are not considered. Scientific achievements are expected more on experimental developments rather than on theoretical aspects. [IC-RAIN]
*The proposal can be strengthened though by better explaining its groundbreaking nature and generalisability. [DiReQT]
+
**Research methods are not well defined. [CATRINA]
 +
*'''Related work:'''
 +
**The proposal does not consider alternative Future Internet approaches. [OCARINA]
 +
**How the project will differentiate from other approaches is not described in detail. [IC-RAIN]
  
 
==== Implementation (management) ====
 
==== Implementation (management) ====
*The number of requested PhD students might be at the upper edge for this work. [OCARINA]
+
*'''PhD student supervision:'''
*This project is an international cooperation (..). However, the proposal does not provide much detail on how this international consortium will be managed. [MOROCCOv2]
+
**The project manager main research so far has been in the area of [different from proposal theme]. He has a strong experience in [irrelevant], but less so in [proposal theme]. More details regarding the supervision of PhD students should have been given in the proposal. [MOROCCOv2]
*The project manager main research so far has been in the area of [different from proposal theme]. He has a strong experience in [irrelevant], but less so in [proposal theme]. More details regarding the supervision of PhD students should have been given in the proposal. [MOROCCOv2]
+
**It is, however, not clear who will advise the researcher to be hired during Phase 1. [IC-RAIN]
 
+
*'''International cooperation:'''
*No work package is dedicated to the integration of WP1 and WP2, the evaluation of the results, the project management and the dissemination, which can constitute a risk for the implementation. [MOROCCOv2]
+
**This project is an international cooperation (..). However, the proposal does not provide much detail on how this international consortium will be managed. [MOROCCOv2]
*The management structure is not sufficiently described. [MOROCCOv2]
+
**It is not clear how they will collaborate in the project, but there is a plan of exchange visits and the intention to submit joint publications. [IC-RAIN]
 
+
*'''Other management issues:'''
*The proposal is rather vague about the division of research tasks and responsibilities between team members. [MOROCCOv2]
+
**The number of requested PhD students might be at the upper edge for this work. [OCARINA]
*It is, however, not clear who will advise the researcher to be hired during Phase 1. [IC-RAIN]
+
**No work package is dedicated to the integration of WP1 and WP2, the evaluation of the results, the project management and the dissemination, which can constitute a risk for the implementation. [MOROCCOv2]
*It is not clear how they will collaborate in the project, but there is a plan of exchange visits and the intention to submit joint publications. [IC-RAIN]
+
**The management structure is not sufficiently described. [MOROCCOv2]
 +
**The proposal is rather vague about the division of research tasks and responsibilities between team members. [MOROCCOv2]
  
 
===== Impact  (dissemination, exploitation, ..) =====
 
===== Impact  (dissemination, exploitation, ..) =====
Line 45: Line 51:
 
*Outstanding impact on research community is not fully convincing in view of the reference list provided, since most of them are of the research team without a significant number of non self-cites. [IC-RAIN]
 
*Outstanding impact on research community is not fully convincing in view of the reference list provided, since most of them are of the research team without a significant number of non self-cites. [IC-RAIN]
 
*There is a lack of KPIs for dissemination activities. [IC-RAIN]
 
*There is a lack of KPIs for dissemination activities. [IC-RAIN]
 +
*Also, the potential exploitation of the research results is not addressed. [IC-RAIN]
 
*But there is a lack of information about contacts to standardisation channels. [IC-RAIN]
 
*But there is a lack of information about contacts to standardisation channels. [IC-RAIN]
*Also, the potential exploitation of the research results is not addressed. [IC-RAIN]
 
 
*Better explaining how this project will generate impact on Internet standards would strengthen the proposal. [DiReQT]
 
*Better explaining how this project will generate impact on Internet standards would strengthen the proposal. [DiReQT]
  
Line 52: Line 58:
 
''Here's some kind of FAQ... not that anyone really asked a question, let alone frequently  :-)  ...but the style seems to fit:''
 
''Here's some kind of FAQ... not that anyone really asked a question, let alone frequently  :-)  ...but the style seems to fit:''
  
* '''Isn't this useless, isn't it just about the quality of the idea? Will good proposals really fail only because of a minor detail?'''  I do believe that it's mainly about the idea quality. Reviewers who don't like an idea will find negative points in a proposal and vice versa, but... why give them negative points to begin with? ...
+
* '''Isn't this useless, isn't it just about the quality of the idea? Will good proposals really fail only because of a minor detail?'''  I do believe that it's mainly about the idea. Reviewers who don't like an idea will find negative points in a proposal, but... why give them negative points to begin with? Only top score proposals tend to get funded - every minimal reason to take a point away can kill your proposal.  
* and then they use minor details are usually just that – minor. Yet, trust me, I've had proposals that received top scores across the board except for one or two of the details above - and bam, no money. Certainly, a successful proposal is about having a good idea and a good plan to work on it, convincingly described! The list above is only a small element of the whole thing - but it's also easy to make this catalogue, and easy (and probably useful) to apply.
+
* '''Why not make this page more general, instead of focusing on NFR proposals only?'''  Papers and other proposals are different. In case of papers, good venues tend to give constructive feedback, and the next opportunity to use this feedback is right ahead - different from NFR, where there may not be another suitable call at all, or the next call is around a year later. European proposals are a different ballgame too: e.g., there is much more focus on the management side of things, proposals dedicate much more space to this kind of text - so, it doesn't seem very useful to me to make (yet another) catalogue of all the general things to consider for EC proposals.
* '''Why not make this page more general, instead of focusing on NFR proposals only?'''  Papers and other proposals are different. In case of papers, good venues tend to give constructive feedback, and the next opportunity to use this feedback is right ahead - different from NFR, where there may not be another suitable call at all, or the next call is around a year later. European proposals are a different ballgame too: there is so much more focus on the management side of things, proposals dedicate more space to this kind of text - so, it doesn't seem very useful to me to make (yet another) catalogue of all the general things to consider for EC proposals.
+
* '''Why this page, are you just a grumpy failure man?'''  No no, not grumpy. Failure, sure, I've had my share. I have experience both in failing and succeeding, in two national funding bodies and the EC. I just felt that this is a constructive (and easy) thing to do.
* '''Why this page, are you just a grumpy failure man?'''  No no, not grumpy. Failure, sure, I've had my share, but I've also had success. So this is based on experience, both failing and succeeding, in two national funding bodies and the EC. I just felt that this is a constructive (and easy) thing to do.
 
 
* '''Who are you anyway?'''  [http://www.welzl.at Michael].
 
* '''Who are you anyway?'''  [http://www.welzl.at Michael].
 
----
 
----
 
General [[Wikiinfo|Wiki info]]
 
General [[Wikiinfo|Wiki info]]

Revision as of 12:56, 16 December 2020

Welcome to the "learning from NFR proposal failures" Wiki !

What's the point of this?

This is a catalogue of statements from NFR proposal rejection letters that are generic, i.e. not about the specific idea being proposed. The idea is that this can become a checklist to avoid making unnecessary mistakes in future proposals. Please feel free to add statements to this list!

The list

Generic reviewer statements from NFR proposal rejections

Note: the headlines below are just my effort to categorize the comments; this is not direct advice to write text addressing them into the respective section of a proposal!

All plain text, per bullet, is directly copy+pasted from a proposal review document.

TO BE REMOVED BEFORE DISSEMINATING WIDER:

Proposal names after the statements below!!

This is currently based on 7 NFR proposals. RAMIFY, OCARINA, CATRINA, IC-RAIN, DiReQT, MOROCCO, MOROCCOv2 (NAMES TO BE REMOVED)

Excellence (the research idea itself, methods, ..)
  • Unclear, not enough detail, not ground-breaking enough:
    • The proposal also mentions analytic evaluations, but does not become specific there. [OCARINA]
    • In a few cases, the concrete results, e.g., simulation model, prototype etc. are not given in sufficient detail. [CATRINA]
    • The quantitative goals defined in the proposal seem realistic, but somewhat vaguely specified not very ambitious (20% improvement over state of the art does not seem a ground-breaking result, nor it is clear what metrics will be used to quantify the improvement). [RAMIFY]
    • The proposal can be strengthened though by better explaining its groundbreaking nature and generalisability. [DiReQT]
    • The proposed research directions are clearly innovative, but the goals seem somewhat incremental rather than truly ground-breaking. [RAMIFY]
    • On the one hand, the existence of a solid set of initial results reduces the risk that the project fails in achieving its planned goals. On the other hand, though, it makes the proposal somewhat incremental, reducing its creativity and originality. [RAMIFY]
    • Some more details on how some of the questions are going to be tackled might have been useful. [DiReQT]
  • Problems with methods:
    • The scientific method limits itself to the experimental space, complementary analytic alternatives are not considered. Scientific achievements are expected more on experimental developments rather than on theoretical aspects. [IC-RAIN]
    • Research methods are not well defined. [CATRINA]
  • Related work:
    • The proposal does not consider alternative Future Internet approaches. [OCARINA]
    • How the project will differentiate from other approaches is not described in detail. [IC-RAIN]

Implementation (management)

  • PhD student supervision:
    • The project manager main research so far has been in the area of [different from proposal theme]. He has a strong experience in [irrelevant], but less so in [proposal theme]. More details regarding the supervision of PhD students should have been given in the proposal. [MOROCCOv2]
    • It is, however, not clear who will advise the researcher to be hired during Phase 1. [IC-RAIN]
  • International cooperation:
    • This project is an international cooperation (..). However, the proposal does not provide much detail on how this international consortium will be managed. [MOROCCOv2]
    • It is not clear how they will collaborate in the project, but there is a plan of exchange visits and the intention to submit joint publications. [IC-RAIN]
  • Other management issues:
    • The number of requested PhD students might be at the upper edge for this work. [OCARINA]
    • No work package is dedicated to the integration of WP1 and WP2, the evaluation of the results, the project management and the dissemination, which can constitute a risk for the implementation. [MOROCCOv2]
    • The management structure is not sufficiently described. [MOROCCOv2]
    • The proposal is rather vague about the division of research tasks and responsibilities between team members. [MOROCCOv2]
Impact (dissemination, exploitation, ..)
  • Dissemination is rather standard, i.e. dissemination activities are mainly based on publications in journals and international conferences. [CATRINA]
  • The technological impact is described, which is however not the case for the impact on / importance for society. [IC-RAIN]
  • Outstanding impact on research community is not fully convincing in view of the reference list provided, since most of them are of the research team without a significant number of non self-cites. [IC-RAIN]
  • There is a lack of KPIs for dissemination activities. [IC-RAIN]
  • Also, the potential exploitation of the research results is not addressed. [IC-RAIN]
  • But there is a lack of information about contacts to standardisation channels. [IC-RAIN]
  • Better explaining how this project will generate impact on Internet standards would strengthen the proposal. [DiReQT]

Final words

Here's some kind of FAQ... not that anyone really asked a question, let alone frequently  :-) ...but the style seems to fit:

  • Isn't this useless, isn't it just about the quality of the idea? Will good proposals really fail only because of a minor detail? I do believe that it's mainly about the idea. Reviewers who don't like an idea will find negative points in a proposal, but... why give them negative points to begin with? Only top score proposals tend to get funded - every minimal reason to take a point away can kill your proposal.
  • Why not make this page more general, instead of focusing on NFR proposals only? Papers and other proposals are different. In case of papers, good venues tend to give constructive feedback, and the next opportunity to use this feedback is right ahead - different from NFR, where there may not be another suitable call at all, or the next call is around a year later. European proposals are a different ballgame too: e.g., there is much more focus on the management side of things, proposals dedicate much more space to this kind of text - so, it doesn't seem very useful to me to make (yet another) catalogue of all the general things to consider for EC proposals.
  • Why this page, are you just a grumpy failure man? No no, not grumpy. Failure, sure, I've had my share. I have experience both in failing and succeeding, in two national funding bodies and the EC. I just felt that this is a constructive (and easy) thing to do.
  • Who are you anyway? Michael.

General Wiki info