Difference between revisions of "Guidelines"

From mn.ifi.proposalfailures
Jump to: navigation, search
(Created page with "===blabla=== ''blabla'' blabla ==Excellence (the research idea itself, methods, ..)== === '''Unclear, not enough detail, not ground-breaking enough:''' === **The proposal al...")
 
(Clarity, being ground-breaking, ..:)
 
(11 intermediate revisions by the same user not shown)
Line 1: Line 1:
===blabla===
+
'''These guidelines are derived as ''subjective'' interpretations of the reviewer statements on [[Main Page|this page]].'''
''blabla''
 
 
 
blabla
 
 
==Excellence (the research idea itself, methods, ..)==
 
==Excellence (the research idea itself, methods, ..)==
  
=== '''Unclear, not enough detail, not ground-breaking enough:''' ===
+
=== '''Clarity, being ground-breaking, ..:''' ===
**The proposal also mentions analytic evaluations, but does not become specific there.
+
*Project outputs: be specific about analytic evaluations.
**In a few cases, the concrete results, e.g., simulation model, prototype etc. are not given in sufficient detail.
+
*Describe concrete project results (e.g., simulation model, prototype etc.) in sufficient detail. Benefits shown as a vision are not concrete enough.
**The quantitative goals defined in the proposal seem realistic, but somewhat vaguely specified not very ambitious (20% improvement over state of the art does not seem a ground-breaking result, nor it is clear what metrics will be used to quantify the improvement).
+
*Be clear about goals. E.g., when describing a % improvement over the state of the art, clarify which metrics will be used to quantify the improvement.
**The proposal can be strengthened though by better explaining its groundbreaking nature and generalisability.
+
*Be ambitious. E.g., 20% better than the SoA is not enough. Explain the ground-breaking nature of the research and its generalisability.
**The proposed research directions are clearly innovative, but the goals seem somewhat incremental rather than truly ground-breaking.
+
*Avoid looking incremental - careful with showing initial results.
**On the one hand, the existence of a solid set of initial results reduces the risk that the project fails in achieving its planned goals. On the other hand, though, it makes the proposal somewhat incremental, reducing its creativity and originality.
+
*Be clear about the research plan: how will the research questions be tackled?
**Some more details on how some of the questions are going to be tackled might have been useful.
+
*Avoid looking too narrow: there should be a reasonably large research community interested in your work.
**Limited novelty due to the relatively small size of the community doing research around [..].
+
*Avoid looking too broad: research objectives should not seem like they could each be a project in its own right.
**A lot of repetitions in the text because the research questions are somehow embedded in the objectives.
+
*Avoid redundancy in the text - e.g., don't embed the research questions in the objectives.
**The scope of the project is very broad. Probably, each of these topics is a project on its own.
+
*Think twice about which field to address (try to aim for new fields, if possible).
 +
*Make sure to motivate and explain all choices well.
  
=== '''Problems with methods:''' ===
+
=== '''Methods:''' ===
**The scientific method limits itself to the experimental space, complementary analytic alternatives are not considered. Scientific achievements are expected more on experimental developments rather than on theoretical aspects.
+
*If the focus is on experiments, 1) provide sufficient details on experimental evaluation, and 2) consider adding complementary analytic methods to also attain more theory-oriented scientific achievements.
**Research methods are not well defined.
+
*Make sure that quantitative research methods are described for all the research activities related to the topics mentioned.
**In some of the research activities related to the topics mentioned, quantitative research methods are missing.
 
**Details on experimental evaluation are not sufficient.
 
  
 
=== '''Related work:''' ===
 
=== '''Related work:''' ===
**The proposal does not consider alternative Future Internet approaches.
+
*Provide a comparison with alternative approaches; clarify how the project will differentiate from them. Highlight originality.
**How the project will differentiate from other approaches is not described in detail.
+
 
**No adequate comparison to existing approaches.
+
=== '''Other matters:''' ===
 +
* Explicitly address ethics, safety and gender issues.
 +
 
 
==Implementation (management)==
 
==Implementation (management)==
  
 
=== '''PhD student supervision:''' ===
 
=== '''PhD student supervision:''' ===
**The project manager main research so far has been in the area of [different from proposal theme]. He has a strong experience in [..], but less so in [proposal theme]. More details regarding the supervision of PhD students should have been given in the proposal.
+
*State who will advise the Ph.D. students.
**It is, however, not clear who will advise the researcher to be hired during Phase 1.
+
*Give enough details regarding the supervision of PhD students, especially when the project manager's own research has previously had a different focus.
  
 
=== '''International cooperation:''' ===
 
=== '''International cooperation:''' ===
**This project is an international cooperation (..). However, the proposal does not provide much detail on how this international consortium will be managed.
+
*When cooperating with international partners, explain how the international consortium will be managed.
**It is not clear how they will collaborate in the project, but there is a plan of exchange visits and the intention to submit joint publications.
+
*When cooperating with industry, engage with other stakeholders from the same industry beyond only one major company.
**Most expertise [on security] lays in the external collaborators – this is a risk as the project investigations focus on this topic.
+
*Clarify how collaboration will work in the project (a plan of exchange visits, and stating the intention to submit joint publications is not enough).
 +
*If external collaborators primarily have the expertise on a topic, this is a risk for the project. Avoid this, or state it as a risk and provide a mitigation plan.
  
 
=== '''Other management issues:''' ===
 
=== '''Other management issues:''' ===
**The number of requested PhD students might be at the upper edge for this work.
+
*Consider including (a) work package(s) for integration, result evaluation, project management and dissemination.
**No work package is dedicated to the integration of WP1 and WP2, the evaluation of the results, the project management and the dissemination, which can constitute a risk for the implementation.
+
*Describe the management structure in sufficient detail.
**The management structure is not sufficiently described.
+
*Describe task dependencies, timing and deliverables.
**The proposal is rather vague about the division of research tasks and responsibilities between team members.
+
*Clarify the division of research tasks and responsibilities between team members.
**The risk management plan appears to be quite generic and the mitigation actions do not appear to be so refined.
+
*Describe WP results not only in terms of of publications but in terms of concrete achievements and developments.
**Work package structure highly simplified in a waterfall arrangement. The timeline and dependencies are not well identified.
+
*Avoid looking too generic with the risk management plan (still, include general risks like failing to recruit the right staff, and delay in hiring people); mitigation actions must be detailed and clear.
 +
*A "waterfall arrangement" work package structure may look too simple. Clarify the timeline and dependencies.
 +
*Search the proposal for "he", "his" and "him" before submitting to say she/he instead, wherever appropriate.
 +
 
 
==Impact (dissemination, exploitation, ..)==
 
==Impact (dissemination, exploitation, ..)==
*Dissemination is rather standard, i.e. dissemination activities are mainly based on publications in journals and international conferences.
+
*If dissemination activities only describe publications in journals and international conferences, consider adding something more. Maybe what the EC calls "communication", i.e. talking to the public, via other media?
*The technological impact is described, which is however not the case for the impact on / importance for society.
+
*In addition to the technological impact, describe the impact on / importance for society at large and/or industry.
*Outstanding impact on research community is not fully convincing in view of the reference list provided, since most of them are of the research team without a significant number of non self-cites.
+
*To convince reviewers that there will be a significant impact on the research community, avoid too many self-cites in the reference list.
*There is a lack of KPIs for dissemination activities.
+
*Provide KPIs for dissemination activities. Plan for enough papers!
*Also, the potential exploitation of the research results is not addressed.
+
*When working across multiple fields, ensure that the publication plans evenly spreads across all of them.
*But there is a lack of information about contacts to standardisation channels.
+
*Address the potential exploitation of the research results.
*Better explaining how this project will generate impact on Internet standards would strengthen the proposal.
+
*In case of doing standardisation, provide information about contacts to standardisation channels, and explain in detail how an impact in standardisation will be achieved.
*Ethics, safety and gender issues are not explicitly detailed in the proposal.
+
*Explicitly address ethics, safety and gender issues.
*The whole impact description appears to be detailed but quite generic.
+
*Explicitly address societal impact.
 +
*Avoid making the impact description too generic looking. It must be specific to the project.
 +
*Avoid road-blocks like requiring a systemic change across different parts of the value chain before the proposed system can become effective.

Latest revision as of 13:56, 30 June 2021

These guidelines are derived as subjective interpretations of the reviewer statements on this page.

Excellence (the research idea itself, methods, ..)

Clarity, being ground-breaking, ..:

  • Project outputs: be specific about analytic evaluations.
  • Describe concrete project results (e.g., simulation model, prototype etc.) in sufficient detail. Benefits shown as a vision are not concrete enough.
  • Be clear about goals. E.g., when describing a % improvement over the state of the art, clarify which metrics will be used to quantify the improvement.
  • Be ambitious. E.g., 20% better than the SoA is not enough. Explain the ground-breaking nature of the research and its generalisability.
  • Avoid looking incremental - careful with showing initial results.
  • Be clear about the research plan: how will the research questions be tackled?
  • Avoid looking too narrow: there should be a reasonably large research community interested in your work.
  • Avoid looking too broad: research objectives should not seem like they could each be a project in its own right.
  • Avoid redundancy in the text - e.g., don't embed the research questions in the objectives.
  • Think twice about which field to address (try to aim for new fields, if possible).
  • Make sure to motivate and explain all choices well.

Methods:

  • If the focus is on experiments, 1) provide sufficient details on experimental evaluation, and 2) consider adding complementary analytic methods to also attain more theory-oriented scientific achievements.
  • Make sure that quantitative research methods are described for all the research activities related to the topics mentioned.

Related work:

  • Provide a comparison with alternative approaches; clarify how the project will differentiate from them. Highlight originality.

Other matters:

  • Explicitly address ethics, safety and gender issues.

Implementation (management)

PhD student supervision:

  • State who will advise the Ph.D. students.
  • Give enough details regarding the supervision of PhD students, especially when the project manager's own research has previously had a different focus.

International cooperation:

  • When cooperating with international partners, explain how the international consortium will be managed.
  • When cooperating with industry, engage with other stakeholders from the same industry beyond only one major company.
  • Clarify how collaboration will work in the project (a plan of exchange visits, and stating the intention to submit joint publications is not enough).
  • If external collaborators primarily have the expertise on a topic, this is a risk for the project. Avoid this, or state it as a risk and provide a mitigation plan.

Other management issues:

  • Consider including (a) work package(s) for integration, result evaluation, project management and dissemination.
  • Describe the management structure in sufficient detail.
  • Describe task dependencies, timing and deliverables.
  • Clarify the division of research tasks and responsibilities between team members.
  • Describe WP results not only in terms of of publications but in terms of concrete achievements and developments.
  • Avoid looking too generic with the risk management plan (still, include general risks like failing to recruit the right staff, and delay in hiring people); mitigation actions must be detailed and clear.
  • A "waterfall arrangement" work package structure may look too simple. Clarify the timeline and dependencies.
  • Search the proposal for "he", "his" and "him" before submitting to say she/he instead, wherever appropriate.

Impact (dissemination, exploitation, ..)

  • If dissemination activities only describe publications in journals and international conferences, consider adding something more. Maybe what the EC calls "communication", i.e. talking to the public, via other media?
  • In addition to the technological impact, describe the impact on / importance for society at large and/or industry.
  • To convince reviewers that there will be a significant impact on the research community, avoid too many self-cites in the reference list.
  • Provide KPIs for dissemination activities. Plan for enough papers!
  • When working across multiple fields, ensure that the publication plans evenly spreads across all of them.
  • Address the potential exploitation of the research results.
  • In case of doing standardisation, provide information about contacts to standardisation channels, and explain in detail how an impact in standardisation will be achieved.
  • Explicitly address ethics, safety and gender issues.
  • Explicitly address societal impact.
  • Avoid making the impact description too generic looking. It must be specific to the project.
  • Avoid road-blocks like requiring a systemic change across different parts of the value chain before the proposed system can become effective.