Thank you for giving us early information about the upcoming NFRF Transformation competition and requesting feedback. I have the following suggestions on the framework:
-
Having both “high risk” and “feasibility” be review criteria seems contradictory and is incredibly confusing for both applicants and reviewers. If by “high risk”, you actually mean “novelty of approach”, please call the criterion “novelty of approach” or “innovation”. These are not synonyms for “high risk”.
-
Make the full peer review manual for all stages available before NOI phase. It is unfair to ask applicants to plan a proposal when they do not know how it will be reviewed.
-
Reduce two phases of adjudication (Phase 1 and 2) to one. Having non-experts select the final recommendations will not lead to the best use of the significant public funds here. It will also add additional administrative costs and delays in the review procedure.
-
It is unclear where applicants are supposed to supply budget and budget justification, and what level of detail is expected. This should not take away from space elsewhere in the proposal.
-
Have all panel discussion in person. Do not attempt virtual online peer review. As CIHR found, it does not work.
-
Provide applicants with anonymized text of the written reviews of their applications, and textual notes summarizing the discussion at the panel. This ensures that effort put into these applications are not wasted, and that the competition itself provides constructive feedback to create better research.
-
Do not use the Canadian Common CV for this competition. The Canadian Common CV adds substantial time and frustration for applicants, makes it more difficult to get indigenous or international team members, and is not useful for reviewers. Instead I would suggest using the biosketch adopted by the CIHR Institute of Indigenous Health Research or the NIH Biosketch to avoid these problems.
-
Require a resource sharing plan in the full application that indicates how the applicants intend to share things created with this funding with the wider community. This would include data, code, and knowledge translation. For funded applications, make this available publicly so others know what to expect.
-
Do not put a limitation on the number of pages for references. Making applicants squeeze references into a smaller space and more compact format does not make things easier for reviewers. Quite the contrary.
-
Do not allow any other appendices for the application such as support letters, publications, or quotations for cost of goods and services. These create an arms race among applicants and are burdensome for reviewers.
-
Explicitly mention that research outputs are not limited to journal articles but also include other things such as preprints, databases, software, and books.
-
Clearly define “early career researcher” as this term is used differently in different fields.
-
Require full and immediate Open Access to all publications from the NFRF Transformation scheme. It is only right that the Canadian public should get immediate access without further payment to any publications produced by this special program.
-
As the final stage appears to only be a recommendation for funding, please specify who will make the final decision.
Please let me know if you would like to discuss any of these points further.