This represents a fundamental failure of CIHR’s primary mandate
July 6, 2016 9:30 PM Subscribe
The Canadian Institutes of Health Research (CIHR) (previously) administers operating grants to researchers in areas of basic, clinical, and population health research. Recent changes to the way CIHR reviews and awards grants have many scientists warning that "there is little hope that the best ideas and projects will be funded" and calling for a rollback of the changes.
In the wake of a decade of Conservative cost-cutting measures and following a 2011 review, a major restructuring of the way grants were awarded is taking place. 45% of the operating grant budget was earmarked for a new "Foundation" scheme, intended to fund entire labs for a longer period of time, rather than individual projects. The remaining 55% would go towards more traditional "Project" grants.
The results of the "pilot" Foundation round in 2015 raised alarm bells. The number and value of grants awarded to early and mid-career scientists fell, with one researcher extrapolating that the total amount of funding awarded to early career scientists (defined as those within 5 years of starting up a lab) would drop by 30%. Female researchers were awarded significantly fewer grants than their male counterparts, but CIHR says it is "too soon to say whether the discrepancy [...] is a sign of systemic bias".
Simultaneous to the restructuring of the granting schemes, CIHR overhauled its peer review process, causing issues with the recently completed Project competition. CIHR cancelled two of the previously twice yearly operating grant competitions, Fall 2014 and Fall 2015, and over 3000 scientists submitted 3800 grants to the most recent competition, the largest CIHR has overseen. For many researchers, this was their last chance to acquire funding, as dropping funding rates and the cancelled competitions had left many labs "running on fumes."
CIHR scrambled to find enough people to act as peer reviewers. Many scientists were assigned grants that were outside their field of expertise, and graduate students, administrative staff, and librarians reportedly received requests to review.
Under the older system, grants were sent to one of 50 panels that met face to face, discussed the applications, scored and ranked them. The new system assigned grants automatically to reviewers with similar expertise, who were supposed to participate in asynchronous online discussion of the grants under the direction of "virtual chairs". Each grant was supposed to have been reviewed and ranked by a minimum of 5 peers, which was reduced to 4 following a larger than expected number of grant submissions. Many reviewers failed to submit their reviews before the 3-day online discussion period, with ~25%-30% of reviews missing on the first day of discussion, and 15% still missing by the time discussion ended.
An open letter to the Minister of Health, Jane Philpott, about the problems with CIHR's changes was signed by over 1200 professors and lab heads, and today she ordered CIHR officials to meet with scientists.
In the wake of a decade of Conservative cost-cutting measures and following a 2011 review, a major restructuring of the way grants were awarded is taking place. 45% of the operating grant budget was earmarked for a new "Foundation" scheme, intended to fund entire labs for a longer period of time, rather than individual projects. The remaining 55% would go towards more traditional "Project" grants.
The results of the "pilot" Foundation round in 2015 raised alarm bells. The number and value of grants awarded to early and mid-career scientists fell, with one researcher extrapolating that the total amount of funding awarded to early career scientists (defined as those within 5 years of starting up a lab) would drop by 30%. Female researchers were awarded significantly fewer grants than their male counterparts, but CIHR says it is "too soon to say whether the discrepancy [...] is a sign of systemic bias".
Simultaneous to the restructuring of the granting schemes, CIHR overhauled its peer review process, causing issues with the recently completed Project competition. CIHR cancelled two of the previously twice yearly operating grant competitions, Fall 2014 and Fall 2015, and over 3000 scientists submitted 3800 grants to the most recent competition, the largest CIHR has overseen. For many researchers, this was their last chance to acquire funding, as dropping funding rates and the cancelled competitions had left many labs "running on fumes."
CIHR scrambled to find enough people to act as peer reviewers. Many scientists were assigned grants that were outside their field of expertise, and graduate students, administrative staff, and librarians reportedly received requests to review.
Under the older system, grants were sent to one of 50 panels that met face to face, discussed the applications, scored and ranked them. The new system assigned grants automatically to reviewers with similar expertise, who were supposed to participate in asynchronous online discussion of the grants under the direction of "virtual chairs". Each grant was supposed to have been reviewed and ranked by a minimum of 5 peers, which was reduced to 4 following a larger than expected number of grant submissions. Many reviewers failed to submit their reviews before the 3-day online discussion period, with ~25%-30% of reviews missing on the first day of discussion, and 15% still missing by the time discussion ended.
An open letter to the Minister of Health, Jane Philpott, about the problems with CIHR's changes was signed by over 1200 professors and lab heads, and today she ordered CIHR officials to meet with scientists.
As a somewhat interdisciplinary researcher, I'm quite open to silo busting when it comes to grant allocation. That said, this scheme is fucking crazy.
posted by ethansr at 9:48 AM on July 7, 2016
posted by ethansr at 9:48 AM on July 7, 2016
This kind of 'silo busting' is something that could only conceivably make sense in boom years with good funding that would make good innovation in structures have more impact and soften the blow of bad structures. Even if the ideas were all great, this would still be a horrifically idiotic idea just for the deeply vulnerable state Canadian science is currently in.
Canadian science desperately needs cash, hard income, to allow what survived way too many years of Conservative leadership to recover - not more meddling.
posted by Blasdelb at 10:11 AM on July 7, 2016
Canadian science desperately needs cash, hard income, to allow what survived way too many years of Conservative leadership to recover - not more meddling.
posted by Blasdelb at 10:11 AM on July 7, 2016
This isn't more meddling, per se. It began under the Conservative government, several years ago.
That said, by all accounts this is another area where simply reverting to the former status quo would be a huge improvement.
posted by Orange Pamplemousse at 10:20 AM on July 7, 2016 [1 favorite]
That said, by all accounts this is another area where simply reverting to the former status quo would be a huge improvement.
posted by Orange Pamplemousse at 10:20 AM on July 7, 2016 [1 favorite]
More money (or maybe I should say, fewer cuts) would certainly have buffered some of the effects. No cancelled competitions would have meant fewer applications so less pressure on the system and I'm assuming the issues with misallocated grants were a result of the record number of applications.
Moving from face to face meetings to online, asynchronous review (a move I am broadly in favour of--in 2016 there's no reason to fly 50 committees' worth of people to Ottawa) without seemingly any plan to adjust the process to account for that, or any direction to the virtual chairs on how to guide a productive discussion, was a mistake. This would have still been a boondoggle.
posted by quaking fajita at 12:16 PM on July 7, 2016
Moving from face to face meetings to online, asynchronous review (a move I am broadly in favour of--in 2016 there's no reason to fly 50 committees' worth of people to Ottawa) without seemingly any plan to adjust the process to account for that, or any direction to the virtual chairs on how to guide a productive discussion, was a mistake. This would have still been a boondoggle.
posted by quaking fajita at 12:16 PM on July 7, 2016
I can't seem to find a cite/link, but iirc, CIHR paid Elsevier $1.7M (sure, Canadian dollars, but still) for their reviewer matching algorithm - which didn't work.
Also, these groups were assigned grants seemingly haphazardly; rankings were between each group's stack of mismatched grants rather than against other similar grants.
Yes, flying reviewers to Ottawa is a big chunk of money, but reviewers typically work really really hard while they're there. Also, no distractions.
posted by porpoise at 1:34 PM on July 7, 2016 [1 favorite]
Also, these groups were assigned grants seemingly haphazardly; rankings were between each group's stack of mismatched grants rather than against other similar grants.
Yes, flying reviewers to Ottawa is a big chunk of money, but reviewers typically work really really hard while they're there. Also, no distractions.
posted by porpoise at 1:34 PM on July 7, 2016 [1 favorite]
Andre Picard Globe & Mail column today: Canada's research regime in need of major surgery
"The real tragedy is that this debacle was largely predictable. After all, the changes were not studied and implemented scientifically; rather, CIHR blindly and foolishly embraced unproven, radical reforms even after being warned of potential problems.
Dr. Beaudet has blamed most of the problems on lack of money, on there being too little money to go around. That’s actually an important but separate issue. So, too, is the penchant for subsidizing business using science research monies. Both these issues will be tackled by a blue-ribbon panel headed by Dr. David Naylor."
posted by chapps at 4:41 PM on July 12, 2016
"The real tragedy is that this debacle was largely predictable. After all, the changes were not studied and implemented scientifically; rather, CIHR blindly and foolishly embraced unproven, radical reforms even after being warned of potential problems.
Dr. Beaudet has blamed most of the problems on lack of money, on there being too little money to go around. That’s actually an important but separate issue. So, too, is the penchant for subsidizing business using science research monies. Both these issues will be tackled by a blue-ribbon panel headed by Dr. David Naylor."
posted by chapps at 4:41 PM on July 12, 2016
« Older When Black Lives Stop Mattering | meanwhile in remix art, video and the radio star... Newer »
This thread has been archived and is closed to new comments
posted by hydrobatidae at 7:25 AM on July 7, 2016 [5 favorites]