Dataset Information

Identification and comparison of key criteria of feedback of funding decisions: mixed-methods analysis of funder and applicant perspectives.

ABSTRACT:

Objective

This study investigated the content, quality and value of feedback given to applicants who applied to one of four research programmes in the UK funded (or jointly funded) by the National Institute for Health Research (NIHR).

Methods

A mixed-method phased approach was conducted using document analysis and an online survey. Phase 1 examined 114 NIHR applicant feedback documents comprised written feedback from funding committees and external peer-reviewers and a conceptual framework of the key components of feedback was developed using content analysis. Phase 2 was an online survey completed by 113 NIHR applicants. Frequencies of responses to closed questions were calculated. Perceptions of quality and value of feedback were identified using content analysis of open-text responses.

Results

In phase 1, a conceptual framework was developed with seven overarching categories: 'Study structure and quality'; 'Team and infrastructure'; 'Acceptability to patients and professionals'; 'Study justification and design'; 'Risks and contingencies'; 'Outputs'; 'Value for money'. A higher frequency of feedback was provided at stage 2 and for successful applications across the majority of components. In phase 2, frequency data showed that opinion on feedback was dependent on funding outcome. Content analysis revealed four main themes: 'Committee transparency'; 'Content validity and reliability'; 'Additional support'; Recognition of effort and constraints'.

Conclusions

This study provides key insights and understanding into the quality, content and value of feedback provided to NIHR applicants. The study identified key areas for improvement that can arise in NIHR funding applications, as well as in the feedback given to applicants that are applicable to other funding organisations. These findings could be used to inform funding application guidance documents to help researchers strengthen their applications and used more widely by other funders to inform their feedback processes.

SUBMITTER: Fackrell K

PROVIDER: S-EPMC8451298 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Importance:It has been nearly 25 years since medical students were queried regarding their perspectives on otolaryngology-head and neck surgery (OHNS) residency selection. Understanding this viewpoint is critical to improving the current application process. Objective:To evaluate the perceptions of 2016 OHNS residency applicants regarding the application process and offer suggestions for reform. Design, Setting, and Participants:In this cross-sectional study of anonymous online survey data, a 14-question survey was designed based on resources obtained from a computerized PubMed, Ovid, and GoogleScholar database search of the English language from January 1, 1990, through December 31, 2015, was conducted using the following search terms: (medical student OR applicant) AND (application OR match) AND otolaryngology. The survey was administered to 2016 OHNS residency applicants to examine 4 primary areas: current attitudes toward the match, effect of the new Otolaryngology Program Directors Organization personal statement mandate, sources of advice and information, and suggestions for improvement. In January 2016, an email was sent to 100 program directors asking them to distribute the survey to current OHNS applicants at their institution. One follow-up reminder email was sent in February 2016. A link to the survey was posted on the Otomatch.com homepage on January 28, 2016, with the last response received on March 28, 2016. Main Outcome and Measures:Survey responses regarding the residency application process. Results:A total of 150 of 370 residency applicants (40.5%) responded to the survey. Of these, 125 respondents (90.6%) noted applying to programs in which they had no specific interest simply to improve their chances of matching. Applicants intended to apply to more programs than they actually did (63.6 vs 60.8; r?=?0.19; 95% CI, -0.03 to 0.40). Program directors advised fewer applications than other sources; however, 58 respondents (38.7%) did not receive advice from a program director. A total of 121 respondents (80.7%) found online program information to be insufficient. Finally, 90 of 140 respondents (64.3%) noted that they would agree to a hard cap on applications, among other suggestions for improvement. Conclusions and Relevance:Several main themes emerged from the data, providing a foundation for process improvement opportunities: careful consideration to applicant mentorship, including peers; uniform set of criteria for residency program websites; and investigating alternative match platforms, which may allow hard caps, flagging programs of higher interest, or wave application cycles. Overall, the otolaryngology applicant provides a unique perspective regarding the current state of the match and potential opportunities for system-wide improvement.

Project description:BACKGROUND:DREAM-GLOBAL (Diagnosing hypertension-Engaging Action and Management in Getting Lower Blood Pressure in Indigenous and low- and middle-income countries) studied a SMS text messaging-based system for blood pressure measurement and hypertension management in Canadian Aboriginal and Tanzanian communities. The use of SMS text messages is an emerging point of interest in global health care initiatives because of their scalability, customizability, transferability, and cost-effectiveness. OBJECTIVE:The study aim was to assess the effect on the difference in blood pressure reduction of active hypertension management messages or passive health behavior messages. The system was designed to be implemented in remote areas with wireless availability. This study described the implementation and evaluation of technical components, including quantitative data from the transmission of blood pressure measurements and qualitative data collected on the operational aspects of the system from participants, health care providers, and community leadership. METHODS:The study was implemented in six remote Indigenous Canadian and two rural Tanzanian communities. Blood pressure readings were taken by a community health worker and transmitted to a mobile phone via Bluetooth, then by wireless to a programmed central server. From the server, the readings were sent to the participant's own phone as well. Participants also received biweekly tailored SMS text messages on their phones. Quantitative data on blood pressure reading transmissions were collected from the study central server. Qualitative data were collected by surveys, focus groups, and key informant interviews of participants, health care providers, and health leadership. RESULTS:In Canada, between February 2014 and February 2017, 2818 blood pressure readings from 243 patients were transmitted to the central server. In Tanzania, between October 2014 and August 2015, 1165 readings from 130 patients were transmitted to the central server. The use of Bluetooth technology enabled the secure, reliable transmission of information from participants to their health care provider. The timing and frequency were satisfactory to 137 of 187 (73.2%) of participants, supporting the process of sending weekly messages twice on Mondays and Thursdays at 11 am. A total of 97.0% (164/169) of the participants surveyed said they would recommend participation in the DREAM-GLOBAL program to a friend or relative with hypertension. CONCLUSIONS:In remote communities, the DREAM-GLOBAL study helped local health care providers deliver a blood pressure management program that enabled patients and community workers to feel connected. The technical components of the study were implemented as planned, and patients felt supported in their management through the SMS text messaging and mobile health program. Technological issues were solved with troubleshooting. Overall, the technical aspects of this research program enhanced clinical care and study evaluation and were well received by participants, health care workers, and community leadership. TRIAL REGISTRATION:Clinicaltrials.gov NCT02111226; https://clinicaltrials.gov/ct2/show/NCT02111226.

Project description:BackgroundIatrogenic hypoglycemia is a common occurrence among hospitalized patients and is associated with poor clinical outcomes and increased mortality. Clinical decision support systems can be used to reduce the incidence of this potentially avoidable adverse event.ObjectiveThis study aims to determine the desired features and functionality of a real-time informatics alert to prevent iatrogenic hypoglycemia in a hospital setting.MethodsUsing the Agency for Healthcare Research and Quality Five Rights of Effective Clinical Decision Support Framework, we conducted a mixed methods study using an electronic survey and focus group sessions of hospital-based providers. The goal was to elicit stakeholder input to inform the future development of a real-time informatics alert to target iatrogenic hypoglycemia. In addition to perceptions about the importance of the problem and existing barriers, we sought input regarding the content, format, channel, timing, and recipient for the alert (ie, the Five Rights). Thematic analysis of focus group sessions was conducted using deductive and inductive approaches.ResultsA 21-item electronic survey was completed by 102 inpatient-based providers, followed by 2 focus group sessions (6 providers per session). Respondents universally agreed or strongly agreed that inpatient iatrogenic hypoglycemia is an important problem that can be addressed with an informatics alert. Stakeholders expressed a preference for an alert that is nonintrusive, accurate, communicated in near real time to the ordering provider, and provides actionable treatment recommendations. Several electronic medical record tools, including alert indicators in the patient header, glucose management report, and laboratory results section, were deemed acceptable formats for consideration. Concerns regarding alert fatigue were prevalent among both survey respondents and focus group participants.ConclusionsThe design preferences identified in this study will provide the framework needed for an informatics team to develop a prototype alert for pilot testing and evaluation. This alert will help meet the needs of hospital-based clinicians caring for patients with diabetes who are at a high risk of treatment-related hypoglycemia.