Dataset Information

A mixed methods evaluation of the Paediatric Musculoskeletal Matters (PMM) online portfolio.

ABSTRACT:

Background

The PMM Portfolio is comprised of the Paediatric Musculoskeletal Matters (PMM) website, the paediatric Gait, Arms, Legs and Spine (pGALS) app and e-learning modules (ELM). The target audiences are non-specialists in paediatric musculoskeletal medicine. Our study aimed to evaluate impact on learning and clinical practice.

Methods

Mixed methods (analytics, online survey, interviews) were used with PMM and ELM registered users and purposive sampling of users using international contacts within paediatrics and paediatric rheumatology. Data was analysed using descriptive statistics and qualitative techniques. A Paired T-Test compared self-rated confidence before and after use of the PMM Portfolio.

Results

There has been wide reach for all the e-resources; PMM website (662,827 hits, 262,476 users, 214 countries, data 31st July 2020); pGALS app (12,670 downloads, 70 countries, data 31st July 2020); ELM (150 users, 30 countries, data 30th May 2019). There were 164 responses (students, trainees and health care professionals) to the survey from 25 countries. Most responders deemed the PMM Portfolio useful / very useful for their learning with significantly increased self-rated confidence in their clinical examination and reasoning skills following access to the PMM website, p = < 0.01, pGALS app, p = < 0.01 and ELM, p = < 0.01. The most popular PMM website pages related to clinical assessment techniques (especially pGALS). There was high uptake of the pGALS app and pGALS ELM especially from trainees and allied health professionals. Many clinicians reported the PMM Portfolio to be useful when used to teach others. User feedback reported that easy navigation, open access, clinical images and cases were the most valued features. User feedback highlighted need to increase awareness of the e-resources through training programmes.

Conclusions

The PMM Portfolio was developed to aid learning for clinicians who are not specialists in paediatric MSK medicine. Our evaluation demonstrates wide international reach and positive feedback on learning. The PMM Portfolio is a highly useful e-resource for paediatric rheumatologists in their teaching of others to raise awareness, facilitate early diagnosis and referral of children with suspected disease. The wide user engagement informed future PMM Portfolio development and the mixed method of evaluation is transferable to other e-resources.

SUBMITTER: Smith N

PROVIDER: S-EPMC8188761 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundDespite the central role of mixed methods in health research, studies evaluating online methods training in the health sciences are nonexistent. The focused goal was to evaluate online training by comparing the self-rated skills of scholars who experienced an in-person retreat to scholars in an online retreat in specific domains of mixed methods research for the health sciences from 2015-2023.MethodsThe authors administered a scholar Mixed Methods Skills Self-Assessment instrument based on an educational competency scale that included domains on: "research questions," "design/approach," "sampling," "analysis," and "dissemination" to participants of the Mixed Methods Research Training Program for the Health Sciences (MMRTP). Self-ratings on confidence on domains were compared before and after retreat participation within cohorts who attended in person (n = 73) or online (n = 57) as well as comparing across in-person to online cohorts. Responses to open-ended questions about experiences with the retreat were analyzed.ResultsScholars in an interactive program to improve mixed methods skills reported significantly increased confidence in ability to define or explain concepts and in ability to apply the concepts to practical problems, whether the program was attended in-person or synchronously online. Scholars in the online retreat had self-rated skill improvements as good or better than scholars who participated in person. With the possible exception of networking, scholars found the online format was associated with advantages such as accessibility and reduced burden of travel and finding childcare. No differences in difficulty of learning concepts was described.ConclusionsKeeping in mind that the retreat is only one component of the MMRTP, this study provides evidence that mixed methods training online was associated with the same increases in self-rated skills as persons attending online and can be a key component to increasing the capacity for mixed methods research in the health sciences.

Project description:BackgroundDigital technologies connect young people with health services and resources that support their self-care. The lack of accessible, reliable digital resources tailored to young people with persistent musculoskeletal pain is a significant gap in the health services in Australia. Recognizing the intense resourcing required to develop and implement effective electronic health (eHealth) interventions, the adaptation of extant, proven digital technologies may improve access to pain care with cost and time efficiencies.ObjectiveThis study aimed to test the acceptability and need for adaptation of extant digital technologies, the painHEALTH website and the iCanCope with Pain app, for use by young Australians with musculoskeletal pain.MethodsA 3-phased, mixed methods evaluation was undertaken from May 2019 to August 2019 in Australia. Young people aged 15 to 25 years with musculoskeletal pain for >3 months were recruited. Phases were sequential: (1) phase 1, participant testing (3 groups, each of n=5) of co-designed website prototypes compared with a control website (painHEALTH), with user tasks mapped to eHealth quality and engagement criteria; (2) phase 2, participants' week-long use of the iCanCope with Pain app with engagement data captured using a real-time analytic platform (daily check-ins for pain, interference, sleep, mood, physical activity, and energy levels; goal setting; and accessing resources); and (3) phase 3, semistructured interviews were conducted to gain insights into participants' experiences of using these digital technologies.ResultsFifteen young people (12/15, 80% female; mean age 20.5 [SD 3.3] years; range 15-25 years) participated in all 3 phases. The phase 1 aggregated group data informed the recommendations used to guide 3 rapid cycles of prototype iteration. Adaptations included optimizing navigation, improving usability (functionality), and enhancing content to promote user engagement and acceptability. In phase 2, all participants checked in, with the highest frequency of full check-ins attributed to pain intensity (183/183, 100.0%), pain interference (175/183, 95.6%), and mood (152/183, 83.1%), respectively. Individual variability was evident for monitoring progress with the highest frequency of history views for pain intensity (51/183, 32.3%), followed by pain interference (24/183, 15.2%). For the goals set feature, 87% (13/15) of participants set a total of 42 goals covering 5 areas, most frequently for activity (35/42, 83%). For phase 3, metasynthesis of qualitative data highlighted that these digital tools were perceived as youth-focused and acceptable. A total of 4 metathemes emerged: (1) importance of user-centered design to leverage user engagement; (2) website design (features) promoting user acceptability and engagement; (3) app functionality supporting self-management; and (4) the role of wider promotion, health professional digital prescriptions, and strategies to ensure longer-term engagement.ConclusionsLeveraging extant digital tools, with appropriate user-informed adaptations, can help to build capacity tailored to support young people's self-management of musculoskeletal pain.

Project description:BACKGROUND:Family-based drug prevention programs that use group-based formats with trained facilitators, such as the Strengthening Families Program (SFP), are effective in preventing underage drinking and youth drug use. However, these programs are resource-intensive and have high costs and logistical demands. Tailoring them for Web-based delivery is more cost-effective and makes it easier to scale these programs for widespread dissemination. This requires the active involvement of all key stakeholders to determine content and delivery format. OBJECTIVE:The aim was to obtain consumer, agency stakeholder, and expert input into the design of a Web-based parenting skills training and youth drug prevention program. METHODS:We conducted 10 focus groups with 85 adults (range 4-10, average 8 per group), 20 stakeholder interviews with family services agency staff, and discussed critical design considerations with 10 prevention scientists and e-learning experts to determine the optimal program content and technology features for SFP Online. Focus group participants also answered survey questions on perceived barriers to use, desired navigational features, preferred course format, desired content, preferred reward structures, course length, interactive components, computer efficacy, and technology use. Descriptive statistics were used to examine consumer characteristics; linear regression was used to examine relations between SFP exposure and four continuous outcome measures, including desired program content, interactive technology, and concerns that may inhibit future use of SFP Online. Logistic regression was used as a binary measure of whether consumers desired fun games in the SFP Online program. RESULTS:Three broad thematic categories emerged from the qualitative interviews enumerating the importance of (1) lesson content, (2) logistics for program delivery, and (3) multimedia interactivity. Among the many significant relations, parents who viewed more SFP lessons reported more reasons to use an online program (beta=1.48, P=.03) and also wanted more interactivity (6 lessons: beta=3.72, P=.01; >6 lessons: beta=2.39, P=.01), parents with less interest in a mixed delivery format (class and online) reported fewer reasons to use the online program (beta=-3.93, P=.01), comfort using computers was negatively associated with concerns about the program (beta=-1.83, P=.01), having mobile phone access was related to fewer concerns about online programs (beta=-1.63, P=.02), willingness to view an online program using a mobile phone was positively associated with wanting more online components (beta=1.95, P=.02), and parents who wanted fun games wanted more interactivity (beta=2.28, P=.01). CONCLUSIONS:Formative evaluation based on user-centered approaches can provide rich information that fuels development of an online program. The user-centered strategies in this study lay the foundation for improving SFP Online and provide a means to accommodate user interests and ensure the product serves as an effective prevention tool that is attractive to consumers, engaging, and can overcome some of the barriers to recruitment and retention that have previously affected program outcomes in family-based prevention.

Project description:IntroductionIncreasing number of Massive Open Online Courses (MOOCs) are being used to train learners at scale in various healthcare-related skills. However, many challenges in course delivery require further understanding, for example, factors exploring the reasons for high MOOC dropout rates, recorded low social interaction between learners and the lack of understanding of the impact of a course facilitators' presence in course engagement. There is a need to generate further evidence to explore these detriments to MOOC course delivery to enable enhanced course learning design. The proposed mixed-methods evaluation of the MOOC was determined based on the MOOC's aims and objectives and the methodological approaches used to evaluate this type of a course. The MOOC evaluation will help appraise the effectiveness of the MOOC in delivering its intended objectives. This protocol aims to describe the design of a study evaluating learners knowledge, skills and attitudes in a MOOCs about data science for healthcare.Methods and analysisStudy participants will be recruited from learners who have registered for the MOOC. On registration, learners will be given an opportunity to opt into the study and complete informed consent. Following completion of the course, study participants will be contacted to complete semistructured interviews. Interviews will be transcribed and coded using thematic analysis, with data analysed using two evaluation models: (1) the reach, effectiveness, adoption, implementation, maintenance framework and the (2) Kirkpatrick model drawing data from pre and post-course surveys and post-MOOC semi-structured interviews. The primary goal of the evaluation is to appraise participants' knowledge, skills and attitude after taking the MOOC.Ethics and disseminationEthics approval for this study was obtained from Imperial College London through the Education Ethics Review Process (EERP) (EERP1617-030). A summary of the research findings will be reported through a peer-reviewed journal and will be presented at an international conference.

Project description:BackgroundEvidence-based medicine (EBM) is a core skillset for enhancing the quality and safety of patients' care. Online EBM education could improve clinicians' skills in EBM, particularly when it is conducted during vocational training. There are limited studies on the impact of online EBM training on clinical practice among general practitioner (GP) registrars (trainees in specialist general practice). We aimed to describe and evaluate the acceptability, utility, satisfaction and applicability of the GP registrars experience with the online course. The course was developed by content-matter experts with educational designers to encompass effective teaching methods (e.g. it was interactive and used multiple teaching methods).MethodsMixed-method data collection was conducted after individual registrars' completion of the course. The course comprised six modules that aimed to increase knowledge of research methods and application of EBM skills to everyday practice. GP registrars who completed the online course during 2016-2020 were invited to complete an online survey about their experience and satisfaction with the course. Those who completed the course within the six months prior to data collection were invited to participate in semi-structured phone interviews about their experience with the course and the impact of the course on clinical practice. A thematic analysis approach was used to analyse the data from qualitative interviews.ResultsThe data showed the registrars were generally positive towards the course and the concept of EBM. They stated that the course improved their confidence, knowledge, and skills and consequently impacted their practice. The students perceived the course increased their understanding of EBM with a Cohen's d of 1.6. Registrars identified factors that influenced the impact of the course. Of those, some were GP-related including their perception of EBM, and being comfortable with what they already learnt; some were work-place related such as time, the influence of supervisors, access to resources; and one was related to patient preferences.ConclusionsThis study showed that GP registrars who attended the online course reported that it improved their knowledge, confidence, skill and practice of EBM over the period of three months. The study highlights the supervisor's role on GP registrars' ability in translating the EBM skills learnt in to practice and suggests exploring the effect of EBM training for supervisors.

Project description:BackgroundPaediatric mortality rates in the United Kingdom are amongst the highest in Europe. Clinically missed deterioration is a contributory factor. Evidence to support any single intervention to address this problem is limited, but a cumulative body of research highlights the need for a systems approach.MethodsAn evidence-based, theoretically informed, paediatric early warning system improvement programme (PUMA Programme) was developed and implemented in two general hospitals (no onsite Paediatric Intensive Care Unit) and two tertiary hospitals (with onsite Paediatric Intensive Care Unit) in the United Kingdom. Designed to harness local expertise to implement contextually appropriate improvement initiatives, the PUMA Programme includes a propositional model of a paediatric early warning system, system assessment tools, guidance to support improvement initiatives and structured facilitation and support. Each hospital was evaluated using interrupted time series and qualitative case studies. The primary quantitative outcome was a composite metric (adverse events), representing the number of children monthly that experienced one of the following: mortality, cardiac arrest, respiratory arrest, unplanned admission to Paediatric Intensive Care Unit, or unplanned admission to Higher Dependency Unit. System changes were assessed qualitatively through observations of clinical practice and interviews with staff and parents. A qualitative evaluation of implementation processes was undertaken.ResultsAll sites assessed their paediatric early warning systems and identified areas for improvement. All made contextually appropriate system changes, despite implementation challenges. There was a decline in the adverse event rate trend in three sites; in one site where system wide changes were organisationally supported, the decline was significant (ß = -0.09 (95% CI: - 0.15, - 0.05); p = < 0.001). Changes in trends coincided with implementation of site-specific changes.ConclusionsSystem level change to improve paediatric early warning systems can bring about positive impacts on clinical outcomes, but in paediatric practice, where the patient population is smaller and clinical outcomes event rates are low, alternative outcome measures are required to support research and quality improvement beyond large specialist centres, and methodological work on rare events is indicated. With investment in the development of alternative outcome measures and methodologies, programmes like PUMA could improve mortality and morbidity in paediatrics and other patient populations.