We have located links that may give you full text access.
Model-Based Operative Performance Expectations for Quantifying Competency in General Surgery.
JAMA Surgery 2023 March 9
IMPORTANCE: Understanding how to translate workplace-based assessment (WBA) ratings into metrics that communicate the ability of a surgeon to perform a procedure would represent a critical advancement in graduate medical education.
OBJECTIVE: To evaluate the association between past and future performance in a comprehensive assessment system for the purpose of assessing point-in-time competence among general surgery trainees.
DESIGN, SETTING, AND PARTICIPANTS: This case series included WBA ratings from September 2015 to September 2021 from the WBA system of the Society for Improving Medical Professional Learning (SIMPL) for all general surgery residents who were provided a rating following an operative performance across 70 programs in the US. The study included ratings for 2605 trainees from 1884 attending surgeon raters. Analyses were conducted between September 2021 and December 2021 using bayesian generalized linear mixed-effects models and marginal predicted probabilities.
EXPOSURES: Longitudinal SIMPL ratings.
MAIN OUTCOMES AND MEASURES: Performance expectations for 193 unique general surgery procedures based on an individual trainee's prior successful ratings for a procedure, clinical year of training, and month of the academic year.
RESULTS: Using 63 248 SIMPL ratings, the association between prior and future performance was positive (β, 0.13; 95% credible interval [CrI], 0.12-0.15). The largest source of variation was postgraduate year (α, 3.15; 95% CrI, 1.66-6.03), with rater (α, 1.69; 95% CrI, 1.60-1.78), procedure (α, 1.35; 95% CrI, 1.22-1.51), case complexity (α, 1.30; 95% CrI, 0.42-3.66), and trainee (α, 0.99; 95% CrI, 0.94-1.04) accounting for significant variation in practice ready ratings. After marginalizing overcomplexity and trainee and holding rater constant, mean predicted probabilities had strong overall discrimination (area under the receiver operating characteristic curve, 0.81) and were well calibrated.
CONCLUSIONS AND RELEVANCE: In this study, prior performance was associated with future performance. This association, combined with an overall modeling strategy that accounted for various facets of an assessment task, may offer a strategy for quantifying competence as performance expectations.
OBJECTIVE: To evaluate the association between past and future performance in a comprehensive assessment system for the purpose of assessing point-in-time competence among general surgery trainees.
DESIGN, SETTING, AND PARTICIPANTS: This case series included WBA ratings from September 2015 to September 2021 from the WBA system of the Society for Improving Medical Professional Learning (SIMPL) for all general surgery residents who were provided a rating following an operative performance across 70 programs in the US. The study included ratings for 2605 trainees from 1884 attending surgeon raters. Analyses were conducted between September 2021 and December 2021 using bayesian generalized linear mixed-effects models and marginal predicted probabilities.
EXPOSURES: Longitudinal SIMPL ratings.
MAIN OUTCOMES AND MEASURES: Performance expectations for 193 unique general surgery procedures based on an individual trainee's prior successful ratings for a procedure, clinical year of training, and month of the academic year.
RESULTS: Using 63 248 SIMPL ratings, the association between prior and future performance was positive (β, 0.13; 95% credible interval [CrI], 0.12-0.15). The largest source of variation was postgraduate year (α, 3.15; 95% CrI, 1.66-6.03), with rater (α, 1.69; 95% CrI, 1.60-1.78), procedure (α, 1.35; 95% CrI, 1.22-1.51), case complexity (α, 1.30; 95% CrI, 0.42-3.66), and trainee (α, 0.99; 95% CrI, 0.94-1.04) accounting for significant variation in practice ready ratings. After marginalizing overcomplexity and trainee and holding rater constant, mean predicted probabilities had strong overall discrimination (area under the receiver operating characteristic curve, 0.81) and were well calibrated.
CONCLUSIONS AND RELEVANCE: In this study, prior performance was associated with future performance. This association, combined with an overall modeling strategy that accounted for various facets of an assessment task, may offer a strategy for quantifying competence as performance expectations.
Full text links
Related Resources
Trending Papers
Proximal versus distal diuretics in congestive heart failure.Nephrology, Dialysis, Transplantation 2024 Februrary 30
World Health Organization and International Consensus Classification of eosinophilic disorders: 2024 update on diagnosis, risk stratification, and management.American Journal of Hematology 2024 March 30
Heart failure with preserved ejection fraction: diagnosis, risk assessment, and treatment.Clinical Research in Cardiology : Official Journal of the German Cardiac Society 2024 April 12
Get seemless 1-tap access through your institution/university
For the best experience, use the Read mobile app
All material on this website is protected by copyright, Copyright © 1994-2024 by WebMD LLC.
This website also contains material copyrighted by 3rd parties.
By using this service, you agree to our terms of use and privacy policy.
Your Privacy Choices
You can now claim free CME credits for this literature searchClaim now
Get seemless 1-tap access through your institution/university
For the best experience, use the Read mobile app