CANDI: an R package and Shiny app for annotating radiographs and evaluating Computer-Aided Diagnosis.

Motivation: Radiologists have used algorithms for Computer-Aided Diagnosis (CAD) for decades. These algorithms use machine learning with engineered features, and there have been mixed findings on whether they improve radiologists' interpretations. Deep learning offers superior performance, but requires more training data and has not been evaluated in joint algorithm-radiologist decision systems.

Results: We developed the Computer-Aided Note and Diagnosis Interface (CANDI) for collaboratively annotating radiographs and evaluating how algorithms alter human interpretation. The annotation app collects classification, segmentation, and image captioning training data, and the evaluation app randomizes the availability of CAD tools to facilitate clinical trials on radiologist enhancement.

Availability: Demonstrations and source code are hosted at {{https://candi.nextgenhealthcare.org}}, and {{https://github.com/mbadge/candi}}, respectively, under GPL-3 license.

Supplementary information: We demonstrate CANDI with chest x-ray datasets and deep learning models for cardiopulmonary diseases. In the online Supplement we describe these datasets, data security, and model training and performance evaluation.