Hyperparameter optimization of YOLO models for invasive coronary angiography lesion detection and assessment

Abstract

Coronary artery disease (CAD) remains the leading cause of mortality, creating an urgent need for reproducible, image-based decision support. Although YOLOv8-based detectors underpin much of today’s state-of-the-art stenosis detection, their accuracy is sensitive to dozens of interacting hyperparameters. We therefore perform a systematic study of hyperparameter optimizers for YOLO-style models, pairing YOLOv8 and its Double Coordinate Attention (DCA) variant with three model-based engines: Covariance-Matrix-Adaptation Evolution Strategy (CMA-ES), Tree-structured Parzen Estimator (TPE), and a Gaussian-process sampler; and contrasting them with Random Search and the mutation-only routine that serves as the default optimizer in the ultralytics package. Optimization targets binary detection and was benchmarked using the CADICA (full sequences) and ARCADE (single key-frames) datasets, maximizing the F1-Score under stratified three-fold cross-validation within a fixed compute budget. Model-based methods consistently lift the F1-speed Pareto frontier: CMA-ES attains on v8l, while Bayesian strategies top the medium and small backbones with (v8m) and (v8s). All surpass the default optimizer and yield more lesion-centric EigenCAM saliency, confirming the value of adaptive probabilistic search for tuning high-dimensional YOLO-based CAD pipelines. The complete code-base is open-source and released at https://github.com/MarioPasc/Coronary_Angiography_Detection.