Visual Object Detection with DETR to Support Video-Diagnosis Using Conference Tools

Abstract

This text discusses the need for real-time multilingual sentence detection during online video presentations, particularly in the healthcare sector for remote diagnosis. The use of visual (textual) object detection and preprocessing is essential for subsequent analysis. The researchers propose using the DEtection TRansformer (DETR) model to achieve accurate and real-time detection of textual objects. The development of real-time videoconference translation supported by artificial intelligence has become especially important during the COVID-19 pandemic. The challenge lies in the variety of languages spoken by specialists, which requires human translators or AI-based technological channels. The accuracy of visual localization of textual elements depends on the complexity, quality, and variety of the training datasets. The researchers compare the performance of the DETR model with other real-time object detectors like YOLO4 and Detectron2, and introduce AI-based innovations through collaborative solutions combined with OCR. The researchers conducted evaluations using training datasets and achieved higher-than-expected accuracy in terms of visual text detection range, with an average accuracy of 0.4 to 0.65.