Early Diagnosis of Lyme Disease by Recognizing Erythema Migrans Skin Lesion from Images Utilizing Deep Learning Techniques

1 Research Problem

Lyme disease is an infectious tick-transmitted disease that manifests itself in most cases with erythema migrans (EM) skin lesions in the early stage. Better diagnosis of these early forms would allow improving the prognosis by preventing the transition to a severe late form thanks to appropriate antibiotic therapy. The unavailability of well-labeled public EM datasets because of privacy concerns of medical data may be the reason for the lack of extensive studies in this field. Existing works on machine learning based Lyme disease prediction only utilize images of EM lesions whereas doctors believe that corresponding patient metadata should be incorporated in the analysis [Burlina et al., 2020; ˇCuk et al., 2014]. Even though images of EM can be collected from the internet and hospitals, it will be time-consuming to collect corresponding patient metadata associated with each image to jointly train a multimodal deep learning model. Skin hair over lesions can be a challenge for real-life application as we do not have sufficient training images with skin hair to cover many real-life skin hair variations. Standard image processing based hair removal is not beneficial for real-time application and removing hair does not give new features to the network. Augmenting images with skin hair can be of interest. So, we are dealing with three research questions: Q1: How to deal with the lack of a well-labeled publicly available dataset? Q2: How to utilize patient metadata in the absence of enough training data? Q3: How to efficiently deal with skin lesion hair?

2 Research Contribution and Plan

Our research contribution and plan for the identified research questions are described in the following subsections.

2.1 Dealing with the Lack of Publicly Available Dataset

As there is no publicly available EM image dataset for Lyme disease predictions, we utilized an EM dataset consisting of images collected from Clermont-Ferrand University Hospital Center (CF-CHU) of France and the internet. CF-CHU collected the images from several hospitals in France. This dataset was labeled by expert dermatologists and infectiologists from CF-CHU. First, we benchmarked this dataset for twenty-three well-known CNN architectures in terms of predictive performance metrics, computational complexity metrics, and statistical significance tests. Second, to improve the performance of the CNNs, we used transfer learning from ImageNet pre-trained models as well as pre-trained the CNNs with the skin lesion dataset "Human Against Machine with 10000 training images (HAM1000)" [Tschandl et al., 2018] as shown in Figure 1. Our study confirmed the effectiveness and potential of even some lightweight CNNs to be used for Lyme disease prescanner mobile applications. We also made all the trained models and part of the dataset publicly available at https://dappem.limos.fr/download.html. The study has been published at [Hossain et al., 2022]. Now, considering the performance and activation map visualization of the different architectures we are planning to create a custom lightweight architecture for our problem utilizing a residual building block consisting of dilated, depthwise separable convolutions, and attention mechanisms.

2.2 Utilizing Patient Metadata in the Absence of Training Data

As we do not have corresponding patient metadata for each of the images of our dataset so initially, we are working on incorporating expert opinion elicitation. The questionnaires and weight attribution were prepared by collecting data from 16 expert dermatologists. As it is difficult for the doctors to provide estimates for the parameters of a probability distribution, they assigned weight values to different symptoms in the range of -1 to +3 (a higher value represents a higher contribution of the symptom towards the possibility of the disease). We summarized each case as the min-max normalized average weight sum. Based on all possible cases we proposed three approaches to the experts for converting the weight sum to a probability score. The approaches are 1. cumulative prob-ability of a normalized weight sum based on the kernel density estimate, 2. cumulative probability of a normalized weight sum based on the density estimate from a Gaussian mixture with two components, and 3. posterior probability of a normalized weight sum belonging to the second component of Gaussian mixture (assuming the second component represents the ill subpopulation) as shown in Figure 2. The experts opted for the second approach. We are still working on finalizing the elicitation process.

2.3 Efficiently Dealing with Skin Lesion Hair

Skin hair augmentation techniques require a hair mask to generate hair in given locations. These masks are created either manually, with random curves or lines and segmentation. Generative Adversarial Network (GAN) can be utilized to automate the creation of hair masks. We are planning to create a GAN to automate the skin hair mask generation process. For that purpose, we are creating a hand-curated skin lesion hair segmentation dataset with a custom segmentation model which can be utilized for preparing the skin lesion hair mask dataset for the GAN training as shown in Figure 3.

Acknowledgements

This research is funded by the European Regional Development Fund, project DAPPEM -AV0021029. This work is su-pervised and co-supervised by Prof. Engelbert MEPHU NGUIFO (https://perso.isima.fr/ enmephun/) and Jocelyn de Go¨er de Herve (https://jgoer.epia.clermont.inrae.fr/) respectively.