A Complete Survey on Automatically Diagnosing COVID-19 In the field of Computer Vision and A Collection of Medical Images

: ​ As COVID-19 is the source of millions of deaths throughout the world, it turned obligatory to fight against the COVID-19 pandemic. Due to the need for expensive equipment, experienced radiologists, and the time-consuming in Reverse Transcription Polymerase Chain Reaction (RT-PCR) test, researchers find out the necessity to embrace X-ray images and Computed Tomography (CT) images based diagnosing. Wreak havoc of COVID-19 ​ instigated me to review current emerging Artificial Intelligence(AI) based automatic diagnosing models through the statistical survey that will pave out the way of research. In this paper, I study different available research resources at the time span from April 2020 to July 2020. In order to help researchers in further research, I presented a statistical survey ​ so that researchers can pick a preeminent diagnosing model. I took a look at 74 papers from April to July and specified preprocessing techniques, feature extraction, classification method, interpretability method, and experimental result. Moreover, I analyze training,testing and validation split ratio, as well as look into the dataset’s availability publicly. Some researchers are able to gain noticeable performance by adopting their own local model. On the contrary, some researchers adopt an existing pre-trained model and achieve the utmost result. Some models need to feed huge data and some models outperform despite having small data. In the following sections, all of the criteria will be illustrated briefly.

1 Introduction COVID-19 is the disease which is caused by severe acute respiratory syndrome coronavirus 2(SARS-CoV-2) that creates a worldwide pandemic of respiratory illness that emerged in Wuhan City, China in December 2019.The first recorded case of COVID-19 outside of China was on 13 January 2020.Within 17 days on 30 January, the World Health Organization(WHO) reported 7818 total confirmed cases worldwide among them China was in majority.WHO made the assessment that COVID-19 is a worldwide pandemic on 11 March 2020.From now on the number of affected persons and death is increasing day by day as it spreads from person to person.Till now COVID-19 affected 213 countries and the number of affected people reached 21,511,064 the 1 number of deaths is 766,394 up to 15 August 2020 globally.The globally recovered people's rate is 14,254,024.Currently, Infected Patients are 6,490,646 among them 99% is in mild condition and 1% is in critical condition.
In these circumstances diagnosing COVID-19 is inevitable otherwise the death toll will go on.RT-PCR is accepted as a standard diagnosing method widely but it only focuses on virus detection that can mislead to detect the people who have recovered from the virus.Moreover, it requires adequate expertise to collect viral RNA which is typically extracted from the nasopharyngeal of patients additionally RT-PCR is also time-consuming.To reach better model researchers focus on diagnosing based on scanning images(X-ray, CT-Scan).Researchers are paying more attention to Artificial Intelligence(AI) based diagnosing models as it requires less time, effort as well they focus on accuracy.RT-PCR test requires expensive equipment whereas image scanning based diagnosing is less expensive and doesn't need the Table 1 presents the available data sources that researchers used in their research to find out the promising model for detecting COVID-19 that requires less time as well as less effort.In addition Table 1 represents papers that make use of datasets along with a number of the papers.Besides, some papers utilized CT images datasource [4], [21], [42], [49], [58] that are not publicly available.Further in this section frequency of papers that used a specific dataset of CT images are illustrated in Fig. 2.  [5], [8], [11], [50], [63], [65], [39] 8 SIRM https://www.sirm.org/[46], [69] 2 COVID-CT https://github.com/UCSD-AI4H/COVID-CT[60], [29] 2 University Hospitals of Geneva (HUG) www.ChainZ.cn, El-Camino Hospital (CA), Zhejiang Province, China University Hospitals of Geneva (HUG) [45] 1 Lung Segmentation and Candidate Points Generation https://www.kaggle.com/arturscussel/lungsegmentation-and-candidate-points-generation [49] 1 LIDC https://doi.org/10.1118/1.3528204[12] 1 CC-19 https://github.com/abdkhanstd/COVID-19[9] 1 Dataset Figure 2: Frequency of papers that used a specific dataset of CT images

X-ray images
Due to the low sensitivity, a high false-negative rate of RT-PCR test, diagnosing using X-ray image gained high popularity because of its low expense, less time requirement, sufficient inexpensive equipment, high sensitivity.Twenty-five papers make use of COVID-Chest X-ray-dataset [3].Chest X-ray Images (Pneumonia) dataset is evaluated by six papers.Evaluating COVID-19 Chest X-ray Dataset Initiative [36] in their system researcher gained accuracy 93%.On the other hand, 97.3% accuracy is obtained using OCT and Chest X-ray images dataset in their model [10].Precise details of available X-ray image data sources are presented in Table 2.Prediction comparison between COVID-19, normal, and viral pneumonia is shown in Fig. 3 that demonstrate three X-ray images.Table 2 presents the available data sources that researchers used in their research to find out the promising model for detecting COVID-19 that requires less time as well as less effort.In addition Table 2 represents papers that make use of datasets along with a number of the papers.Besides, some papers utilized X-ray images datasource [41], [47], [49], [51], [52] that are not publicly available.Further in this section frequency of papers that used a specific dataset of X-ray images are illustrated in Fig. 4.
Flipping or Rotating performs image enhancement in datasets.Vertically or horizontally image pixels can be reversed.Additionally, images can be rotated at different angles for instance: 90, 180, or 270 degrees.
Cropping or Scaling is performed by resampling the image thus the size of the whole image changes.Scaling can enlarge or lessen an image in size.Cropping reduces redundancy and shrinks unwanted areas on the various scales.Researchers many times use random cropping in various scales [2 1], [55].
Brightness or Intensity adjusting increases or reduces the brightness of an image by substituting pixel values with a constant.Brightness can be increased or reduced by subtraction or addition.Generative Adversarial Network(GAN): Due to the scarcity of Chest X-ray and CT image dataset GAN is widely adopted in several datasets to generate images at the same time it demises overfitting.Sometimes interference caused by image enhancement using GAN, to demise interference researchers performed histogram equalization [47] on the datasets.However, histogram equalization may impact on image details and show unexpected noise to resolve the problem Contrast Limited Adaptive Histogram Equalization (CLAHE) was proposed in paper [41].
In this section, I review preprocessing methods that are depicted in Table 3 used by researchers in their paper including resizing, flipping or rotating, scaling or cropping, contrast adjusting, brightness, or intensity adjusting, GAN, and so on.Resize a common feature of data preprocessing used in 31 papers.Next to it, Flipping is used in 29 papers.Along with that scaling or cropping, Contrast adjusting, Brightness or Intensity adjusting, and GAN is used in 20,14,7,4 papers respectively.The author used both resizing, flipping or rotating, and scaling or cropping in paper [2], and in paper [21] author used four preprocessing methods respectively Flipping or Rotating, Scaling or Cropping, Contrast adjusting, and Brightness or Intensity adjusting.Like this many authors used more than one preprocessing method in their research.Oppositely some authors used only one preprocessing method as an example in paper [9] author used scaling, in paper [11] the author used resizing 352 x 352, in paper [16] author used rotation, and so on.Moreover, some researchers also use an adaptive winner filter for noise reduction [59], Affine Transformation [31] in their research.
Except for the pre-trained CNN models that are already presented in Table 4 and Table 5, several specialized diagnosing models using those pre-trained models researchers adopt nowadays to enlarge accuracy .I represent the researchers' proposed feature extraction method based on both CT images and X-ray images in Table 6.

Papers
Interpretability method COVID-Net specifically proposes a neural network to detect COVID-19 using chest X-ray images.Based on the advantages of CXR imaging for the rapid outcome of COVID-19 screening, tangibility, mobility, the author makes predictions through the COVIDNet interpretability method.The author makes use of projection-expansion-projection design patterns in COVID-Net architecture [36].The proposed COVID-Net was pre-trained on the ImageNet dataset and then assigned to the COVIDx dataset and achieved accuracy about 93.3% on the COVIDx dataset.
CovXNet is a multi-dilation convolutional neural network which detects COVID-19 and other pneumonia automatically using chest X-ray images that utilize depthwise convolution for efficiently extracting diversified features.In the paper [26] author uses a total of 6161 datasets among which the first dataset consists of 5856 images (1583 normal X-rays, 1493 non-COVID viral pneumonia X-rays, and 2780 bacterial pneumonia X-rays) and the 2nd dataset comprises 305 images.
ChexNet CoroNet is a Deep Convolutional Neural Network model that is based on Xception architecture.In paper [54] authors collect two different datasets that are publicly available then create their own dataset to detect COVID-19 infection automatically using chest X-ray images.In the paper the author also uses softmax to predict and it is inspected from that CoroNet has 33,969,964 parameters in total out of which 33,969,964 trainable and 54528 are non-trainable parameters.Author achieves promising results through CoroNet in spite of using a small dataset(COVID-19 284, Normal 310).

COVID-CAPS is a Capsule Network-based Framework that identifies COVID-19 cases from X-ray
Images which consists of 4 convolutional layers and 3 Capsule layers.It is noticeable COVID-CAPS shows higher performance dealing with a small dataset [57].The number of trainable parameters using COVID-CAPS without pre-training and with pre-training is 295,488.
Detail-Oriented Capsule Networks (DECAPS) combine Capsule Networks (CapsNets) that is basically based on ResNet to identify discriminative image features to detect COVID-19 patients using CT images.
In paper [60] the author gained accuracy 87.60% using DECAPS model which comprises a smallest amount of dataset.Because of the scarcity of sample image authors applied conditional adversarial network and also adopted other preprocessing techniques for instance Rescaling(286X286), Cropping(256X256).Among the total dataset 391 images are labeled as COVID-19 and 339 images are labeled as Normal.ResNet has three residual blocks and, outcome 1024 feature maps, with a 1 × 1 convolutional layer.It contains a ReLU non-linear layer.Author also shows using Peekaboo with DECAPS accuracy of the diagnosing model increased.

Classification
The classification process took place in the softmax layer, and a fully connected layer whereas the Convolutional layer proceeds as a feature extractor in a pre-trained CNN.Some researchers proposed improvements based on pre-trained CNN along with a Support Vector Machine(SVM) classifier [46], [66].
The SVM classifier uses deep features that are extracted from each CNN network for detection.Researchers combine CNN with KNN and a support estimator network [14] that requires huge data to train.In paper [6] researchers use a COV-ELM classifier that classifies COVID-19 cases from the chest x-ray images using an extreme learning machine (ELM) and lessens training time with the least interventions required to tune the networks.
Researchers developed an end to end web-based detection system with a Bagging trees classifier to replicate a digital clinical pipeline and ease the screening of suspicious cases [67].Researchers achieved satisfactory performance despite the limited number of image samples using the Bagging trees classifier.
In [34], researchers proposed Adaptive Feature Selection Guided Deep Forest(AFS-DF) as a classification method in the meanwhile they compared AFS-DF with Logistic Regression (LR), Random Forests (RF), Neural Networks (NN), SVM classifiers and AFS-DF attain higher accuracy.
It is obvious from Table 7 Binary class is mostly used by researchers than Multi-class.However, Binary classification may create ambiguity while detecting COVID-19 as it can not distinguish between other Viral Pneumonia and COVID-19.

Figure 8: A relationship of Lung Diseases
A straightforward way of COVID-19 diagnosing system tasks is binary classifying the scanning images into COVID-19 class and normal class, and that is adopted by many papers [21].[19], [23].As shown in Fig. 8., test images of other groups of abnormal Lung Diseases can be miscategorized as COVID-19.
Lung diseases belonging to the same subclass share similar patterns in diagnosing using X-ray image or CT images and contain a higher probability to be miscategorized to resolve this dilemma researchers adopt multi-class classification in many papers [7], [12], [17].

Experimental results
Several pre-trained models are proposed by the researchers; perhaps specific methods are enough qualified to gain the highest accuracy depending on its total dataset size, classifier, number of convolutional layers in a CNN, and so on.Experimental results are measured in terms of accuracy in this survey, when researchers don't state accuracy as an alternative sensitivity, Area Under Curve (AUC) or specificity is picked up that illustrates how precisely and accurately that model can predict the result.

Experimental results for CT images
Abbreviated evaluation of COVID-19 diagnosing models using CT images presented in Table 8.I also mentioned the size of the train, test, and validation set.Some authors in their papers [11], [53], [59] did not apparently provide the size of train, test, and validation set, I compute them with corresponding train, test, and validation split ratio.On the contrary, some papers apparently stated the size of the train-test-validation data of COVID-19 images [5], [68], [69] and for those papers, I computed the ratio according to the split that was provided.Even so, for some papers, the distribution of the dataset [50], [55], [34] was not clearly mentioned.Moreover, some papers apparently mentioned the use of data for validation [5], [8], [11] along with some papers do not state exact data instead provide a comparison based on 10-fold cross-validation [50], 5-fold cross-validation [53] for performance assessment.Along with the best experimental results using CT images feature extraction methods corresponding to the accuracy and total dataset(COVID-19 patients, non affected patients, and other pneumonia diseases) also reviewed in Table 8.In terms of sensitivity, accuracy, and AUC VGG, IRRCNN, ResNet scores higher 97%, 99.56% and99.40%respectively.In this section, the average result is reviewed over the extraction method.It is sharply clear from Fig. 9 that the average accuracy of Inception is far exceeding above 90% than the other 4 models and the average accuracy of SqueezeNet is not up to the mark.

Experimental results for X-ray images
Abbreviated evaluation of COVID-19 diagnosing models using X-ray images presented in Table 9.I also stated the size of the train, test, and validation set.Some authors in their papers [19], [18] did not apparently provide the size of the train, test, and validation set, I compute them with corresponding train, test, and validation split ratio.On the contrary, some papers apparently stated the size of the train-test-validation data of COVID-19 images [16], [20], [22] and for those papers, I computed the ratio according to the split that was provided.Even so, for some papers, the distribution of the dataset [7], [10], [26] was not clearly mentioned.Moreover, some papers apparently mentioned the use of data for validation[1], [16], [25], [30] along with some papers do not state exact data instead provide a comparison based on 10-fold cross-validation [6], 5-fold cross-validation [3,[18] for performance assessment.
Along with the best experimental results using X-ray images feature extraction methods corresponding to the accuracy or other metrics and total dataset(COVID-19 patients, non affected patients, and other pneumonia diseases) also reviewed in Table 9.In terms of accuracy and AUC [14], [72] performance of DenseNet is up to the mark 99.49%,88.04%respectively whereas NasNetLarge scores 100% sensitivity [23], f1 score of FFT is 95%.Additionally in respect of accuracy Xception+ResNet [61] is significantly the highest scorer 99.56%.Class activation mapping is a method to generate heatmaps of images that is a visualization and can be interpreted as telling researchers wherein the image the neural net is (metaphorically) looking to make its decision indicating a highly important area.Several variations of the method including Score-CAM and Grad-CAM (Gradient Weighted Class Activation Mapping) is accepted widely.CAM is proposed by researchers to examine overfitting occurrence even CAM is capable to classify relevant portions of the image for CNN.Actually, when the diagnosing model has significant accuracy on the training data, but unmarked accuracy on the Test dataset, CAM helps to verify whether the CNN is biased or not while predicting on the features of the images.
Perhaps, because of its nonlinearity vanishing property of the classifiers nowadays Gradient Weighted Class Activation Mapping (Grad-CAM) [47] is broadly by researchers.As an interpretability method gradient-guided class activation maps (Grad-CAM++) and layerwise relevance propagation (LRP), Local Interpretable Model-Agnostic Explanations (LIME) are widely adopted by many researchers for explaining the predictions and to identify the critical regions on patient's chest besides generating class-discriminating attention maps.10.
Table 11: Interpretability methods used in the X-ray images based works

Interpretability method Papers Total
Grad-Cam [1], [6], [10], [13], [15], [16], [20], [26], [31], [32], [47], [52], [56], [70], [73] 15 Cam [22], [41], [72], [74] 4 The entire use of Grad-Cam and Cam based on both CT-image and X-ray image based diagnosing systems in this survey are interpreted in Fig. 12 to deliver a sharp concept of interpretability.In this survey, I reviewed 74 automatic COVID-19 diagnosing models and overseen the characteristics of this diagnosing model.It is obvious from this survey, the average accuracy of X-ray image based diagnosing is better than CT image-based diagnosing 96.5%,94.5% respectively.The reason could be a massive amount of X-ray training data than CT training data.The average training data size of the CT image-based diagnosing system is 609.9 whereas the average training data size of the X-ray image-based diagnosing model is 2857.9.But many authors don't clarify their dataset size [10], as well as many authors, do not provide a clear indication of training dataset size in their literature [36], [40].
For transfer learning, some diagnosing models adopted pre-trained CNN model [46] even showed high performance 98.27% accuracy, while others proposed various architectures to detect COVID-19 cases from Chest X-ray images to obtain more discriminative features as in paper [3] author propose CovMUNET, which is a multiple loss deep neural network approach, in paper [25] author proposed a deep learning CAD system to simultaneously recognize the COVID-19 9 among the other eight lung diseases: Atelectasis, Infiltration, Pneumothorax, Mass, Effusion, Pneumonia, Cardiomegaly, and Nodule using chest X-ray images, in paper [53] author propose a weakly supervised deep learning strategy for detecting and classifying COVID-19 infection from CT images.The proposed model can reduce the demands of manual labeling of CT images.But still, researchers are unable to detect infection and discriminate COVID-19 from non-COVID-19 cases.To enhance the performance of the classifiers, researchers proposed [16] multi-class classification.We also abbreviated the results of diagnosing models.Some of the models report very good performance, but the size of some test sets are not sufficient [19].

Conclusion
In this survey, I provide the overview of research works that detect COVID-19 using CT image and X-ray image on the basis of feature extraction method, classification, interpretability, and model accuracy.Early quarantine is a must for an affected person in order to decrease the infection rate and to get clinical support diagnosing COVID-19 as early as possible is mandatory.AI-based diagnosing models using Chest X-ray images and CT-Scan images have a higher potentiality to support radiologists in COVID-19 detection rapidly.
The rate of death can be decreased by recognizing patients that are affected so that they can get immediate medical attention and that is possible only by adopting AI-based diagnosing systems as it can diagnose at a very limited time and precisely.It is noticeable in this survey that we analyze different research work so that researchers can pick out the best one or discover any model based on our data that we analyze considering different aspects.I hope this survey can provide valuable insight into computer vision efforts against the COVID-19 pandemic and assist the researchers in new work.

Figure 4 :
Figure 4: Frequency of papers that used a specific dataset of CT images

Figure 5 :
Figure 5: Different Types of Preprocessing Methods

Figure 6 :
Figure 6: Ratio of Different preprocessing methods

Figure 7 :
Figure 7: Basic Pre-trained CNN architecture 4.1 Feature Extraction methods for CT To recognize patterns from the CT images between COVID-19 patients and nonaffected persons, various pertained models(ResNet, VGG, DenseNet, etc) availed the researchers nowadays.A few pre-trained models mostly used by researchers are delimited in Table4, among all ResNet is used largely then VGG, DenseNET, and others.Researchers use ResNet as a pre-trained model[42] for binary classification of CT scan images that is 71 layers deep and requires an input image size of 224x224x3.As an extension of CNN researchers uses DenseNet[29] to boost computational efficiency reducing image dimension and obtain 90.61% accuracy.As a feature extraction method researchers employ a modified version of inception V3 (IV3*) then train the extracted features using layers of the capsule network[9] that achieved the highest sensitivity and lowest specificity compared to other pre-trained models.Researchers use eight different Deep Learning Models in the paper[4] and found NasNet and MobileNet performed better than the other six models.

Figure 10 :
Figure 10: Average Accuracy for X-ray imageIn this section, average accuracy is reviewed over the best 5 extraction methods.It is sharply clear from Fig.10that the average accuracy of VGG is far exceeding above 95% than the other 4 models and the average accuracy of Xception is not up to the mark.The average accuracy of VGG and Inception is immensely close 96.5%,96.3%individually.X-ray image based diagnosing is superior rather than CT image-based diagnosing clearly appreciable from Fig.9and Fig.10 However, In this survey, I oversee two interpretability methods that are Grad-Cam and CAM.Both the X-ray image based diagnosing model and CT-image based diagnosing model used Grad-Cam more in comparison to CAM.Besides in some literature[4],[22] authors also utilize LIME with the purpose of rectifying misclassification.Chest X-ray images of a patient In Fig.11are displayed to output heatmaps for interpretation of the ultimate result and represent an intuitive understanding of which area is the model focusing on.

Figure 11 :
Figure 11: Chest X-ray images of a patent in three points in the upper row and in a lower row their heatmaps using Grad-Cam In CT image-based diagnosing model researchers use Grad-Cam at 3 literature, CAM at 2 literature depicted in Table10.

Figure 12 :
Figure 12: Total number of Grad-Cam and CAM for both CT and X-ray based works

Table 1 :
Publicly available Dataset for CT Images

Table 5 :
Feature Extraction methods for X-ray images used by the papers [35]asically a DenseNet-121 type of deep network trained on the ChestX-ray14 dataset.They used the Softmax activation function To classify COVID-19 the Softmax activation function is used(Normal, Viral Pneumonia, and Bacterial Pneumonia).The number of trainable parameters in this model is 6,955,906.Before the classification layer, the author used pre-trained CheXNet to extract 1024-D feature vectors by taking the output after global pooling in paper[35].The sensitivity achieved by ChexNet is noticeable.In this paper, the author utilizes the largest Covid-19 dataset named QaTa-Cov19 that constitutes 6286 images among them 5028 is used to train the dataset.

Table 8 :
Summary of experimental result along with the extraction method using CT images

Table 9 :
Summary of experimental result along with method using X-ray images