Author(s): Vijaykanth Reddy T* and Sashi Rekha K
The advent of deep learning has paved way for more efficient computer vision applications. In agricultural crop monitoring with technology driven approaches, it is indispensable to have plant disease identification. Recent research reveals that Convolutional Neural Network (CNN) is most suitable deep learning method to process leaf images for detecting diseases. As symptoms of leaf disease appear in specific area, considering entire leaf for processing incurs more computational cost and time besides deteriorating performance due to inadequate quality of training. To overcome this problem, we proposed a framework that considers extraction of ROI using deep CNN prior to prediction of pre-trained deep learning models such as VGG13, Res Net34, Dense Net19, Alex Net, Sqeezenet1_1 and Inception_v3. An algorithm named ROI Feature Map Creation (ROI-FMC) is defined to extract ROI for given input image. This will be given as input to another algorithm proposed namely ROI based Deep CNN with Transfer Learning for Leaf Disease Prediction (ROIDCNN-LDP). The latter is used to predict leaf diseases. Plant Village dataset is used for empirical study. The experimental results revealed that with ROI awareness, all models could perform well. However, Inception_v3 is the deep CNN model that outperforms other models.
Agriculture is very crucial domain across the globe as it takes care of human survival process from time immemorial. In fact, in India, it is very crucial domain as majority of population depend on agriculture or its related industries. The advent of deep learning has paved way for more efficient computer vision applications. In agricultural crop monitoring with technology driven approaches, it is indispensable to have plant disease identification. Recent research reveals that Convolutional Neural Network (CNN) is most suitable deep learning method to process leaf images for detecting diseases. As symptoms of leaf disease appear in specific area, considering entire leaf for processing incurs more computational cost and time besides deteriorating performance due to inadequate quality of training. Deep learning models have been around for improving image processing based leaf disease detection. As explored in many CNN based models in and it is observed that CNN models are most appropriate for disease detection in agricultural crops [1-4].
Leong and Tze proposed a Region of Interest (ROI) based approach along with ResNet-50 CNN for identification of plant leaf diseases [5]. Zhang et al Proposed an improved faster RCNN model for object detection towards tomato disease detection [6]. They also make use of ROI towards improving performance. From the literature it is understood that there are different approaches in deep learning models and pre-trained CNN models found. However, there is little research on ROI based approach with CNN. It is the motivation behind the work in this paper. Our contributions in this paper are as follows.
The remainder of the paper is structured as follows. Section 2 reviews literature on deep learning methods for leaf disease detection. Section 3 covers the methodology used for the plant disease detection using advanced convolutional neural networks with region of interest awareness. Section 4 presents’ experimental results and Section 5 concludes the paper and gives directions for future scope of the research.
This section reviews literature on deep learning models for leaf disease identification. It focuses on neural network models, deep CNN models and pre-trained deep CNN models.
Neural network models are good for processing hyper spectral data. Golhani et al [7]. Made a review of different models such as Back-Propagation Neural Network (BPNN), Feed-Forward Neural Network (FFNN) and Generalized Regression Neural Network (GRNN) and their role in early detection of leaf diseases in agriculture.
Deep CNN models are widely used for image processing based approach for leaf disease identification. Lu et al [8]. Proposed a methodology based on deep CNN for identification of rice diseases. The CNN architecture is made up of different layers appropriately configured in order to have better performance. In future, they intend to extend their model for fault detection. Zhong and Zhao focused on DenseNet-121 for apple leaf disease identification [9]. Multi-label classification approach is used to have more flexible identification of diseases. The diseases thus identified include “Healthy Apple, General Apple Scab, Serious Apple Scab, Apple Gray Spot, General Cedar Apple Rust and Serious Cedar Apple Rust”. Priyadharshini et al [3]. Proposed a deep CNN model by modifying LeNet for maize leaf disease classification. The convolutional layers, pooling layers and fully connected layers are configured in such way that their method could improve performance. Militante et al [10]. Proposed CNN based deep learning model for sugarcane disease identification. Their methodology includes pre-processing, feature extraction and classification. In future, they intend to apply different learning rates to evaluate their method further. Jiang et al [4]. Proposed apple leaf disease prediction model based on improved deep CNN. Their real time approach has both disease localization and classification. They also experimented with Google Net inception structure along with Rainbow concatenation for disease detection.
Sardogan et al [11]. Exploited Learning Vector Quantization (LVQ) and deep CNN model for plant leaf disease detection. The usage of LVQ as part of the CNN architecture could leverage classification accuracy. They intend to improve it with different size of convolutions and different filters. Geetharamani and Pandian proposed a deep CNN model with nine layers and six kinds of data augmentation methods are employed including Principal Component Analysis (PCA) [12]. They intend to improve it to the study of disease using flowers and stem also in future. TM et al. [13] used CNN model such as LeNet for leaf disease prediction in Tomato crop. Their methodology includes data acquisition, data pre-processing and classification. Barbedo studied different deep learning models in the usage of plant disease prediction [14]. They could different factors that affect deep learning models. They include disorders with same symptoms, simultaneous disorders, symptom variations, symptom segmentation, conditions of image capture, image background, and covariate shift and symptom representations. Sladojevic et al [14]. Studied on deep CNN models for leaf image classification. Wang et al [15]. Focused on deep learning models for disease identification in terms of severity estimation. Francis and Deisy investigated on different operations involved in deep CNN based model for leaf disease detection [16]. They classified them into two phases such as feature extraction phase and classification phase.
Luna et al [17]. Proposed an automatic approach for leaf image capture and identification of disease based on CNN models. They built a tool named automaton with GUI for visualization of the detection process. Wiesner-Hanks et al [18]. Used CNN and Conditional Random Field (CRF) approach towards plant disease detection. Karthik et al [19]. Proposed an attention embedded residual CNN for tomato leaf disease detection. They also proposed a method known as Residual Progressive Feature Extraction (RPFE) for improving quality of training and result in higher level of accuracy. Xie et al [20]. Proposed an improved CNN for real time detection of plant diseases. Their model is named as DR-IACNN which has better feature extraction capability to influence accuracy.
Sharath et al [2]. Investigated on deep CNN model for disease detection in plants. Cap et al. [21] focused on real time plant disease detection using deep learning models. Zhang et al [22]. Focused on improved deep CNN models for maize leaf disease detection. Atila et al. [23] proposed a deep learning architecture known as EfficientNet for plant leaf disease classicisation. Tan et al [24]. Explored deep learning and machine learning methods for plant species classification. They found that deep learning models could perform well comparatively. Yadav et al [25]. Proposed CNN models for plant disease identification while Li et al [26]. Reviewed CNN models for different plant disease detection. Luo et al [27]. Focused on Apple diseases using multi-scale feature fusion and sub-class categorization using ResNet model. Singh and Sabrol proposed CNN based model for crop disease identification [28]. Zhang et al. proposed a deep learning based model known as R-CNN comprising of joint segmentation and leaf skeleton identification [29]. Tavakoli et al [30]. Proposed discriminative CNNs for image classification for plant disease diagnosis.
Pre-trained deep learning models help in reusing knowledge models in leaf disease detection. Rangarajan et al [1]. Explored two pre-trained deep models such as VGG16 net and AlexNet. They used PlantVillege dataset for tomato crop disease prediction. VGG16 net showed better performance over AlexNet in terms of accuracy. However, in terms of computational load AlexNet showed better performance. Too et al [31]. Investigated on deep learning models such as ResNet, Inception V4 and VGG16. The models are evaluated with different layers and also Dense Nets. It is observed that the usage of Dense Nets consistently increased performance. However, they took more computational time. Shijie et al [32]. Studied VGG16 along with SVM model for detection of tomato diseases based on leaf images. With fine-tuned experiments, they could achieve better performance. Ferentinos used deep learning models such as VGG and Alex Net OWTBn for plant disease detection. They found the potential of such models in the research carried out [33].
Sujatha et al [34]. Investigated on CNN based deep learning models like VGG16, VGG19 and Inception V3 for leaf disease detection. VGG16 showed highest accuracy when compared with other deep models and also ML models like RF, SVM and SGD. Leong and Tze proposed a Region of Interest (ROI) based approach along with ResNet-50 CNN for identification of plant leaf diseases [5]. Hernández and Lopez focused on Bayesian deep learning models to know and uncertainty and quantify the same [35]. Zend and Li proposed a CNN based approach is known as self-attention CNN for crop leaf disease identification to leverage robustness in the process [36]. Zhang et al [6]. Proposed an improved faster RCNN model for object detection towards tomato disease detection. They also make use of ROI towards improving performance. Sambasivam and Opiyo used CNN model to have a strategy in Cassava disease detection. However, they intend to improve their
method to diagnose multiple co-occurring diseases [6]. Patayon and Crisostomo used different deep models like Alex Net, ZF Net, VGG16, and VGG19 for automatic identification of Abaca Bunchy Top disease [37]. From the literature it is understood that there are different approaches in deep learning models and pre-trained CNN models found. However, there is little research on ROI based approach with CNN. It is the motivation behind the work in this paper.
This section presents the proposed methodology of detecting leaf diseases using advanced CNN with region of interest awareness.
The methodology for effective leaf disease detection is based on deep learning models and extraction of ROIs. Deep CNN model is configured along with an algorithm for extracting region of interest. It results in the realization of plan leaf disease prediction model. With transfer learning, the framework is optimized in training and testing phases. Plant Village dataset is used in order to evaluate the proposed model. The framework is as presented in Figure 1.
Figure 1: Overview of the Proposed Methodology
As deep CNN model is found suitable for processing image based inputs like plant leaves, it is used along with the algorithm known as ROI Feature Map Creation (ROI-FMC). This algorithm is reused in the deep learning process in order to improve the accuracy of prediction.
ROI Feature Map Creation (ROI-FMC) is the algorithm defined to facilitate quality in disease prediction.
The algorithm ROI-FMC takes a leaf image and ROI maps training set as input and produces ROI feature map for given image. The ROI feature map contains background information, leaf area information and spot area information. The extraction of ROI feature map is based on training ROI maps. It is done by a deep CNN model. The output of ROI-FMC is used as input in the leaf disease prediction algorithm provided in the next slide.
ROI based Deep CNN with Transfer Learning for Leaf Disease Prediction (ROIDCNN-LDP) is the mail algorithm used towards leaf disease prediction.
Algorithm 2 takes leaf test image, ROI maps as training set and pre-trained deep CNN models such as AlexNet, DenseNet19, Inception_v3, ResNet34, Sqeezenet1_1 and VGG13 as input. This algorithm reuses ROI-FMC algorithm to extract feature map from given test image. Then it is subjected to testing with all pre-trained models with transfer learning. The deep learning models could perform well with the proposed methodology
The proposed framework is implemented and the results of experiments are provided in this section. The results include visual inputs, outputs in terms of heatmap and disease detection. Besides, this section evaluates the proposed deep learning based model and compares with the state of the art.
As presented in Figure 2, the input leaf and the detection of early blight disease is visualized. The results also include heat map in different dimensions.
Figure 3: Detection of Late Blight Disease
As presented in Figure 3, the input leaf and the detection of late blight disease is visualized. The results also include heat map in different dimensions.
Figure 4: Detection of Early Blight Disease
As presented in Figure 4, the input leaf and the detection of early blight disease is visualized. The results also include heat map in different dimensions.
| Deep Learning Model | Evaluation Time (Without Retraining) | Accuracy (Without Retraining) | Evaluation Time (With Retraining) | Accuracy (With Retraining) |
|---|---|---|---|---|
| alexnet | 56.440 | 95.747 | 56.477 | 99.245 |
| densenet169 | 93.009 | 98.664 | 93.622 | 99.727 |
| inception_v3 | 140.008 | 97.437 | 244.491 | 99.763 |
| resnet34 | 74.923 | 98.482 | 73.176 | 99.672 |
| squeezenet1_1 | 56.995 | 92.493 | 57.213 | 99.200 |
| vgg13 | 95.137 | 97.955 | 95.122 | 99.491 |
Figure 5: Evaluation time Comparison of Deep Learning Models
As presented in Figure 5, the leaf disease detection models are shown in horizontal axis while the vertical axis shows the evaluation time in seconds. The evaluation time is measured with and without retraining. With retraining, it is observed that there is performance enhancement as it could reduce time taken for evaluation.
Figure 6: Accuracy comparison of deep learning models
As presented in Figure 6, the leaf disease detection models are shown in horizontal axis while the vertical axis shows the accuracy percentage. The accuracy is measured with and without retraining. With retraining, it is observed that there is performance enhancement as it could improve accuracy. With the proposed methodology Inception_V3 with transfer learning showed highest performance with 0.997637223.
We proposed a framework that considers extraction of ROI using deep CNN prior to prediction of pre-trained deep learning models such as VGG13, ResNet34, DenseNet19, AlexNet, Sqeezenet1_1 and Inception_v3. An algorithm named ROI Feature Map Creation (ROI-FMC) is defined to extract ROI for given input image. This will be given as input to another algorithm proposed namely ROI based Deep CNN with Transfer Learning for Leaf Disease Prediction (ROIDCNN-LDP). The latter is used to predict leaf diseases. Plant Village dataset is used for empirical study. Deep CNN models could perform well with the proposed methodology. ROI awareness included in the architecture led to performance improvement. Particularly accuracy of the models has been improved. With the proposed methodology Inception_V3 with transfer learning showed highest performance with 0.997637223. Therefore, deep CNN models with transfer learning and ROI awareness are found to be suitable for leaf disease detection. However, Inception_v3 is found better among the other models. In future, we intend to investigate further on the deep learning models other than CNN [38, 39].
