Weed Detection in Crops Using Computer Vision Presenter: Dr. Yasir - PowerPoint PPT Presentation

Centre of Robotics Weed Detection in Crops Using Computer Vision Presenter: Dr. Yasir Niaz Khan Researchers: Taskeen Ashraf, Danish Gondal, Novaira Noor. http://cs.ucp.edu.pk/index.php/robotics-security/

UCP Robotics Group Centre of Robotics ◦ Faculty ◦ Researchers ◦ Dr. Yasir Niaz Khan ◦ Aamir Ishaq ◦ Dr. Syed Atif Mehdi ◦ Sibtain Abbas ◦ Dr. Musharraf Hanif ◦ Ruhan Asghar ◦ Dr. Oumeir Naseer ◦ Hamad ul Qudous ◦ Muhammad Awais ◦ Noman Saleem ◦ More than 50 undergrad students

Road Map Centre of Robotics ◦ Introduction ◦ Problem Statement ◦ Methodology ◦ Experimentation & Results ◦ Comparison ◦ Conclusion ◦ Future Work

Centre of Robotics INTRODUCTION

Importance of Rice Crop 1,2 Centre of Robotics ◦ Feeds over 50% of World’s population ◦ Pakistan 13 th world wide in Rice production ◦ Pakistan 4 th in Rice Exports ◦ Stands second in terms of staple food in Pakistan ◦ 13% to the total value of Exports ◦ Stands third in terms of cultivation area 3 1. Old.parc.gov.pk, "NARC-Rice||Introduction", 2015. [Online]. Available: http://old.parc.gov.pk/NARC/RiceProg/Pages/intro.html. [Accessed: 20- Dec- 2015]. 2. Bayercropscience.com.pk,. 'Bayer Cropscience - Pakistan : Rice'. N.p., 2015. Web. 14 May 2015. 3. Fao.org,. 'Fertilizer Use By Crop In Pakistan'. N.p., 2015. Web. 18 June 2015

What are Weeds and Weed Control? 1 Centre of Robotics Biological Means 2 1. biological spray -spore suspension of Weeding an endemic Approaches fungus 2. a fish, the white amur Partially Manual automated Hand Hand Herbicides Biological weeding hoeing application means 1. Pakissan.com,. 'Integrated Weed Management In Rice :: Pakistan Agricultural News Chennal-:PAKISSAN.Com:- '. N.p., 2015. Web. 17 May 2015. 2. Eap.mcgill.ca,. "Biological Control Of Weeds". N.p., 2015. Web. 31 sep. 2015.

Issues with Weed Control Methods Centre of Robotics ◦ Difficult to harvest ◦ Disadvantage of uniform spraying ▫ Uneconomical ▫ Affects crop health ▫ Environmental Pollution ▫ Resistance to sprays

Resistance to sprays Centre of Robotics Survey website at http://www.weedscience.org on September 13 th , 2015.

Centre of Robotics Weeds are a Problem o Weed destroys 15-20% or in some cases up to 50% of the crop 1 o Uniform spraying is uneconomical o Control Period of weeds is first 40-50 days 1. Pakissan.com,. 'Integrated Weed Management In Rice :: Pakistan Agricultural News Chennal-:PAKISSAN.Com:-'. N.p., 2015. Web. 17 May 2015.

Centre of Robotics Problem Statement

Problem Statement Centre of Robotics “ Automated localized weed detection in rice fields to avoid excessive uniform spraying; that will result in high, good quality yield with low production cost.”

Problem Statement Centre of Robotics “ Automated localized weed detection in rice fields to that will result in high, good quality yield with low production cost.”

Problem Statement Centre of Robotics “ in rice fields to that will result in high, good quality yield with low production cost.”

Centre of Robotics

Initial Experimentation Centre of Robotics ◦ Testing conducted with few images (DS-1) (1,2) ◦ Broadleaf and sedges 1. Jircas.affrc.go.jp,. 'JIRCAS cyperus Difformis plants In Lowland Savanna Of West Africa'. N.p., 2015. Web. 10 May 2015. 2. Mikobi.deviantart.com,. 'Water Lily In The Rice Paddies Around Angkor Wat'. N.p., 2015. Web. 10 May 2015.

Initial Experimentation ◦ Three techniques Centre of Robotics ▫ Based on localized FFT and Edge Detection 1 ▫ Based on localized Entropy  ▫ Based on Wavelet Transform 2 Accuracy: 89.60 % Comparison Using Accuracy and FPR 100 Techniques Accuracy FPR 90 80 70 60 FFT 74.85% 27.83% 50 Accuracy 40 FPR Entropy 76.66% 24.09% 30 20 Wavelet 89.60% 17.50% 10 0 Localized FFT Localized Discrete Wavelet Entropy Transform 1. Nejati, Hossein, Zohreh Azimifar, and Mohsen Zamani. "Using fast fourier transform for weed detection in corn fields." Systems, Man and Cybernetics, 2008. SMC 2008. IEEE International Conference on. IEEE, 2008 2. Noor, Novaira, and Yasir Niaz Khan. 'Weed Detection In Wheat Fields Using Computer Vision'. Graduate. FAST-NU Lahore, 2014. Print.

Centre of Robotics Experimental setup & Dataset

Experimental Setup Centre of Robotics ◦ Setup ▫ MATLAB 2014 64 bit ▫ Windows 8 64 bit ▫ 4 GB RAM ▫ Core i5 1.70GHz Processor ▫ LibSVM and RF ◦ Dataset ▫ Images taken height of 2-4 ft. ▫ Angle of capture is 90 degrees ▫ Image resolution is 1920x1080

Centre of Robotics Technique 1 Using Wavelet Transform involving Blur Detection

Overall Technique Centre of Robotics Video Extract Every Nth Frame(Image) Blur Trained SVM Blur Detection Module Model Non-Blur Weed Detection Module Output image Calculate Weed Coverage

Blur Detection Centre of Robotics Dataset blur/Non-Blur labelled images Get image one by one Convert RGB to Gray Calculate Discrete Laplacian Extract Features Min, max, std Train SVM (Batch Training) Linear SVM Model

Weed Detection Centre of Robotics Input image Excessive green image Wavelet Transform Thresholding on Diagonal Coefficients Inverse Wavelet Transform Dilation Remove small regions Output image

Steps 1-3 Centre of Robotics Original Image Excessive Green Image Diagonal Coefficient Diagonal Coefficient(Filtered)

Steps 4-5 Centre of Robotics Dilation

Accuracy & FPR Centre of Robotics ◦ Total Frames = 1717 ◦ Total Frames processed = 172 ◦ Non-blur frames detected = 67 ◦ Accuracy of blur detection = 84.88% ◦ FPR of blur detection = 18.46% ◦ Weed Detection Accuracy = 68.95% ◦ FPR = 12.69% ◦ Weed Detection Accuracy after blur removal = 76.16% (8% increase) ◦ FPR = 13.38%

Centre of Robotics Weakness Accuracy drops drastically when texture difference decreases with the growth of grass

Centre of Robotics Technique 2 Using SVM and Random forest with Moments

Dataset-2 Density Based Centre of Robotics

Weed Detection Centre of Robotics Density Based Dataset Extract Green channel from RGB Calculate complex Calculate Mean,variance,kurtosis,skew moments Train Classifier (Batch Training) Calculate n-fold cross validation

Accuracy Using First Four Moments Centre of Robotics 88.00 86.00 84.00 82.00 80.00 Accuracy Linear kernel 78.00 RBF kernel 76.00 Random Forest 74.00 72.00 70.00 68.00 1 2 3 4 5 No. of Iterations Accuracy: 82.22% RBF Kernel SVM C=8, g=0.25

Accuracy Using Complex Moments Centre of Robotics 86.00 84.00 82.00 80.00 78.00 Accuracy Linear kernel 76.00 RBF kernel 74.00 Random Forest 72.00 70.00 68.00 66.00 1 2 3 4 5 No. of Iterations Accuracy: 81.42% random forests with 300 trees

Accuracy Using Combined Moments Centre of Robotics 90 88 86 84 82 Accuracy 80 Linear kernel RBF kernel 78 Random Forest 76 74 72 70 1 2 3 4 5 No. of Iterations Accuracy: 86.06% RF With 300 trees

Centre of Robotics Comparisons Accuracy and Execution Time

Accuracy Centre of Robotics 100 90 80 70 60 Accuracy 50 Moments Feature set GLCM feature set 40 30 20 10 0 Linear kernel RBF kernel Random Forest Type of classifiers Accuracy: 86.06% RF With 300 trees Moments Feature Set

Execution Time Centre of Robotics 35 30 25 Execution time in seconds 20 Linear SVM kernel RBF SVM kernel 15 Random forests 10 5 0 Wavelet Transform with Moments GLCM features blur detection Wavelet Transform Less feature extraction Time

Conclusion Centre of Robotics ◦ Strengths • Different densities of grasses • Multiple backgrounds (dry soil, muddy soil, straw/stalk) • Grasses are a common weed in other crops such as cotton. ◦ Limitations • First technique dependents on growth stage • Threshold of dilation, area removal needs to determined. • Limited to a single type of weed

Topic Centre of Robotics Plants Classification using Hough Line Transform & Support Vector Machine(SVM) Researcher: Umar Muzaffar 52

Tools & Technology Centre of Robotics ◦ Visual Studio ◦ Image Processing( Opencv, C++) 53

Data set Collection Centre of Robotics ◦ All dataset collected from:  University of Central Punjab  Fields  Nurseries 54

Sample Images Centre of Robotics Kangi Palm’s Plant Potato’s Plant Pea’s Plant (Captured from UCP) (Captured from Fields) (Captured from Nursery) 55

Training of data Centre of Robotics ◦ There were total of 9 species which I classified successfully ◦ There were total of 300 images collected ◦ Each specie consist of 33 images. ◦ 31 images were used for testing purpose ◦ 2 images were used for validation purpose 56

Techniques Used Centre of Robotics ◦ Hough Line Transform (To extract different shapes) ◦ SVM (Support Vector Machine) 57