Low cost computer vision implementations for plant - PowerPoint PPT Presentation

New Technologies for Plant Phenotyping Unidad Integrada Balcarce (INTA-UNMDP) 4 de mayo de 2016 Low cost computer vision implementations for plant phenotyping/identification problems Pablo M. Granitto Centro Internacional Franco Argentino de Ciencias de la Información y de Sistemas Consejo Nacional de Investigaciones Científicas y Técnicas Universidad Nacional de Rosario

Outline  Our path here: – Weed seeds – Green seeds – Plant identification using veins – Counting seeds in pods – Stripes in apples Conclusions  The Future 

The beginning: Weed seeds identification (~2000) Colaboration :

Weed seeds identification: Hardware  High-End Equipment – Frame grabber – Special camera – Light source – Etc. Pro: High  Performance Con:High cost! 

Weed seeds identification: Software • Imaging + segmentation • Measurement of diverse features: – Morphological – Textural – Color • Classification with Neural Networks ensembles • Very good results: – +95% correct recognition on 250 species – +99.5 accuracy using the 5 most probable species

Weed seeds identification: The problems • Nobody was willing to pay the cost of the equipment! • High-End video equipment also have problems – Drivers – Replacements – Aging of lamps (COLOR!)

Second attemp: Green levels in soybeans (~2008) Colaboration :

Green levels in soybeans: How to measure color? • We gave up on special hardware! • Low cost solution: – Of-the-shelf imaging device with calibration standard – Software implemented as a web service

Green levels in soybeans: Software • Calibrated Scanner + Segmentation • Feature extraction – Morphological – Color • Clasification with Random Forest (Ensemble of classification trees) • All project based on Open Software (Open CV - R)

Green levels in soybeans: Problems! • Color is really difficult! • Even for us! • We can control the ilumination easily with a flatbed scanner, but translating colours from diverse equipments with high accuracy is very difficult

Green levels in soybeans: Results • Average human accuracy: 65% • Best result for automatic system: 85% • But: – Using a single scanner – Translating from other scanners decrease accuracy to near random results

Cultivar identification using leaf veins (2012) Colaboration :

Cultivar identification using leaf veins

Cultivar identification using leaf veins: pipeline

Cultivar identification using leaf veins: results • Average human accuracy: 45% • Best result for automatic system: 60% • Automatic methods outperforms humans (on cultivars and species) • But results are not good enought as to develop a portable device

Cultivar identification: can we improve? Deep learning

Cultivar identification with Deep learning

Phenotyping: counting seeds in pods (2015) • Semi-automatic procedure: pods are colected from the plant by hand and counted automaticaly with a vision system • Regular camera, cheap ilumination device and a computer • Segmentation + feature extraction • Classification with SVM

Phenotyping: counting seeds in pods

Phenotyping: counting seeds in pods

Phenotyping: counting seeds in pods

Phenotyping: Results • Accuracy +90% • Limits: pods with “new” shapes and size lead to errors • Proposed solution: using deep learning (working now...)

Phenotyping: stripes on apples (2015) • Work in progress with FEM (Trento, Italy) • Goal: develop a low cost device to grade apples according to stripes quality

Conclusions  Machine vision systems based on low cost hardware are useful and easy to develop  Many agricultural applications known  Measuring color in practice is difficult – But you hardly need color in phenotyping  Lots of potential phenotyping applications

The (near) future  Phenotyping – Counting seeds (pods) in live plant  Identification – Identifying weeds in real time video – Collaboration in the development of a weed control autonomous robot

The team  Dra. Mónica Larese  Dr. Rafael Namías  Dr. Pablo Verdes  Dr. Guillermo Grinblat  Dr. Lucas Uzal  Dr. Ariel Baya  Dra. Belén Bernini (former)  Dr. Alejandro Ceccatto (former)  Dr. Hugo Navone (former)  Dr. Roque Craviotto and gruop (INTA OLIVEROS)  Dr. Eligio Morandi and group (UNR – Zaballa)  Dr. Eugenio Aprea (FEM – Trento - Italy)