Developing Robust Systems for Lettuce Thinning and Phenotyping Jim Ostrowski 2015-04-28
Outline What is Blue River? What is Lettuce Thinning? Why Lettuce Thinning? Blue River Automated Thinning Key lessons What’s next for lettuce? Other ventures: High throughput Phenotyping Working with breeders Weeding for other crops Plant-by-plant care 1
Blue River…? • “Advanced Technology for Better Agriculture” • Focus on computer vision and robotics in agriculture • Co-founders Jorge Heraud and Lee Redden • Met at Stanford and developed idea in Steve Blank’s Entrepreneurship class • Researched a variety of ideas, including automated mowing (e.g., golf courses) and carrot weeding • Based in Sunnyvale, CA: • Started in 2011 • 31 employees • Initial funding from Khosla Ventures • NSF SBIR Grant • First commercial revenue in May, 2013 • Goal of reducing chemical usage and improving existing agricultural practices • No one wants a Green River… 2
31 world-class employees Diversity of backgrounds & education 3
Lettuce thinning ����������� �������������� • Farmers overplant lettuce and then thin • Lettuce grown locally in Salinas Valley • E.g., plant seed every 2” and thin to • Existing, but very slow, manual process plant every 10” • Labor shortages • Driven by poor germination in lettuce • Grown year-round plants • Operate as a service • Thinning traditionally a dull, slow manual • Stay close to customers process using a hoe • Don’t have to perfect a robotic system • ~40 people / 25 acre field / day • About 20-30,000 plants per acre 7 1 2 3 4 5 6 8 4
Our approach • Camera used to identify plants • Determine plants versus weeds and best plants to keep • Spray fertilizer on early stage plants • This is toxic to young plants and kills them • Provides some residual fertilizer to remaining plants • AVOID spraying on plants you want to keep…! 5
Decision-making overview Target spacing 7 1 2 3 4 5 6 8 9 10 11 12 13 14 15 Target spacing (inches): 10.0 Minimum spacing (inches): 9.0 6
Decision-making overview Target spacing 7 1 2 3 4 5 6 8 9 10 11 12 13 14 15 Never keep these plants Target spacing (inches): 10.0 Minimum spacing (inches): 9.0 7
Target spacing 7 1 2 3 4 5 6 8 9 10 11 12 13 14 15 Blue River’s ¼” precision minimizes doubles left in the field 8
Target spacing 7 1 2 3 4 5 6 8 9 10 11 12 13 14 15 Killed plants Killed plants Blue River’s ¼” precision minimizes doubles left in the field 9
Target spacing 7 1 2 3 4 5 6 8 9 10 11 12 13 14 15 Keep the next plant Optimize Eliminate spacing all doubles 10
Target spacing 7 1 2 3 4 5 6 8 9 10 11 12 13 14 15 Killed Killed plants plants Keep the next plant Optimize Eliminate spacing all doubles 11
Target spacing 7 1 2 3 4 5 6 8 9 10 11 12 13 14 15 Balance spacing and doubles in each decision Optimize Eliminate spacing all doubles 12
Target spacing 7 1 2 3 4 5 6 8 9 10 11 12 13 14 15 Balance spacing and doubles in each decision Optimize spacing 13
14
Main vision system • Shrouded system to control lighting (mostly) • RGB GigE camera • Processes plants at roughly 5-10 Hz • Required at speeds of 2.5 – 3.0 MPH • Use color, gradients, shape, size to identify plants • Classifies plants: identifies plant centers and boundaries • SVM used in classification; exploring deep learning • Filters out most weeds • Better than 98% detection rates • Less than 5% false positives (weeds as plants) 15
About our machine • In Ag parlance, our system is called an “Implement” • High-speed vision system • Takes action on ~1,000,000 plants per day per machine • NOT fully autonomous • Implement is pulled by a tractor (which is manually driven) • Autonomy would be cool (challenging to navigate turns), but not economical • 100-fold increase in speed and width since first prototype 16
About our machine • Modular system • Complete processing and spray system for each row of lettuce • Reconfigurable in the field to different planting formats − From 40” beds with 2 seedlines per bed to 80” beds with 6 seedlines per bed 17
About our fleet Six fully capable machines Yield increases of ~10% • Each is capable of thinning 15-30 acres / • More precise spacing day • No damage to root structure by hoe- − Roughly 40 times faster than manual strikes on neighboring plants thinning • Centralized, remote management software • People, trucks, trailers, etc. operate as a daily service to the growers 18
Challenges • Engineering challenges • Rugged outdoor system with a LOT of computing power • System simple enough to be fully operated by tractor driver • Fertilizer destroys everything that’s not stainless steel (or plastic/rubber) • Sprayer’s solenoids get tens of millions of cycles per year and get sticky/slow • Anything that can be walked on or hit with a hammer will be… • Economic risk • At 20,000+ plants per acre, each acre is worth about $10K • At 4-5 acres per hour, machines can do a LOT of damage very quickly • How to safeguard against this? We employ a secondary vision system to self-monitor and self- calibrate our systems “Collinear” sprays 19
Image pipeline walk-through 20
Image pipeline walk-through 21
Image pipeline walk-through 22
Image pipeline walk-through 23
Image pipeline walk-through 24
Next opportunity: Weeding • Natural next step is to do weeding in lettuce and other vegetable crops • Requires several small changes to system, e.g., spray accuracy, material, vision • Approx. $25B spent annually in US on weeding (including corn and other commodity crops) Spray weeds without harming stand 25
Other work: High-throughput phenotyping Breeding is the key to higher productivity & phenotyping is the key to breeding “ Phenotyping limits the ability to derive full value from the DNA ” – N. America Field Breeding Leader, Dow Agro Science, @ corn breeding school 2014 Most phenotypic info gathered by hand today 26
Phenotyping as a service High-throughput, in-field data Multiple sensors collect robust collection plant-by-plant data PASSIVE Visual + NIR + Thermal ACTIVE Incoming wide spectrum radiation Reflected (LEDs) wavelengths Scanning laser Absorbed (LiDAR) wavelengths 27
CONFIDENTIAL Multiple metrics can be gathered from every plant Planned metrics Visible light Nitrogen Leaf Area Index Time of measurement Geospatial position Distance to nearest plant #174 #174 N = 1.7% LAI = 1.2 Planting density Height of top leaf Number of leaves Leaf angle #175 #175 Leaf width N = 1.8% LAI = 1.4 Projected leaf area LAI #176 #176 Lodging (angle) N = 1.6% LAI = 1.3 Stalk diameter Tassel size # of ears Ear length #177 #177 Ear diameter N = 2.2% LAI = 1.6 Water potential #178 #178 Nitrogen content N = 2.3% LAI = 1.7 Stomatal conductance NDVI TCARI/OSAVI CCCI CWSI PRI … open to others 28 28
Recommend
More recommend
