I feel it’s important to occasionally remind myself why I do what I do, and who I’m doing it for. With that, let me tell you a story.
I was recently asked to give a presentation about spray coverage and drift mitigation to an arborist organization. I agreed but harboured reservations. I’ve given talks of this nature many, many times, but I rarely work with arborists. In preparation I looked back through my files and discovered I’d spoken to them 10 years ago. Coincidently, that was also the last time I’d encountered an arborist.
So, what value could I possibly offer? My concern was that all I’d leave them with was a few “factoids” and the vague sense that they’d been entertained. But would I leave things better than I found them? What could I say that would move the needle and give them something actionable?
Fortunately, I was paired with a veteran sprayer operator and together we worked out a presentation / demonstration. It went over very well, and I was relieved that people were engaged and asked insightful questions. Crisis averted.
I believe the reason it worked was because I asked the operator about the real-world challenges (however unpopular) that he faced. We discussed and agreed upon a few lesser-of-two-evils solutions to share with the group. It was authentic, it was pragmatic, and it was appreciated.
As a result, I decided to dedicate some time this spring/summer to riding along with a variety of sprayer operators as they perform their jobs. If they’d have me, I’d promise to stay out of their hair, acting only as an observer. I wouldn’t make suggestions and I wouldn’t criticize. I would ask the occasional question and I’d watch to see where policy and reality crossed paths.
I was hoping for a few educational experiences that would inform my research trajectory and teach me a few tips and tricks to share at winter meetings. Perhaps I’d reinforce my understanding of spray application, or maybe I’d be forced to re-evaluate my position on what is a technical truth and what is a practical truth. At the very least I would get to see how professionals did their jobs, and which best practices got sacrificed when things didn’t go to plan.
And, while I was at it, I decided to keep a journal to create articles in the vein of “A Day in the Life”. You’re reading the first one right now and I hope you find it as interesting to read about as I did to live it. It’s unlikely you work in all the agricultural spaces I’ll be writing about in this series but keep an open mind. The potential for cross-pollination is enormous; Perhaps your “cousin” sprayer operator has solved a problem you face in your own operation.
And so, given our recent success, my first victim will be my new arborist-friend. You can read all about it here.
A quick selfie in suburbia as I’m guided through a day in the life of an arborist.
One of the fastest moving new agricultural technologies is spray drones. Hardly a month goes by without some sort of new capability, some new features. It’s truly an exciting space to watch.
As with all things, there are good news and bad news to share. First the good news.
Drone capacity is on the rise. The early drones shipped with hoppers of 8 to 10 litres. Part of the reason was to keep weight below 25 kg. Below this weight, pilot licensing requirements and flight restrictions are easier. Anyone with a Basic RPAS license (RPAS is the official term for drones, Remotely Piloted Aircraft Systems) can operate drones up to 25 kg. Above this weight, one requires an Advanced license, which is much more difficult to obtain. Current drones like the DJI T40 have a hopper capacity of 40 L, allowing more area to be covered per flight.
The new DJI T40 holds 40 L of liquid and has a claimed swath width of 36 feet (Source: DJI)
Swath widths are increasing with drone size. The limiting factor for electric drones is still battery power. Flight times of 15 to 20 minutes are possible, depending on the ferrying distance. As a result, larger drones don’t necessarily fly longer, but they spray wider, up to a claimed 30 feet for the DJI T30, and 36 feet for the T40.
Atomizers are improving. The trusty flat fan nozzle certainly works on a drone, but its proper operation depends on spray pressure. And spray pressure is not currently reported by drones. Instead, their application software relies on flow rate, and pressure is adjusted in the background in response to changes in travel speed, swath width, or nozzle size. Although drone flow meters are remarkably accurate, the operator could inadvertently operate the drone at a pressure that produces the wrong spray quality for the conditions.
Enter the rotary atomizer. Long a darling of the thinking applicator, these atomizers use centrifugal energy to create a spray with a tighter span, meaning fewer fine and fewer large droplets. Spray quality still depends on pressure-generated flow rate, but droplet size can additionally be altered with rotation speed. This means that if a faster travel speed increases the spray pressure, the effect on spray quality can be counteracted with a changed rotational speed to keep everything more uniform.
Rotary atomizers, like this one from XAG generate more uniform droplet sizes and can alter droplet size without changing spray pressure.
Hybrid systems are entering the market. Rotary wings allow for precise positioning of aircraft and they provide downwash that helps spread the spray pattern out. Downwash also improves canopy penetration and could reduce drift, like air-assist, if used properly. But rotary wings use a lot of energy, limiting battery life. When flown at the wrong height or speed, deposit patterns, drift, and swath width will change. That has to be managed and requires experience.
In comparison, hybrid drones have fixed wings for flight and rotary wings for take-off and landing. The rotors just rotate into the position needed at the time. Fixed wing drones will fly faster, possibly improving capacity and also reducing the effect of the downwash. These systems are new, and much needs to be learned before we understand their various characteristics. But they offer a nice avenue into more productivity.
Hybrid drones like this one from Advanced Robotics can cover more ground with less turbulence than a rotary wing drone.
Drones are multi-purpose. Virtually all drones have interchangeable wet and dry hoppers so they can be used to apply dry nutrients or seed as needed. That makes them quite versatile. But the newest spray drones have scouting-quality cameras on board and can be asked to take high resolution images while they’re spraying. At the end of the mission, a very detailed picture of the crop emerges, with much higher resolution than the higher-elevation scouts produce. Other sensors on the drones can be used for variable rate application of nutrients, or even for spot spraying weed patches.
Scouting camera takes pictures while conducting a spray mission (Source: DJI)
Now for the bad news. It’s still not legal to apply mainstream pesticides using drones in Canada, and it may stay that way for a while yet.
Pesticide application by drones remains illegal in Canada. The main reason is that the Pest Management Regulatory Agency (PMRA) has declared drones to be unique application method, separate from ground sprays and aerial sprays from piloted aircraft. This has triggered the need for risk assessment data for spray drift, efficacy, bystander exposure, crop residue. It’s a fair decision – drones produce finer sprays than any other existing system, they potentially use lower water volumes by necessity, and they create a less predictable deposit due to rotor downwash. The majority of current pesticide formulations are designed for 5 to 10 gpa, this creates a certain concentration of surfactants and products that interact with plant surfaces or that change the potency of drift. Altering this by a factor of 5 can have undesirable outcomes. Yes, aircraft also use lower volumes, but more in the area of 2 to 5 gpa. Drones could cut that in half again, and that warrants study.
Registrants haven’t rushed to study drones. Most major manufacturers of pesticides have a small drone program to get their feet wet, and most have applied for Research Authorization (RA) from the PMRA to study them. But the decision to register a drone use for a pesticide has much to consider. Is it worth it to generate the required dataset for the regulators? Will drones amount to a lucrative new market for product? Do we have the resources and expertise to service this new market? The answers to such questions are clearly complex and much remains unknown. The registrants’ caution is understandable.
There may be a small portfolio of available products. Anyone thinking that a fleet of inexpensive, nimble drones will replace their ground sprayer is banking on the registration of the products they need in their operatioin by the registrants. The most likely products to be registered are fungicides, for which drones would offer several advantages in canopy penetration and spraying in tight time windows due to, say, wet weather.
Another obvious use is in industrial vegetation management where rough terrain or remote locations make it difficult to use wheeled sprayers. Or vector control with larvicides, which, incidentally, comprise the first pesticide registrations for drones in Canada (two microbial mosquito larvicides were approved for drone use in October 2022).
But it seems unlikely in the short term that a producer would have their pick of products to apply by drone anytime soon. And this means that a drone would remain a supplemental tool on the farm, not the main workhorse.
Regulatory hurdles are substantial. Not only is a pilot required to be licensed to use drones, a pesticide application also requires a Specialized Flight Operations Certificate (SFOC). In general, SFOCs are required if:
you are a foreign operator (i.e., not a Canadian citizen or permanent resident);
you want to fly at a special aviation event or an advertised event;
you want to fly your drone carrying dangerous or hazardous payloads (e.g. chemicals);
you want to fly more than five drones at the same time.
SFOC applications are fairly easy to fill out. Aside from identifying the drone and the pilot, the application needs the purpose of the mission, the location of the mission, and the time period of the mission. The problem is that it may take up to 30 days to hear back for simple missions, 60 days for complex mission. And if the SFOC is not granted, you can’t fly. You can’t decide to spray a field at the last minute.
The news is clearly a mixed bag. We have it all – exciting technology, obvious niche in the marketplace, significant regulations, slow process. In the meantime, spray drones are legal to purchase and relatively inexpensive. And we know they are being purchased. Canada doesn’t have a strong compliance system within the PMRA, so it’s hard to know how much pesticide spraying is being done illegally, or how perpetrators will be treated by the law.
The reputation of the industry once again rests with hope that good decisions are being made by conscientious individuals.
No matter where you live, it’s been a long time since the last “normal” ag trade show. The pandemic forced a break in this long-standing tradition, and it wasn’t easy. Trade shows are an integral part of doing farm business. At a time when there are fewer equipment dealers offering a lower diversity of makes, a trade show may be the only place to see and thoroughly inspect what other manufacturers offer. With that in mind, I spent a full three days at the 2022 Ag in Motion Show near Langham, Sask. to see the sights.
On the sprayer front, a lot has happened in the last few years. The strength within the “new kids on the block,” the Netherlands’ Agrifac sprayer and Germany’s Horsch Leeb, is noteworthy. Even with fledgling dealer networks, they’ve managed to sell dozens of units based on the strength of convincing features. These go beyond exceptionally wide booms and large tanks, and include smart plumbing designs with recirculating booms and easier cleanout. They also weigh less than their North American counterparts despite standard features like four-wheel steering and superior boom stability. Most importantly, both manufacturers are committed to innovation and offer quick iterations within their models, incorporating new ideas constantly. Often, it’s the small thoughtful things that save time and create endearment.
The Horsch-Leeb sprayer
Case IH showed their new generation 4450 series sprayer. The machine has many improvements over the 4440 edition, focusing on operator comfort, digital integration and hydraulic capabilities. It weighs more than its predecessor, as to be expected. The Millennium spray boom is great, but unfortunately, there is no factory-designed recirculation, leaving owners to go to the after-market for solutions–a missed opportunity.
The Case 4450 with Millennium boom
Since acquiring Miller Sprayers in 2014, New Holland has had a unique front-mount boom design, and one was on display. Nicely engineered, I’ve always liked the concept of a front-mounted boom for better visibility of the things that matter. But again, there’s no recirculation. Did they consider it yet feel it wasn’t a high priority? We know that recirculating booms perform a waste-free prime while avoiding cleanout problems with boom-ends. These are important issues for applicators. I’m also curious why NH hasn’t gone to the Wilger boom components and nozzle bodies for IntelliSpray, since the Wilger tips are still the best choice for PWM.
Front-mounted boom on New Holland SP410F
John Deere’s newly named sprayers were at the show, the new models also offering improvements in operator comfort. The one on display was fitted with its recirculating boom, a nice addition to the feature set. Like the Raven retrofit kit, it appears to be designed with 200 gpm flow needs in mind, necessitating additional pipes and hoses that other systems avoid. Such high flows are very rarely needed except for 28-0-0 topdressing. Still, kudos for having a factory installed, controller integrated option.
AGCO showed off its Fendt Rogator and my heart went pitter patter as I approached it, recalling seeing its namesake at Agritechnica 2019. At the time, I felt it was one of the top sprayers at the German show owing to its innovative frame and suspension systems (independent wishbone), unique pump system (centrifugal pump that can run dry and never lose prime) and efficient plumbing. But this Fendt Rogator was a traditional Rogator frame with a Fendt tractor hood and cab to fool the passer-by. It’s no slouch, with optional four-wheel steering, adjustable clearance, and Liquid Logic recirculating booms. I felt cheated, nonetheless.
Fendt Rogator, sort of.
A visit to the Apache display showed why they remain a valid sprayer option. Built on mechanical two-wheel drive, they save weight and have better fuel economy than their hydrostatic counterparts. Yet, they still offer large tanks and aluminum Pommier booms when requested. Like most other brands, Pulse-width Modulation (PWM) is an available option. Personally, my soft spot for lighter sprayers is due to analysis of logistical efficiency–fill, clean and transport times. These create competent productivity values even with smaller sprayers that are more affordable to own and operate.
Light and nible Apache with strong Pommier aluminum boom
That led me to PhiBer Industries, creators of the DASH tender system. The family-owned firm from Crystal City, Man. creates custom solutions for hauling water and metering pesticides to sprayers. Thoughtful designs and use of air-driven product pumps with volumetric metering creates an efficient system that’s easy to understand and use. PhiBer joins existing products from Pattison Liquid, Free-Form Plastics or The Handler to offer a basic inductor and pump system. They will also build a complete custom trailer incorporating numerous additional features.
Phiber DASH tender system
Weed detection was shown by three exhibitors, but only one demonstrated it in action. Croplands had its field-proven green-on-brown WEEDit unit installed unit on a customer’s John Deere R4045, available for a test drive, as well as offering technical support at a booth. WEEDit remains the standard for green-on-brown and works very reliably out of the box. John Deere had an installed See & Spray Select, also green-on-brown, on a sprayer, but did not run it.
The most exciting new item sat at the Agrifac booth with their AiCPlus feature courtesy of Bilberry. Bilberry is a small French company, recently acquired by Trimble, selling green-on-green spot spray retrofits in Australia for over a year now. In Canada, they offer four algorithms at this time: green on brown for general burndown or desiccation; green-on-green for broadleaf weeds in cereals; grassy weeds in canola; and, both broadleaf and grassy weeds in corn. Powered by colour cameras and NVIDIA processors, continuous advancements in the software will improve performance as more weeds are added to its list of capabilities. Two AICPlus machines are running in Manitoba, and it’s good to see the company focusing on this market. John Deere and Greeneye, for example, are focused on the US row-crop market for the time being.
Weed detecting cameras and processors by Bilberry, featured on Agrifac AIC Plus
Perhaps the most ambitious spray technology was shown by Precision AI. Utilizing a large-fixed wing drone, the company aims to detect and spray weeds while operating at 60-to-70 km per hour. A hybrid power unit has an internal combustion motor that generates electrical power for rotors. This type of design offers significantly longer flight times than the 15 minutes currently possible with battery-driven units. Fascinating.
Precision AI concept spot spray drone, fixed wing with vertical take-off and landing
Spray drones also featured in the demonstration space at Ag in Motion despite the fact that this application method remains illegal for pesticides in Canada. It’s astounding what types of advancements each year brings. DJI is the global leader with three models available in Canada, including the 8-litre capacity T10, the 20-litre T20, 30-litre T30, and soon, the 40-litre T40. XAG, another leading manufacturer, showed a new two-rotor unit with rotary atomizers for excellent droplet size control. Flow management is nicely handled by accurate flow meters, but pressure sensors and displays are lacking in all units I’ve seen. Given that pressure is a big determinant of spray quality for hydraulic nozzles, that’s an oversight. We’ll continue to wait for this application method to be legal for pesticides in Canada.
XAG drone with two rotors and two rotary atomizers
As always, the show was full of old friends and catching up was welcome after the recent isolation. The opinions and advice from our peers play a big role in agriculture, where relationships and reputation still govern business alongside product features and cost. Trade shows are the perfect place for all of that to come together. Welcome back, old friend.
“Precision agriculture” is many things to many people. In the context of spraying, let’s define it as “detecting and responding to variability”. One example of precision ag is the use of crop-sensing optics to efficiently and accurately direct spray application. This is nothing new to field sprayer operators, but did you know that before Ken Giles published the first paper on pulse-width modulating nozzles in 1989, airblast sprayers already had crop-sensing technology?
In the 1970s, Bert Roper noted the wastefulness inherent to citrus spraying. Losses to the ground of 30-50% and off-target drift of 10-20% of applied volume were (and still are) not uncommon for airblast sprayers. So, using Polaroid’s autofocus technology, and enlisting the help of a few engineers, they developed an ultrasonic sensor system that enabled a computer to “see” the target tree and engage nozzles accordingly. He and son Charlie built prototypes in their kitchen before proving it in their family groves, spraying 10 gal/ac instead of the usual 250 gallons. The first Tree-See system was sold to Cola-Cola in 1984.
Figure 1 Tree-See on a Swanson sprayer (www.treesee.com)
This technology is still used today; Sensors detect specific zones on the canopy and actuate boom sections, or individual nozzles, to only spray the target zone. But optics and machine learning are evolving. Now they can modulate flow from individual nozzles in response to changes in canopy density. To be clear, that’s not just “on/off”, but variable flow.
Eventually, these systems will be able to identify and respond to specific pests (or pest damage) and adjust plant growth modifier rates based on canopy density or bloom counts. The possibilities are amazing. As an aside, interested readers can learn more about airblast sensors in this excellent article from Oregon State University which one of the authors later summarized for us here.
Figure 2 LiDAR and control interface for a Smart Apply system fitted to a Turbomist sprayer
However, as operators embrace this technology, they should be aware of the current limitations. Canopy-sensing optics are great at managing waste (their primary selling point seems to be pesticide savings), but this depends on crop morphology and planting architecture. It makes sense to not spray what isn’t there, but the gaps may not be as big as you think.
Non-continuous canopies require the spray to lead and lag to some extent before and after passing the target to ensure sufficient coverage. Given the difficulty inherent to spraying to the tops of tall canopies, some specialists believe the top nozzles should never disengage. And, in the case of uniform canopies that form continuous hedge-like rows, the potential savings is greatly reduced.
Further, all of these systems assume that application efficiency is primarily dependent on matching liquid flow rates to the profile (or perhaps density) of the target canopy. I don’t believe that’s true. At least, not entirely true. The impact of air settings on coverage efficiency and efficacy seems to have been marginalized.
For example, these sprayers do not account for the spray’s ability to span the distance from nozzle to target (i.e., transfer efficiency). That depends on the droplet size, sprayer air settings and the environmental conditions – none of which are monitored by sprayer optics. They also cannot “know” if the spray gets intercepted by the target (i.e., catch efficiency) or if it deposits a biologically-active residue on the target surface (i.e., retention). Droplets must be retained by the target surface and not bounce or run off.
What this means is that these sprayers, like any sprayer, can only promise “coverage potential”. Operators are still required to perform the following tasks:
Optimize air direction and air energy in relation to canopy size, travel speed and environmental conditions.
Use water-sensitive paper, or some other means of quantifying coverage, to ensure your target receives threshold coverage.
Monitor and adjust practices throughout the season in response to changing conditions.
HOL’s Intelligent Spray Application (I.S.A.) system employing Weed-It sensors.
So what’s missing? How do we progress beyond what is arguably a sophisticated rate-controller?
In my opinion, I believe the pitcher needs a catcher – a closed-loop feedback system. Optics would identify the target, nozzle flow would respond, and then a digital spray sensor in the target canopy would detect and report coverage back to the sprayer so machine learning could make iterative adjustments in real-time.
Spray-sensors are not a new idea, as wetness-detection systems have been used in forestry since the 70s. But, a sensor that can discern spray coverage would yield far more detail, and once again it seems Ken Giles is a pioneer in this concept. Such a sensor, integrated with sprayer optics and machine learning, could summarily account for all the unknowns that interfere with spray from the moment it’s released to the point that it (hopefully) lands. That’s some serious crop-adapted spraying.
And yes, it would be fantastic if there were some manner of anemometer tied to a baffle or louvers in the spray head. Air energy could be balanced between up- and down-wind sides, and further adjusted to compensate for the distance to the canopy… but I’m dreaming in technicolour, now.
Until then, sprayer eyes can only blindly dictate the release of spray into the airstream based on an assumed coverage constant (e.g., 1.2 oz./ft3). It remains for the sprayer operator to act as the brain, optimizing sprayer settings, quantifying coverage, and making changes to reflect conditions.
Learn more about how to optimize the fit between your airblast sprayer and your target by downloading a free copy of our Airblast 101 textbook.
The idea of controlling weeds individually instead of treating the whole field uniformly makes a lot of sense. Why waste herbicides, till soil unnecessarily or use other weed control methods on areas without weeds? Besides reducing unintended environmental impacts, it means cost savings for the farmer, reduced crop stress and the opportunity to concentrate efforts where they are needed.
In this era of widespread herbicide resistance, the latter opens the door for new weed control tools – such as lasers or electrical weeding (Carbon Robotics as one example). Unfortunately, all the above absolutely relies on the ability to reliably recognize weeds in all manner of crop-weed conditions. Enter cameras, computers and cake recipes.
Besides alliteration, they are all connected by their role in real-time, in-crop weed recognition. So why isn’t weed recognition already widely available and how will this trio change that? Well, the way I look at it, if our faces were weeds, this problem would have been solved a long time ago. Said differently: new technologies mean that this challenge can be addressed with the necessary focus, investment, and research.
In the last few years, there have been step changes in research and development for real-time weed recognition, which are driving rapid gains for in-crop site-specific weed control. We are seeing this with the emergence of image-based green-on-green (GoG) see-and spray systems globally, many of which are listed in the table below.
Yet, as the last 50 years of plant detection, identification and recognition research have shown, reliable weed recognition is a challenging problem to target. The aim of this article is to take you on the journey of weed recognition – from simple plant detection for thinning in the 1970s to every metre of a boom equipped with camera-computer-cake recipe combinations. Fortunately, we are well on this path toward more effective weed recognition.
Green-on-brown weed detection
As far as the available research shows, the first attempts at plant detection were made for thinning sugar beets in the early 1970s. The method is impressive in its simplicity – two sensors (photodiodes) that generate a signal based on incoming light intensity, are each covered by a filter that only allows specific wavelengths of light through. By knowing the reflectance spectrum for plants and comparing the ratio/output of these two sensors means you can detect if a plant has entered the field of view, but not necessarily exactly where it is.
In this case the weed detection ‘algorithm’ is a ratio of sensor values and some predefined threshold, which can be adjusted as a form of ‘sensitivity’. The concept is largely the basis of WEEDits and WeedSeekers today. This system has all the principles of advanced image-based systems: (1) data stream from a sensor/camera + (2) computer running a weed recognition algorithm + (3) some form of actionable output (e.g. turn on a nozzle).
Image-based weed recognition
With the fundamentals of SSWC largely consistent between sensor and image-based systems, the interesting details and drivers for GoG technology emerge if we dive into the data stream (images) from the camera and the algorithm running on a computer.
At its most basic, a digital colour camera is a sensor that generates signals based on incoming light intensity. The difference with the photodiodes is that a camera records this reflectance intensity information for every pixel in the camera across the red, green and blue (RGB) parts of the spectrum. For example, a 12-megapixel camera has 12 million pixels reporting reflectance intensity for each RGB channel. That means 36 million individual numbers generated for every photo. Learn more about the basics of digital imaging, here.
When you bring this together in an image, you have information on object relationships in space, providing not just a ‘spectral’ dimension (RGB) but also a ‘spatial’ dimension. The use of computers to understand image content is known as computer vision. Having all this data (colour and spatial information) means there is a lot more to work with when differentiating two plants, increasing your chances of success. The downside of having more to work with, is having to work with more! In this case, the computer needs to deal with the 36 million numbers it receives 30 times per second.
The next part of this weed recognition puzzle is the computer and associated weed recognition algorithms, which receive the incoming images and determine if there is a weed in the image. In the case of ‘conventional’ or non-convolutional neural network (CNN) methods (we’ll get to those later), this analysis process is largely formed of four stages shown in Figure 2 – (1) pre-processing, (2) segmentation, (3) feature extraction and (4) classification. If you’re interested in the details on the volumes of research done in this space, I’d highly recommend this review by Wang et al., 2019.
Figure 2 Overview of the image analysis process including convolutional neural networks (CNNs) that automate much of what is done by hand in the more conventional methods with manual feature extraction. Adapted from Wang et al. 2019.
In the case of a simple colour-based detection system that just needs to find green plants in fallow, like our DIY weed detector the OpenWeedLocator (OWL), the algorithm is largely a threshold on the green colour channel using the RGB colour space. This carries some risk – for example if the lighting changes substantially or if weeds aren’t green the method can break down. Yet we went down this path because of the simplicity and speed with which it can be used on the rather computationally constrained Raspberry Pi.
Our field testing also showed acceptable levels of performance in variable fallow conditions. We managed these issues by combining multiple colour-based algorithms; relying on greenness in the RGB colour space from the ‘excess green’ (ExG) vegetation index, combined with thresholds in the hue, saturation, and value (equivalent to brightness) (HSV) colour space to avoid false detections on over/underexposed regions, which often occur in stubble. Even with these adjustments, the system is prone to errors, but the benefit of using cameras is that they can be levelled up for in-crop detection to more advanced algorithms.
The ‘green detection only’ approach without machine learning that effectively exits Figure 2 at the segmentation stage is also likely in the initial launch of John Deere’s green-on-brown See & Spray Select ™. Probably for the reasons above, they also warn against use close to sunrise and sunset where lighting is changing rapidly. As expected, the system pivoted to in-crop detection with deep learning in early 2022 because images and embedded processors allow software-only changes for green-on-green.
If weed species classification or crop-weed discrimination is needed for green-on-green use, then the remaining two stages of feature extraction and classification are required. In the conventional process, someone selected which plant attributes (known as image features) you wanted to use, trained an algorithm on those features and then ran it in the field, a method generally known as machine learning. In spite of this more advanced approach the performance drop between the test dataset and the variable field conditions meant the method was still commercially unusable in large-scale systems. So, what has changed?
Well, in 2012 a research group managed to substantially outperform all these other methods with an algorithm known as a convolutional neural network (CNN). Instead of an ‘expert’ identifying which plant attributes in an image were important, the algorithm itself could select and learn which features were most important, making it more robust. The CNN effectively skips all the steps in the conventional process (Figure 2), instead replacing them with having large quantities of training images with weeds manually highlighted – a newfound bottleneck itself, but not insurmountable.
Part of CNNs robustness comes from the algorithms being capable of analyzing many dozens of features and combinations of features that wouldn’t necessarily be obvious to humans. In the training process, it tests one combination of features before correcting itself based on the training dataset you’ve provided and testing another. One of the biggest improvements is that by doing this feature extraction and selection process automatically, it removes the slow and somewhat qualitative process of deciding which features/plant attributes were most important. Besides this, the algorithms are incredibly large. Some of the modern CNNs have over 100 million (!) different dials that can be automatically tuned to learn the patterns of a weed.
Combined with these algorithms, we now also have low-cost (<US$150) credit-card sized computers such as the Raspberry Pi (when it has additional support) and Jetson Nano that can run these algorithms real time, or around 15 – 20 frames analyzed per second. Even when processing millions of pixels through CNNs with over a 100 million parameters 15 times per second, they consume very little power and can fit easily on agricultural equipment. Genuinely mind boggling every time I think of the sheer scale of it.
Figure 3 The fully assembled OpenWeedLocator without the cover showing all the parts necessary for a site-specific weed control system. Camera at the front, a credit card-sized computer in the middle (Raspberry Pi) and a relay control board at the back to activate solenoids for spot spraying. The OWL is an open-source, DIY weed detection system and can be accessed here.
That leaves us with the cake recipe – how does that fit? Well, the step-changes in technology, particularly deep learning for image analysis can be largely attributed to the use of open-source software, data and hardware. Accessible datasets gave rise to the first effective CNN; open-source deep learning libraries (e.g. Tensorflow and Pytorch) to widespread adoption and development and open-source and/or low-cost hardware to field-scale implementations of the work. The best analogy I have heard used to describe an open-source approach is that it’s like sharing the recipe for a cake – except the code/assembly guide are the recipe and the ingredient list all the tools/languages/packages/components required to make it work.
Even though I could make an average chocolate cake with the ingredients in my pantry, I’ll still go and buy one for many different reasons – quality, convenience or support/returns in case it doesn’t quite live up to standards. The emphasis in this approach is on the quality of the entire product experience not necessarily a secret combination or method of combining ingredients. Plus, it means everyone with the basic tools can try making the cake or training the algorithm, discovering opportunities for fixes, optimization or low-hanging fruit that may change its use case entirely.
I mean, Australians took sponge cake and made lamingtons! In my own experience with the OpenWeedLocator, we built a device for detecting green weeds in large-scale fallow situations. But in true open-source fashion, this has now been used for site-specific fungicide sprays, desiccant application and under trees for weed control. A Canadian innovation – AgOpenGPS – developed by Brian Tischler is an open-source GPS steering system for tractors, similarly enabling farmer-driven development. The examples of different uses are quite extraordinary.
One of the main tenets of open-source technology is that by allowing people to see the details of software and hardware, a larger and more diverse array of people can examine the code and any inefficiencies and errors can be picked up faster. Besides this, it makes research and development accessible to those that might need the technology – the farmers – instead of it being locked away in large companies with inaccessible customer support. Farmer-driven innovation has a long and successful history and open-source development facilitates this continuing to occur in the era of agritech.
Over these last 50 years of development, the cameras, the computers and the open-source recipes have each contributed at different points to site-specific weed control. It seems that now they are converging in agriculture in a storm of interest and development for weed recognition and targeted application.
