Introduction

Across the growing season, fruit yields fluctuate. Fruit loads vary as the plants respond to changes in temperature, disease load, humidity and other factors. These fluctuations present several challenges for growers. If actual yield exceeds what the grower expected, they may need to:

• Add members to their picking team
• Fold additional boxes
• Request packing line staff to undertake overtime
• Revoke an agreement to allow another grower to use their in-house packing line
• Secure an additional truck at short notice
• Find a buyer to purchase the additional fruit, often at a lower price.

The dynamics of the labour and produce markets vary between different crops and regions. So, the relative cost of these actions will vary. Accurate yield estimates are particularly important on large farms and in industries where selling fruit at short notice requires heavy discounting.

Image 1: Ambit Robotics trial at Bundaberg eggplant farm

A similar but inverted situation arises when yields underperform on expectations. The labour, transport and packaging capacity that the grower planned to use is partly underutilised, resulting in losses. The grower also faces repercussions from any customers whose orders can’t be fulfilled.

The issue yield variability

While yield swings have a significant bearing on operational costs, they also have an impact on capital requirements. Costly capital items, including packing and harvesting equipment, must be sized to the greatest possible yield discrepancy from the initial budget projection. That is, a grower that can achieve high accuracy estimates requires less equipment, as they can allocate what they have more efficiently.

To limit the impact of these yield swings, most growers will undertake a manual count in the lead-up to harvest to support decisions about transport, labour and packing, but also to reach contractual agreements with buyers. However, consistent hand counting is costly, and not always accurate.

Hand counting suffers from a variation in counts between team members. In many cases, only one or two team members are qualified to perform manual counting. This is particularly true in the case of longer-term projections, as they typically require significant experience. Such a limited pool of qualified counters presents a challenge when experienced counters are on leave and introduces an operational risk when these people exit the company.

A potential robotic solution

Image 2: An imaging robot was used in a trial to estimate eggplant yield near Bundaberg.

Automated yield projection technologies have been developed over several decades. The most widely used are based on mathematical crop models, which use statistical techniques to infer yields from measurements such as humidity, temperature, leaf area and carbon dioxide availability. These models provide some guidance to growers about likely yields.

Throughout 2023, the VegNET Queensland Wide Bay Burnett program set out to trial a new approach to yield estimation that showed the potential to add significant additional information to these statistical models, by automating the direct measurement of fruit load. The trial was established to determine if direct imaging of the fruit had the potential to improve yield estimates.

Ambit Robotics, a Melbourne-based start-up, was selected to provide the imaging technology and test its viability at an eggplant farm located at Moore Park, near Bundaberg in Queensland.

The Ambit team provided and installed an imaging robot at the site. The Basis+ robot was located in a ‘hutch’ near the door of the greenhouse. It was programmed to wake up each night and image hundreds of pieces of fruit.

About the Basis+ robot

Image 3: The robot was designed to operate at night and used artificial lighting to illuminate the eggplant fruit.

The robot is a 4WD unit with a camera fixed above the main chassis. The camera can be tilted to capture the fruit.

The robot uses a set of laser-based sensors to navigate down the crop rows, and remain central within the row. Consistency of navigation proved to be important, allowing each piece of fruit to be imaged in the same way each night. This proved critical to spotting trends in fruit size and quantity.

The system is designed to operate at night to avoid colliding with staff and equipment during the day. Nighttime operation also allows for the use of artificial lighting to illuminate the fruit. For the eggplants, the light proved beneficial because each piece of fruit reflected the light, which improved fruit detection even when the fruit was partly covered by leaves.

Fruit detection and calibration

The Ambit team hand-labelled over 100 pieces of fruit to train a computer vision model to recognise eggplants. As size is the criteria that the picking team uses to determine pickability, it was important that the model could accurately size the fruit.

The model was therefore trained to project the size of the fruit, even when partially covered by leaves. Image 3 demonstrates how the model could project a fruit’s total size using only an image of the exposed region of the fruit.

Image 3: The total size of an eggplant could be estimated using only an image of the exposed region of the fruit.

The team also physically tagged 10 pieces of fruit, and measured each piece’s length before it was imaged by the robot. On the trial site the grower harvests at 185mm, so fruit was chosen that was both longer and shorter than this threshold. This allowed the relationship between the actual size and the size of the fruit in the image (the pixel size) to be calibrated.

The data demonstrated a strong (and perhaps obvious) relationship between actual size and pixel size. The accuracy of measurement of each piece of fruit varied; the worst error was 15mm (9 per cent) while the average was 11mm (6 per cent). These estimates are shown in red in Figure 1.

By adjusting the measured size to account for the fruit’s distance from the camera, the error in fruit sizing reduced from an average of 6 percent down to 3 percent, of just 5mm. These estimates are shown in yellow in Figure 1.

Figure 1: Eggplant sizing estimates taking into account distance from the camera.

The level of accuracy the robot achieved provides a solid basis on which to observe growth across hundreds of pieces of fruit and track fruit growth in real-time.

Observing growth

Figure 2 demonstrates how the robot mapped eggplant growth over two nights. The blue line shows the distribution of fruit on the first night, and red on the second. The daily growth varied between 5-8mm for smaller fruit and 4-10mm for larger pieces.

Improving grower productivity, profitability, preparedness and competitiveness

The trial successfully demonstrated that a robot can be programmed to navigate a greenhouse and image fruit, providing accurate counts and sizing of the fruit.

Figure 2: Comparison of eggplant growth across two nights.

The Ambit team is beginning to explore how to add value to forecasts through disease tracking, fruit grading and performance management.

The VegNET Queensland Wide Bay Burnett project will continue to work with AgTech companies and growers who are looking at technology solutions that will increase on-farm efficiencies to support their uptake into the future.

While the data showed strong potential to provide insights to growers, a significant challenge remained at the conclusion of the trial: robot traction issues on loose areas of weed-mat, difficulty avoiding obstacles within the rows and navigational issues. These issues prevented long-term (multi-week) data observation as they introduced critical gaps in the data stream from the robot.

The Ambit team has responded to these challenges by focusing on greenhouses with pre-installed heating rails. These rails provide certainty of navigation, avoiding gaps in the final data stream.

“The VegNET project gave us the opportunity to expand into a new crop; the highlight was gaining the perspective of new growers, and the different ways that yield forecasting helps them deal with customers and manage their farm,” Ambit Robotics team member Paul Voutier said.

“For crops such as eggplants which are sold based on size, accurate measurement on the plant could have a real impact for growers. The rate at which eggplants grow each day varies significantly, and the robot showed significant potential to monitor this in real time each day.”

Next steps

Consistent, daily observation of fruit has the potential to generate yield insights that provide a ‘step change’ in yield projection accuracy. While achieving higher accuracy, automation allows for constant monitoring of trends.

On many farms, the value of these insights in terms of operational savings is at least an order of magnitude greater than the cost of the hardware and computing power needed to deliver these estimates.

‘Delivering’ the camera on a fixed path night after night presents the most significant challenge to adoption. We are likely to see significant growth of this technology on railed farms where navigational issues are already solved. From there, expansion into the more traditional greenhouses is only a matter of time.

The trial was a meaningful learning opportunity for the VegNET program and Ambit team. This case study provides an opportunity for other technology providers and growers to also learn from the trial.

Further information
Contact VegNET Queensland Wide Bay Burnett Regional Development Officer Jessy Logan at vegnet@bfvg.com.au or 0407 366 797. Find out more about the technology from Ambit Robotics at https://ambitrobotics.com/