Cyth Pairs AI Software with Robotic Arm to Pick, Place & Orient Organic Seedlings
While agriculture is one of the largest and most profitable industries in California, many nurseries are hesitant to introduce new technology into their handling processes and have continued to stick with the traditional manual methods. The client for this project is an automation entity representing multiple seedling nurseries within California, and their goal is to automate the handling, sorting, inspection and processing of raw plants. They were looking to leverage new technology for the organic identification of key plant features for controlled packaging and post-processing. They sought out help from Cyth to optimize their current inspection technologies and take them to the next generation, getting ahead of the curve.
The company plants fruit seeds and nurtures them to a midlife point just before they bear fruit. While in this stage of development they are removed from the ground and guided along a conveyor line to sort which seedlings are considered good enough for distribution and planting at nurseries around the country. The identification process has traditionally consisted of manual labor with workers stationed on an assembly line as they pick and choose adequate plants for packaging.
A decade ago, the client had implemented an early stage attempt at artificial intelligence into their automation system with the goal of increasing speed and decreasing the need for human labor. Though there was marginal success, there was a limitation with its ability to handle variation or different plant breeds. Limited to a 100-image template, the original system was restricted in its capabilities and lacked customization. The customer was looking to the future for next-level, cutting-edge technology that would give them greater control, visibility, trainability and handling when paired with a pick-and-place robot for seedling processing. When they approached Cyth asking for their partnership in creating this new system, Cyth knew that they had just the right technology for this endeavor: their own AI- based machine vision software, Neural Vision.
Client Request & Cyth’s Solution The client wanted the construction of a new imaging system to be built onto an existing conveyor line and paired with a robotic arm for the pick-and-place of fruit seedlings. The project started with a tabletop Phase Zero imaging system to begin capturing images of raw seedlings for the testing and evaluation of a Neural Vision solution. Neural Vision differs from traditional machine vision tools in that it does not rely on templates or a string of vision algorithms to handle vision inspections, but instead leverages Deep Learning capabilities. Just like a human would, Neural Vision builds a mental model to sort, grade, and classify objects. This way, the client has total control of their system performance without having to be dependent on external vision engineers to make adjustments or incorporate new product lines. The software was designed to allow a person with no machine vision experience to be able to program their system to inspect and classify products with a simple click.
When the Phase Zero was successfully completed and the customer was able to see how Neural Vision could work for their product, Cyth’s engineering team moved to Stage 2: designing a modular vision and robot subassembly on a test conveyor. Cyth’s engineers began the mockup of the imaging frame to be installed onto the conveyor as well as the pairing of a Denso robot. Cyth took the findings from the Phase Zero to determine optimal lighting, cameras, and lenses to capture ideal images for the Neural Vision dataset. Cyth then moved on to begin Stage 3: transitioning this modular system from a simulated environment to a true production line for testing and refinement.
Challenge The most challenging part of the project was pairing Neural Vision with high accuracy conveyer tracking. The Denso Robotics technical team visited Cyth’s workshop for a week while the teams worked collaboratively to learn how the Neural Vision software could be integrated with their robot and the client’s conveyer. By working together with Denso’s team and the client, Cyth’s engineers were able to develop a fully integrated solution.
Outcome Much of the agricultural industry is seasonal work; sometimes messy, organic, unpredictable and constantly changing. It is also difficult to find labor and often expensive to bring in a work force from far off locations. By bringing an autonomous inspection system with pick-and-place sorting to the client, Cyth could provide them with a robot handling system that is easily programmed to recognize and adapt to the inevitable change that will occur next season. The client’s previous application leveraged old technologies in an effort to automate sorting, and while it partially met their needs, it wasn’t true Deep Learning or Artificial Intelligence. This meant there was a definitive ceiling to their old system’s ability to learn or handle difficult and subjective situations.
The client was thrilled to see how Neural Vision could work for their raw plants as well as open the door to many new automated sorting and grading needs. Rather than having a single model based on just one image dataset, the client can now continue to create various solutions for their product lines or even expand to different types of plants. With Cyth’s successful integration of a Neural Vision robot system on an initial full processing line, the client aims to scale the technology and platform to over 80 lines internationally. They continue to voice their excitement about the capabilities that this technology can and will accomplish.
Above is an example of what the GUI for client looked liked once Cyth's Neural Vision software was implemented
Cyth’s Neural Vision Software
LabVIEW 2017 32-bit
2 Basler 5MP, 17fps, Area Scan, CCD Global Shutter, Color Camera
2 Edmund Optics 16mm, 300-2000mm Primary WD, HP Series Fixed Focal Length Lens
Medium Size 4-Axis SCARA HM-G Series Denso Robot
Incremental, Resolution 1000P/R, 40-dia., 5 to 5 VDC, Line driver output, Pre-wired models (2 m) OMRON Encoder