The SWEEPER sweet pepper harvesting robot was tested for four weeks in a commercial greenhouse, on 262 fruits across different varieties and growing conditions. Its developers report in the Journal of Field Robotics that the harvest success rate was 61% for the best fit crop conditions and 18% in current crop conditions. The robot was the same in both cases. The crop was not. For anyone building greenhouse robots, that is the core problem: the plant is part of the robot's environment, and it changes every day.
A greenhouse robot must be tested against the crop at each growth stage, because a route, map or clearance margin that works among young plants can fail at full canopy. In a virtual greenhouse, a calibrated crop model predicts plant height, canopy width and leaf area per stage, and a simulator checks the robot's swept volume against that geometry.
Why a growing crop breaks static maps
A greenhouse looks like a structured environment. The rows are straight, the paths are concrete and the building does not move. The difference from a warehouse is the crop. In a tomato or sweet pepper house the plants go from small transplants to a dense canopy within weeks, and growers then manage that canopy continuously by training, lowering, de-leafing and harvesting. The free space around the robot is set by biology and crop management, not by the building, and it changes daily.
Clearance is the first assumption to fail. The usable aisle width is not the distance between rows on the drawing. It is that distance minus the leaves and side shoots that grow into the aisle from both sides, and that encroachment rises as the canopy develops. Fruit trusses hang at heights that move as plants are lowered along the wire. A clearance margin that passed acceptance in the first week can be negative in the same row six weeks later, with no change to the robot or the building.
Static maps fail next. Off the pipe rails, many greenhouse robots localise against a lidar map. Harik and Korsaeth combined Hector SLAM with an artificial potential field for greenhouse navigation with a single 2D lidar, in what they call a GPS-denied environment typical for a greenhouse. An occupancy grid built at transplanting shows open aisles. Weeks later, leaves fill cells the map marks as free. Repetitive rows add a second problem: one row looks very like the next to a lidar, so scan matching can settle on the wrong row.
Fixed routes fail last, usually at the row ends. A turn from one row into the next sweeps the corners of the chassis through space beside the first plants of each row. With small plants that space is empty. At full canopy it holds leaves and fruit, and a turn that worked in week one now makes contact. The planned waypoints have not changed, but the obstacles around them have.
The usual answer is to test on real crops, and every iteration is then tied to the plant's biological clock. Even the Harik and Korsaeth trials ran in an office-like building, because the greenhouses had been dismantled for the winter. Demand makes the delay costly: the International Federation of Robotics reports almost 20,000 agricultural professional service robots sold in 2023, up 21%. Growers buying them need machines validated for the whole season.
The greenhouse geometry a robot must fit
A common structure in Dutch glasshouse horticulture is the Venlo-type glasshouse: a multi-span house built from repeated bays, with gutters running its length. Crop rows run in one direction, a concrete path crosses the house, and heating pipes run along the rows. In many houses those pipes double as a rail for trolleys and robots. The SWEEPER platform, for example, is mounted on a cart that "autonomously drives on pipe rails and concrete floor in the end-user environment".
The rail solves lateral guidance inside the row, but it fixes the robot's track width to the pipe spacing of that greenhouse. It also makes the row end the critical part of the route. There the robot leaves the rail, drives or turns on the concrete path and enters the next rail, often with plants on both sides of the manoeuvre. Most geometry failures in a pipe-rail house happen in those few metres.
A small set of dimensions decides whether a robot fits. Row spacing and path width between double rows set the lateral space. Gutter height and crop wire height set the vertical working zone for harvesting and de-leafing arms. Path width at the row end decides whether a given footprint and turning radius can turn without touching the next row. Posts, benches, irrigation lines and substrate slabs are fixed obstacles at known positions. These dimensions vary between greenhouse builders and growers, so a robot that fits one house may not fit the next.
Lettuce and other bench or gutter crops change the question. When the robot works over the crop rather than beside it, what matters is the canopy envelope from above: how high the heads stand and how far leaves spread across the gap between gutters. The simulation method stays the same; only the binding dimensions change.
Crop growth models: the published starting point
Predicting how a crop will look in four weeks is not a new problem. Horticultural science has decades of process-based greenhouse crop models, with Wageningen University & Research central to much of that work. For tomato, the main families are TOMSIM, TOMGRO and the yield model developed by Vanthoor and colleagues for model-based greenhouse design. They share a structure: climate drives photosynthesis, photosynthesis produces assimilates, and assimilates are partitioned to leaves, stems and fruit.
Light is the main driver. The canopy intercepts a fraction of the photosynthetically active radiation (PAR), usually through an exponential extinction law in which interception rises with leaf area index (LAI) and levels off as the canopy closes. Intercepted light becomes dry matter through leaf photosynthesis or a light use efficiency term. Temperature sets the rate of development, such as how fast nodes and trusses appear, and higher CO2 raises photosynthesis. Growers have long assumed that 1% more light gives 1% more yield; Marcelis and colleagues found that for most crops a 1% light increment gives 0.5 to 1% more harvestable product.
The reduced TOMGRO model shows what a model gives a simulation team. In a comparison of dynamic tomato growth models for greenhouse control, its inputs are the daily average temperature, the daytime average temperature, solar radiation and CO2 concentration. Its state variables are the number of main stem nodes, LAI, total plant weight, fruit dry weight and mature fruit dry weight. Node number and LAI are the two states a robot engineer needs most, because they drive plant height and canopy density.
These models are not ready to use straight from the paper. The same comparison concludes that the models it could apply to optimal control "require more accurate parameter identification and calibration tailored to greenhouses". Another study combined TOMGRO and the Vanthoor model into one yield model, sorted parameters by sensitivity analysis and tuned them by Bayesian optimisation, using four years of data from a 680 square metre Venlo-type glasshouse at 2.5 plants per square metre. The calibrated model had a far lower error than either model alone.
Growing medium and nutrient solution are inputs to the predictor alongside climate. Reduced TOMGRO has no explicit term for them, so where a robot maker's customers grow in soil, use a different substrate or run a different feeding strategy, their effect is carried by the parameters calibrated for that crop and greenhouse. This keeps the model small enough to calibrate with the data a grower already holds.
From biomass to plant geometry
A crop growth model outputs dry matter per square metre per day. A robot simulator needs geometry in metres. The step that joins the two converts biomass and development state into plant height, canopy width and leaf area per plant at each growth stage. Most crop models stop before this step, and a robot simulation cannot start without it.
Some conversions are direct. Leaf area per plant is LAI divided by plant density: at an LAI of 3 and 2.5 plants per square metre, each plant carries 1.2 square metres of leaf. Plant height is node number multiplied by internode length, which depends on variety, temperature and light. Canopy width follows from leaf number, leaf length and leaf angle, fitted as a relation between leaf area and width. All three relations are fitted from simple measurements of height, leaf number and canopy width taken on known dates.
Crop management then shapes the plant. In high-wire tomato and cucumber, growers lower plants along the wire and remove lower leaves, so the height band of the active canopy above the floor is set by the cropping schedule as much as by growth. The geometry model therefore takes the lowering and de-leafing schedule as an input. It outputs, per stage, a stem, a canopy envelope described as width per height band, and truss positions where the task needs them.
Plants also vary around the model mean. The calibration measurements give that spread, so each stage is built twice: a mean crop and an upper-envelope crop at the wide end of the measured canopy widths. A check that passes on the mean but fails on the upper envelope is marginal, and it tells the team where real plants will make contact on a bad day.
| Growth stage | Model state | Geometry in the scene | Checks most likely to fail |
|---|---|---|---|
| Young plants | Low LAI, few nodes | Narrow stems, small envelopes, aisles set by structure | Rail entry, row-end turns against posts, pipes and benches |
| Canopy closure | LAI rising fast, first trusses | Envelopes widening into the aisle, lower leaves present | Row-end turns into the first plants, arm approach paths |
| Full canopy and production | Highest LAI, lowering and de-leafing active | Widest envelopes, trusses in the arm's working zone | Minimum aisle margin along the row, arm and boom reach into dense canopy |
The output is not a photorealistic plant. For clearance checks, a leaf-by-leaf mesh adds collision cost without changing whether a chassis passes a row. For camera or lidar testing, simplified leaves with the correct total area can be added inside the envelope.
Building the virtual greenhouse in OpenUSD
The scene is built in OpenUSD. Universal Scene Description is Pixar's open-source framework for assembling many assets into one scene and editing them non-destructively through overrides. Data is organised as a hierarchy of prims, attributes can vary over time, and layers combine in a strength order stated in the files. That fits a greenhouse well: the building is one layer, the row layout another and the crop at each growth stage another again.
Layout comes from what the robot maker or grower already has. A DWG or DXF drawing gives the positions of bays, gutters, posts, paths and pipe rails, and a PDF plan can be traced. Where neither exists, a short questionnaire on bays, row spacing, gutter height, paths and benches generates a parametric greenhouse. The stage is set to metres and Z-up, the same convention as ROS, so the robot, the rows and the plants share one reference frame.
The crop layer is generated per stage from the geometry model. Switching from week two to week eight means swapping one crop layer while the greenhouse and robot stay unchanged. Edge cases, such as a dense patch or a row where de-leafing is overdue, are added as separate layers or variants, so they can be tested without rebuilding the scene.
The first target simulator is NVIDIA Isaac Sim. Its documentation describes it as USD-native, with PhysX or Newton physics, RTX and physics-based sensors, and ROS 2 integration for validating robot stacks. For Gazebo teams the scene can be exported: the gz-usd tools from the Gazebo project convert between SDFormat and USD. The conversion aims for approximately equivalent results rather than identical ones, so the exported world is checked against the USD original before results are compared.
How the checks are computed
The robot enters the scene as collision geometry. The chassis is a box or convex hull matching its footprint, and each arm or boom link is a capsule or convex hull taken from the robot's URDF or CAD model, or from a simplified envelope if no model is available. Plants are collision geometry too: a capsule for the stem and the canopy envelope from the geometry model, split into convex pieces by height band. Coarse convex shapes keep queries fast and conservative.
Each planned route is sampled into poses at a step smaller than the clearance margin, and the shapes are swept between consecutive poses so that contact between samples is not missed. The union of the robot's collision shapes along the route is its swept volume. Row-end turns are simple and unforgiving: for a robot turning about a point on its rear axle line, at radius R to the centreline, the outer front corner traces a circle of radius √((R + w/2)² + a²), where w is the robot's width and a the distance from the rear axle to its front edge. The rear overhang swings out on the other side.
Arm and boom motions are handled the same way in joint space. Each reach motion for harvesting, de-leafing or spraying is sampled along its joint trajectory, and every link pose is tested against the canopy. The tests use the physics engine's scene queries. PhysX supports raycasts, sweeps and overlaps: a sweep moves a shape from a pose along a direction and reports the first hit, and an overlap reports any shape inside a region. A distance query between hulls then gives the clearance at each pose.
Clearance is compared with a margin the robot maker sets, normally the sum of localisation error, path-tracking error and a safety allowance. Each segment of the route is classed as pass when the minimum distance exceeds the margin, marginal when it lies between zero and the margin, and fail on contact. Canopy contact is reported by row, position along the row, height and robot part, run against both the mean crop and the upper-envelope crop.
The scope is deliberately geometry and movement. The simulation does not predict whether a leaf tears or a fruit bruises, because that needs material models for stems, petioles and fruit that vary with variety and turgor. It answers the question robot makers need first, where and when the robot touches the crop, and real-plant damage tests can then target those exact locations. The same logic applies to the sim-to-real gap: the virtual greenhouse decides where real trials go, and the real trial confirms a margin rather than discovering a failure.
What you provide and get back
The inputs are few, and most already exist in a robot maker's design files and a grower's climate computer and crop registration. Collecting them rarely needs new sensors.
From the robot side:
- footprint, turning radius and rotation centre, in metres
- track width, if the robot rides pipe rails
- arm or boom reach and working height band
- planned routes, including row-end manoeuvres
- URDF or CAD model if available; a simplified envelope is enough for a first pilot
From the greenhouse and crop side:
- layout as DWG, DXF or PDF, or answers to the layout questionnaire
- crop and, ideally, variety, plus growing medium and nutrient strategy
- climate setpoints or logged temperature, light and CO2
- lowering and de-leafing schedule
- plant height, leaf number and canopy width measured on a few known dates
What you get back:
- a crop growth model calibrated to your crop and greenhouse type
- plant geometry per growth stage, as a mean crop and an upper-envelope crop
- an OpenUSD greenhouse scene ready for Isaac Sim and exportable to Gazebo
- a per-stage report on route feasibility, row clearance and canopy contact, with the robot parameter behind each failure
The scene stays with you for your own perception testing or synthetic data generation.
Keeping the twin current with robot data
A crop model predicts the average plant. The robot sees the real ones. Every pass a scouting, harvesting or spraying robot makes along a row produces camera and lidar data on the actual canopy: its height, its density and where it encroaches on the aisle. That data is the best calibration source a growth model can have, and it arrives every time the robot drives.
A defined update path back to reality is what separates an operational digital twin from a 3D model, and we are building towards a twin that closes this loop. The next layer of our Smartout platform is a robot perception engine that generates robot-ready twins from CAD and BIM, syncs them to robots, consumes camera and lidar feeds, detects local changes and updates the shared twin in near real time. For greenhouses, that would let robot measurements correct the plant geometry row by row, so next week's prediction starts from this week's observed crop.
For a fleet this matters more than for one machine. A canopy change measured by one robot is useful to every robot that drives that row next, and a shared twin is where those observations meet. Longer term, that model is the basis for fleet mission planning and route optimisation, which are on our roadmap, not in the current offering. More on this work is on our robotics solutions page.
The Kaiinos approach and the pilot
Kaiinos builds operational digital twins: continuously updated models of real places, assembled from CAD and BIM, lidar, drone, satellite and sensor data. Our technology covers that pipeline from data integration to spatial queries and change history. The virtual greenhouse applies the same discipline to a place where the main object grows. Agriculture is a long-standing part of our delivery work, as our agriculture industry page describes.
Two pieces of delivered work are directly relevant. The first is a plant protection and surveillance application. Pests are sampled along transects, counts are compared with thresholds that trigger spray decisions, and fields are re-sampled afterwards to check the effect. The same loop maps onto a monitoring robot: its route through the rows becomes the sampling transect, and its observations feed the threshold and the follow-up check.
The second project related long-term weather and satellite indicators to crop condition and suitability for about 20 crops. In a greenhouse the climate is controlled and logged rather than observed from orbit, but the modelling task is close: relate climate to crop state, then calibrate against observations. The growth predictor itself starts from the published greenhouse crop models described above.
The pilot is kept small: one crop, two to three growth stages and one greenhouse type, in six steps.
- Confirm the setup: crop, variety, greenhouse type, robot parameters and the growth stages to test.
- Prepare the crop model: select the published model for the crop and calibrate it with the grower's climate data and plant measurements.
- Create the plant geometry: convert biomass and development state into height, canopy width and leaf area per plant for each stage.
- Build the greenhouse scene: generate the OpenUSD scene from CAD, PDF or the questionnaire, with a crop layer per stage.
- Run the movement checks: drive the robot through the rows at each stage and record route feasibility, row clearance and canopy contact.
- Review: go through the results with your engineering team and decide what to change in the robot, the route or the operating window.
Frequently asked questions
What is greenhouse robot simulation?
Greenhouse robot simulation tests a harvesting, de-leafing, scouting or spraying robot in a virtual greenhouse before or alongside real trials. A useful simulation includes the greenhouse structure, the crop rows and plant geometry that matches a real growth stage. Without the growing crop, it only tests the building, the part of the environment that rarely causes failures.
Which crop growth model is used?
We start from established published greenhouse crop models. For tomato, candidates include the TOMGRO, TOMSIM and Vanthoor model families from greenhouse horticulture research. The chosen model is then calibrated to your crop, variety and greenhouse type using climate data and a small set of plant measurements, because published parameters rarely match a specific greenhouse without adjustment.
How are clearance and canopy contact computed?
The robot and plants are represented as collision geometry. Each route is sampled into poses to form the robot's swept volume, and each arm or boom motion is sampled along its joint trajectory. Physics engine sweep, overlap and distance queries give the minimum clearance at every pose, which is compared with your safety margin and classed as pass, marginal or fail.
Does the simulation predict plant damage?
No. The scope is geometry and movement: route feasibility, row clearance and canopy contact. The simulation reports where the robot touches or enters the canopy, but it does not model tearing, bruising or stem breakage. Those effects depend on material properties that vary by variety and plant condition, and are better tested on real plants at the contact points the simulation finds.
Which simulators are supported?
The greenhouse scene is built in OpenUSD and runs first in NVIDIA Isaac Sim, which is USD-native and supports physics-based sensors and ROS 2. For Gazebo teams, the scene can be exported to SDFormat with the Gazebo project's USD conversion tools. Because that conversion is approximate, the exported world is checked against the original, and your existing stack stays in place.
What does a greenhouse robot maker need to provide?
The robot footprint, turning radius, arm or boom reach and planned routes, plus the greenhouse layout as CAD, PDF or questionnaire answers. On the crop side, the crop and variety, growing medium and nutrient strategy, climate data, the lowering and de-leafing schedule and a few plant measurements on known dates. A detailed robot model helps but is not needed for a first pilot.
Test your robot against next month's crop
A greenhouse robot that only works in the week it was tested is not ready for a full season. If you build robots for greenhouse crops and want to see how your platform performs at full canopy before the crop gets there, book a technical call and we will scope a pilot for one crop and one greenhouse type.