How to perform on-site greenhouse product trials

Most growers have conducted product trials in their commercial greenhouses and nurseries at one time or another. The objective of on-site trials is usually to compare a product or practice that is currently being used to a product or practice that is presented as being better. The suggestion of something better comes from sources such as a competing manufacturer, a fellow grower, a conference speaker, or university extension agent. 

What’s your experience with trials?
Have you ever run a trial in a greenhouse or nursery? Was it executed in a way that produced meaningful results? How confident were you that the results were accurate? Upon completion, did you change a product or practice based on the results generated? 

These questions all address a common challenge in conducting a trial…the confidence factor. A product trial today might focus on something basic like comparing a fertilizer formulation currently in use to a similar formulation of another brand that is cheaper. This type of trial is fairly easy to interpret, if both products produce the same quality crop the cheaper fertilizer saves money and boosts profit margin. 

However, a product trial tomorrow may focus on a different production factor and the decision could impact the entire operation rather than a single crop. Consider the growing media category. It doesn’t appear to be that far removed from fertilizer trials but how does one conduct valid growing media trials to produce accurate results with a high confidence factor? 

What’s being compared? 
There are realities to understand before running any trial. As stated above, trialing two fertilizers on the same crop growing in the same mix in the same production area is fairly easy. With the other production factors the same (growing mix, irrigation frequency, greenhouse temperature) using a trial fertilizer on one bench or in one bay is pretty simple (Figure 1). Just apply the same strength at the same frequency without contaminating either group of plants with the other fertilizer and the confidence level in the results will be high. Think of this trial as comparing apples to apples.  

 
Figure 1. Geranium trial with each tray containing a different treatment. 

But, what if we trial a new growing mix? The new mix could have a different profile for water holding capacity, drainage, and aeration. We can set up a similar trial as above and plant one bench or bay with the new mix next to the rest of the production area using the old mix. But if a characteristic of the new mix is that it holds more water without sacrificing aeration (sounds contradictory but precisely the benefit coconut coir offers), the trial bench or bay may require less frequent irrigation. If liquid fertigation is used, fewer irrigation cycles means less fertilizer will be applied. 

The results of such a trial will likely be that the trial plants are either of lower, equal, or higher quality than the standard plants. Let’s assume the trial plants were irrigated on the same schedule as the standard plants. If they end up being of lower quality the reason might not be the mix but rather that they were overwatered. In this scenario we’re left trying to compare apples to oranges and the confidence factor drops. 

Automation and trials can conflict
What if such a trial is run using one bench with the new mix and a matching bench next to it with the standard mix, isolated from the rest of the production block. How is the trial irrigated? Do we water both benches when the standard bench is dry or both when the trial bench is dry. To run the trial correctly, each bench should be watered as needed and observations made about irrigation frequency. 

If an automated irrigation system is used with only one zone available for the trial one of the groups of plants could be at a disadvantage. It is likely that final crop quality will be lower due to a factor other than the actual growing mix. Again, the confidence factor drops.    

Propagation trials
The previous article in this series covered the importance of managing the environment in a propagation area. Optimizing temperature, relative humidity, moisture, light, and  propagation medium are associated with best practices regarding seed germination and cutting rooting. The more automated the systems are the better our control is. That said, all that automation can also create challenges for trialing new products in a propagation setting.     

One propagation medium doesn’t serve all
Continuing the theme of growing media trials, growers have many choices of propagation media to consider. Categories include loose mix of traditional ingredients such as peat, coir, and perlite in a plug tray, stabilized products (not loose) comprised of peat and coir, and inert products such as foam and rock wool. Each has its benefits and limitations and each can have subtle requirements that need to be managed differently.

Most growers prefer to settle on a single propagating medium for system simplicity. But propagating a wide range of crop species using a one-size-fits-all medium is bound to fail in a few crops due to different requirements those species have during propagation (Figure 2). If those sensitive crops are important to the overall product mix it makes sense to tailor the medium they are propagated in to optimize their performance. Tailoring the mix might be as simple as having access to media that does not contain a starter fertilizer charge. 

 
Figure 2. Seed germination trial using two propagating mixes, left and right. 

One mist cycle doesn’t serve all
Compared to trialing growing media in final production containers, trialing different propagation media can be even more challenging. An attempt to trial loose mix next to stabilized plugs next to foam next to rock wool could turn into a disaster if side-by-side means under the same mist zone. 

Different propagating media can differ significantly in their water and air holding capacities. While all are designed to deliver a balance between water and air, each may need a slightly different mist regime in order to perform optimally. With this in mind it is generally not a good idea to trial different categories of products under a single mist zone, the result will likely be closer to our apples to oranges comparison than apples to apples. 

One approach to conducting a fair trial of different propagation media categories involves using two or more mist zones each with the range of products being trialed. If a loose mix is being compared to a stabilized peat/coir product include both products on each zone. One mist zone should be set to the loose mix’s specifications and the other zone set to the stabilized peat/coir product. If the two products perform similarly under each regime, fine. If one does better under one regime than the other, that’s valuable knowledge to learn from the trial.  

Don’t be a scientist, do be scientific 
You may be thinking why set up this trial to create failure for one product under another’s mist regime? Why sacrifice seedlings and cuttings? The answer is to insure having a high level of confidence in the results. Sacrificing a few plug trays can be helpful in identifying the differences between trial treatments. 

While a few plug trays may need to be sacrificed now and then, a huge difference between conducting a trial in a commercial setting vs a university or private research setting is that in the commercial setting there is intense pressure to generate profit on every square foot of space. We researchers don’t have that pressure to sell the plants after a trial is completed, our research funding relieves it. 

Let us be the scientists, you be the grower. We can include some treatments that will likely fail in order to gain more knowledge. Your trial will be simpler, more direct. But while you don’t have to be a scientist do make sure to conduct trials scientifically. 

Sounds crazy…mix things up!
Getting things mixed up during research usually results in chaos and little if any knowledge gained. But there’s one context of mixing things up in a trial or experiment that means quite the opposite. The concept of randomization as an experimental tool is powerful. 

To eliminate variability of the mist pattern across a bench include multiple plug trays of each product in a trial and randomly place them in different locations across the bench (Figure 3). This way both media products being compared experience all mist variations. Randomizing the trays rather than grouping each product allows for insightful observation during the trial. If one product performs better at a certain location on the bench compared to another it’s worth noting.    

 
Figure 3. Standard five foot wide bench with single row of mist nozzles. 

Why is replication important? 
The basic premise to trialing something is that we test a product or practice on a few plants before deciding to use it on an entire crop. It’s pretty risky to implement a change without having a high level of confidence that it will work. 

Without being a scientist but still being scientific, we use a rule called ‘the power of three’. Let’s say we include only one plant in every trial treatment. However that plant grows, who’s to say it grew that way because of the trial treatment or because of something else. A lot is riding on that one plant if the trial results will be applied to the entire crop in the future. 

The next step is to increase to two plants per treatment. These two plants are averaged together at the end of the trial and the average becomes the treatment result. Let’s say the two plants grow differently, which one is the true result of the trial treatment? Picking one over the other introduces a strong bias and a 50/50 chance of making the wrong choice. 

Increasing from two to three plants per treatment is magical from a statistical perspective. The power of three becomes clear because if the three plants are different chances are the third plant will be similar to one of the other two. A trend appears and the average of the three plants moves toward the two similar plants. Increasing beyond three is even better, the larger the sample size the more confident we are in the average. If you have room for more than three pots, plug trays, or benches go for it. But if space is limited remember to trust the power of three rule.  

Last but not least
The last point in this discussion is as important as any covered so far. Many of my colleagues will agree that it is perhaps the most important. The concept of a control treatment is much too often misunderstood by growers. In any experiment one treatment is compared to another and to another. A reference point is always needed, a treatment that can be described as untreated (Figure 4). 

Keeping with this article’s theme of trialing fertilizers and growing media, the currently used fertilizer is our control treatment. That’s the reference the competing brand’s product will be measured against. In the growing media trial the standard product being used is the control, the new mix will be compared against what’s currently in use.  

 
Figure 4. Individual nutrient film channels set up for various trial treatments including a control treatment.  

Other descriptions of control treatments include not pinching a plant if some aspect of pinching is being trialed. When using chemical growth regulators the control often is a plant not treated with the product being trialed. Take a moment and think about trials you’ve run in the past and identify what the control treatment was. 

A question I routinely ask growers who call or email me with a question about what they’re seeing in a trial is “how do the treated plants look compared to the control plants?” If I had a dime for every time this question has been answered with some version of “Oh, I just sprayed the whole crop”…