True holdout tests

A holdout test is a type of A/B Test that lets test whether a message is useful or not. In a holdout test, some members of your audience receive a message and some don’t—but we still calculate conversions for the message, helping you see whether your message affects your conversion rate.

In the past, we had a complicated recipe to perform holdout tests, but it involved sending real messages to a black hole address, which wasn’t ideal. Now you can set up holdout tests with a simple checkbox!