# Cohort tests

When you want to A/B test something other than email content, try a cohort test! Randomly assigning people to cohorts gives you a way to perform tests independently of individual messages.

Cohort testing is great for things like:

*   Testing timing: different delay times before sending a message
*   Single vs multi-touch: sending a user one email vs. several emails
*   Blackhole/holdout testing: where you want to send a message or no message at all. Check out our recipe specifically for [holdout tests](/journeys/holdout-test).

## Ingredients[](#ingredients)

You’ll need:

*   A basic understanding of [liquid](/journeys/using-liquid/)
*   [The ability to create campaigns](/accounts-and-workspaces/add-remove-team-members/#journeys-permissions)

If you decide to assign cohorts via API, you’ll also need:

*   A terminal window and basic knowledge of cURL (or a friendly developer)
*   [Your site ID and API key](/journeys/managing-credentials/)

## Method[](#method)

To create a cohort test:

1.  Randomize and assign cohorts to your users via a campaign or [our API](/integrations/api/track/#operation/identify).
2.  Create a campaign to run your cohort test. For our example, we’ll run a campaign that sends an email five minutes or ten minutes after a person performs an event.

### Step 1: assign a `cohort` attribute[](#step-1-assign-a-cohort-attribute)

In this example, we’ll trigger a campaign for users who signed up for your product and assign them a `cohort` attribute to divide them into groups. This can be used for any campaign moving forward!

 You can use our [API](/integrations/api/track/#operation/identify) to create and assign a `cohort` attribute instead of a campaign. If you use our API, skip to [creating your test campaign](#step-2-create-a-campaign-to-test-your-cohorts).

1.  Go to Campaigns and click **Create Campaign**. Enter a name.
2.  Click **Choose trigger** and select **Segment change**.
3.  Specify users in the segment **Signed Up** and click **Save**.
    
    [![recipe_send-a-welcome-email_trigger-1.png](https://docs.customer.io/images/recipe_send-a-welcome-email_trigger-1.png)](#e06df25b8674fbfe4edcda825cd217d2-lightbox)
    
4.  Drag a [**Create or Update Person**](/journeys/create-update-person) block onto the canvas. Click the block then click **Add details**.
    
    [![cohort-test-campaign-1.png](https://docs.customer.io/images/cohort-test-campaign-1.png)](#b82bd96e88c73bce2ce13f99a6e3f9b3-lightbox)
    
5.  Under “Who do you want to update?” select “the person in the workflow.”
6.  Under “Which attributes do you want to add, change, or remove?” add liquid to randomize cohort assignment.
    
    [![cohort-test-campaign-1-liquid.png](https://docs.customer.io/images/cohort-test-campaign-1-liquid.png)](#22bf916480e0898cce74201ebe7f501e-lightbox)
    
    1.  To create two groups:
        
        *   Set a new attribute called `cohort` with a *Liquid* value and use the code below to generate a number, 0 or 1. If the outcome is 0, it sets the `cohort` attribute to A. If it’s 1, it’s set to B.
        
        ```fallback
        {% capture rValue %}{% random 1 %}{% endcapture %}{% case rValue %}{% when '0' %}A{% else %}B{% endcase%}
        ```
        
    2.  To create more than two groups:
        
        *   Here’s an example that assigns one of 10 possible options (A to J):
        
        ```fallback
        {% capture rValue %}{% random 9 %}{% endcapture %}{% case rValue %}{% when '0' %}A{% when '1' %}B{% when '2' %}C{% when '3' %}D{% when '4' %}E{% when '5' %}F{% when '6' %}G{% when '7' %}H{% when '8' %}I{% when '9' %}J{% endcase %}
        ```
        
7.  Click **Save Changes** then **Done**.
8.  On the review step, let’s choose to update **Current people and future additions**, though you can choose “Future additions only” if you don’t want to update users already in your database.
    
    [![cohort-test-campaign-1-review.png](https://docs.customer.io/images/cohort-test-campaign-1-review.png)](#3c648ed61ec25c9a8d50bebd5ba11a90-lightbox)
    

This sets us up for future cohort testing, as anyone who signs up in the future will randomly get an A or B value, in addition to our existing people.

Now, we’re ready to test!

### Step 2: create a campaign to test your cohorts[](#step-2-create-a-campaign-to-test-your-cohorts)

In this example, we’ll create a campaign to test how our cohorts’ behaviors change when we delay sending an email at different intervals. We’ll send the same email to people who view a product, but for one cohort, we’ll send after five minutes, and for the other, after 10 minutes.

We’ll create a campaign triggered by the event `viewed_product`, and the workflow will contain two time delays before an email, where each time delay has a condition targeting a different cohort.

1.  Go to Campaigns and click **Create Campaign**. Enter a name.
2.  Select **Event** as the trigger.
3.  Enter the event name, in this case, `viewed_product`.
4.  Specify your [message settings](/journeys/campaigns-in-customerio/#settings) and your [goal and exit criteria](/journeys/campaigns-in-customerio/#campaign-conversion-goal).
5.  On the workflow step, drag two **Time Delay** blocks onto the canvas followed by an **Email** block.
    
    [![cohort-test-campaign-2.png](https://docs.customer.io/images/cohort-test-campaign-2.png)](#d9a3df10a0f22282ef81712076b4a9d8-lightbox)
    
6.  Click a delay block and specify “Wait 5 minutes.” Then add an attribute condition where `cohort` is equal to the value A. Save your changes.
    
    [![cohort-test-campaign-2-delay-1.png](https://docs.customer.io/images/cohort-test-campaign-2-delay-1.png)](#99d1397e20d51d522620b746f44008e1-lightbox)
    
7.  Click the other delay block and specify “Wait 10 minutes.” Then add an attribute condition where `cohort` is equal to the value B.
8.  Click the email block and create your message. Remember to update the sending behavior to **Send Automatically**.
    
    [![cohort-test-campaign-send-auto.png](https://docs.customer.io/images/cohort-test-campaign-send-auto.png)](#f3007f98731ea9bd1bd494de5217c9ca-lightbox)
    
9.  Click **Review items** at the top if you have outstanding steps to complete then click **Start Campaign** for a final review.

As soon as people view a product, they’ll enter the campaign and wait to receive the email based on the time delay that matches their `cohort` attribute.

## Notes[](#notes)

### When to assign the `cohort` attribute[](#when-to-assign-the-cohort-attribute)

You *can* assign a `cohort` value within your testing campaign by combining both campaigns into one. This might be useful if you want to run A/B tests in some campaigns and A/B/C/D in others. However, assigning cohorts separate from testing cohorts lets you trigger tests based on different criteria.

### Testing one cohort per campaign or newsletter[](#testing-one-cohort-per-campaign-or-newsletter)

If you want to test multiple campaigns against each other, you can create a segment for each cohort you want to test then add this as a segment trigger condition or filter.

If you want to test multiple newsletters against each other, add an attribute condition and target a specific cohort or add a segment condition that targets a specific cohort.