> This page is part of the [Customer.io documentation](https://docs.customer.io). For the complete index, see [llms.txt](https://docs.customer.io/llms.txt).
> Last updated: November 16, 2026
> To take actions in Customer.io from an agent, use the [Customer.io CLI](https://docs.customer.io/ai/cli/get-started.md) or the [MCP server](https://docs.customer.io/ai/mcp/get-started.md).
> This page has tabbed examples for: Amazon S3, Google Cloud Storage, Azure Blob Storage. Add `?tab=<name>` to the URL to get only that tab, and repeat it for more than one, for example `?tab=amazon-s3&tab=google-cloud-storage`.

# Keep a collection up to date from cloud storage

Keep collections in Customer.io up to date by exporting data from your systems to a bucket in Amazon S3, Google Cloud Storage, or Azure Blob Storage, and Customer.io automatically pulls in the changes.

## How it works

To keep a [collection](https://docs.customer.io/messaging/objects-data/collections/) up to date with data that changes in your systems, like a product catalog or a list of store locations, your systems can write files to a bucket. Customer.io then loads each file to add, update, and delete rows in the collection.

Customer.io can load files from your bucket into a collection in two ways:

*   **A sync**: Customer.io checks a folder in your bucket on a schedule and loads each new or changed file that your systems write there. Use a sync when your systems export on a schedule. Most of this page covers syncs.
*   **A write request**: you use the API to load a specific file into a collection. Use a write request when your systems need to know that a particular file loaded before they do something else. See [Load a file when you’re ready](#load-a-file-when-youre-ready).

When Customer.io loads a file, it doesn’t replace your collection. Instead, Customer.io uses the collection’s [key field](https://docs.customer.io/messaging/objects-data/collections/#key-fields), which works like a primary key in a database table, to find the row that each row in the file changes:

*   **Add**: If a row’s key doesn’t match an existing row, Customer.io adds the row to the collection.
*   **Update**: If a row’s key matches an existing row, the new row replaces the whole existing row.
*   **Delete**: If a row sets the collection’s [delete field](https://docs.customer.io/messaging/objects-data/collections/#key-fields) to `true`, Customer.io removes the row with that key.
*   **No change**: Rows that a file doesn’t include stay in the collection as they are.

## Before you begin

You need the following things:

*   **A value that’s unique to each row.** This becomes the collection’s key field, like a product’s `sku`. The value can’t be empty, and no two rows can share a value.
*   **Credentials that can list and read the bucket.** Customer.io only reads your bucket. It never deletes, moves, or renames your files.
*   **An export job that writes files to the bucket**, or access to change the one you have. You’ll set the job up to match the steps on this page.

## Prepare your exports

Before you set up a sync, set up your export job, the process in your own systems that writes the collection’s data to your bucket. A sync can only load the files that your export job writes, so the job needs to follow the rules in this section.

### How a sync reads your files

A sync works like a reader that keeps a list of the files it already loaded. At each scheduled sync interval, Customer.io lists every file in the folder you tell Customer.io to check, including the folders inside it. Customer.io loads any new or updated files since the previous sync. Your export job depends on these rules:

*   **Customer.io loads files one at a time, in alphabetical order of their paths.** When two files change the same row, the file that loads last wins.
*   **Customer.io loads every file that’s in the folder when you start the sync.** The folder’s existing files load first, in the same alphabetical order.
*   **An overwritten file loads again.** If you overwrite a file more than once between checks, Customer.io only loads the latest version.
*   **A file that already loaded doesn’t affect the collection after that.** Deleting the file from the folder doesn’t remove its rows. If you put a deleted file back, Customer.io loads it again.
*   **Customer.io loads up to 2,000 new or changed files per check.** If more than 2,000 files are waiting, Customer.io loads the first 2,000 and loads the rest at later checks.
*   **A file that Customer.io can’t load stops the sync.** Customer.io doesn’t load the files that sort after it until you [fix the failed file](#fix-a-failed-sync). If only some rows in a file are invalid, like a row without a key, Customer.io loads the rest of the file and records the [rejected rows](#monitor-a-sync).

### Mark deleted rows

Customer.io only changes the rows that a file includes, so leaving a row out of an export doesn’t remove it from the collection. To remove a row, your export needs to include it with a field that marks it as deleted.

If you use a field called `deleted` to mark deleted rows, you can delete rows by including `"deleted": true` in the row:

```
{ "sku": "A-100", "deleted": true }
```

Customer.io removes the row with that key and never stores the row that marked it. Rows that set `deleted` to `false`, or leave it out, load as usual. Customer.io only removes rows this way after you [set the same field as the collection’s delete field](#step-2-set-a-delete-field).

We recommend that each export includes only the rows that your systems added, changed, or deleted since the last export. Files stay small, so Customer.io loads them quickly.

### Organize your files

Pick a folder for the collection’s files, like `products/`, and keep only your collection’s files in it. Customer.io tries to load every file in the folder and the folders inside it, so a file for another system can stop your sync. Then choose a naming convention for your files:

*   **Write a new file each time.** Start each filename with the date and time, like `products/2026-11-04T0600-products.csv`, so that alphabetical order matches the order that you wrote them. This approach keeps a history of your exports in the bucket.
*   **Overwrite the same file each time.** For example, your export job can replace `products/latest.csv`. Customer.io loads the file again at the next check.

A folder can hold up to 100,000 files

If your folder is over this limit, Customer.io stops the sync and doesn’t load any files until you move files out of the folder. Moving loaded files out of the folder doesn’t change the collection, so you can remove or archive loaded files on a schedule with a lifecycle rule in your bucket.

### File requirements

Each file needs to follow these rules:

*   **Use JSON or CSV.** Customer.io reads every CSV value as a string.
*   **Include each key once per file.** If more than one row has the same key, Customer.io loads the first row and rejects the rest.
*   **Be less than 500 MB and 1,000,000 rows.** If a file is over these limits, Customer.io can’t load it and stops the sync at that file. If you have larger files, you can split them into smaller files with names that sort in order, like `2026-11-04T0600-part-01.csv` and `2026-11-04T0600-part-02.csv`.

Export when your messages aren't using the collection

Customer.io loads a file at the next check after you export it, not the moment the file lands in your bucket. Customer.io also saves a file 500 rows at a time. While a large file loads, a message that queries the collection can get some rows from the new file and some from your previous export. To avoid sending a mix of old and new data, schedule your exports for a time when few messages use the collection. If a message depends on data from a specific export, send it after the **Last file loaded** in the **Sync** panel shows that export’s file.

## Set up a sync in Customer.io

After your export job is ready, you set up the sync in Customer.io:

1.  [Set a key field](#step-1-set-a-key-field) so that Customer.io can match each row in a file to a row in the collection.
2.  [Set a delete field](#step-2-set-a-delete-field) so that your exports can remove rows.
3.  [Connect your bucket and start the sync](#step-3-connect-your-bucket-and-start-the-sync).
4.  [Check that your first export loaded](#step-4-check-that-your-first-export-loaded).

### Step 1: Set a key field

The key field tells Customer.io which value represents a unique row, like a product `sku` or a store `id`. You must set a key field to sync a collection.

1.  Under *Configure data*, click *Collections*, and go to your collection.
2.  Click next to **Key field**.
3.  Choose a **Key field** from the attributes in your collection’s rows, like `sku`.
4.  Leave the window open to [set a delete field](#step-2-set-a-delete-field), or click **Save**.

When you set the key field, Customer.io checks every row in the collection first. If any row is missing a key or repeats another row’s key, Customer.io keeps the old key field and tells you which rows you need to fix.

### Step 2: Set a delete field

A sync only deletes rows if you set a delete field. Set it to the field that your exports use to [mark deleted rows](#mark-deleted-rows), like `deleted`.

1.  In the **Edit key fields** window, type the name of the field that your exports use, like `deleted`, in **Delete field**. The field doesn’t need to be in your rows yet.
2.  Click **Save**.

 ![The Edit key fields window, with sku as the key field and deleted as the delete field.](https://docs.customer.io/images/collection-edit-key-fields.png)

When you set a delete field, Customer.io permanently deletes every row in the collection where that field is already `true`. Check your rows for the field before you save.

### Step 3: Connect your bucket and start the sync

1.  On the collection’s page, click **Set up sync** in the **Sync** panel.

2.  Under **Bucket**, choose a bucket that you connected before, or choose **Connect a new bucket**. For a new bucket, choose the **Storage** provider and give the bucket a **Name**. Then enter credentials that can list and read the bucket:

    <!-- tabs -->

    <!-- tab: Amazon S3 -->

    #### Amazon S3

    Enter the bucket name, its **Region**, and the **Access key ID** and **Secret access key** of an AWS user.

    <!-- /tab -->

    <!-- tab: Google Cloud Storage -->

    #### Google Cloud Storage

    Enter the bucket name and the **Service account key**, which is the JSON key file of a service account.

    <!-- /tab -->

    <!-- tab: Azure Blob Storage -->

    #### Azure Blob Storage

    Enter a **SAS URL** for the container that grants read and list permissions.

    <!-- /tab -->

    <!-- /tabs -->

3.  In **Folder**, enter the folder that your export job writes to, like `products/`. If you leave it empty, Customer.io loads files from the whole bucket.

4.  Under **Check for new files**, choose how often Customer.io checks the folder, every 1 to 24 hours.

5.  Click **Save**. Customer.io reads a file from a new bucket before it saves the connection, so you find out right away if your credentials don’t work. If the bucket doesn’t have any files yet, Customer.io can only check that your credentials can list the bucket.

 ![The Set up sync window, connecting a new Amazon S3 bucket and loading files from the products/ folder every hour.](https://docs.customer.io/images/collection-set-up-sync.png)

### Step 4: Check that your first export loaded

1.  Run your export job, if it hasn’t written a file to the folder yet.
2.  Click **Check now** in the **Sync** panel, so that you don’t have to wait for the next scheduled check.
3.  Check that the **Last file loaded** shows your export’s file, and that **Rejected rows** is 0. If it isn’t, see [Monitor a sync](#monitor-a-sync).

## Monitor a sync

The **Sync** panel shows the sync’s status, the **Last file loaded** and when it loaded, the **Next check**, and the number of **Rejected rows**.

 ![The Sync panel on a collection, showing an active sync with its bucket, folder, last file loaded, next check, and one rejected row](https://docs.customer.io/images/collection-sync-panel.png)

The panel can show you problems with your sync:

*   **The status is `Failed`.** The sync stopped at a file that Customer.io couldn’t load. The panel shows which file failed and why. Customer.io doesn’t load the files that sort after it until you [fix the failed file](#fix-a-failed-sync).
*   **Your latest export didn’t load.** If a check passes after your export and the **Last file loaded** still shows an earlier file or time, Customer.io didn’t find your file. Usually, your export job wrote the file outside the sync’s folder.
*   **The sync rejected rows.** Customer.io loads the rest of a file when some rows are invalid, like a row without a key, so the status stays Active. Click **View rejected rows** to see each row’s file, row number, key, and the reason Customer.io rejected it. In a CSV file, the header row doesn’t count, so row 1 is the first row of data. When Customer.io loads a file again, it clears the rejected rows from the file’s earlier version. Customer.io keeps rejected rows for 90 days.

Customer.io doesn’t send you a notification when it loads a file. To alert on these problems automatically, you can [check the sync with the API](#check-the-sync-with-the-api) after each export.

### Fix a failed sync

When a sync fails, the **Sync** panel shows why. A sync can fail for these reasons:

*   **Customer.io can’t load a file.** Customer.io stops at the file and doesn’t load the files that sort after it. At each check after that, Customer.io tries the same file again. To get the sync going again, overwrite the file with one that Customer.io can load, or delete the file.
*   **The folder holds more than 100,000 files.** Move files that Customer.io already loaded out of the folder.
*   **You deleted the bucket connection that the sync uses.** Click **Edit sync** and choose another bucket.

Then click **Check now** to check the folder right away.

## Change or stop a sync

To change a sync, click **Edit sync** in the **Sync** panel. Your changes affect which files Customer.io loads:

*   **Changing how often Customer.io checks** keeps the sync’s place.
*   **Changing the bucket or the folder** starts the sync over. Customer.io loads every file in the folder again, starting from the first one. While the earlier files load, rows can show older values, and a large folder can take several checks to load. After Customer.io loads the newest file again, the rows match your latest export. If you changed any of those rows with write requests since then, the files overwrite your changes. To avoid loading every file again, move the files that Customer.io already loaded out of the folder before you make the change.

To stop a sync, click > **Stop sync**. The rows that the sync already loaded stay in the collection. Customer.io also clears the list of rejected rows, so look at them before you stop the sync.

### Rotate your bucket’s credentials

You can’t change the credentials of a bucket that you connected. When you rotate a key, you need to connect the bucket again with the new credentials. If you want to avoid loading every file again, move the files that Customer.io already loaded out of the folder.

1.  Go to your sync and click **Edit sync**.
2.  Choose **Connect a new bucket**, and enter the new credentials.
3.  Delete the old connection with `DELETE:/v1/collections/sources/:id`. You can only delete a connection with the API.

If you only changed the bucket’s permissions, use `POST:/v1/collections/sources/:id/check` to confirm that Customer.io can still read the bucket.

If you delete a connection that a sync uses, the sync stops loading files, and the **Sync** panel tells you that the bucket was removed. Edit the sync to choose another bucket.

## Use the API

Everything on this page also works with the [App API](https://docs.customer.io/integrations/api/app/), so you can set up syncs from your own deployment scripts and monitor them. Use a private key (`sk_`) from the [API Credentials page](https://fly.customer.io/settings/api_credentials?keyType=app) as a bearer token. Use `https://api.customer.io` in the US region or `https://api-eu.customer.io` in the EU region.

### Set up a sync with the API

1.  Set the key field and delete field with `PUT:/v1/collections/:id`:

    ```
    {
      "key_field": "sku",
      "delete_field": "deleted"
    }
    ```

    Customer.io queues the change as a write request and responds with a `write` that has an `id`. Check `GET:/v1/collections/:id/writes/:write_id` until the `state` is `applied`. If the `state` is `failed`, the `message` names a row that’s missing a key or repeats another row’s key.

2.  Connect the bucket with `POST:/v1/collections/sources`. You can set a `prefix` on the connection, so that Customer.io can only read files in that folder of the bucket. This example connects an Amazon S3 bucket:

    ```
    {
      "name": "Product exports",
      "provider": "s3",
      "bucket": "acme-exports",
      "region": "us-east-1",
      "prefix": "collections/",
      "credentials": {
        "access_key_id": "AKIAIOSFODNN7EXAMPLE",
        "secret_access_key": "<your secret access key>"
      }
    }
    ```

    The response includes the connection’s `id`. See the [API reference](https://docs.customer.io/integrations/api/app/tag/collections/createCollectionSource/) for the fields that Google Cloud Storage and Azure Blob Storage need.

3.  Start the sync with `PUT:/v1/collections/:id/sync`. Set `prefix` to the folder that holds this collection’s files, relative to the connection’s `prefix`. Set `poll_hours` to how often Customer.io checks the folder, from 1 to 24 hours:

    ```
    {
      "source_id": "01JD8X4Q6R2W7N3K5M9P1T0VZC",
      "prefix": "products/",
      "poll_hours": 1
    }
    ```

To check the folder right away, use `POST:/v1/collections/:id/sync/poll`. To stop the sync, use `DELETE:/v1/collections/:id/sync`.

### Check the sync with the API

`GET:/v1/collections/:id/sync` shows the sync’s `state`, the last file that Customer.io loaded, and the most recent error:

```
{
  "sync": {
    "collection_id": 12,
    "source_id": "01JD8X4Q6R2W7N3K5M9P1T0VZC",
    "prefix": "products/",
    "poll_hours": 1,
    "state": "active",
    "last_file": {
      "path": "products/2026-11-04T0600-products.csv",
      "applied_at": "2026-11-04T06:31:12Z"
    },
    "next_run_at": "2026-11-04T07:31:12Z"
  }
}
```

Have your monitoring alert when:

*   The `state` is `failed`. The `last_error` shows which file failed and why.
*   More than `poll_hours` passed since your export, and `last_file.applied_at` is still earlier than your export.
*   `GET:/v1/collections/:id/sync/rejects` returns rejected rows. Each one includes the row’s `file`, `row` number, `key`, and `code`.

## Load a file when you’re ready

If your systems need to know when a particular file loads, send a write request for the file instead of using a sync. For example, your system might need to wait for a price update to load before it triggers a sale announcement. You need the `id` of a connected bucket, which you can get from `GET:/v1/collections/sources`.

1.  Upload the file to a folder in the bucket that a sync doesn’t check, like `on-demand/`. A sync loads every file in its folder, so it would load your file again at its next check and could undo changes that you made since.

2.  Send the connection’s `id` and the file’s `path` to `POST:/v1/collections/:id/writes`. The `path` is relative to the connection’s `prefix`.

    ```
    {
      "source_id": "01JD8X4Q6R2W7N3K5M9P1T0VZC",
      "path": "on-demand/2026-11-04-price-update.csv"
    }
    ```

3.  Check `GET:/v1/collections/:id/writes/:write_id` with the `id` from the response until the `state` is no longer `queued`.

When the `state` is `applied`, Customer.io loaded the file. When the `state` is `failed`, the `code` explains why. Customer.io checks every row before it saves anything, so if the `code` is `items`, the `items` list shows each invalid row, and Customer.io didn’t save any rows from the file. Fix those rows, upload the file again, and send another write request. Because each row replaces the row with the same key, you can send the same file more than once, and the collection ends up the same.

A write request can read a file from any folder under the connection’s `prefix`, and it loads a file even if you loaded it before. Customer.io keeps the result of each write request for 24 hours.

## Use a sync and write requests together

You can send write requests to a collection that has a sync. Customer.io applies every change to a collection one at a time, in the order that it receives them. A sync loads a file as a series of writes of 500 rows, so a write request that you send while a file loads can land in the middle of the file. When a write request and a file change the same row, the change that Customer.io receives last wins.

To keep the results predictable, have the sync and your write requests change different rows. For example, let the sync load the products from your nightly catalog export, and use write requests for promotional rows that your export doesn’t include.
