Push metrics and message statuses
UpdatedHow it works
While we track the status of each individual message as it goes through Customer.io, push notifications can be trickier to track than other message channels for two main reasons:
- A person can have multiple devices, and you can send a message to all of a person’s devices. When you send to All Devices, we aggregate metrics across all of a person’s devices because we (and you!) monitor engagement at a personal level, not a device level.
- Push notifications go through an intermediary—the push provider (APNS or FCM). We report statuses between us and your push provider; devices using our SDK report back
deliveredandopenedmetrics back to Customer.io—but there can be a slight delay in reporting these metrics. And Customers who have an earlier version of your app that doesn’t have our SDK installed may not report these statuses back at all.
notification) a-->b{Is push finished
sending to all devices} b-.->|No|c(Push Attempted) b-->|Yes|i{Is at least one
send successful?} i-->|Yes|d(Push Sent) i-.->|No, liquid
failure|j(Push Failed) i-.->|No, bounced
tokens|k(Push Bounced) d-->e{Is device online?} e-->|Yes|z(Push Delivered) z-->g{Does person
tap push?}-->|yes|h(Push Opened) e-.->|No|x(Wait for device
to come online)-.->z
Mobile metrics
In general, you’re concerned with a few metrics for push notifications:
- Sent: We’ve sent a message to the push provider.
- Delivered: The device reported that it received the message.
- Opened: The user tapped the notification; if the notification includes a deep link, this takes the user into the app.
- Converted: The user achieved a goal and you want to report that back to Customer.io.
You can measure success of push notifications against the number of Sent or Delivered messages, but there’s an important distinction between the two: Sent is reported by the push provider (APNS or FCM) and Delivered is reported by the device.
When you send a push notification, it’s much more likely that it’ll report Sent right away, but it may not report Delivered (or other metrics on the device) until the user opens the app. When you measure the success of messages, just remember that your Sent is immediately trustworthy, and the Delivered count may change over time as people open your app.
What push metrics do we capture from devices and when?
Devices report message deliveries, opens, and conversions. But unlike other channels, push notification reporting depends on the status of your app on the recipient’s device.
- If your app is in the foreground, we’ll typically report metrics (if you show push notifications when your app is in the foreground).
- If your app is in the background, we might be able to report metrics based on the device’s operating system and your SDK implementation. See Reporting metrics while apps are in the background for more information.
- If the app is closed (sometimes called the killed or cold state), we can’t report metrics until the user opens your app.
Android and iOS have strict rules about what we can do when apps are in the background. This can cause a delay between when you send a message and when we can report metrics relating to deliveries, opens, and so on.
In general, this is an issue for deliveries, which happen independently of your app’s state on a recipient’s device—as opposed to an Opened or Converted events. For example, if your push notifications include deep links, your link will open the app and report metrics. Your notification is only likely to be opened in the background if it doesn’t include a deep link into your app.
| Event | Reported in Foreground | Reported in Background | Reported in Closed |
|---|---|---|---|
| Sent | N/A | N/A | N/A |
| Delivered | ✅ | ✅* | ❌ |
| Opened | ✅ | ✅** | ❌ |
| Converted | ✅ | N/A | ❌ |
Reporting metrics when apps are in the background
Android and iOS both have strict, but slightly different rules for what our SDKs are allowed to do when apps are in the background.
For iOS, the SDK has 30 seconds after receiving a notification to report metrics back to Customer.io. Typically that means that we’ll report a delivery as long as your app is opened.
For Android, apps in the background will report metrics back to Customer.io if:
- The device isn’t in Doze mode.
- The device and/or your app isn’t in Deep Sleep mode.
- The user hasn’t otherwise optimized battery usage in a way that blocks background activity.
Why are messages sent but not delivered?
We mark a message sent when we send it to the push provider (APNS or FCM). But we can’t verify delivery until a device reports a delivered event back to Customer.io.
Because the SDK reports delivery, a message might show sent but not yet show delivered when:
- The app is closed.
- The recipient’s device is offline.
- The recipient doesn’t have a version of your app containing the Customer.io SDK.
In many cases, the push notification probably made it to your recipient’s device, but we won’t know about it because the app can’t report back delivered metrics until it’s opened or put in a state that allows it to report back metrics (like foregrounding the app).
unsubscribed true?}
a-.....->|yes|c(push not sent)
a-->|no|b(push sent)
b(push sent)
b-->d{Is person opted-intonotifications?} d-...->|no|h(push not delivered) d-->|yes|f{Does the app include
the Customer.io SDK?} f-..->|no|i(push delivered but delivery
not reported to Customer.io) f-->|yes|k{Is device online and
able to report?} k-.->|no|l(Wait for app
to come online) l-.->k k-->|yes|j(push delivered and marked
delivered in Customer.io)Differences in delivered metrics for Android and iOS
Android and iOS handle deliveries differently, which means that you’ll see a significant difference in delivered metrics between the two platforms.
iOS devices only report delivery when a user has opted into notifications and a push notification appears to the user (whether in the notification tray, lock screen, etc). But Android reports the delivery of notifications regardless of opt-in status. This means that a message can be delivered to a device, suppressed because a user hasn’t opted into notifications, and the device will still report back that the message was delivered.
Because Android reports delivery regardless of opt-in status, you’ll likely see higher delivered metrics for Android devices than iOS devices. We’re working on ways to reconcile the difference between the two platforms in the future, so that we can give you a better understanding of your audience’s engagement.
Push reporting when migrating from another provider
If you migrated a push audience from another provider, your audience may have versions of your app that don’t include our SDK. These devices can still receive notifications (assuming you use the same certificates in Customer.io and your previous service provider), but they won’t report back delivered metrics to Customer.io unless you gather this data using a third party like Segment or Rudderstack and send these events to Customer.io.
Push notification statuses
You can look up the status of each push notification by clicking on the message in the Deliveries page. This can help you understand whether a message was sent, delivered, opened, or converted.
Note that statuses are affected by the All Devices and Last Used Device settings. See Push notification status when targeting all devices for more information.
Attempted: We’re in the process of sending a message to the push notification service.
This status means that your push notification has been created and we’ve tried to send it via the appropriate service—Apple’s Push Notification Service (APNS) or Firebase Cloud Messaging (FCM)—which then would send to your customer’s device. If there are errors with our attempt, we’ll try to send it again up to ten times.
Sent: We’ve successfully sent your message to the push notification service.
This status does not indicate that a message has been successfully delivered. It only means that the push payload was valid and we were able to pass it to the push notification service for valid push tokens and filter out suppressed devices.
If you’ve sent your message to all of a person’s devices, this indicates that we’ve successfully sent your message to the push notification service for all of a person’s devices and filtered out suppressed devices.
Note that the total number of “Sends” in your campaign metrics includes people whose messages bounced and were suppressed. We do this to help calculate your total audience size when you send a message: sends minus suppressed and bounced messages equals total recipients.
Delivered: The device reported back that it received the message.
There are a number of things that can affect or delay us reporting message delivery. See What push metrics do we capture from devices and when? for more information.
Opened: The user tapped the notification.
If you’ve sent your message to all of a person’s devices, we report a message as Opened when a person taps on a push notification sent to any of their devices. This metric is reported back to Customer.io when a person opens your app (or already has the app open).
Failed: Our attempts to send a push notification were unsuccessful.
If you tried to send a notification to all of a person’s devices, then this status occurs when none of the attempts to send a push notification to the end-user’s devices are successful. You’ll be able to see the failure reasons on the Delivery page. If we’re able to send a push notification to at least one of a person’s devices, we’ll report the message as Sent.
Converted: The user achieved a goal and your app reported that back to Customer.io.
This is treated the same as emails; when a user matches your campaign’s conversion criteria, the message nearest the conversion is marked as Converted.
Opens and clicks are the same things in push notifications
Unlike other channels, where people have to open a message before they can click a link, people can read a notification right when it’s delivered—they don’t have to open it. So, in push notifications, we track Opened in the same way that most other messages track Clicked.
Push notification status when targeting all devices
For other message channels, you send your message to an individual email address, phone number, etc—a message is one-to-one with a recipient. But for push notifications, you can target all of a person’s devices, which affects how we track the status of messages and associated metrics.
For some statuses, like Sent, we count a status when it has occurred for all of a person’s devices. For others, like Opened, we count it when it has occurred for any of a person’s devices. For metrics, we’ll display an aggregate metric for the person rather than each device they own, because we (and you!) monitor engagement at a personal level, not a device level.
In general, we recommend that you use Last Device when you send a message so you don’t overwhelm your audience with messages across all their possible devices.
For example, when you use Last Device, a sent message is a single push notification sent to the push provider. When you use All Devices, we don’t count a message as being sent until we’ve sent a message to the push provider for each of a person’s devices.
| Status | All Devices | Any Device | Description |
|---|---|---|---|
| Attempted | ✅ | We're in the process of sending messages. | |
| Sent | ✅ | We've sent messages to the push provider. | |
| Delivered | ✅ | Your app reported back that it received the message. | |
| Opened | ✅ | Your app reported back that the message was opened. | |
| Converted | ✅ | The person matched your campaign's conversion criteria. | |
| Failed | ✅ | We were unable to send messages to the push provider. This is typically due to a liquidA syntax that supports variables, letting you personalize messages for your audience. For example, if you want to reference a person’s first name, you might use the variable {{customer.first_name}}. error. See Failed for more information. | |
| Undeliverable | ✅ | We couldn't attempt delivery. This happens when a user has no devices associated with them. | |
| Bounced | ✅ | There's a temporary delivery issue, like a soft bounce in an email. This usually happens when a user has uninstalled your app, their device token has expired or their device token is invalid. |
Message failures
If you see a message Failed, you can see the failure reasons on the Delivery page. Here’s a list of failure reasons you might see, what they mean, and how to fix them.
| Failure Reason | Description |
|---|---|
| Liquid error: variable is missing | The recipient didn’t have a value for a variable you used in your message. For example, if your message used {{customer.email}} and the customer didn’t have an email attribute, the message will fail. Check that you’re using the correct variable name and either set a fallback or that the recipient has a value for that variable. |
| Customer doesn’t exist | A rare error indicating that the value you’ve used for the recipient doesn’t exist. Make sure you’re using the correct customer ID or email for a message. |
| Customer is unsubscribed | This error indicates that the customer has opted out of receiving messages. Verify the customer’s subscription status to make sure they can receive messages. |
| Credentials not found | Your push credentials failed or were otherwise empty. Check that you’ve provided your push certificate and that it hasn’t expired. |
Set campaign goals to measure engagement
Unlike email, recipients can read and react to your push notification without “opening” it.
While you can measure engagement with Opened metrics—where a person taps on a push notification to go to a link—you may find more success setting a campaign goal to measure whether or not a message helps you achieve a specific business goal.
Ideally, your messages have a call to action, a result you want your audience to achieve. You want your audience to finish setting up their profile in your app, complete a purchase, sign up for a class, and so on.
If you capture these actions as goals, you’ll be able to understand how many people engage with your message and achieve your goal—which is a much better metric of success than raw sent and delivered metrics.
We track metrics at the message and journeyA person or data object’s path through your campaign. level. Setting a campaign goal as an event that you want your audience to achieve—a completed purchase, visiting a specific page, etc—can help you understand how your push notifications contribute to your business goals, even when it’s difficult to track the status of individual push notifications.
Campaign Overview data
On the Overview page, you can see how your push notifications perform in your campaign.
Delivered
You can track delivery using our SDKs, or by adding custom code to your app to report delivery metrics back to Customer.io when a customer receives a push.
Opened
To track opened metrics, you need to either integrate with our SDKs or add some code to your app to detect when opens occur and then send “open” events to Customer.io as documented in our Technical Integration Guide. Note, you need to include the CIO-Delivery-ID and CIO-Delivery-Token parameters from the push when sending open events to Customer.io.
Delivery Logs data
When filtering deliveries, you can choose to only see push notifications. In this view, you’ll be able to see:
- The name of the push notification and a link to the message
- Which customer triggered that message, and how many of their devices we attempted to send to
- When the push notification was attempted
- The push notification status (sent, failed, attempted, etc.)
- Conversion information where applicable
You’ll also have the option to retry sending the push notification if its status is ‘failed’:
