Push metrics and message statuses
UpdatedHow it works
While we track the status of each individual message as it goes through Customer.io, push notifications can be trickier to track than other message channels for two main reasons:
- A person can have multiple devices, and you can send a message to all of a person’s devices. When you send to All Devices, we aggregate metrics across all of a person’s devices because we (and you!) monitor engagement at a personal level, not a device level.
- Push notifications go through an intermediary—the push provider (APNS or FCM). We report statuses between us and your push provider; devices using our SDK report back
deliveredandopenedmetrics back to Customer.io—but there can be a slight delay in reporting these metrics. And Customers who have an earlier version of your app that doesn’t have our SDK installed may not report these statuses back at all.
notification) a-->b{Is push finished
sending to all devices} b-.->|No|c(Push Attempted) b-->|Yes|i{Is at least one
send successful?} i-->|Yes|d(Push Sent) i-.->|No, liquid
failure|j(Push Failed) i-.->|No, bounced
tokens|k(Push Bounced) d-->e{Is device online?} e-->|Yes|z(Push Delivered) z-->g{Does person
tap push?}-->|yes|h(Push Opened) e-.->|No|x(Wait for device
to come online)-.->z
Do the All Devices and Last Used Device settings affect metrics?
When you send a message, you can determine whether you send it to all of a person’s devices, their last-used device, or a custom device token. This setting affects how we track metrics for your push notifications. In general, we recommend that you use Last Device when you send a message so you don’t overwhelm your audience with messages across all their possible devices.
The Last Device and Custom Device Token settings work like our traditional messages—each person gets a single message, so it’s easier to understand what each metric represents.
When you send to All Devices, we display an aggregate metric for the person rather than each device they own, because we (and you!) monitor engagement at a personal level, not a device level.
For example, when you use Last Device, a sent message is a single push notification sent to the push provider. When you use All Devices, we don’t count a message as being sent until we’ve sent a message to the push provider for each of a person’s devices.
Push notification statuses
Because you can send to all devices, there’s a slight difference as to when we count a metric. For some statuses, like Sent, we count a status when it has occurred for all of a person’s devices. For others, like Opened, we count it when it has occurred for any of a person’s devices.
| Status | All Devices | Any Device | Description |
|---|---|---|---|
| Attempted | ✅ | We're in the process of sending messages. | |
| Sent | ✅ | We've sent messages to the push provider. | |
| Delivered | ✅ | Your app reported back that it received the message. | |
| Opened | ✅ | Your app reported back that the message was opened. | |
| Converted | ✅ | The person met your campaign conversion goal. | |
| Failed | ✅ | We were unable to send messages to the push provider. This is typically due to a liquidA syntax that supports variables, letting you personalize messages for your audience. For example, if you want to reference a person’s first name, you might use the variable {{customer.first_name}}. error. See Failed for more information. | |
| Undeliverable | ✅ | We couldn't attempt delivery. This happens when a user has no devices associated with them. | |
| Bounced | ✅ | There's a temporary delivery issue, like a soft bounce in an email. This usually happens when a user has uninstalled your app, their device token has expired or their device token is invalid. |
Attempted
We show messages as Attempted until we’ve either succeeded or failed to send messages for all of a person’s targeted devices.
This status means that your push notification has been created and we’ve tried to send it via the appropriate service—Apple’s Push Notification Service (APNS) or Firebase Cloud Messaging (FCM)—which then would send to your customer’s device. If there are errors with our attempt, we’ll try to send it again up to ten times.
Sent
We show a status of Sent when we’ve sent messages for all of a person’s devices to the delivery provider and filtered out suppressed devices.
This status does not indicate that a message has been successfully delivered. It only means that the push payload was valid and we were able to pass it to the push notification service for valid push tokens and filter out suppressed devices.
Note that the total number of “Sends” in your campaign metrics includes people who whose messages bounced and or were suppressed. We do this to help calculate your total audience size when you send a message: sends minus suppressed and bounced messages equals total recipients.
Delivered
If you use our SDK, we report a message Delivered when the push provider delivers it to any of a person’s devices and our SDK reports back a delivered metric.
Differences in delivered metrics for Android and iOS
Android and iOS handle deliveries differently, which means that you’ll see a significant difference in delivered metrics between the two platforms.
iOS devices only report delivery when a user has opted into notifications and a push notification appears to the user (whether in the notification tray, lock screen, etc). But Android reports the delivery of notifications regardless of opt-in status. This means that a message can be delivered to a device, suppressed because a user hasn’t opted into notifications, and the device will still report back that the message was delivered.
Because Android reports delivery regardless of opt-in status, you’ll likely see higher delivered metrics for Android devices than iOS devices. We’re working on ways to reconcile the difference between the two platforms in the future, so that we can give you a better understanding of your audience’s engagement.
Opened
We report a message as Opened when a person taps on a push notification sent to any of their devices. This metric is reported back to Customer.io when a person opens your app (or already has the app open) and taps on a push notification.
Opens and clicks are the same thing in push notifications
Unlike other channels, where people have to open a message before they can click a link, etc, people can read a notification right when it’s delivered—they don’t have to open it. So, in push notifications, we track Opened in the same way that most other messages track Clicks.
Failed
If none of the attempts to send a push notification to the end-user’s device(s) are successful, then we mark the message as Failed. You’ll be able to see the failure reasons on the Delivery page.
Here’s a list of failure reasons you might see, what they mean, and how to fix them.
| Failure Reason | Description |
|---|---|
| Liquid error: variable is missing | The recipient didn’t have a value for a variable you used in your message. For example, if your message used {{customer.email}} and the customer didn’t have an email attribute, the message will fail. Check that you’re using the correct variable name and either set a fallback or that the recipient has a value for that variable. |
| Customer doesn’t exist | A rare error indicating that the value you’ve used for the recipient doesn’t exist. Make sure you’re using the correct customer ID or email for a message. |
| Customer is unsubscribed | This error indicates that the customer has opted out of receiving messages. Verify the customer’s subscription status to make sure they can receive messages. |
| Credentials not found | Your push credentials failed or were otherwise empty. Check that you’ve provided your push certificate and that it hasn’t expired. |
Converted
This is treated the same as emails; when a user meets your campaign conversion goal, the message nearest the conversion is marked as Converted.
Why are messages sent but not delivered?
We mark a message sent when we send it to the push provider (APNS or FCM). We consider a message delivered when the push provider delivers it to the device and our SDK reports back a delivered metric.
Because the SDK reports delivery, a message might show sent but not yet show delivered when
- The recipient’s device is offline.
- The recipient doesn’t have a version of your app containing the Customer.io SDK.
In the second case, the push notification very well have made it to your recipient’s device, but we won’t know about it because the app doesn’t report back delivered metrics.
unsubscribed true?}
a-....->|yes|c(push not sent)
a-->|no|b(push sent)
b(push sent)
b-->d{is person opted-intonotifications?} d-..->|no|h(push not delivered) d-->|yes|f{Does the app include
the Customer.io SDK?} f-.->|no|i(push delivered but delivery
not reported to Customer.io) f-->|yes|j(push delivered and marked
delivered in Customer.io)If you migrated a push audience from another provider, your audience may have versions of your app that don’t include our SDK. These devices can still receive notifications (assuming you use the same certificates in Customer.io and your previous service provider), but they won’t report back delivered metrics unless you gather this data from a third party (like Segment or Rudderstack) and report it back to Customer.io.
Set campaign goals to measure engagement
Unlike email, recipients can read and react to your push notification without “opening” it.
While you can measure engagement with Opened metrics—where a person taps on a push notification to go to a link—you may find more success setting a campaign goal to measure whether or not a message helps you achieve a specific business goal.
Ideally, your messages have a call to action, a result you want your audience to achieve. You want your audience to finish setting up their profile in your app, complete a purchase, sign up for a class, and so on.
If you capture these actions as goals, you’ll be able to understand how many people engage with your message and achieve your goal—which is a much better metric of success than raw sent and delivered metrics.
We track metrics at the message and journeyA person or data object’s path through your campaign. level. Setting a campaign goal as an event that you want your audience to achieve—a completed purchase, visiting a specific page, etc—can help you understand how your push notifications contribute to your business goals, even when it’s difficult to track the status of individual push notifications.
Campaign Overview data
On the Overview page, you can see how your push notifications perform in your campaign.
Delivered
You can track delivery using our SDKs, or by adding custom code to your app to report delivery metrics back to Customer.io when a customer receives a push.
Opened
To track opened metrics, you need to either integrate with our SDKs or add some code to your app to detect when opens occur and then send “open” events to Customer.io as documented in our Technical Integration Guide. Note, you need to include the CIO-Delivery-ID and CIO-Delivery-Token parameters from the push when sending open events to Customer.io.
Delivery Logs data
When filtering deliveries, you can choose to only see push notifications. In this view, you’ll be able to see:
- The name of the push notification and a link to the message
- Which customer triggered that message, and how many of their devices we attempted to send to
- When the push notification was attempted
- The push notification status (sent, failed, attempted, etc.)
- Conversion information where applicable
You’ll also have the option to retry sending the push notification if its status is ‘failed’:
