Critical alert delay

I am sending push notifications to the app with critical alerts, but there is a significant delay. If the number of target devices is 1000 or less, notifications will be received normally within a few seconds to a minute. Once the number of target devices exceeds 1000, some devices will arrive quickly (normally within a few seconds to 1 minute) and others will arrive late (3 minutes to 15 minutes, divided into hundreds of items). In severe cases, notifications to more than 80% of devices will be delayed.

Example: If you send 3000 notifications at once,  1 minute: Notify 400 items  5 minutes: Notify 1000 items  10 minutes: Notify 1000 items  13 minutes: Notify 600 items *The timing of 5 minutes, 10 minutes, and 13 minutes changes every time and is not at regular intervals.

We understand that according to the push notification specifications, sending several thousand messages at once is not a problem. Please let me know if there is a rule, such as sending 1000 items at a time, in order to deliver quickly and with minimal delay.

Answered by Engineer in 814659022

There is no limit or rule that says you can’t send more than 1000 at one time, and APNs is capable of handling way more than that. So the issue is likely on your end.

1000 is a curious number, because that’s the maximum number of HTTP/2 channels APNs will offer per connection.

My guess is will be that your push servers are not configured to adequately handle this volume. Things to look for are:

  • you have adequate connections open to handle the volume
  • your various connections are not hitting the same APNs host. Use uncached DNS lookups to get a fresh and balanced span of hosts
  • keep your connections open as long as you can. Make sure you are not closing and reopening them
  • make sure your push software is not DoS’ing itself by waiting for the response from APNs on a single thread before sending the next one.

These are the basic guidance points I can give. You may want to talk to your server people to see where exactly the bottleneck is. Then I can help with more specific guidance to clear that.


Argun Tekant /  DTS Engineer / Core Technologies

There is no limit or rule that says you can’t send more than 1000 at one time, and APNs is capable of handling way more than that. So the issue is likely on your end.

1000 is a curious number, because that’s the maximum number of HTTP/2 channels APNs will offer per connection.

My guess is will be that your push servers are not configured to adequately handle this volume. Things to look for are:

  • you have adequate connections open to handle the volume
  • your various connections are not hitting the same APNs host. Use uncached DNS lookups to get a fresh and balanced span of hosts
  • keep your connections open as long as you can. Make sure you are not closing and reopening them
  • make sure your push software is not DoS’ing itself by waiting for the response from APNs on a single thread before sending the next one.

These are the basic guidance points I can give. You may want to talk to your server people to see where exactly the bottleneck is. Then I can help with more specific guidance to clear that.


Argun Tekant /  DTS Engineer / Core Technologies

Notifications are not split into 1000 items each time. The number of cases changes every time, sometimes reaching 1,500 or 2,000 cases, so I don't think it's a problem with the HTTP channel. Also, even if you send the same number of items, the number of times they will be divided will be different each time.

The notification was sent to APNS from MSAzure's NotificationHubs, and we asked MSAzure to investigate. We have confirmed that several thousand items can be sent from NotificationHubs to APNS at once. However, due to a delay on the APNS side, there was no response returned to NotificationHubs, and the response was that it was waiting. Is there anything we can learn from this?

Number and frequency of notification splits.

Critical alert delay
 
 
Q