macOS wifi ping spikes

Before we start:

• I’m presuming that by “spikes” you’re referring to the sections of increased latency.

• Also, make sure to read Wi-Fi Fundamentals because I’ll assume terms from there.

Most Wi-Fi hardware only has a single radio, and that radio must be tuned to a specific channel. Thus, the hardware can only work on one channel at a time.

However, there are situations where a STA need to switch channels. For example:

• When actively scanning for networks

• When an Apple STA is coordinating peer-to-peer Wi-Fi work

While it’s on that temporary channel, the STA can’t be on the channel it’s using to communicating with the AP, and thus you see spikes in latency.

If you want to investigate this further you should read Investigating Network Latency Problems. That shows how to isolate the cause of the latency to one specific subsystem. And, in the C and D case, you can use a Wi-Fi level packet trace to see what’s really happening on the ‘wire’.

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Hmm, what do you think about switching the Synergy protocol to UDP?

what do you think about switching the Synergy protocol to UDP?

I presume you’re asking about what effect this will have on this specific latency issue? That really depends on the cause of the latency. If it is caused by channel switching, changing to UDP won’t have any effect.

It’s a common misconception that UDP has lower latency than TCP. From a latency perspective the only benefit that UDP has is that, if the network drops packet N, you don’t have to wait for a retransmissions before you see packet N+1. In your case, however, there’s no indication that packets are being dropped (which makes sense because Wi-Fi has its own link-level flow controller and retransmission mechanism).

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

I am also seeing this annoying problem on my latest gen MacBook Pro running 10.13.4, and it is affecting basic ssh sessions in terminal to the point of making them very unresponsive and frustrating to use at times..

It doesn't happen on other platforms, so Mr. Eskimo, Instead of finding excuses please ESCALATE THIS PROBLEM so that it gets properly investigated and fixed ASAP.

Marc

The problem is even better illustrated with a shorter interval value when pinging a local peer (such as the default gateway IP):

% ping -i 0.1 <IP_of_local_peer>

Then simultaneously starting an application such as Google Earth Pro causes not only increased latency but many request timeouts.

Marc

I have similar problem with slow ssh connection. Moreover, ping depends on interval value. If you soluted your problem, can you help me?

Mikhail

No solution yet. I'm astonished that this major bug still hasn't been resolved. It's still present even in the latest 10.14.1 beta.

I suspect that there is an interaction between apps such as Google Earth constantly querying for geolocation which leads to wifi scans which then cause the increased latency and packet loss on the wifi interface.

Perhaps even having browser windows open on sites that constantly poll for user location could also induce scans causing the same detrimental effect on wifi performance / latency / packet loss .

Apple, could you please FIX THIS PROBLEM ? It is a significant issue !

turns out Google Earth is a QT app. QT has a "bearer" plugin which can frequently request wifi scans, killing network performance. See:

https://bugreports.qt.io/browse/QTBUG-65586?gerritReviewStatus=All

and https://lostdomain.org/2017/06/17/qt-qnetworkaccessmanager-causing-latency-spikes-on-wifi/

Until Apple and QT developers get their act together, possible workarounds are setting the QT_BEARER_POLL_TIMEOUT environment variable to -1 or disabling the wlan bearer plugin altogether

in the QT app.

(for Google Earth Pro: "sudo chmod 0 /Applications/Google\ Earth\ Pro.app/Contents/plugins/bearer/libqcorewlanbearer.dylib")

I agree that it's probably wifi scanning that causes the latency. It seems that the locationd process triggers this approximately once a minute. If you start a ping in the terminal:

ping -i 0.25 192.168.1.1 --apple-time

...and watch the activity in Console.app, you'll notice that the ping response time rises as described just as locationd starts doing it's once-per-minute update. If you disable Location Services in the Privacy pref pane the periodic ping latency issues go away completely.

This latency causes major issues in audio quality with apps like GoToMeeting and causes video corruption and lag in streaming video game apps like Google's Project Stream. These issues go away when locationd isn't scanning!

Seems like there's a bug here somewhere in locationd... if it's not possible to avoid the latency during a wifi scan then perhaps locationd needs to either skip those altogether or perform them far less often. As it is today, I need to disable Location Services on my MBP if I'm joining a conference call on wifi.

Seems like there's a bug here somewhere in

locationd

.

If this problem is causing you significant grief that you should file a bug about it. Make sure to include the extra Wi-Fi logging info per the instructions on our Bug Reporting > Profiles and Logs.

Please post your bug number, just for the record.

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Dear Eskimo,

rdrom, myself and others have extensively described the issue. Is that not sufficient for Apple to take action? You are the Apple employee, please do your work, file whatever bug report(s) are needed and above all, have this long standing issue fixed once and for all.

Is that not sufficient for Apple to take action?

It is best if you file your own bug reports because:

• You are the one seeing the problem, so you can describe it accurately

• If Wi-Fi Engineering has follow-up questions about the issue, they can contact you directly

• You can see the status of bug reports that you file

You are the Apple employee, please do your work …

The vast majority of my “work” involves responding to official DTS tech support incidents [1]. My time here on DevForums is roughly split 50:50 between my work time and my personal time. I choose to spend that time answering questions because I think that’s a more valuable use of that time than filing bug reports.

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

[1] You can learn more about this service on our page on the developer web site.

Even if location services are disabled, wifi scanning can destroy wireless performance. Scanning will essentially pause communications on your current wifi connection, as the hardware has to hop to other channels to scan for available access points. Depending on how long this takes, you could see an adverse impact on your current connection.

Applications and plugins can trigger this behavior by trying to interact with the locationd service, such as web browsers or QuickTime (it apparently has known bugs that will cause the application using qt libraries to trigger wifi scanning). This will manifest itself as frequent or periodic ping spikes and streaming performance drops (audio or video conference calling will drop quality or cut out entirely).

Video and voice calling, particularly over wireless, can be severely impacted. You should disable all wireless devices and use ethernet only if you require anything related to audio or video over the network. If your office is wireless only, you may need to re-evaluate your hardware / OS options.

One way to confirm this is to do the following:

1) open Apple's "Console" Application

2) add "scan" as asearch term to the top-right

3) scroll to the bottom of the log pane so new events show up at the bottom as they occur

4) open the terminal / iterm2 app and ping either '1.1.1.1' or your local router

5) click on your wireless icon and see if your ping lag spikes occur at the same time as the "locationd" messages

If you have location services disabled, you will see the following:

default 12:01:58.360773 -0400 locationd WIFI_LOC: location services are disabled, ignore scan result

default 12:01:58.907701 -0400 locationd WIFI_LOC: location services are disabled, ignore scan result

default 12:01:59.469546 -0400 locationd WIFI_LOC: location services are disabled, ignore scan result

default 12:02:00.089787 -0400 locationd WIFI_LOC: location services are disabled, ignore scan result

default 12:02:00.922026 -0400 locationd WIFI_LOC: location services are disabled, ignore scan result

If location services are enabled, you will see the following:

default 12:03:25.995118 -0400 locationd WIFI_LOC: onWifiNotification, notification, <private>, kNotificationScan, scanType, <private>, <private>, inject, <private>, inProg, <private>

default 12:03:25.995167 -0400 locationd WIFI_LOC: scan result, scanType, <private>, <private>, currentRequest, <private>

default 12:03:26.546485 -0400 locationd WIFI_LOC: onWifiNotification, notification, <private>, kNotificationScan, scanType, <private>, <private>, inject, <private>, inProg, <private>

default 12:03:26.546536 -0400 locationd WIFI_LOC: scan result, scanType, <private>, <private>, currentRequest, <private>

default 12:03:26.705880 -0400 locationd WIFI_LOC: processScanResult, aps, <private>, fIsUseNetworkLocationProvider, <private>, fIsUseTileManager, <private>

. . .

The ping latency may improve with location services disabled, but the problem likely will not be eliminated. You must identify the application, service, or plugin that is triggering wifi scanning.

Hopefully someone from Apple can chime in with a simple method of doing so (like monitoring / tracing to capture apps). In the event they don't respond, you might try taking a look at capturing additional logging...

https://www.howtogeek.com/211034/troubleshoot-and-analyze-your-mac%E2%80%99s-wi-fi-with-the-wireless-diagnostics-tool/

https://superuser.com/questions/585473/debugging-osx-airport-wifi-connection

Related links:

https://superuser.com/questions/1142798/experiencing-high-latency-on-wifi-every-other-second-with-macos-sierra

https://superuser.com/questions/1142798/experiencing-high-latency-on-wifi-every-other-second-with-macos-sierrahttps://apple.stackexchange.com/questions/310218/stop-destroying-network-performance-every-time-an-app-scans-for-wifi-networks/312388

https://lostdomain.org/2017/06/17/qt-qnetworkaccessmanager-causing-latency-spikes-on-wifi/

https://bugreports.qt.io/browse/QTBUG-65586?gerritReviewStatus=All

https://discussions.apple.com/thread/8523377

Thank you! You saved my day.

Here is what I did:

System Preferences -> Security and Privacy -> Location Services (left) -> System Services (at the bottom of the list) -> Details -> Turn off "Wi-Fi Networking"

And after that - reboot the computer.

The problem went away.

+1 on this issue. I've been chasing the ping-spikes for about a month after moving a new office. I hadn't experienced this in other networks. However, at this location, I was having high-latency spikes every ~5min. Very frustrating when you're in meetings and things freeze. Even worse, I was also experiencing packet loss when this happened.

In my case, similarly, the fix was to fully turn off location services. I don't want to given apps like Find My Mac, but I can't live with the ping spikes.

@Eskimo - can you let me know if this is already logged as a bug? Otherwise, I'll be happy to create one.

can you let me know if this is already logged as a bug?

Realistically, you should file your own bug about this even if it’s others have already done so. These problems are often specific to both your hardware and your environment.

Before filing a bug, review the comments in my 2 Mar 2018 and 25 Oct 2018 posts. Investigating bugs like this is hard, and it’s best if you can provide detailed information about the problem. IMO it’s worth going through the debugging procedure described in Investigating Network Latency Problems to help isolate the hop that’s causing the delay. And retain the evidence you collect during that process so that you can attach it to your bug report.

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

I just came across this thread and here's an update ...

My problem

I'm using Airfoil to transmit music playback from the Music to the Sonos One speaker and I'm experiencing lot of sound drops (playback interruptions). This problem occurs only and only when my MacBook Pro is connected via WiFi. There's no issue if I turn WiFi off and use an USB dongle with ethernet cable.

I can easily reproduce this problem by holding down the Option key and start rapidly clicking on the WiFi menu bar icon. The problem appears almost immediately and while I'm rapidly clicking on the icon there's basically no music playback.

Network traffic

Airfoil sends +- 4MB of data per minute, roughly 125 UDP (IPv4) packets/second. When this problem occurs, packet rate drops down to +- 60 packets/second, immediately goes up to 600 packets/second and then back to normal 125 UDP packets/second.

Other notes

When I open the Wireless Diagnostics, I see that my Tx rate is about 600Mbps & MCS Index is 7. But when I'm rapidly clicking on the WiFi menu icon, MCS Index circulates 7 -> 9 -> 7 -> 9 -> ... and the Tx rate goes up to 800Mbps and then back to 600Mbps. Not sure if it's related to this issues, but these (Tx rate & MCS Index) changes appear at the same time when the sound drops out.

Comparison to the Music.app

The official Music isn't affected. One can quit the Airfoil and transmit music via Airplay in the Music directly (there's a button on the right side of the volume slider). No matter what I do, there's no sound drop at all. The difference here is that the Music:

• utilizes AirPlayXPCHelper,
• amount of data sent per minute is +- 2MB (half of the Airfoil traffic),
• more importantly, Instruments & Wireshark tells me that the music is sent via TCP (IPv6) and there's also lot of PTPv2 traffic.

Other network traffic

There's no other network traffic. While this is happening, no uploads, no downloads, just the music transmission.

Workaround

I don't have a solution for this problem, but when I disabled Location Services completely (System Preferences - Security & Privacy - Location Services and uncheck the Enable Location Services) & rebooted, the problem disappeared completely. I can rapidly click on the WiFi menu icon, no sound drops at all. Thanks for the tip in this thread!

HW & OS

• macOS Catalina (10.15.3 -19D76)
• MacBook Pro (15-inch, 2016 - MacBookPro13,3)
• tp-link Archer VR300 v1 VDSL modem (1.0.0 0.8.0 v009c.0 Build 181214 Rel.51117n)

This was reported on Mar 1, 2018. Writing this down on Mar 6, 2020 and the issue is still there. 2 years, pretty sad for such a crucial component like networking.

What's next?

I'll test it for the whole day with disabled Location Services (so far good) and then I'll start enabling apps/system services one by one to see if there's a difference and to find which one is causing this issue. Then I'll gather wireless diagnostics, system logs & Airfoil logs and will report it to the Apple & Rogue Amoeba (Airfoil).

I'm in touch with Rogue Amoeba support, but at this stage, I can't really say if there's anything they can do in the Airfoil to prevent this or not. I still don't know what's causing this. Disabled Location services is a hint, but it's still too broad.

In the meanwhile, no location services, no Find My MBP, ...

Reported as FB7619830 in case anyone reads it.

Reported as FB7634737.

I was experiencing the exact same thing on an early 2015 MBP w/ Catalina. It was extremely aggravating as the 2.4GHz radio seems dead and 5GHz wasn't working well due to location relative to AP. Timestamping ping and running "/System/Library/PrivateFrameworks/Apple80211.framework/Versions/A/Resources/airport logger" showed that there was dependence on ping spikes (~1.5ms jumping up to ~200ms) to the local gateway with this airportd autojoin message, "Wed Jan 27 10:53:47: AutoJoin: <airportd[219]> Successful cache-assisted scan request for locationd" The MBP was constantly scanning, and without a decent WiFi connection, networking would ultimately fail.

The ping spikes were also dependent on the "kernel_task" process running, which probably means that networking was blocked in the kernel during the time airportd was doing a scan for locationd.

So, observing that all this was dependent and locationd had way more CPU time then one would imagine, it quickly led to the "Find My Mac" feature. Turning this off quieted locationd and the MBP completely stopped doing a WiFi scans.

Hopefully this helps others.

Hi,

here is my experience with the new MacbookPro from 2021 and macOS Monterey 12.3.

The ping spikes are still an annoying problem especially if you are working remotely via SSH. After endless debugging and tracing I solved the problem by deactivating the "Find My Mac" feature in the iCloud settings and restarting. I don't get why Apple is not addressing this problem. In my case I had the ping spikes every 5 seconds causing SSH, RDP connections to lag big time. This needs to be fixed because the "Find My Mac" feature is a huge plus and I don't want to deactivate it every time I leave the house.

Greetings

This bug has been driving me absolutely nuts for about 6 months. I was sure it was my router or my ISP (who would suspect a brand new $2500 macbook pro??). Spent countless hours trying to troubleshoot a nonexistent issue with those. I finally dug out my old 2014 macbook pro and the connection is absolutely perfect. The ping is an extremely consistent 20-30ms. On my new mac running monterey, every 30-60 secons or so, the ping will spike up to 1000ms+ or completely drop packets for ~2-3 seconds. Apple has built up a tremendous loyalty from me over the years, but these are the sorts of issues that will very quickly erode all of that.

I found a workaround which worked for me. If you have an iPad connected to the same network, disable Bluetooth and the lag spike goes away on the Mac.

This problem still exists on 2021 Macbook Pro M1 laptop. Every ~10 seconds there's a spike in network latency raising the ping responses from the router up to 300ms. Disabling the Location Services has no impact, but disabling Bluetooth solves the issue and I'm getting consistent ping responses. iPhone connected to the same router doesn't have this problem, same for a couple of Windows laptops - all can work fine with Bluetooth AND Wifi enabled at the same time.

I realize this is an old thread, but this (disabling awdl0) has been the sure-fire solution for me: https://medium.com/@mariociabarra/wifriedx-in-depth-look-at-yosemite-wifi-and-awdl-airdrop-41a93eb22e48

The WiFriedX app is optional, but is nice if you don't want to have to monitor the active state of awdl0.