16 Replies
      Latest reply on Sep 5, 2019 1:40 AM by eskimo
      nbolton Level 1 Level 1 (0 points)

        Hey Apple devs,

         

        Has anyone encountered ping spikes on macOS when using WiFi? When you run a constant ping, it looks like this…

         

        64 bytes from 192.168.0.30: icmp_seq=502 ttl=128 time=1.443 ms

        64 bytes from 192.168.0.30: icmp_seq=503 ttl=128 time=3.007 ms

        64 bytes from 192.168.0.30: icmp_seq=504 ttl=128 time=1.684 ms

        64 bytes from 192.168.0.30: icmp_seq=505 ttl=128 time=186.861 ms

        64 bytes from 192.168.0.30: icmp_seq=506 ttl=128 time=69.545 ms

        64 bytes from 192.168.0.30: icmp_seq=507 ttl=128 time=141.434 ms

        64 bytes from 192.168.0.30: icmp_seq=508 ttl=128 time=24.043 ms

        64 bytes from 192.168.0.30: icmp_seq=509 ttl=128 time=2.961 ms

        64 bytes from 192.168.0.30: icmp_seq=510 ttl=128 time=1.827 ms

        64 bytes from 192.168.0.30: icmp_seq=511 ttl=128 time=1.171 ms

         

        Seems to happen reliably when you open the WiFi menu bar on High Sierra. Other apps can cause this too.

         

        I think others are seeing this issue too (especially gamers):

        https://apple.stackexchange.com/questions/263638/macbook-pro-experiencing-ping-spikes-to-local-router

         

        As well as games, real time network apps like Synergy suffer too:

        https://symless.com/help/mac-wifi-lag

         

        Cheers,

        Nick

        • Re: macOS wifi ping spikes
          eskimo Apple Staff Apple Staff (12,295 points)

          Before we start:

          • I’m presuming that by “spikes” you’re referring to the sections of increased latency.

          • Also, make sure to read Wi-Fi Fundamentals because I’ll assume terms from there.

          Most Wi-Fi hardware only has a single radio, and that radio must be tuned to a specific channel.  Thus, the hardware can only work on one channel at a time.

          However, there are situations where a STA need to switch channels.  For example:

          • When actively scanning for networks

          • When an Apple STA is coordinating peer-to-peer Wi-Fi work

          While it’s on that temporary channel, the STA can’t be on the channel it’s using to communicating with the AP, and thus you see spikes in latency.

          If you want to investigate this further you should read Investigating Network Latency Problems.  That shows how to isolate the cause of the latency to one specific subsystem.  And, in the C and D case, you can use a Wi-Fi level packet trace to see what’s really happening on the ‘wire’.

          Share and Enjoy

          Quinn “The Eskimo!”
          Apple Developer Relations, Developer Technical Support, Core OS/Hardware
          let myEmail = "eskimo" + "1" + "@apple.com"

            • Re: macOS wifi ping spikes
              nbolton Level 1 Level 1 (0 points)

              Hmm, what do you think about switching the Synergy protocol to UDP?

                • Re: macOS wifi ping spikes
                  eskimo Apple Staff Apple Staff (12,295 points)

                  what do you think about switching the Synergy protocol to UDP?

                  I presume you’re asking about what effect this will have on this specific latency issue?  That really depends on the cause of the latency.  If it is caused by channel switching, changing to UDP won’t have any effect.

                  It’s a common misconception that UDP has lower latency than TCP.  From a latency perspective the only benefit that UDP has is that, if the network drops packet N, you don’t have to wait for a retransmissions before you see packet N+1.  In your case, however, there’s no indication that packets are being dropped (which makes sense because Wi-Fi has its own link-level flow controller and retransmission mechanism).

                  Share and Enjoy

                  Quinn “The Eskimo!”
                  Apple Developer Relations, Developer Technical Support, Core OS/Hardware
                  let myEmail = "eskimo" + "1" + "@apple.com"

                • Re: macOS wifi ping spikes
                  marcbou Level 1 Level 1 (0 points)

                  I am also seeing this annoying problem on my latest gen MacBook Pro running 10.13.4, and it is affecting basic ssh sessions in terminal to the point of making them very unresponsive and frustrating to use at times..

                   

                  It doesn't happen on other platforms, so Mr. Eskimo, Instead of finding excuses please ESCALATE THIS PROBLEM so that it gets properly investigated and fixed ASAP.

                   

                  Marc

                  • Re: macOS wifi ping spikes
                    rdrom Level 1 Level 1 (0 points)

                    I agree that it's probably wifi scanning that causes the latency. It seems that the locationd process triggers this approximately once a minute. If you start a ping in the terminal:

                     

                    ping -i 0.25 192.168.1.1 --apple-time 

                     

                    ...and watch the activity in Console.app, you'll notice that the ping response time rises as described just as locationd starts doing it's once-per-minute update. If you disable Location Services in the Privacy pref pane the periodic ping latency issues go away completely.

                     

                    This latency causes major issues in audio quality with apps like GoToMeeting and causes video corruption and lag in streaming video game apps like Google's Project Stream. These issues go away when locationd isn't scanning!

                     

                    Seems like there's a bug here somewhere in locationd... if it's not possible to avoid the latency during a wifi scan then perhaps locationd needs to either skip those altogether or perform them far less often. As it is today, I need to disable Location Services on my MBP if I'm joining a conference call on wifi.

                      • Re: macOS wifi ping spikes
                        eskimo Apple Staff Apple Staff (12,295 points)

                        Seems like there's a bug here somewhere in locationd.

                        If this problem is causing you significant grief that you should file a bug about it.  Make sure to include the extra Wi-Fi logging info per the instructions on our Bug Reporting > Profiles and Logs.

                        Please post your bug number, just for the record.

                        Share and Enjoy

                        Quinn “The Eskimo!”
                        Apple Developer Relations, Developer Technical Support, Core OS/Hardware
                        let myEmail = "eskimo" + "1" + "@apple.com"

                          • Re: macOS wifi ping spikes
                            marcbou Level 1 Level 1 (0 points)

                            Dear Eskimo,

                             

                            rdrom, myself and others have extensively described the issue. Is that not sufficient for Apple to take action? You are the Apple employee, please do your work, file whatever bug report(s) are needed and above all, have this long standing issue fixed once and for all.

                              • Re: macOS wifi ping spikes
                                eskimo Apple Staff Apple Staff (12,295 points)

                                Is that not sufficient for Apple to take action?

                                It is best if you file your own bug reports because:

                                • You are the one seeing the problem, so you can describe it accurately

                                • If Wi-Fi Engineering has follow-up questions about the issue, they can contact you directly

                                • You can see the status of bug reports that you file

                                You are the Apple employee, please do your work …

                                The vast majority of my “work” involves responding to official DTS tech support incidents [1].  My time here on DevForums is roughly split 50:50 between my work time and my personal time.  I choose to spend that time answering questions because I think that’s a more valuable use of that time than filing bug reports.

                                Share and Enjoy

                                Quinn “The Eskimo!”
                                Apple Developer Relations, Developer Technical Support, Core OS/Hardware
                                let myEmail = "eskimo" + "1" + "@apple.com"

                                [1] You can learn more about this service on our page on the developer web site.

                        • Re: macOS wifi ping spikes
                          neurogenesis Level 1 Level 1 (0 points)

                          Even if location services are disabled, wifi scanning can destroy wireless performance. Scanning will essentially pause communications on your current wifi connection, as the hardware has to hop to other channels to scan for available access points. Depending on how long this takes, you could see an adverse impact on your current connection.

                           

                          Applications and plugins can trigger this behavior by trying to interact with the locationd service, such as web browsers or QuickTime (it apparently has known bugs that will cause the application using qt libraries to trigger wifi scanning). This will manifest itself as frequent or periodic ping spikes and streaming performance drops (audio or video conference calling will drop quality or cut out entirely).

                           

                          Video and voice calling, particularly over wireless, can be severely impacted. You should disable all wireless devices and use ethernet only if you require anything related to audio or video over the network. If your office is wireless only, you may need to re-evaluate your hardware / OS options.

                           

                          One way to confirm this is to do the following:

                           

                          1) open Apple's "Console" Application

                          2) add "scan" as asearch term to the top-right

                          3) scroll to the bottom of the log pane so new events show up at the bottom as they occur

                          4) open the terminal / iterm2 app and ping either '1.1.1.1' or your local router

                          5) click on your wireless icon and see if your ping lag spikes occur at the same time as the "locationd" messages

                           

                          If you have location services disabled, you will see the following:

                           

                          default 12:01:58.360773 -0400 locationd WIFI_LOC: location services are disabled, ignore scan result

                          default 12:01:58.907701 -0400 locationd WIFI_LOC: location services are disabled, ignore scan result

                          default 12:01:59.469546 -0400 locationd WIFI_LOC: location services are disabled, ignore scan result

                          default 12:02:00.089787 -0400 locationd WIFI_LOC: location services are disabled, ignore scan result

                          default 12:02:00.922026 -0400 locationd WIFI_LOC: location services are disabled, ignore scan result

                           

                          If location services are enabled, you will see the following:

                           

                          default 12:03:25.995118 -0400 locationd WIFI_LOC: onWifiNotification, notification, <private>, kNotificationScan, scanType, <private>, <private>, inject, <private>, inProg, <private>

                          default 12:03:25.995167 -0400 locationd WIFI_LOC: scan result, scanType, <private>, <private>, currentRequest, <private>

                          default 12:03:26.546485 -0400 locationd WIFI_LOC: onWifiNotification, notification, <private>, kNotificationScan, scanType, <private>, <private>, inject, <private>, inProg, <private>

                          default 12:03:26.546536 -0400 locationd WIFI_LOC: scan result, scanType, <private>, <private>, currentRequest, <private>

                          default 12:03:26.705880 -0400 locationd WIFI_LOC: processScanResult, aps, <private>, fIsUseNetworkLocationProvider, <private>, fIsUseTileManager, <private>

                          . . .

                           

                          The ping latency may improve with location services disabled, but the problem likely will not be eliminated. You must identify the application, service, or plugin that is triggering wifi scanning.

                           

                          Hopefully someone from Apple can chime in with a simple method of doing so (like monitoring / tracing to capture apps). In the event they don't respond, you might try taking a look at capturing additional logging...

                           

                          https://www.howtogeek.com/211034/troubleshoot-and-analyze-your-mac%E2%80%99s-wi-fi-with-the-wireless-diagnostics-tool/

                          https://superuser.com/questions/585473/debugging-osx-airport-wifi-connection

                           

                          Related links:

                           

                          https://superuser.com/questions/1142798/experiencing-high-latency-on-wifi-every-other-second-with-macos-sierra

                          https://superuser.com/questions/1142798/experiencing-high-latency-on-wifi-every-other-second-with-macos-sierrahttps://apple.stackexchange.com/questions/310218/stop-destroying-network-performance-every-time-an-app-scans-for-wifi-networks/312388

                          https://lostdomain.org/2017/06/17/qt-qnetworkaccessmanager-causing-latency-spikes-on-wifi/

                          https://bugreports.qt.io/browse/QTBUG-65586?gerritReviewStatus=All

                          https://discussions.apple.com/thread/8523377

                            • Re: macOS wifi ping spikes
                              ethaniel Level 1 Level 1 (0 points)

                              Thank you! You saved my day.

                               

                              Here is what I did:

                              System Preferences -> Security and Privacy -> Location Services (left) -> System Services (at the bottom of the list) -> Details -> Turn off "Wi-Fi Networking"

                              And after that - reboot the computer.

                               

                              The problem went away.

                            • Re: macOS wifi ping spikes
                              togume Level 1 Level 1 (0 points)

                              +1 on this issue. I've been chasing the ping-spikes for about a month after moving a new office. I hadn't experienced this in other networks. However, at this location, I was having high-latency spikes every ~5min. Very frustrating when you're in meetings and things freeze. Even worse, I was also experiencing packet loss when this happened.

                               

                              In my case, similarly, the fix was to fully turn off location services. I don't want to given apps like Find My Mac, but I can't live with the ping spikes.

                               

                              @Eskimo - can you let me know if this is already logged as a bug? Otherwise, I'll be happy to create one.

                                • Re: macOS wifi ping spikes
                                  eskimo Apple Staff Apple Staff (12,295 points)

                                  can you let me know if this is already logged as a bug?

                                  Realistically, you should file your own bug about this even if it’s others have already done so.  These problems are often specific to both your hardware and your environment.

                                  Before filing a bug, review the comments in my 2 Mar 2018 and 25 Oct 2018 posts.  Investigating bugs like this is hard, and it’s best if you can provide detailed information about the problem.  IMO it’s worth going through the debugging procedure described in Investigating Network Latency Problems to help isolate the hop that’s causing the delay.  And retain the evidence you collect during that process so that you can attach it to your bug report.

                                  Share and Enjoy

                                  Quinn “The Eskimo!”
                                  Apple Developer Relations, Developer Technical Support, Core OS/Hardware
                                  let myEmail = "eskimo" + "1" + "@apple.com"