How do I implement low-latency streaming between two local iOS devices?

Hello,

I'd like to know whether Multipeer Connectivity (MPC) between two modern iOS devices can support cross-streaming of video content at low latency and relatively high resolution. I learned from Ghostbusters that crossing the streams is a bad idea, but I need this for my app so I'm going to ignore their sage advice.

The application I have in mind involves one iOS device (A) live-streaming a feed to another iOS device  (B) running the same app. At the same time, B is live streaming its own feed to A. Both feeds don't need to be crystal clear, but there has to be low latency. The sample code for "Streaming an AR Experience" seems to contain some answers, as it's based on MPC (and ARKit), but my project isn't quite AR and the latency seems high.

If MPC isn't suitable for this task (as my searches seem to indicate), is it possible to have one device set up a hotspot and link the two this way to achieve my cross-streaming ambitions? This seems like a more conservative method, assuming the hotspot and its client behave like they're wifi peers (not sure). I might start a new thread with just this question if that's more appropriate.

A third idea that's not likely to work (for various reasons) is data transfer over a lightning-lightning or lightning-usb-c connection.

If I've missed any other possible solutions to my cross-streaming conundrum, please let me know.

I've been reading around this subject on this forum as well and would be hugely grateful if Eskimo would grace me with his wisdom. 

Answered by DTS Engineer in 703756022

I should ask for one clarification regarding APs not forwarding STA-to-STA traffic

The problem here is that Personal Hotspot is a user-facing feature, not a developer API. Last I checked it will forward STA-to-STA traffic [1]. However, that’s not really a part of Personal Hotpot’s core mission, which is to provide Internet connectivity to the STAs. You could imagine a world where Apple changed this behaviour. The vast majority of Personal Hotspot users wouldn’t notice, but your use case would be completely toast.

So, is this behaviour going to change? I can’t predict The Future™ in general, and that’s doubly so for user-level features like Personal Hotspot.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] Including the not-really-a-STA case of an app running on the device running Personal Hotspot.

Multipeer Connectivity only works on the local network, so I presume that’s a constraint that you’re prepared to accept. However, there’s one wrinkle here. Multipeer Connectivity supports two local network scenarios [1]:

  • Infrastructure Wi-Fi, where both devices are on the same infrastructure Wi-Fi network

  • Peer-to-peer, where they are not

Do you care about the latter?

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] There’s nothing special about Multipeer Connectivity here. Many other networking APIs, including Network framework, support peer-to-peer Wi-Fi.

I only care about local peer-to-peer for this specific use-case.

The term “local peer-to-peer” means different things to do different people. Do you expect these devices to be on the same infrastructure Wi-Fi?

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Don’t worry about not understanding this stuff. Peer-to-peer Wi-Fi is a complex topic. Even if you limit yourself to the Apple ecosystem, there are two implementations in play! [1] And there many more outside of the Apple ecosystem.

I do not expect the devices to be connected to the same infrastructure Wi-Fi

OK.

with the session being purely and directly between them

Just for context, that’s not how Apple peer-to-peer Wi-Fi works. Apple peer-to-peer Wi-Fi is not point-to-point. Rather, the peers negotiate and then use a shared Wi-Fi network.

There’s a trade-off in play here. In general, infrastructure Wi-Fi is faster than peer-to-peer Wi-Fi. Moreover, the use of peer-to-peer can degrade the performance of infrastructure. So, if you’re looking for best performance, it’s better to require infrastructure Wi-Fi.

This is one (of the many :-) place where Network framework is better than Multipeer Connectivity because Network framework gives you control over whether to use peer-to-peer on not (via its includesPeerToPeer property). If you using Multipeer Connectivity, you always opt in to peer-to-peer.


Coming back to your original question, you wrote:

I'd like to know whether Multipeer Connectivity (MPC) between two modern iOS devices can support cross-streaming of video content at low latency and relatively high resolution

I don’t think Multipeer Connectivity is an issue here; it adds some overhead but not a substantial amount [2].

The real issue is whether the underlying peer-to-peer network can deal with your load. It’s hard to answers that for two reasons:

  • You didn’t post any concrete info about your load.

  • Even if you had, network performance tends to vary a lot based on the situation, both with the devices themselves and the local RF environment.

What I recommend is that you set up some performance tests. First look at the video side of this, so you can get a handle on your latency and bandwidth requirements. Then create a tiny test app that transfers data using Network framework, run it in realistic circumstances, and see if it gets anywhere near those requirements.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] You could see evidence for this in older versions of macOS where the Finder > Go > AirDrop window had a “search for older devices” option. Fortunately that older protocol has now fallen by the wayside.

[2] Assuming you restrict yourself to its datagram features. The high-level stream and resource transfer facilities are more complex and thus its hard to evaluate their performance.

Oh, I should’ve posted this link earlier: Wi-Fi Fundamentals. It defines the terms that I use in answers like this.

If I have one device set up a hotspot, and have the other connect to it, would I be able to use the resulting network as though both devices are sharing an infrastructure Wi-Fi?

Hmmm, tricky. Personal Hotspot certainly does create an infrastructure Wi-Fi network. The problem with relying on it is that it doesn’t guarantee connectivity between your code running on a STA and your code running on the AP. I believe that this connectivity works in practice, but if you build a product around it then you’re relying on it working that way forever.

This is an issue with infrastructure Wi-Fi in general. To quote the post I referenced about: Some APs refuse to forward STA-to-STA traffic.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

Edit:

@Eskimo

Err come to think about it I should ask for one clarification regarding APs not forwarding STA-to-STA traffic before I burn myself:

In your previous response you were ambiguous about the device brand, probably due to all the Peer-to-Peer Wi-Fi specifications out there.

My use case is only between Apple products. Period.

Does the phenomenon of some APs not forwarding STA-to-STA traffic apply to iPhone Hotspots talking with iPhones or iPads or Macs? E.g. Can an iPhone hosting a Hotspot Connection (being both AP and STA) and the iPhone being hosted (the pure STA) communicate with each other (e.g. cross streaming videos)?

I did a double take and came back to ask because this is critical to me.

Accepted Answer

I should ask for one clarification regarding APs not forwarding STA-to-STA traffic

The problem here is that Personal Hotspot is a user-facing feature, not a developer API. Last I checked it will forward STA-to-STA traffic [1]. However, that’s not really a part of Personal Hotpot’s core mission, which is to provide Internet connectivity to the STAs. You could imagine a world where Apple changed this behaviour. The vast majority of Personal Hotspot users wouldn’t notice, but your use case would be completely toast.

So, is this behaviour going to change? I can’t predict The Future™ in general, and that’s doubly so for user-level features like Personal Hotspot.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] Including the not-really-a-STA case of an app running on the device running Personal Hotspot.

How do I implement low-latency streaming between two local iOS devices?
 
 
Q