I have an app where we want to stream video frames locally from one device to another. We were initially using Multipeer, but wanted to force the user to have to be using infrastructure wifi rather than something like p2p BT.
Using Network.framework, I start a udp NWListener and send the host/port to the client which then connects. This all works well. On the app we compress the frames, send them to the other device, and decompress them on the other end. This all works fine except there are sometimes periods of freeze (oddly when the camera is trying to focus). There are no explicit errors from the framework, but we are getting this in the console during the freeze:
2021-05-19 20:04:57.870871-0400 App Name[17116:5431473] [] nw_protocol_ipv4_frame_output_finalizer [C3:2] Not enough bytes to fragment 70806
Although we don't get an explicit error from Network.framework, I'm assuming that the message size is too big. I decided to just create an empty project and attempt to send one (uncompressed) frame across. I set it up just like in the WWDC 2018 video using bonjour and udp instead of host port and it got me:
2021-05-20 15:42:59.656575-0400 NetworkTest[65379:1172081] [connection] nw_flow_prepare_output_frames [C1.2.1 fe80::9:9e05:f383:58a0%en10.52624@en10 ready socket-flow (satisfied (Path is satisfied), interface: en10, expensive)] Data won't fit in frame (9216 84272)
2021-05-20 15:42:59.656930-0400 NetworkTest[65379:1172081] [connection] nw_write_request_report [C1] Send failed with error "Message too long"
2021-05-20 15:42:59.657056-0400 NetworkTest[65379:1172081] [connection] nw_flow_prepare_output_frames [C1.2.1 fe80::9:9e05:f383:58a0%en10.52624@en10 ready socket-flow (satisfied (Path is satisfied), interface: en10, expensive)] Failed to use 1 frames, marking as failed
Failed to send data with error The operation couldn’t be completed. (Network.NWError error 0.)
So I guess my question is, in my first error is this a problem with the messages being too long? It seems like in general my message sizes are short enough but the occasional spike is causing a decent freeze, here is an output of printing data.count:
22713
23313
23236
22278
22545
22342
22920
21996
70798
2021-05-19 20:04:57.870871-0400 Encore Artist Dev[17116:5431473] [] nw_protocol_ipv4_frame_output_finalizer [C3:2] Not enough bytes to fragment 70806
And for the second error in my test project, I was wondering why it is explicit and different. Perhaps because bonjour/udp instead of hostname/port/udp?
Finally, is there demonstration or something I could see that would help me learn how to split up these larger messages with Network.framework over udp? WWDC 2018 has a demo of doing exactly what I want to do, but they gloss over the splitting of frames parts and there's no source code that I could find. Thank you in advance!
Post
Replies
Boosts
Views
Activity
Xcode Version 16.0 (16A242d)
iOS18 - Swift
There seems to be a behavior change on iOS18 when using AppShortcuts and AppIntents to pass string parameters. After Siri prompts for a string property requestValueDialog, if the user makes a statement the string is passed. If the user's statement is a question, however, the string is not sent to the AppIntent and instead Siri attempts to answer that question.
Example Code:
struct MyAppNameShortcuts: AppShortcutsProvider {
@AppShortcutsBuilder
static var appShortcuts: [AppShortcut] {
AppShortcut(
intent: AskQuestionIntent(),
phrases: [
"Ask \(.applicationName) a question",
]
)
}
}
struct AskQuestionIntent: AppIntent {
static var title: LocalizedStringResource = .init(stringLiteral: "Ask a question")
static var openAppWhenRun: Bool = false
static var parameterSummary: some ParameterSummary {
Summary("Search for \(\.$query)")
}
@Dependency
private var apiClient: MockApiClient
@Parameter(title: "Query", requestValueDialog: .init(stringLiteral: "What would you like to ask?"))
var query: String
// perform is not called if user asks a question such as "What color is the moon?" in response to requestValueDialog
// iOS 17, the same string is passed though
@MainActor
func perform() async throws -> some IntentResult & ProvidesDialog & ShowsSnippetView {
print("Query is: \(query)")
let queryResult = try await apiClient.askQuery(queryString: query)
let dialog = IntentDialog(
full: .init(stringLiteral: queryResult.answer),
supporting: .init(stringLiteral: "The answer to \(queryResult.question) is...")
)
let view = SiriAnswerView(queryResult: queryResult)
return .result(dialog: dialog, view: view)
}
}
Given the above mock code:
iOS17:
Hey Siri
Ask (AppName) a question
Siri responds "What would you like to ask?"
Say "What color is the moon?"
String of "What color is the moon?" is passed to the AppIntent
iOS18:
Hey Siri
Ask (AppName) a question
Siri responds "What would you like to ask?"
Say "What color is the moon?"
Siri answers the question "What color is the moon?"
Follow above steps again and instead reply "Moon"
"Moon" is passed to AppIntent
Basically any interrogative string parameters seem to be intercepted and sent to Siri proper rather than the provided AppIntent in iOS 18