IOS13 - Help with using NWProtocolFramer - using delimiter to retrieve messages across TCP packets

I have a device that my app needs to communicate with over TCP. It uses a simple communication protocol that separates messages with a single character delimiter. I have been attempting to use the NWProtocolFramer implementation attached to my connection and am having problems when messages are across TCP packets.


In my case, once I've sent the command to retrieve the messages, the first item the device sends back is the number of expected messages. I then retrieve this number and then in a loop attempt to retrieve the remaining messages. The pseudo code looks like this:

let command = opCode.commandString.data(using: .utf8)
connection.send(content: command, completion: .contentProcessed({ (error) in
     // handle error
}))

connection.receiveMessage(completion: { (content, context, isComplete, error) in
    let numberOfMessages = Int(String(data: content!, encoding: .utf8))

    for count in 0..<numberOfMessages {
        connection.receiveMessage(completion: { (content, context, isComplete, error) in
            let message = String(data: content!, encoding: .utf8)
            // process message
        })
    }
})


The NWProtocolFramer.handleInput method psuedo code looks like this (thanks to Quinn's post : https://forums.developer.apple.com/thread/118686

func handleInput {
    while true {
        var didFindDelimiter = false
        var countBeforeDelimiter = 0
        let parseResult = framer.parseInput(minimumIncompleteLength: 1, maximumLength: 65535) {
            … search the data for the delimiter then
            set up `didFindDelimiter` and `countBeforeDelimiter` …
            return 0
        }
        guard
            parseResult, didFindDelimiter || countBeforeDelimiter != 0
        else {
            // Nothing to deliver right now.
            return 0
        }
        let didDeliver = framer.deliverInputNoCopy(length: countBeforeDelimiter, message: …, isComplete: didFindDelimiter)
        guard didDeliver else {
            return 0
        }
  
        if didFindDelimiter {
            let _ = framer.parseInput(minimumIncompleteLength: 1, maximumLength: 65535) { (buffer, isComplete) -> Int in
                // move the buffer pointer down by 1 to discard the delimiter
                return 1
            }
        }
    }
}

This works succssfully if all messages appear within a single TCP packet, but I have a situation where the messages go across the TCP packets as follows:

Total number of messages expected: 78

Packet 1: data length: 1514 bytes, contains 54 messages

Packet 2: data length: 749 bytes, contains 24 messages


The handleInput method parses 53 messages and sends them back to the connection.receiveMessage. The 54th message though is only partially contained in the input buffer (cannot see its delimiter in the current input buffer) and so the handleInput does not send anything back (ie didFindDelimiter is always false).


Essentially, the handleInput is in a continuous loop and the connection.recieveMessage just sits and waits and the remaining (1 + 24 messages) are not processed and essentially this thread is stuck.


How should I handle this situation?

a) How do I ask for the next TCP packet to appear in the inputBuffer, or

b) Should I send back an incomplete message and somehow append the results together


Any advice or guidance here would be much appreciated.


Kind regards


Doug

Answered by dougie@wwp.com.au in 641128022
I eventually resolved this issue, although not sure if the approach I've taken is the best method:

Thanks to Matt Eaton for his assistance. To help explain how I solved this, this is an example of the network bytes that arrive at handleInput
Code Block
|        PACKET A        |        PACKET B        |
|<message>|<message>|<message><mess...age>|<message><message><message><m...essage>
    1     2     4    5a   5b    6     7     8    9a   9b


Some notes:
  • Used class level var to store the bytes from any partial messages (eg., lhsMessage and rhsMessage ; used to store 5a and 5b from above example)

  • when in the parseInput closure, if no delimiter found, store 5a in lhsMessage and return the (lhsMessage.count) to move the buffer cursor to the end.

  • At this point, PACKET B then arrives in handleInput

  • Again, while in parseInput closure, if there exists a lhsMessage, then the next bytes from 0..<countBeforeDelimiter will be rhsMessage. Return (rhsMessge.count + 1) to again move the buffer cursor along.

  • to deliver the custom message (lhsMessage + rhsMessage), used deliverInput rather than deliverInputNoCopy


I have been attempting to use the

NWProtocolFramer
implementation attached to my connection and am having problems when messages are across TCP packets.

You should open a DTS tech support incident so that I can take an in-depth look at your problem (unfortunately that won’t be until next year because DTS is officially closed for the winter break).

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

ps DTS is closed 21 Dec through 1 Jan.

Were you able to resolve this issue?
Hi Damiaan, No I had not resolved it... I am having trouble understanding how the NWProtocolFramer works.

I had to put a hack (external to the the NWProtocolFramer) in place to get it working... not ideal.

I've started looking into it again now.
Accepted Answer
I eventually resolved this issue, although not sure if the approach I've taken is the best method:

Thanks to Matt Eaton for his assistance. To help explain how I solved this, this is an example of the network bytes that arrive at handleInput
Code Block
|        PACKET A        |        PACKET B        |
|<message>|<message>|<message><mess...age>|<message><message><message><m...essage>
    1     2     4    5a   5b    6     7     8    9a   9b


Some notes:
  • Used class level var to store the bytes from any partial messages (eg., lhsMessage and rhsMessage ; used to store 5a and 5b from above example)

  • when in the parseInput closure, if no delimiter found, store 5a in lhsMessage and return the (lhsMessage.count) to move the buffer cursor to the end.

  • At this point, PACKET B then arrives in handleInput

  • Again, while in parseInput closure, if there exists a lhsMessage, then the next bytes from 0..<countBeforeDelimiter will be rhsMessage. Return (rhsMessge.count + 1) to again move the buffer cursor along.

  • to deliver the custom message (lhsMessage + rhsMessage), used deliverInput rather than deliverInputNoCopy


IOS13 - Help with using NWProtocolFramer - using delimiter to retrieve messages across TCP packets
 
 
Q